> I thought using loops was cheating, so I programmed my own using samples. I then thought using samples was cheating, so I recorded real drums. I then thought that programming it was cheating, so I learned to play drums for real. I then thought using bought drums was cheating, so I learned to make my own. I then thought using premade skins was cheating, so I killed a goat and skinned it. I then thought that that was cheating too, so I grew my own goat from a baby goat. I also think that is cheating, but I’m not sure where to go from here. I haven’t made any music lately, what with the goat farming and all.
Just import "AmenBrother-drums" or something and start from there, because obviously you're not going to use Zoidberg's leftmost tentacle in your cool new sound.
Need to do X? If you write your own library that does X, chances are you'll be the only one to ever work on it. Need a new feature? You have to stop working on your actual project and implement that feature. Found a bug? No one else will fix it for you.
If you depend on a library that thousands of other people also use that does X, if you do find a feature you need that it doesn't have, open a ticket, someone will likely do that work for you. More often than not, that feature already exists, all you have to do is read the docs. If you find a bug, report it and and wait for it to get fixed, but there's also nothing stopping you from fixing it yourself and submitting a pull request.
http://fdiv.net/2012/04/01/objectivist-c
"In Objectivist-C, software engineers have eliminated the need for object-oriented principles like Dependency Inversion, Acyclic Dependencies, and Stable Dependencies. Instead, they strictly adhere to one simple principle: No Dependencies."
— Carl SaganThings are different if we’re talking live performance. You’d be fine with that multi-terabyte cello library on stage, but not if sampling plugin brought opaque nth-party dependencies you can’t vouch for and have no time to properly wrap your head around. You don’t want surprises and you want to know for sure there’s enough performance leeway on your concert laptop.
There are steps you’d take to reduce the error margin while you perform that have no parallels in software engineering realm. Maybe you’d bounce your cello loops to audio to never have that computation happening in real time. Maybe you’d bring analog synths, which are bulky and pricey but also simpler conceptually, self-sufficient, well-tested and never give blue screen of death.
Your song is essentially being created every moment your program is running or being developed. You’re putting even more trust in what you’re pulling—the black box abstraction boundaries around your dependencies can contract, the input you receive from audience is more direct, and importantly there’s no you actually playing instruments and directing the performance because the whole behavior is defined by algorithms that often end up partly delegated to dependencies.
now, onto the the problem I have with the actual issue at hand: it's not like the author is saying to write your libraries in assembler. he's saying that maybe you don't need to include gems upon gems that themselves reference other gems, as the dependencies pileup and get ridiculous and the performance suffers, which is actually a major concern with Rails and other monolithic frameworks with single-threaded processing etc. for crying out loud, he's even giving you permission to specify your behaviors with a ridiculously high-level concise language aka Ruby... how much more direct and obvious can a point be and still be missed?
Actually there are songs using samples that are 10 times more original and unique than songs other people have totally recorded and played themselves.
You can be inventive and original using samples and you can be a copy-cat bore writing your own stuff.
For example, let's compare a cheesy, but still huge classic and immediately recognizable "U can't touch this" with tons of stupid-ass Michael Bolton cliched ballads or MOR rock.
http://www.nizkor.org/features/fallacies/slippery-slope.html
Edit: Dart is an interesting case. It has dynamic typing, but it's static enough that tree-shaking is feasible. Seth Ladd's blog post about why tree-shaking is necessary [1] makes the same point that I'm making here.
[1]: http://blog.sethladd.com/2013/01/minification-is-not-enough-...
For that matter, there's no guarantee that tree-shaking would even have fixed the referenced issue; if the library preloaded 10MB of stuff, like a Unicode definition table, that you didn't use, but the tree shaker couldn't quite prove you never would, you'll still end up with it loaded at runtime. (For that matter, you may very well be using such code even though you don't mean to, if, for instance, you have code that attempts to load the table, and uses it if it loads for something trivial, but will just keep going without it if it is not present. The tree shaker will determine (correctly!) that you're using it.)
Basically, tree shaking only sort of kind of addresses one particular problem that deep dependencies can introduce, and that one not even necessarily reliably and well.
About your 10 MB Unicode table example, did you have in mind the ICU data table that's included with Chromium and, more recently, all the Electron-based apps?
There's still no substitute for good code hygiene and knowing exactly what you're using and what additional bloat you're bringing in when you add a library.
(... note that one could make a similar argument for more runtime-dynamic languages. I won't disagree, other than to observe that as a lone engineer, I've managed to code myself into a corner with dependencies in Rails ;) ).
That aside, that is the third time in a couple of months that I have heard people mention situations where they ran out of RAM without explaining why swap could not at least work as a stop gap measure.
You need both tree shaking and a community dedicated to keeping code small.
Javascript has the latter; it's not universally true, but lots of JavaScript libraries pride themselves on small code size and few dependencies.
That's great. But you can't stop doing that work just because you have a tree-shaking compiler. For example, there's a lot of work going into making Angular 2 apps reasonably sized and dart2js doesn't magically make it go away.
When building Android applications, it's common to process the JVM bytecode with ProGuard before converting it to Dex bytecode. ProGuard includes a tree-shaking step. Sometimes it's necessary to tell ProGuard about specific classes or class members that it should leave alone, if they're accessed dynamically (e.g. using java.lang.reflect or Class.forName). But it's still better than nothing.
Likewise, .NET applications for the Windows Store are compiled to native code using .NET Native, and that compilation includes a tree-shaking step. This introduces some limitations on the use of reflection. I'm guessing similar limitations will apply to the native compilation option of .NET Core.
As for JavaScript, Google's Closure Compiler can do tree-shaking. But I don't know if the Closure Compiler's advanced mode works with any of the popular JavaScript libraries or frameworks, or just Google's Closure Library.
Tree shaking doesn't help you when you are pulling in every HTTP client in existence transitively. It is still code that is being run, so can't be automatically optimized away, but it is unnecessary.
Tell that to anyone using NPM.
You're doing a quick MVP to demonstrate that your idea is working? Fuck it, just throw in dependencies for everything, just care about solving the problem you're trying to solve and proving/disproving your point.
Once you verified it, then go and kill your dependencies. But don't do it just because you want to do it. If in the end the users doesn't benefit from you optimizing your dependencies, why do it? (Speaking from a product side rather than a OSS project used by other projects)
Not sure KILL ALL DEPENDENCIES is helpful, but I'm not sure that MAKE EVERYTHING A DEPENDENCY is helpful either so...
The big thing is transitioning to any kind of final production code. The rules for clean code apply as much for Ruby as it does for any other language.
But its a good post, it can easily be something you overlook, due to gems being so damn convenient.
But, the advice is really important for gem writters. As a gem author, I think you really need think a little more about our dependencies as you do with our public interface.
As time approaches infinity, the number of magic "I use this package and it does something in my code, and then it all just works" dependencies you pull in should approach 0.
Your app uses to much memory? Improve a dependency, you have now improved other peoples apps, too.
Your app uses to much dependencies in total? Try to get all your first-level dependencies to standardize on the best http-client. (Which he is partially doing with his post.)
Dependencies may have problems, but shared problems are better than problems only you have.
I used to bring in dependencies with the "don't reinvent the wheel" mentality. Then I realized how much trust I'm giving to the authors of all dependencies I pull in. Now I tend to do my best to understand the dependencies I bring so I can improve them if I can.
The only problem I find with this decision is when I make an improvement/fix a bug on a dependency, and the project is either inactive or the authors don't give a crap about your work.
The only problem I find with this decision is when I make an improvement/fix a bug on a dependency, and the project is either inactive or the authors don't give a crap about your work.
Dependencies are great for the reasons you specified, and I saw nothing in that article suggesting otherwise. The part that feels the worst to read is:
> Can I implement the required minimal functionality myself? Own it.
This is largely a judgement call; "can I" and "minimal functionality" are subject to change based on many external circumstances. "Own it" also seems to imply owning it not as a dependency, based on the context, but rather as a part of a monolithic whole.
It is also interesting that the sidekiq product makes use of gem dependencies. At top level 5 without platform dependencies, which (mostly due to rails) expands out to many more. The message should not be to "kill your dependencies", because that mindset is outdated and slow.
So tired of hearing about how bad dependencies or scripting languages are. Would be much more excited to hear about how to contribute to open source dependencies, and how to write efficient scripts.
This goes for client-side JavaScript too. XSS holes are one of the worst web app vulnerabilities out there and could easily be introduced accidentally by a simple mistake in a library. And this stuff is incredibly hard to audit these days thanks to the JavaScript community's cultural trend towards deeply nested dependencies.
I.e. I'd 100% use libxml to sanitize xml rather than trying and reimplementing xml parsing myself.
As always, trade offs.
OpenSSL has major security issues encountered on a relatively regular basis.
Do not do your users the disservice of rolling your own SSL implementation. ;)
Consider these three statements:
- No code runs faster than no code. - No code has fewer bugs than no code. - No code is easier to understand than no code.
For a language like scala where there is no json processing in the standard lib, if there is a json library that is battle tested, then by removing my own json code and leaning on that well tried and tested code for serialization/de-serialization, I've removed a whole bunch of code from my own library. The whole point of having modules as abstractions is to keep concerns neatly tucked in their own places to to increase re-use. By subscribing to the idea that my module should implement all of the functionality it needs, we're loosing the benefits of modularization.
I just went through this exercise myself in a library I maintain - I removed my own json code and put a library in. I removed a bunch of code and made the whole thing simpler by leaning on that abstraction.
Your example sounds like a situation where a dependency certainly makes sense.
You removed a bunch of code you understood, and added a bunch more code you don't understand, along with whatever technical debt, edge cases, and performance issues which are lingering in that library.
Adding a library is never removing code from your project, it's adding code you don't yet understand to your project. It can still be a net win, but it's not less code for you to maintain.
> You removed a bunch of code you understood, and added a bunch more code you don't understand, along with whatever technical debt, edge cases, and performance issues which are lingering in that library.
Not all code you've written is good code. Hell, not all code you've written you actually understand. Libraries and dependencies make sense in many cases. Don't write yet another JSON parsing library unless you really need to.
> Adding a library is never removing code from your project, it's adding code you don't yet understand to your project. It can still be a net win, but it's not less code for you to maintain.
It's referencing code that you don't maintain. If the maintainer is bad, use a different library.
It seems like this could also be cast as a major success for "semi" standard dependencies.
However when discussing languages with no specific standard library or languages who's standard library is missing feature y, then it's quite understandable to use a 3rd party battle tested dependency. In fact I'd go further and say it would be advisable to use a respected 3rd party library when dealing with code which handles security or other complex concepts with high failure rates.
There are lots of libraries that just put one interface on top of another interface. They don't do much actual work. Pulling in shims, especially if they pull in lots of other stuff you're not using, should be avoided.
If the dependency does real work you'd otherwise have to code, then use it.
Bugs happen, though. If you see a bug in a dependency, it is your job to report it at the very least, if not make an attempt to fix it. Without this community of people helping to improve a common codebase, we'd all be writing everything from scratch, and progress would move a lot slower.
http://www.joelonsoftware.com/articles/fog0000000007.html
Apparently Microsoft's Excel team had even written their own C compiler.
http://queue.acm.org/detail.cfm?id=2349257
History continues to repeat itself. Fake reuse and proliferation of unnecessary bloat are two of those recurring themes. Fight it whenever you can. The old TCL, LISP, Delphi, REBOL, etc clients and servers were tiny by modern standards. They still got the job done. Strip out bloat wherever you can. Also, standardize on one tool for each given job plus hide it behind a good interface to enable swapping it out if its own interface isn't great.
A lot of my projects are just wrappers around one main gem. Rails, Nokogiri, Roo, API wrapper gems. These are 'project gems'. If they give me problems, I'll re-evaluate the scope of the project and perhaps pick another gem to orient the project around. Once the project reaches maturity, I'll default to fixing the problem rather than re-engineering it unless the problems run deep.
Sometimes I'll use gems like Phoner to handle datatypes that are too tricky to do with regular Ruby. I'll call these 'utility gems'. When I include a utility gem, generally it has one job and one job only, it's invoked in exactly one place in the code and gets included in that file. I can generally replace a utility gem with stdlib Ruby code if I really need to.
I also have what I call 'infrastructure gems'. These are gems like pry, capistrano, and thor that I tend to include in every project where it seems they would be useful. These are gems that are worth getting to know very well because they solve really hard problems that you don't want to use stdlib for. If these give me problems I will do whatever I need to to resolve them and understand why the problem exists, because the costs of migrating off of them would be steep.
The decision to use a gem should not be taken too lightly, but nor should it weigh large on the mind. Be quick to try it out, but also quick to take it out.
Would you want to have one namespace for "official" modules and heavily influence everyone to use them? That's centralization (of governance). But, it's not centralization of a process that requires high availability. So the "drawback" is only that you centralize control and can make certain guarantees to developers on your platform.
When you're starting an ecosystem, you can choose a "main namespace" as yum, npm etc. does or you can choose the more "egalitarian" convention of "Vendor/product" as github and Composer do. I think, in the end, the latter leads to a lot more proliferation of crap, and as the articls said, multiple versions of everything existing side-by-side.
I have to deal with these issues when designing our company's platform (http://qbix.com/platform) and I think that having a central namespace is good. The platform installer etc. will make it super easy to download and install the "official" plugins. You can distribute your own "custom" plugins but the preferred way to contribute to the community would be to check what's already there first and respect that. If you REALLY want to make an alternative to something, make it good enough that the community's admins protecting the namespace will allow it into the namespace. Otherwise, promote it yourself, or fork the whole platform.
This is why every 1.0 product I work on I include every dependency that speeds up my development. In 2.0 the first things to do is prune all unnecessary dependencies and start minor rewrites when a dependency can be done in house (yeah yeah reinventing the wheel is a problem but most npm dependencies are small and many can be recreated internally without issue).
This is even more important if you're creating a library / module. My msngr.js library uses zero dependencies and yet can make http calls in node and the browser because it was easy to implement the minimal solution I needed without bringing in dependencies to support a single way of calling http.
Request itself in my opinion is a great piece of work, it does pretty much anything one would want to do with an HTTP client.
I've been using request and its many transitive dependencies for years without hitting any real problems due to this.
The dependencies you decide to implement yourself in a minimal fashion are code though. I generally agree with the article, but in the end It Depends™
It itself does not have any dependencies that aren't in core:
http://deps.cpantesters.org/?module=Try%3A%3ATiny;perl=lates...
I've made a point not to add any third parties references and packages I can avoid. I went ahead and got a third party scheduling engine, and the SQLite provider, but beyond that, I'm writing everything else myself so far.
First of all, I'm learning a lot in having to write stuff myself. At the very least, it's a great educational experience. I've worked with a lot of code samples, so I'm not going totally from scratch, but they're all at the very least tailored to my needs.
But for me, the big thing is keeping everything thin. The program loads in milliseconds. Almost all of the reference data for what it's built on is in one place (the .NET Framework Reference). And key, is that the features my program supports are the features I want and need, not the features some dependency has told me to have.
The biggest dependency I have, Quartz.NET, is actually the most confusing part. It's not structured like the rest of my program is, it's documentation leaves some things to be desired, and it does a lot more than I need it to. There's a lot of bloat I could cut out if I wrote my own scheduler, and maybe someday I will.
If your app has a long shelf time, the less deps you rely on, the easier to manage from what I've seen.
For some reason Golang feels like it makes sense here. Pretty much everything you need is in core. *Disclaimer, I don't have any Golang apps in prod but I'd love to hear from those that do.
I would revise this to: Don't bring in more code than you need. But if the choice is between writing something yourself and using someone else's well-tested, heavily-used library, always go for the latter.
How much time will it take to implement each option?
How much time will it take in the future to support it?
What security risk does each option incur?
What is the risk of the project being abandoned?
What is the risk of the project changing in non-backwards compatible ways?
What are the performance characteristics of each option?
On the other hand, I've been learning to use GSON's parser in my Sponge plugin (a Minecraft server) because the SpongAPI dependency pulls that in anyway.
Albeit, both libraries are dead simple to use so it's a bit contrived, but I see a lot of projects that would pull in Spring's RESTTemplate into a Camel project when they've already pulled in CXF or have Apache's HTTPClient readily available via other dependencies.
(And no, URLConnection is terrible. TERRIBLE.)
I also spend more time actually reading through specs to see how well they exercise the code.
That's probably standard procedure for a lot of people, but it's something that I had to learn to always do.
Absolutely. However, there are plenty of situations where what you pull down from npm or rubygems isn't actually all that well-written or well-tested.
When I first started programming I kind of had this impression that if an open source library is published on a package repo and people are using it then it must be much better than anything I could write. I have learned the hard way over the years that is not always true.
Mycode -> facade -> library
Navigating code becomes more cumbersome and stack traces longer. I do like this approach too but it isn't free.
A lot of the dependencies discussed in this blog post are libraries that aren't actually adding anything useful: the various JSON parsing libraries that should be replaced with the parser in stdlib, or rspec testing libraries that shouldn't have ever been a regular runtime dependency.
[1] Managing complexity and dependencies is probably the most important concern going into the future, not just in programming, but also in every other complex system.
I see it as a sliding scale. If I'm parsing 1 string with the same date format into 1 object, I'm not going to pull in some general purpose time parsing library - I'll write the 10 lines of code myself, a few unit tests, and be happy.
If in the future I start having to deal with different date strings and some need to do more than just throw up a single date on a page somewhere, I'll get a date/time library.
> Clear is better than clever.
> Reflection is never clear.
At the end of the day, the only person who is responsible for the quality of your proje t is you. You have to figure out which parts of your project are key to your operation and which are just window dressing. If your job is to make a blogging platform, I would expect many features of formatting documents to be reimplementations of other people's work, because you have to know you are relying on yourself for your core purpose.
I also don't understand the heartburn other developers have over knowing "OMG L, THERE IS SOMEONE ON TEH INTARWEBS AND THEY ARE REINVENTING THE WHEEL". It smacks of fear, a fear that is ultimately rooted in insecurity. If a person was secure in their knowledge of their skills, their ability to understand problems and fix them, then there should be nothing to fear from a dozen or a million different libraries doing the same thing and running into one or two of them on one's next project. It is just a matter of course.
I actually do fear a million reimplementations of, say, RSA.
My rule of thumb is if the stuff we need can be reduced to a few functions, it's better to copy so you at least know which code you are using and don't have thousands of lines of code in you repo where you don't know if they are ever being used,
If you take the time to do your research, the choice between rolling your own, copying or adding a dependency will become clear. If it's not becoming clear, then you haven't finished your homework. Learning is a good thing, yes it takes time, but it's time well spent, and it's fun above all.
You may discover that this thing that you thought was hard and needed a dependency is really a few lines of code (a good example is a graph implementation). It might even change your career path. At least that's been my experience in the nearly two decades of writing software.
"Can I implement the required minimial functionality myself? Own it" is advice one gives if one can't trust the libraries one depends upon to stay healthy, performant, and applicable to your use-case. Nobody'd recommend re-implementing readline or printf; if you have some heavy-lifting mathematics to do in Python, use numpy.
There's a natural tension between code reuse and avoiding dependencies. If you can avoid a big dependency by writing a couple hundred lines of low-maintenance code, its probably worth it.
Being an old fart Java developer I generally prefer things where you can plugin your own implementation (ie agnostic).
That is there is an extreme for killing your dependencies of either extreme copy'npaste OR which every library offers a plugin SPI (ie inversion of control) (or a combo of both).
The problem with the dependency injection above approach (aka Spring prior to Boot) is that you have developers doing lots of custom crap, bloated/overly engineered libraries, increased ramp up time, and configuration hell.
But I still think this is probably better than ole copy'n paste.. most of the time. I do hate dependencies though.
Take for example Hystrix. I'm just now fixing that the thing loads up its own configuration framework (Archaius) which uses static initialization. Archaius needed like 10 other dependencies. This is all really because Hystrix uses static singleton (HystrixPlugins) and many frameworks need this or else is incredibly difficult to get an implementation up (ie using pseudo singleton to avoid excessive passing of a context).
I'm looking to getting back into Ruby at some point later this year, but I might ignore Rails altogether so I don't miss out on learning a new fun language.
From the description all glslify does is look for files with the extensions .glsl, .frag, and .vert and lets you get their contents with `content = require(filename)`.
Sounds like it would be at most 10-30 lines of code. Nope
npm install --save glslify-loader
webpack-glsl-test@1.0.0 /Users/gregg/temp/webpack-glsl-test
└─┬ glslify-loader@1.0.2
└─┬ glslify@2.3.1
├─┬ bl@0.9.5
│ └─┬ readable-stream@1.0.33
│ ├── core-util-is@1.0.2
│ ├── isarray@0.0.1
│ └── string_decoder@0.10.31
├─┬ glsl-resolve@0.0.1
│ ├── resolve@0.6.3
│ └── xtend@2.2.0
├─┬ glslify-bundle@2.0.4
│ ├─┬ glsl-inject-defines@1.0.3
│ │ └── glsl-token-inject-block@1.0.0
│ ├── glsl-token-defines@1.0.0
│ ├── glsl-token-depth@1.1.2
│ ├─┬ glsl-token-descope@1.0.2
│ │ ├── glsl-token-assignments@2.0.1
│ │ └── glsl-token-properties@1.0.1
│ ├── glsl-token-scope@1.1.2
│ ├── glsl-token-string@1.0.1
│ └── glsl-tokenizer@2.0.2
├─┬ glslify-deps@1.2.5
│ ├── events@1.1.0
│ ├─┬ findup@0.1.5
│ │ ├── colors@0.6.2
│ │ └── commander@2.1.0
│ ├── graceful-fs@4.1.3
│ ├── inherits@2.0.1
│ └─┬ map-limit@0.0.1
│ └─┬ once@1.3.3
│ └── wrappy@1.0.1
├── minimist@1.2.0
├── resolve@1.1.7
├─┬ static-module@1.3.0
│ ├─┬ concat-stream@1.4.10
│ │ ├── readable-stream@1.1.13
│ │ └── typedarray@0.0.6
│ ├─┬ duplexer2@0.0.2
│ │ └── readable-stream@1.1.13
│ ├─┬ escodegen@1.3.3
│ │ ├── esprima@1.1.1
│ │ ├── estraverse@1.5.1
│ │ ├── esutils@1.0.0
│ │ └─┬ source-map@0.1.43
│ │ └── amdefine@1.0.0
│ ├─┬ falafel@1.2.0
│ │ ├── acorn@1.2.2
│ │ ├── foreach@2.0.5
│ │ └── object-keys@1.0.9
│ ├─┬ has@1.0.1
│ │ └── function-bind@1.0.2
│ ├── object-inspect@0.4.0
│ ├─┬ quote-stream@0.0.0
│ │ ├── minimist@0.0.8
│ │ └─┬ through2@0.4.2
│ │ └─┬ xtend@2.1.2
│ │ └── object-keys@0.4.0
│ ├── shallow-copy@0.0.1
│ ├─┬ static-eval@0.2.4
│ │ └─┬ escodegen@0.0.28
│ │ ├── esprima@1.0.4
│ │ └── estraverse@1.3.2
│ └─┬ through2@0.4.2
│ └─┬ xtend@2.1.2
│ └── object-keys@0.4.0
├── through2@0.6.5
└── xtend@4.0.1
---
update: I think maybe I misunderstood the description. glslify actually parses GLSL and re-writes it in various ways so maybe this is a bad example.
I've seen other though. Like 40k+ lines of deps for an ANSI color library or 200k+ lines deps and native node plugins for launching a browser from node.
An enormous portion of that code lives under static-module, which appears to do some kind of bundling/codegen/code swapping logic. So it depends on a parsing library for JavaScript.
I don't know if this is a good design, I don't follow the exact functions of this library, but I'm completely unimpressed by you having a drive-by reaction of "4 megs for this library?" It feels like you just scanned the library and decided it was wrong, without asking what it was trying to do.
In the same vein, what I want to see from this discussion is people delving into specifics. The original article did that a little, but I want to know more. What are the actual costs of tearing our dependencies? Which dependencies are worth suffering through? Perham uses the example of Net.HTTP. What do the various HTTP clients add to it? Are they just more terse, do they help to avoid various pitfalls, or do they actually "abstract" in such a way that they lead you to write bad code?
Not all free software is free.
The big change in mime-types 3 is using the columnar store by default, which is where the memory savings come from. It's opt-in from mime-types 2.6 onwards because it's a breaking change. Mail and afaik most other gems have opted in already.
I agree with the article, the less dependencies the better. GitLab's gemfile.lock https://gitlab.com/gitlab-org/gitlab-ce/blob/master/Gemfile.... has over 1000 lines and GitLab uses a lot of memory. We try to be careful what we pull in but if anyone has suggestions which can be removed please let us know. Recently we found out that we still had to remove Redcloth as a dependency, it will be gone in GitLab 8.5.
The garbage collector in Ruby 1.8 / 1.9 negated the benefits of copy-on-write forking, but that's fixed since Ruby 2.0
Unfortunately, in some programming language ecosystems where having many small and transitive dependencies on modules from an non-curated repository is common, none of those three things is necessarily true.
Code reuse is not a trivial problem, and you always have to weigh the benefits against the costs and risks to decide whether it’s worth it. If we’re depending on GitHub repositories with a dozen files and three subdirectories just to provide some simple functionality that any junior programmer could implement directly in five lines of code, we’ve probably lost the plot. On the other hand, if we have a full in-house implementation of encryption algorithms we use to throw sensitive customer data around between the browser and our servers, we’ve also probably lost the plot.
I wonder how much traffic could be saved by optimizing npm packages... probably on terabyte scale at github alone, methinks.
I'm worried this sort of "screw it just add a library" is going to spread further in my language of choice: Java.
In my time doing open source programming on the side, I've found that it has become more common with the advent of things like mvn and gradle to just slather on layers to your stack even for the simple tasks.
Need a function to turn a byte buffer into a string? Download these 3 Apache commons libraries and their dependencies.
I understand if you are relying on a large portion of a library and you need to use it, but why bring an entire library in for one function.
Me, I very often prefer to write things myself, in a way that can get labelled as NIH. My inclination is based on bad experiences with trying to debug external libraries. Sometimes I look at open source library code and find staggering complexity that I have no need for. Yes, maybe the library is great, but if its combinatorial size is 10,000 times the functionality that we need, then depending on its correctness becomes scary to me. And when I need to customize it, due to some requirements alteration, I will find it difficult and tedious.
Black-box type libraries for isolated complicated tasks like codecs and crypto I will happily use.
Otherwise, I'm a fan of the "design patterns" approach to reuse, which is all about learning from others, but without creating reusable formal abstractions in library form. So if you teach me how to write an URL router, I can then use your insights without depending on your code base, and I can adapt the idea so it fits my application perfectly.
It's like pornography. I don't know how to define it, but I definitely know when I see it.
There are correct times to use libraries. But pulling a 15meg for some simple functionality is not a good practice in my opinion.
Why not? An extra 3MB of disk space? A few milliseconds more of compile time? Maven makes it very easy to work with dependencies (and you want to use maven anyway, even if you have no dependencies, so that you have a proper release process). At this point I think of basic utility libraries like guava-collections, commons-io, httpclient and junit as part of the standard library. If I need even one function from one of them I'll add the dependency. Better that than have two inconsistent ways of doing the same thing.
Still, it isn't hard to bring in a chain accidentally. I have a program than needs to do a query against the local LDAP system to extract members of a specified group. The LDAP library brings in five more libraries for parsing all the various bits of LDAP. Since this isn't C, I'm a bit less nervous about pulling in, say, a BER decoding library, because at least Go is generally memory-safe, but, still, that's a somewhat large stack for such a simple query. (Traditionally in C, you might as well just expect any library that decodes anything remotely binary-esque will have buffer overflows. C is a DSL for writing buffer overflows.)
And yet, I'd be insane to try to implement some sort of just-barely-minimal LDAP client to do it myself.
Looking at my local godoc instance's full set of packages that have gotten pulled in one way or another is still sort of intimidating. Some of them are cases where I'm just pulling in a subdir and got an entire large repo (the golang experimental repos do that a lot), but, still, I've got a lot of stuff in there. If you're a go programmer and you haven't run godoc locally and had a look at the packages page, have a look. You may be surprised.
- Test gems loading in production.
That does not sound familiar! Is this a thing which happens with Rails?
Nokogiri.
"no code" - well, it's there for a reason. "own it" - do i really want to write my own minimal implementation?
i understand that dependency is a pita but this post doesn't provide anything worthwhile.
Never do they ask the opposite: what kind of foundations do I need? What elementary blocks do I need to have or learn in order to make a JSON parser in 5 lines of code? Is it possible to do logging without all the cruft? Can I write the library in the same amount of time as I can read the docs? Could the code I write be the docs?
Similar line of reasoning: can I leverage my OS to do scheduling/IPC/monitoring/security? If it can't, should we lobby for better OSes (that might scale over multiple machines?) Does Linux/Docker offer the right fundamentals?
Dijkstra was truly right: the art of programming is the art of managing complexity.