undefined | Better HN

0 pointsjmillikin11y ago0 comments

  > every other language which uses a package manager has
  > one that works out of the box without any hassle and
  > without any sandbox tricks.

I have programmed professionally with C, C++, Go, Haskell, Java, Javascript, Python, and Ruby. While this is not "every other language", I feel it's a sufficient cross-section to be useful for the purposes of this thread.

Trying to do any sort of open-source development without a library sandbox, in any language, is madness. OS package managers are completely unable to deal with multi-version dependency graphs. NixOS is no different, unless you want to install the cartesian product of the possible combinations -- how much SSD space do you have?

---

My experience with Haskell leads me to believe that "Cabal hell" is an artifact of certain library developers' API versioning philosophies. Namely, they release numerous libraries all depending on each other with tight version bounds, and change the API regularly. This behavior is fundamentally incompatible with dependency resolution in a compiled language.

Say you have four libraries:

  foo-1.0 depends on bar==3.8 and qux==1.5
  bar-3.8 depends on baz==0.2 and qux==1.5
  baz-0.2 has no dependencies
  qux-1.5 has no dependencies

Then tomorrow you want to release an API-incompatible version of qux and use those new features in foo:

  foo-1.1 depends on bar==1.1 and qux==1.6
  bar-1.1 depends on baz==1.1 and qux==1.5

What do you do here? There's no good choice. Two versions of qux can't be linked into the same binary, and the tight API versioning prevents Cabal from being able to construct a coherent package set.

Now imagine that instead of four libraries, it's O(50), and they're all changing regularly.

Writing lots of tooling and writing manifestos against Cabal sort of helps, for a bit, but it's a lot of work and is deeply unsatisfying for the kind of person who likes to make progress on other goals.

IMO the only practical solution is to loosen the dependency version bounds, and commit to maintaining API compatibility for several releases. I have never encountered Cabal hell when using packages with reasonable versioning philosophies.

0 comments

berdario11y ago

> Trying to do any sort of open-source development without a library sandbox, in any language, is madness.

I agree, but there're some interesting differences in how different language tools implement a sandbox.

Python and Haskell (with Virtualenv and Cabal-dev/sandbox) have separate local repo/caches in which packages are installed (this is useful if you want packages in a reliable location to install executables)

Ruby and Clojure (with Bundler and Leiningen) have a common repo/cache with multiple libraries version, and the version resolution is handled inside the project itself (e.g. when you need to whip up a repl or build the project)

(I'm not mentioning rbenv or the like, since that doesn't handle version graphs by itself... but obviously it can be used just like Virtualenv)

IMHO, the second approach is better suited for a language (implementation) that has an explicit compilation step in which an artifact/binary is built, just like for Haskell/GHC

(then again, something like the first approach is still useful for executables and maybe also to try out different compiler versions)

> Two versions of qux can't be linked into the same binary

It might be possible, but yes... it's a world of pain

http://stackoverflow.com/questions/3232822/linking-with-mult...

dllthomas11y ago

I think a big part of the issue is that - because of the way Haskell's cross-module inlining works - you need to generate different artifacts not just for each version you use but for each set of dependencies chosen for each set of flags set for version you use. I think this strips away much of the benefit of the second approach.

tome11y ago

This sounds like something Backpack might be able to help with! (I don't actually know. I'm speculating. And hoping ...)

1 more reply

zak_mc_kracken11y ago

> What do you do here?

Pick a language where developers are not so cavalier with backward compatibility?

Java has Maven which allows you to exclude transitive dependencies for such situations, and in more than a decade of developing with the language, I've only had to use this functionality a handful of times.

Developers sometimes mess up and break backward compatibility but this should be an extremely rare event. That or the compiler itself is terribly implemented and creating incompatible binaries just by bumping up versions (looking at you Scala).

progman11y ago

Thanks for your detailed answer. I think you are right in recommending a way of less tight version bounds.

platz11y ago

If you need it, cabal takes an option --allow-newer which can selectively or globally ignore upper bounds, in case the package owner specified too tight a bound

frowaway00111y ago

> Trying to do any sort of open-source development without a library sandbox, in any language, is madness.

Really?

Even Java people have been perfectly fine without it. Sandboxing is just a hack around Cabal/GHC not having any workable version support.

j / k navigate · click thread line to collapse

0 pointsjmillikin11y ago0 comments

  > every other language which uses a package manager has
  > one that works out of the box without any hassle and
  > without any sandbox tricks.

---

Say you have four libraries:

  foo-1.0 depends on bar==3.8 and qux==1.5
  bar-3.8 depends on baz==0.2 and qux==1.5
  baz-0.2 has no dependencies
  qux-1.5 has no dependencies

Then tomorrow you want to release an API-incompatible version of qux and use those new features in foo:

  foo-1.1 depends on bar==1.1 and qux==1.6
  bar-1.1 depends on baz==1.1 and qux==1.5

What do you do here? There's no good choice. Two versions of qux can't be linked into the same binary, and the tight API versioning prevents Cabal from being able to construct a coherent package set.

Now imagine that instead of four libraries, it's O(50), and they're all changing regularly.

0 comments

berdario11y ago

> Trying to do any sort of open-source development without a library sandbox, in any language, is madness.

I agree, but there're some interesting differences in how different language tools implement a sandbox.

(I'm not mentioning rbenv or the like, since that doesn't handle version graphs by itself... but obviously it can be used just like Virtualenv)

IMHO, the second approach is better suited for a language (implementation) that has an explicit compilation step in which an artifact/binary is built, just like for Haskell/GHC

(then again, something like the first approach is still useful for executables and maybe also to try out different compiler versions)

> Two versions of qux can't be linked into the same binary

It might be possible, but yes... it's a world of pain

http://stackoverflow.com/questions/3232822/linking-with-mult...

dllthomas11y ago

tome11y ago

This sounds like something Backpack might be able to help with! (I don't actually know. I'm speculating. And hoping ...)

1 more reply

zak_mc_kracken11y ago

> What do you do here?

Pick a language where developers are not so cavalier with backward compatibility?

progman11y ago

Thanks for your detailed answer. I think you are right in recommending a way of less tight version bounds.

platz11y ago

If you need it, cabal takes an option --allow-newer which can selectively or globally ignore upper bounds, in case the package owner specified too tight a bound

frowaway00111y ago

> Trying to do any sort of open-source development without a library sandbox, in any language, is madness.

Really?

Even Java people have been perfectly fine without it. Sandboxing is just a hack around Cabal/GHC not having any workable version support.

j / k navigate · click thread line to collapse