Ask HN: Is Knuth's TAOCP worth the time and effort?

In 1956, Knuth graduated high school and entered college, where he encountered a computer for the first time (the IBM 650, to which the series of books is dedicated). He took to programming like a fish to water, and by the time he finished college in 1960, he was a legendary programmer, single-handedly writing several compilers on par with or better than professionals (and making good money too). In 1962 when he was a graduate student (and also, on the side, a consultant to Burroughs Corporation), the publisher Addison Wesley approached him with a proposal to write a book about writing compilers (given his reputation), as these techniques were not well-known. He thought about it and decided that the scope ought to be broader: programming techniques were themselves not well-known, so he would write about everything: “the art of computer programming”.

This was a time when programming a computer meant writing in that computer's machine code (or in an assembly language for that machine) — and some of those computers were little more than simple calculators with branches and load/store instructions. The techniques he would have to explain were things like functions/subroutines (a reusable block of assembly code, with some calling conventions), data structures like lists and tries, how to do arithmetic (multiplying integers and floating-point numbers and polynomials), etc. He wrote up a 12-chapter outline (culminating in "compiler techniques" in the final chapter), and wrote a draft against it. When it was realized the draft was too long, the plan became to publish it in 7 volumes.

He had started the work with the idea that he would just be a “journalist” documenting the tricks and techniques of other programmers without any special angle of his own, but unavoidably he came up with his own angle (the analysis of algorithms) — he suggested to the publishers to rename the book to “the analysis of algorithms”, but they said it wouldn't sell so ACP (now abbreviated TAOCP) it remained.

He polished up and published the first three volumes in 1968, 1969, and 1973, and his work was so exhaustive and thorough that he basically created the (sub)field. For example, he won a Turing Award in 1974 (for writing a textbook, in his free time, separate from his research job!). He has been continually polishing these books (e.g. Vols 1 and 2 are in their third edition that came out in 1997, and already nearly the 50th different printing of each), offering rewards for errors and suggestions, and Volume 4A came out in 2011 and Volume 4B in 2023 (late 2022 actually).

Now: what is in these books? You can look at the chapter outlines here: https://en.wikipedia.org/w/index.php?title=The_Art_of_Comput... — the topics are low-level (he is interested in practical algorithms that one could conceivably want to write in machine code and actually run, to get answers) but covered in amazing detail. For example, you may think that there's nothing more to say about the idea of “sequential search” than “look through an array till you find the element”, but he has 10 pages of careful study of it, followed by 6 pages of exercises and solutions in small print. Then follow even more pages devoted to binary search. And so on.

(The new volumes on combinatorial algorithms are also like that: I thought I'd written lots of backtracking programs for programming contests and whatnot, and “knew” backtracking, but Knuth exhausted everything I knew in under a page, and followed it with dozens and dozens of pages.)

If you are a certain sort of person, you will enjoy this a lot. Every page is full of lots of clever and deep ideas: Knuth has basically taken the entire published literature in computer science on each topic he covers, digested it thoroughly, passed it through his personal interestingness filter, added some of his own ideas, and published it in carefully written pages of charming, playful, prose. It does require some mathematical maturity (say at the level of decent college student, or strong high school student) to read the mathematical sections, or you can skim through them and just get the ideas.

But you won't learn about, say, writing a React frontend, or a CRUD app, or how to work with Git, or API design for software-engineering in large teams, or any number of things relevant to computer programmers today.

Some ways you could answer for yourself whether it's worth the time and effort:

• Would you read it even if it wasn't called “The Art of Computer Programming”, but was called “The Analysis of Algorithms” or “Don Knuth's big book of super-deep study of some ideas in computer programming”?

• Take a look at some of the recent “pre-fascicles” online, and see if you enjoy them. (E.g. https://cs.stanford.edu/~knuth/fasc5b.ps.gz is the one about backtracking, and an early draft of part of Volume 4B. https://cs.stanford.edu/~knuth/fasc1a.ps.gz is “Bitwise tricks and techniques” — think “Hacker's Delight” — published as part of Volume 4A. Etc.)

• See what other people got out of the books, e.g. these posts: https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... are by someone who read the first three volumes in 3 years. For a while I attended a reading group (some recordings at https://www.youtube.com/channel/UCHOHy9Rjl3MlEfZ2HI0AD3g but I doubt they'll be useful to anyone who didn't attend), and we read about 0.5–2 pages an hour on average IIRC. And so on.

I find reading these books (even if dipping into only a few pages here and there) a more rewarding use of time than social media or HN, for instance, and wish I could make more time for them. But everyone's tastes will differ.

I think you can read at various levels though.

When I say I've read most of Volumes 1-3, I mean literally that, I read the text through. At that level it isn't too difficult to read, and you'll learn a lot like that.

However some bits I've studied in much greater detail, like for instance the multi-precision arithmetic algorithms which I had to implement. For that I read it multiple times, implemented the algorithms, did some of the exercises. That takes much much longer than just reading it through.

It is rather like reading a research paper which you do roughly like this, stopping after any step where you think it is no longer providing value to you.

    1. Read the title
    2. Read the abstract
    3. Read the paper through quickly.
    4. Read the paper through slowly being more critical, making notes.
    5. Read the paper through again working through the maths, implementing the algorithms, making sure you understand everything.

So I've read most of TAOCP at level 3, half a volume at level 4 and only 50 pages at level 5.

If nothing else I've loaded the index in my head!

PS the examples are notorious in TAOCP. The easy ones will take a few minutes and the hard ones are unsolved research problems!

> "Nobody ever taught me about a B-tree. I had heard of it. When I went to write my own B-tree, on the bookshelf behind me, I’ve got Don Knuth’s The Art of Computer Programming, so I just pulled that down, I flipped to the chapter on searching and looked up B-trees and he described the algorithm. That’s what I did. Funny thing, Don gives us details on the algorithm for searching a B-tree and for inserting into a B-tree. He does not provide an algorithm for deleting from the B-tree. That’s an exercise at the end of the chapter, so before I wrote my own B-tree I had to solve the exercise at the end. Thanks, Don. I really appreciate it."

https://corecursive.com/066-sqlite-with-richard-hipp/#b-tree...

I read through volume 1 several times. I then skimmed through the arithmetics part of volume 2 and read only the section about random numbers, which is very fascinating but you will be able to live without unless you are doing simulations or cryptography/cryptoanalysis.

Whether you will need volumes 4a, b, c depends on whether your work has much to do with combinatorics. I'm a professor and I did not need most of it, but had a good laugh about the hidden gems and jokes like the exercise about Knuths personal pipe organ.

As a software engineer, you may occasionally consult volumes 3 and 1 (searching, sorting, data structures).

[Side node: If I was Knuth, I don't think I would have created three subvolumes instead of the planned single volume 4. While it is true that the material exploded, that will always be the case, and is also true for the other chapters. In his shoes, I would have worked on succession planning and trying to map out the remaining volumes so that the book can be completed one day. Under Knuths approach (esp. going back to update the existing volumes repeatedly on the way) TAOCP will unlikely ever be completed, which would be a shame.]

That's probably true, in the sense that it deals largely with the kind of functionality modern programmer should be calling from a library, not implementing by hand. But I think it's hugely enriching to read these books. I read 1-3 back in the 70s, and was particularly inspired by Sorting and Searching, and especially the chapter on random numbers. RNGs have intrigued me ever since.

I find Knuth's writing style very pleasant. He's funny. So is it worth the time and effort? Not if "worth it" means "makes you a better Javascript programmer". But it's worth it in the sense that reading books about the history and geography of a country is "worth it" before visiting; your appreciation of the place is deepened, possibly transformatively. It could even be that you would never have visited had you not read those books.

I found TAOCP easy to read, much more so than most textbooks. I don't think I tackled many of the exercises; certainly none of the challenging ones. So for me, the "effort" aspect of "is it worth the time and effort" was quite low, so the ROI was quite high. If you find the books hard to read, then don't bother (and perhaps you aren't really interested in low-level algorithms).

TAOCP is extremely rigorous and detailed, the discussion of any topic there will cover every detail. For practical software engineering, it's not useful, you don't deal need that level of theoretical understanding as a software engineer (if you're one of the few who need it, you'd know you are). And you need a very solid mathematical background to actually understand it.

If you want to understand the theory of CS algorithms, I think the CLRS "Introduction to Algorithms" is an excellent textbook. It covers about everything most people are likely to encounter, has comprehensive mathematical analysis but is much easier to follow than TAOCP. For an understanding of the mathematics involved, Concrete Mathematics is a good book, it teaches the mathematical concepts and tools that are the foundation of other, more rigorous books.

For instance, I just opened randomly to the binary tree traversal material in Vol. I. It's very mathematic or formal, e.g. "T2. [P = Λ?] If P = Λ, go to step T4." and so on. To fish out some of the more interesting details, you have to look through the questions and then flip to the answers.

Meanwhile, https://en.wikipedia.org/wiki/Tree_traversal has the same information but somehow more clearly presented.

If I flip to other random pages and scan for 30 seconds, I can't find much that is actually interesting---particularly in the early volumes.

-----

The Art of Electronics is more my speed, as it's a collection of real-world applications, deep dives into each component type, quick comparison tables, and clear overview of a specific design's pros and cons. It's as much interesting to read through as it is useful as a reference.

I can flip to basically any random page of The Art of Electronics and find something eliciting, "Oh cool." I can't say the same for TAOCP.

> he told me the book may be trying too hard

Knuth isn't trying to write a novel or something to learn from, rather a concise, comprehensive attempt to cover the field of Computer Science. It's a repository of knowledge. He marks some exercises at 50 which, from memory, means that if you can solve it fully you are in the running for a Turing award. One of the Python core developers was once asked what the developers did in their sparetime: We read Knuth so you don't have to [0].

[0] https://www.norvig.com/quotations.html

They do make a good doorstop though.

If you're looking for some browser scripting code to cut and paste, you can go to Stack Overflow or Chat GPT to have your thinking done for you. If you want to understand what you're doing, pull out your dog-eared well-worn copy of TAoCP and spend some warm quality time with D. E. Knuth.

TAOCP is in that vein. As you have identified, it is not a programming book, and it's not a computer engineering book, and it's not really a college textbook either. It is a computer science book in the old tradition, where computer science is treated as a branch of mathematics.

It won't teach you to program. As another user said, you should probably be using the algorithm your language's standard library provides. Now, if you want to know why a library's author chose a particular algorithm, the trade-offs between the options, and an open-ended and in-depth exploration of such topics, then this is the book for you. The author of that optimized tree search function you're using probably read TAOCP at some point, so you don't have to.

For example i never thought i would use the basic compiler knowledge i gained during my bachelor. But at one of my first consulting gigs it did helped knowing about abstract syntax tree, where you can add and remove nodes. This knowledge helped me create a highly automated refactoring tool for an old but massive enterprise code base.

This was the only situation in my whole career for now where i needed this knowledge, the rest has all been about creating webpages and rest apis :p

[1] https://wiki.c2.com/?MappersVsPackers

What books if you liked will you like TAOCP

If you hated skip TAOCP.

Me:

Loved SICP

totally wore out Godel, Escher, Bach

Somewhat angry at A New Kind of Science

- For expanding my conceptual understanding of algorithms. I still remember the thrill of learning how the Joseph's problem can be modeled as base-n modulo arithmetics. Or how powerful it is to use generating functions to compute algorithm complexities. And I read the vol 4 for a pleasant study of how we can code back-tracking in an almost mechanical way.

- For historical context. Knuth is amazing at tracing the roots of algorithms, and show how the solutions to a problem evolve over time. You'll get high when seeing how a seemingly impossible problem gets solved in such elegance and ingenuity over generations of attempts by great minds. Check Vol 4 for all the wonders of combinatoric algorithms, for example.

- For thorough study and pure awe. For instance, one probably learns a bit about ordered binary decision diagram (OBDD) to reduce state space by thousands of times in a formal verification system. But what if you want to see all the amazing details of OBDD and therefore getting hours of dopamine high? Vol 4 is the go-to place.

- For reference. At least for a student. I got stuck implementing a fibonacci heap for my project on memory management in my OS class. And Knuth's books came to rescue. I guess professionals would still refer to Knuth for their implementation from time to time.

I think the reader is improved for having read it, because it can open one's eyes to the fractal detail of solutions to coding problems.

Take mathematics as an example. It has a rich history of primary sources written by famed mathematicians. Most people never even glance at these sources and gain a practical understanding of mathematics strictly through secondary sources like textbooks. This is perfectly fine for most people that only need to use math mechanically and don't care too much about a deep theoretical understanding. If you want the latter however, imo reading primary sources and understanding the history of a field is really important.

To me, TAOCP has always been more of a primary source. If you're just looking for surface level knowledge and advice that's immediately applicable, look elsewhere. If you're looking to understand some of the fundamental concepts in the field of computing and their history, it might be a good fit.

https://theory.stanford.edu/~tim/won1samplefinal.pdf

1. Most of the time you are not implementing foundational algorithms like sorting or SAT solving. You use mature implementation in libraries.

2. If you are in fact implementing foundational algorithms, then the existing volumes of Knuth cover only a very limited set of problem areas.

3. If you are implementing something in an area covered by Knuth, the books are worth looking into as a reference, but often writing performant and robust code requires considerations not in Knuth. This is because Knuth works with an idealised machine ignoring things like parallelism, memory hierarchy etc. and understandably does not get into software design issues.

Keep them in the bathroom and open them at random at idle moments. Like the Bible or a "Complete Shakespeare" there's something worth your time on almost any page you pick.

I'm a big fan, but I think you already have to be pretty deep into the world of algorithms and discrete math to get a lot out of them. I cheated my way into that world by starting my math minor a few years ago with Combinatorics, specifically because I felt like learning how to count things in weird ways would help my autrocious mental math (it did!).

Had a giggle. Sounds exactly like something a mathematician would say.

- pragmatic programmer

- code complete

- hackers and painters

That's not to say his works are 'bad' -- but they are very specialized.

you've told us about your friend (hello there friend), but not said anything about your background / the context in which you're "starting". So...

It's generally aimed for those studying Computer Science and/or have deep interest in Theory of programming + computation.

Knuth loves algorithms, he even has algorithm on how to Read the book and approach the exercises (difficulty, mathematical complexity etc).

I read TAOCP out of curiosity long time ago. Has it been useful for me? Very much. But the usefulness came from abstract connections that clicked later in my life almost randomly. I am not a common individual.

If you expect a direct practical application of what you read, I believe you better choose other material that suits your needs better.

yes, i have TAOCP on the shelf and yes guilty of "one day i'll ready it".

but every now and then i open them up and just flip thru that magnificent typography.

Knuth has not just written up all these things, he has developed an entire typesetting system (complete with fonts) to bring technical publishing screaming and kicking into the 20th century (when other software thought kerning and hypenation were creatures from space). it's the only program deserving a version number approximating PI.

Volume 2 is within arms reach on my desk. After reading the book and additional literature over the course of several years, the book is filled with sticker notes. Over time my usage shifted from reading TAOCP to rereading (and changing) my notes.

> Would you still recommend this book, and if yes, in what circumstances?

Everyone in math/cs should at least know about the existence of theses books. A rough, cursory knowledge of the content is essential, so that one can look up things if there is the need to really understand a specific topic. TOACP should be considered as one of the best and most important/influential science books of the last century.

Read from a certain perspective, TAOCP is less a book about how to do things better (the focus of many CS courses), and more about "wow, who would have imagined ...". I think CS students might find their curricula more engaging if, rather than having a set of concepts to master, there were actually interesting puzzles to solve. Thus, I suggest reading TAOCP not as a text book, but more as a novel. Take a look at the parts that interest you (how many ways of sorting 10 numbers can you think of just using paper and pencil?). CS is a lot easier to learn as a process of surprising discovery, than as a set of barriers that must be overcome.

"One of the CS bestsellers which is seldom read. That puts it right next to Bible"

From the practical element, mostly No. People won't use this as a first book - or the second or third. But if you wanted a reference on something deeper, which missed any of the standard textbooks, this is the book for you.

As an illustrative layman example, lets say you are modeling server usage with a mathematical model, most textbooks will tell you "hey, the pings and handshakes closely follow Poisson distribution". You pick from there and build on that information or mental model. But why Poisson model happens to be that way, why not something else - and how closely the server model cross-correlates will be realm of where TAOCP starts building the problem. It goes a layer or two deeper of yak shaving IMHO

    If all you want to do is write CRUD apps, the book is not for you. That being said...

    To quote directly fromt a quote in the book, "Now I see with eye serene, the very pulse of the machine." I finally jumped the gun and started reading it during the summer after my acquiring my bachelors. I finished the first volume during that summer. Here are some notes:

    The Mathematical premliminaries reads like a novel if you have some mathematical maturity, you don't dwell too much on the sections that are over your head, and you treat the exercises as optional. This is because the book can be read from a HS level all the way through the Post-Grad level. Choose your depth.

    The MMIX approach was eye-opening because it opened my eyes to see computers as the 'computing' machines that they are. I indirectly gained a fundamental understanding how everything starts with the CPU. I started seeing all the high-level languages (C++, Python, Rust, Java, etc...) as the abstractions that they are. It is liberating if you have ever felt the uneasiness when you are programming in a high level language when you are nagged by the question "how is the computer doing this?"

    Algorithms and Data Structures( a.k.a Information Structures in vol 1): It is like going to grammar school for algorithms! Have you ever asked yourself "Why are we learning all of these abstract things about algorithms and data structures?" I got my answer from the book. What I mean by this is that after studying Knuth's approach to algorithms, you gain a firm, fundamental, and concrete understanding of their necessity, their role. Afterwards ANYTHING I would encounter on Hackerank or Leetcode became digestible. You gain a "first principles" understanding of algorithms and information structures.

    I believe that the book stands the test of time largely because of these points.

    If you have programmed to the point where you've been exposed to some sorting algorithms and some data structures and you want to continue the road to becoming an expert programmer on a firm footing, it is worth the time and effort. The caveat that I would add is do not wait until you finish all the books before you enter the job market or write software out of your own interest.

https://news.ycombinator.com/item?id=10897460

I've enjoyed dipping in. Sometimes found it directly useful to work. But the best of his works I read (in full) was the MMIX book that is a literate program of the interpreter. Second to that was the typography book, a total fascinating delight.

In reality there's probably almost no one who reads any volume of TAOCP in full.

The only way to answer your question is probably to start reading and see if you enjoy it. But expect it take some effort.

Does that mean you shouldn't read it? Maybe. Honestly, it doesn't matter if you read it or not. Do it if you want to, but do it without the expectation of getting something out of it.

If you want a return on investment for your time, you're better of building something with the knowledge you already have. Put the books away and create something.

I really enjoyed working through them - I keep meaning to get around to volume 4, but can't seem to find the time to do so.

I didn't think of studying them as "effort", though, because they were (and are) extremely interesting to me.

Personally, I think the TeX (that Knuth designed because he could not typeset the TAOCP to his liking) is a much more valuable side effect of Knuth writing the TAOCP than the books themselves, but this is not a popular opinion, at least within my Stanford-educated friends.

Understanding your std library performance is lot more important