I'm really annoyed by this kind of things like they have any meaning except "I don't know shit about how abstractions work in computers"
%s/you/picture author
You even mentioned it yourself: this is the additional overhead added by the Java ecosystem on top of what may already be a pretty large stack of stuff. (From what I've seen, libc is pretty shallow. The majority are either leaf functions like strlen(), or <5 levels from a leaf/before execution disappears into a system call.)
Even if the overhead is not much to a machine, it is certainly going to have an effect on the human who has to figure out what's going on. When you write such code you may not think this way, but every piece of code is a possible place for a bug to be.
From my short experience with Enterprise Java years ago, such deep callstacks are usually symptomatic of code that does far more indirection than actual work --- hundreds of tiny methods that have maybe a statement or two and then call another one. I understand that it might feel good and even better to write code like this, but that simplicity is deceptive: you aren't looking at the whole picture and just focusing on micro-simplicity, when it's macro-simplicity that really counts.
This means that when you're trying to track down a bug, the "interesting parts" are scattered in tiny pieces across dozens of files, and it increases the cognitive overhead significantly in having to piece everything back together. You might not realise that something is wrong until you're deep inside, and then you have to jump back several levels above in the callstack to figure out where things went wrong, seeking in and between files to trace out the execution flow.
I'm glad I don't work on code like this anymore.
This isn't just a Java thing. I've seen plenty of TDD code written with other languages where there is basically the same thing (so the code can be "testable"), but also with heaps of redundant tests that just add dependencies and maintenance overhead without providing much value.
I'd wager that the JIT in OP's picture will transform that huge call stack into something <5 levels from CPU instructions/syscalls.
What remains though, is the huge context carried around such a stack; and I'm quite excited at what the Loom project [1] could yield to tackle that.
Such huge call stacks are a good sign to me. I don't want to implement all the corner cases of all the RFC I rely on; and it is great that library implementors can organise their code well. In the end the JIT will adapt my program to the currently used corner case, and prune everything else.
[1] http://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.htm...
"Java EE is a lot about abstractions which I have grown to appreciate over the years. However, I find this very difficult to explain to my colleague who sits just across the room – he is a Mainframe veteran with tons of experience :)"
In my experience, the code you write within a third-party framework rarely works the way you expect it to the first time. So you often do "debug" the framework, just to see where your code goes while in the framework to see what you missed.
That's not a bad thing, though--it's a great way to learn any new framework!
Two function calls for business logic, a hundred dependency libraries, thousands of lines of stateful "initialization" code outside of the stack and a fuckzillion XML files for configuration. But yeah, everything's great!
And instead you want to switch to what? Writing 20 lines of that same logic + implementing stuff that has already been better implemented in all of the libraries, by better programmers (because they were able to focus and specialize on that one thing)?
For what? A 2% increase in wall-clock speed? There is a 1 in a hundred applications where this matters. What matters in others is maintainability and development speed. Development is expensive. CPU time is cheap.
The opposite is not, and should not, necessarily that you write your own versions of things; after all, if you did so, you'd still have as deep of a stack trace as you do with the third-party code!
I guess, if I encountered this stack trace, I might say there's something of a code smell there. But, hey--this is also fairly old code, and I suspect things are Better Now, so let's not be so quick to cast stones!
Object Oriented Programming has been mostly replaced by Annotation Oriented Programming in modern Java web development.
These can often be very powerful, but of course debugging is more complicated.
I just counted the frames from my typical stack, a Guice/Hibernate project. It was about 100 lines from a constraint violation handler in the postgres JDBC driver up to Thread.run(). This seems pretty reasonable, especially with the various layers I've added for authentication, declarative transaction management, AOP logging, etc.
Furthermore, IntelliJ is pretty good about only showing me what I care about. Overall I find the debugging experience no worse than Python or Ruby, and much better than exceptions from Python or Ruby native libraries. It's orders of magnitude better than debugging Go, which destroys stack context with every "if err != nil return err".
This huge call stack has been designed to make your life as a developer easy but the price you pay is an enormous amount of complexity.
I've been working a lot with a similar Java web stack and I feel how painful this complexity is. What is worse, is that I think that a lot of this complexity is incidental. There are libraries and frameworks designed to make some things easier, but in the process end up creating a lot of problems that then requires another library or framework to overcome that problem which also has other problems and so on... The result is a huge stack like this.
One concrete example of this is Hibernate. A tool designed to make it easier (apparently) to work with databases, but in the end create so many problems that the medicine ends up being much worse than the disease.
Resolving an HTTP request that returns a the result of a database call should not be this complicated! HTTP is simple! Why do we need so many calls to so many things. I'm not advocating for a flat stack of course, but certainly a stack this deep is a clear sign that something is wrong.
I very much agree with Rich Hickey, we need to stop thinking about how to make things easier and start thinking how to make them simpler.
Http is pretty simple, executing sql queries against a database is simple-ish (close those connections!). Authentication, authorization, marshalling, unmarshalling, transaction boundaries, ..., are not so simple, especially not when all taken together.
People bemoan java as you are doing here, but the reality is other languages and frameworks, any that attempt to address the same problems and concerns have the same level of complexity. Java has the advantage of kick ass tooling, debugging, and monitoring infrasture, a lot in the jvm itself (visualvm).
I like Java. It's simple and performant and has excellent tooling. I just don't like that sometimes I see a lot of incidental complexity in its ecosystem.
Sure.
At our startup we had the choice to let 20 programmers write custom individual SQL statements for 100s of CRUD operations, or create entities and let Hibernate generate them for us.
We used hibernate and it has worked out well.
I can't imagine how it would have been to debug 100s of bespoke SQL queries and associated object mapping code, each written in the developers unique style after a few years.
That would have been fun.
Hibernate does not save you from writing queries. You are still writing queries, just in a language different than SQL (e.g. JPA). It's an abstraction layer. The problem is that this abstraction is very leaky, so if you really want to write performant code with Hibernate you do need to understand how SQL and your database works. And if you really understand how it works, you end up realizing that the abstraction is kinda pointless because SQL is already a really fine abstraction over your database.
And if you need to scale, for example working with a replication setup with multiple db servers and having to deal with eventual consistency, then Hibernate really complicates things.
I think Hibernate is a good example of something that makes things easier at the beginning. At the cost of enormous complexity and difficulty in the long term.
This particular problem could be solved by just having a good filtering UI.
You don't have to analyze the stack in its raw text form.
That being said, I agree that complexity in the Java world is often much higher than it needs to be, and sometimes the tradeoffs are not worth it.
Anyway while the stack (hibernate, etc) shown in this stacktrace is still heavily in use, newer async java frameworks/tools usually result in very short stacks for me. Sometimes I get a trace and it’s 5-6 frames. Of course then you may not know how you got there. So it’s really not all that great.
I'm a big fan of the JVM and the Java ecosystem but in many ways the JVM ecosystem is split into two worlds: frameworks or libraries. This would be an example of a framework heavy development model where god knows what is going on between the outside world, your biz logic and the database calls.
[1] java 8 makes thinks bearable, but even then you feel like sculpting chapter 1 lisp/haskell idioms in granite with a spoon.
Can you mention which frameworks and glues are common today ?
Personally moving from a mostly Java background to entirely typescript on the backend and front-end.
function HandleRequest(request : Request) : Response
{
LogRequest(request)
CheckAuthentication(request)
CheckAuthorization(request)
var response = DispatchAndHandleRequest(request)
LogResponse(response)
return response
}
This way you would have no deep stack traces and all would be fine. But only until you need to do some additional work somewhere right in the middle, say patch malformed requests between authorization and dispatch. With the above implementation you would be somewhat screwed, the best you could do would be reimplementing HandleRequest() with your additional step in the middle.But with implementations like those show in the article you just have a list of steps with a common interface, each step does its work and then the next step gets called. If you need to do something new somewhere in the middle, you just add a new step to the list of steps in the right place and you are done, possibly simply by putting the class name of the new step in a configuration file.
var steps =
{
new LogRequestStep(),
new CheckAuthenticationStep(),
new CheckAuthorizationStep(),
// This stupid XYZ client always sends broken requests.
new PatchMalformedRequestFromXyzStep(),
new DispatchAndHandleRequestStep(),
new LogResponseStep()
}
function HandleRequest(request : Request) : Response
{
var context = new Context(request)
foreach (var step in steps)
{
step.Execute(context)
}
return context.Response
}
This would still avoid deep stack traces because we are iterating over all the steps but note that with this implementation we would not really be able to abort processing the request somewhere in the middle, say if the authorization check failed, but we could fix this by adding a flag to the context and check it inside the loop after a step executed. But a more serious limitation is that every step only gets one chance to act, note for example that we have two separate steps for logging the request and the response.Imagine we wanted to log the request duration, then we would need a step getting the current time at the beginning and another one getting the current time after the request was handled at the end. And the first step would have to somehow communicate the processing start time to the second one, possibly by storing it in the context. A much more elegant solution is to organize the list of steps as a linked list with all steps looking like this.
function Execute(request : Request) : Response
{
PreprocessRequest(request)
var response = GetNextStep().Execute(request)
PostprocessResponse(response)
return response
}
This creates those deep stack traces but it also creates a huge amount of flexibility and extensibility. It surly looks crazy if you do not need it, but when you need it, it is really easy with this model.And an integrated distributed async-to-async database that is just as shallow: http://root.rupy.se
Feel free to use it: http://github.com/tinspin/rupy
I'm saying that with all that overhead, people still routinely make mistakes using static typing systems. They get the compiler to tell them that they're correct but their code is still not working.