I found it a little funny that their big "win" for the nilness checker was some code logging nil panics thousands of time a day. Literally an example where their checker wasn't needed because it was being logged at runtime.
It's a good idea but they need some examples where their product beats running "grep panic".
The advantage of NilAway is not just detecting nil panic crashes after the fact (as you note, we should always be able to detect those eventually, once they happen!), but detecting them early enough that they don't make it to users. If the tool had been online when that panic was first introduced, it would have been fixed before ever showing up in the logs (Presumably, at least! The tool is not currently blocking, and developers can mistake a real warning for a false positive, which also exist due to a number of reasons both fundamental and just related to features still being added)
But, on the big picture, this is the same general argument as: "Why do you want a statically typed language if a dynamically typed one will also inform you of the mismatch at runtime and crash?" "Well, because you want to know about the issue before it crashes."
Beyond not making it all the way to prod, there is also a big benefit of detecting issues early on the development lifecycle, simply in terms of the effort required to address them: 'while typing the code' beats 'while compiling and testing locally' beats 'at code review time' beats 'during the deployment flow or in staging' beats 'after the fact, from logs/alerts in production', which itself beats 'after the fact, from user complains after a major outage'. NilAway currently works on the code review stage for most internal users, but it is also fast enough to run during local builds (currently that requires all pre-existing warnings in the code to either be resolved or marked for suppression, though, which is why this mode is less common).
Nilability of return values should be part of functions public interface. It shouldn't come as a surprise under certain circumstances of using the code. The problem of global inference is that it targets both the producer and the consumer of the interface at the same time, without a mediating interface definition deciding who is correct. If a producer starts returning nil and a consumer five levels downstream the call-stack happens to be using it, both the producer and caller is called out, even if that was documented public api before, just never executed. Or vice versa.
For anyone who had the great pleasure of deciphering error messages from C++ templates, you know what I'm talking about.
I understand the compromises they had to take due to language constraints and I'm sure this will be plenty useful anyway. Just sad to see that a language, often called modern and safe, having these idiosyncrasies and need such workarounds.
Hi! I use global type inference and I love it.
$ nilaway ./...
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x100c16a58]
Maybe we’ll get a Golang 3 with sum types…
If you don't squint, then I don't think so.
Otherwise, since pointers are frequently used to represent optional parameters, generics + sum types would get the job done; for that use case, it's one of two steps to solve the problem. I don't foresee Go adding sum types, though.
First we define the type, hiding the pointer/non-existent value:
type Optional[Value any] struct {
value Value
exists bool
}
func (o Optional[Value]) Get() (Value, bool) {
return o.value, o.exists
}
if value, ok := optional.Get(); ok {
// value is valid
}
// value is invalid
Here's a full implementation I wrote a while back: https://gist.github.com/MawrBF2/0a60da26f66b82ee87b98b03336e...
What answer do you expect?
Partly this is out of memory of the good/bad old newsgroup days where this kind of thing somehow worked ok, until it didn't, but it definitely doesn't work on the sort of forum that HN is. We'd like a better outcome than scorched earth for this place.
I am toying around with a similar project, with the same goal, and it is DIFFICULT.
I'll definitely get to learn from their implementation.
Hopefully with time, when exploring union types and perhaps a limited form of generalized subtyping (currently it's only interface types) we'll be able to deal with nil for good.
Nil is useful, as long as correctly reined in.
A good way to rein in behaviour is with types. If you need Nil in your domain, great! Give it type 'Nil'.
The untyped nil type is just not a first-class citizen nowadays.
But with type sets, we could probably have ways to track nillables at the type system level through type assertions.
And where nillables are required such as map values it would be feasible to create some from non nillables then ( interface{T | nil})
But that's way ahead still.
Tested via vim and looks good!
We definitely have gotten some useful reports there already since the blog post!
We are aware of a number of sources of false positives and actively trying to drive them down (prioritizing the patterns that are common in our codebase, but very much interested in making the tool useful to others too!).
Some sources of false positives are fundamental (any non-trivial type system will forbid some programs which are otherwise safe in ways that can't be proven statically), others need complex in-development features for the tool to understand (e.g. contracts, such as "foo(...) returns nil iff its third argument is nil"), and some are just a matter of adding a library model or similar small change and we just haven't run into it ourselves.
- You're confident that a flagged value is actually non-Nil?
- A value was Nil but you prefer it that way?
I have some code that eventually core dumps and honestly I don't know what I'm doing wrong, and neither do any golang tools I've tried :(
maaaaaybe there's something that'll check that your code never closes a channel or always blocks after a specific order of events happens...
But I think that focusing on nils is a wrong analysis. The problem is the default zero-values dogma, and that is not going to change anytime soon.
Sometimes you also need a legitimate empty string or 0 integer, but the language cannot distinguish it from the absence of value.
In my codebase, I was able to improve the readability of those cases a lot by using mo.Option, but that has a readability cost and does not offer the same guarantees than a compiler would. The positive side is that I get a panic and clear stack trace whenever I try to read an absent value, which is better than nothing, but still not as good as having those cases at compile time.
No amount of lint checkers (however smart) will workaround the fact that the language cannot currently express those constraints. And I don't see it evolving past it's current dogmas unfortunately, unless someone forks it or create something like typescript for go.
The Go team is very careful to avoid breaking changes (cue all the usual Well Actually comments regarding breaking changes that affected exactly zero code bases) and rightfully so. Their reputation as a stable foundation to build large projects upon has been key to the success and growth of the language and its ecosystem.
I have about a million and one other issues I'd like to see resolved first that don't involve breaking changes. It's a known pain point, the core maintainers acknowledge it, but suggestions to fundamentally derail the entire project are ludicrous.
Focusing on nils is fine. NilAway is fine. It's a perfectly reasonable approach and adds a lot of value. This solves a real problem in real code bases today. There is no universe wherein forking to create a new language creates remotely equivalent value.
For example we could have a new non-nilable pointer type (that would not have any default value), or an optional monad natively in the language (or any other thing in-between, there are many possibilities). That would allow the compiler to statically report about missing checks, without breaking backward compatibility.
But we all know that it's not going to happen soon because while not breaking any existing code, it goes against the "everything has a zero-value" dogma. That was the meaning of my message.
https://github.com/golang/go/issues/57644
https://github.com/golang/go/issues/19412
I interpret your comments as propagating FUD in bad faith.
Insane that Go had decades of programming mistakes to learn from but it chose this path.
Anyway, at least Uber is out there putting out solid bandaids. Their equivalent for Java is definitely a must-have for any project.
Yup, every time I write some Go I feel like it's been made in a vaccum, ignoring decades of programming language. null/nil is a solved problem by languages with sum types like haskell and rust, or with quasi-sums like zig. It always feels like a regression when switching from rust to go.
Kudos to Uber for the tool, it looks amazing!
True, and because of this, the language can be learned over a weekend or during onboarding, new hires can rapidly digest codebases and be productive for the company, code is straightforward and easy to read, libraries can be quickly forked and adapted to suit project needs, and working in large teams on the same project is a lot easier than in many other languages, the compiler is blazing fast, and it's concurrency model is probably the most convenient I have ever seen.
Or to put this in less words: Go trades "being-modern" for amazing productivity.
> It always feels like a regression when switching from rust to go.
It really does, and that's what I love about Go. Don't get me wrong I like Rust. I like what it tries to do. But I also love the simplicity, and sheer productiveness of Go. If I have to deal with the odd nil-based error here and there, I consider that a small price to pay.
And judging by the absolute success Go has (measured by contributions to Github), many many many many many developers agree with me on this.
> code is straightforward and easy to read
I have to disagree. I don't want to read 3 lines out of four that are exactly the same. I don't want to read the boilerplate. I don't want to read yet another abs or array_contains reimplementation. Yes it's technically easy to read, but the actual business logic is buried under so much noise that it really hinders my capacity to digest it.
> the compiler is blazing fast
much agreed, that is my #1 pain point in rust (but it's getting better!)
> and it's concurrency model is probably the most convenient I have ever seen
this so much. this is what I hate the most with go: it pioneered a concurrency model and made it available to the masses, but it has too many footguns imho. this is no surprise other languages picked channels as a first class citizen in their stdlib or core language.
> Go trades "being-modern" for amazing productivity.
I don't think those two are incompatible. If we take the specific point of the article, which is nil pointers, Go would only have to import the sum types concept to have Option and maybe Result as a bonus. Would this translate to a loss of productivity? I don't think so. (oh and sum types hardly are a modern concept)
Also, there may be a false sense of productivity. Go is verbose, and you write a lot. Sure if you spend most of your time typing then yes you are productive. But is it high-value productivity? Some more concise languages leave you more time to think about what you are writing and to write something correct. The feeling of productivity is not there because you are not actively writing code most of the time. IIRC plan9 makes heavy use of the mouse, and people feel less productive compared to a terminal because they are not actively typing. They are not active all the time.
This may be ok, as you say, if you allow errors here and there because you are fine dealing with those problems. But at the other end, it may be a user that is affected by the error. Which may be ok as well, but why should it be? We lament the quality of software all the time.
Compare this to other engineering fields: unless you study the knowledge of those who came before you may not even be allowed to practice in the field. I would not want to use a bridge built by someone who learned bridge building in a weekend.
Software is different though, it's rarely a matter of life or death. Given that, maybe it's ok to not have the highest quality in mind, because the benefit of productivity far outweighs the alternative.
I'm torn.
Yeah, I truly hate this field
You can't make up that other devs' opinions / preferences are identical to yours just because they use the same language, there are other important factors in play (e.g., if your company is using Go, then you'd be more productive in it and be more likely to choose to contribute in it even it Go is less productive as a language)
This is a false dichotomy. One does not imply the other.
Go is also not a simple language. It is deceptively difficult with _many_ footguns that could have easily been avoided had it not ignored decades of basic programming language design.
Many things also aren't straightforward or intuitive. For instance, this great list of issues for beginners: http://golang50shad.es/
“Optimising your notation to not confuse people in the first 10 minutes of seeing it but to hinder readability ever after is a really bad mistake.” — David MacIver
Go is a simple language that anyone can pick up in a weekend, but productivity plateaus once you’re doing anything that requires hard constraints or complex systems (the same is true for JS, Python, and other scripting languages).
None of the Google fresh hires I know personally are stupid. They are talented people who could be just as productive in a C++ or Java codebase. Maybe even better when you have features like Java's streams or C++ templates to throw at non-trivial problems. They might need more time, but it's something easily budgeted for. If new hires have to be productive from the first day, that's a problem the company has created and not the employee. If other languages have too many ways to do something, just enforce only using a few of them, teams have and continue to do that.
I use Golang in my current job. The library ecosystem seems fine. But even as a "new hire", the language frustrates me sometimes. Go's concurrency is "easy", but has a minefield of problems. Just off the top of my head, for-loop semantics [which to Golang's credit, is being fixed but it is absolutely a breaking change], just being able to copy a mutex by accident. These are bugs I've written and not had fun tracking down. In a year I'll have all these footguns memorized, but I could also have spent a year getting better at any other language. Even at my experience level, the Rust compiler gives me enough grief for me to know that when it's happy, whatever I've written will work. Nothing about Golang gives me that confidence.
Go is just obstinately living in the 90s. I guess that's not really a surprise. It's pretty much C but with great tooling.
For java projects I think NullAway has gotten so good that it really takes the steam out of the Kotlin proponents. Hopefully NilAway will get there too.
Like, one of the first files has only .unwraps in the comments (like a dozen of them in a file), some are infallible uses, some are irrelevant-to-runtime tooling, etc.
But anyway, "some" is a lot smaller than "all". Just like some of memory safety issues would also have happened since you can still use unsafe in Rust, yet it's still a big step forward in reducing those issues in the ugly real world
Getting good type errors without requiring type annotations seems like a win over languages that are annotation-heavy. Normally I’d be skeptical about relying on type inference too much over explicit type declarations, but maybe it’s okay for this problem?
This is speculative, but I could see this becoming another win for the Go approach of putting off problems that aren’t urgent. Sort of like having third-party module systems for so many years, and then a really good one. Or like generics.
Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?
There's a lot of overlap and some invalid combinations, but you're still left with a huge number of combinations where Uber must simply work. And every time you add a new thing to this list, the total number of combinations grows polynomially.
(Also, Go is slightly more verbose than most languages. I think that's a feature and not a bug, but it's one more reason.)
A lot of people seems to gravitate toward languages with less dense cognitive load. I have learned to love kotlin, but its also a super dense set of syntax to power it's very expressive language.
I appreciate both languages, and of course Swift feels like what you’d pick any day.
But, after using both nearly side by side and comparing the experience directly, I’ve got to say, I’m so much more productive in Go, there’s SO much less mental burden when writing the code, — and it does not result in more bugs or other sorts of problems.
Thing is, I, of course, am always thinking about types, nullability and the like. The mental type model is pretty rich. But the more intricacies of it that I have to explain to the compiler, the more drag I feel on getting things shipped.
And because Go is so simple, idiomatic, and basically things are generally as I expect them to be, maintenance is not an issue either. Yes, occasionally you are left wondering if a particular field can or cannot be nil / invalid-zero-value, but those cases are few enough to not become a problem.
Effectively,
instead of
result, err := doSomething()
if err != nil {
return nil, err
}
result := doSomething()?Types for which Try is implemented can Try::branch() to get a ControlFlow, a sum type representing the answer to the question "Stop now or keep going?". In the use you're thinking of where we're using ? on a Result, if we're Err we should stop now, returning the error, whereas if we're OK we should keep going.
And that's why this works in Rust (today), when you write doSomething()? the Try::branch() is executed for your Result and resolves into a Break or a Continue which is used to decide to return immediately with an error or continue.
But this is also exactly the right shape for other types in situations where failure has the opposite expectation, and we should keep going if we failed, hoping to succeed later, but stop early if we have a good answer now.
...and then they add more syntax sugar to partly sweep the complexity under the rug. I like Rust as much as the next person, but I'm apprehensive about how this will play out.
I have worked with Rust Option/Rust types and found them extremely unergonomic and painful. The ?s and method chains are an eyesore. Surely PLT has something better for us.
Hence why the language is full of gotchas like these.
Had it not been for Docker and Kubernetes success, and most likely it wouldn't have gotten thus far.
They made the language easier and quicker to write a compiler, but harder to write programs in, and it doesn't look like that will change in Go 2.0.
But if you really cannot afford to return more than one bit of information, do `func foo() (*T, bool)`.
Result<T,E> does this. I forget exactly why Result is actually different from, and in fact superior to, `func foo() (*T, error)` but IIRC it has to do with function composition and concrete vs generic types.
(*T, error) is either T (non-nil, nil), or error (nil/undefined, non-nil), or both (non-nil, non-nil), or neither (nil, nil). By convention usually only the first two are used, but 1) not always, 2) if you rely on convention why even have type system, I have conventions in Python.
Leaving aside pattern matching and all other things which make Rust way more ergonomic and harder to misuse, Go simply lacks a proper sum type that can express exactly one of two options and won't let you use it wrong. Errors should have been done this way from the start, all the theory was known and many practical implementations existed.
The second part is that it's reified as a single value, so it works just fine as a normal value e.g. you can map a value to a result, or put results in a map, etc... , language doesn't really care.
And then you can build fun utilities around it e.g. a transformer from Iterator<Item=Result<T, E>> to Result<Vec<T>, E> (iterates and collects values until it encounters an error, in which case it aborts and immediately returns said error).
Genuinely curious what's so much of business logic is for.
There are entire teams that are working on just internal services that connect some internal tools together.
There was also very little effectivity and efficiency in the era of cheap capital so there were tons of talent wasted on nonsense. Uber built their own slack for a while!! (before just going to mattermost)
People always ask who actually makes money on Uber... I think it's not the cab drivers, not the investors, who makes money is the programmers. It's a transfer of money from Saudis to programmers.
Well it was, anyway.
"All" a kernel does (for some very large value of "all") is schedule userspace programs and manage the system's physical resources (memory, disk, devices). You can reach a point where a kernel is done, in the sense that it meets those basic needs with an acceptable level of performance. Kernel developers don't make extra money for every new feature they add - if the system is good enough, then it's good enough.
As for "per locality business rules differ that's why so many lines of code.." seems like you can have a policy engine+DSL (JSON or YAML or custom policy language and engine) thus your code base shouldn't balloon to almost 100 million limes of code...
What I would really like golang to have is way to send a “last gasp” packet to notify some other system that the runtime is panicing. Ideally at large scales it would be really nice to see what is panicing where and at what time with also stack traces and maybe core dumps. I think that would be much more useful for fixing panics in production.
There was a proposal to add this to the runtime, but it got turned down: https://github.com/golang/go/issues/32333 Most of the arguments against the proposal seem to be that it is hard to determine what is safe to run in a global panic handler. I think the more reasonable option is to tell the go runtime that you want it to send a UDP packet to some address when it panics. That allows the runtime to not support calling arbitrary functions during panicing as it only has to send a UDP packet and then crash.
I could see the static analyzer being useful for helping prevent the introduction of new panics, but I would much rather have better runtime detection.
I tried this with a medium sized project and some unexpected code that could panic 3 functions away from the nil.
Link to the source, or better yet, never link at all to anything related to Uber.
I do recommend the Go team to find a way to these tools to run before it complies, just doing go build while going through these tools first goes a long way than just using scripts