That they also allow mapping, filtering, etc, isn't about 'removing ifs' or 'hiding ifs' so much as they are about writing more readable code, imho.
Optional<Foo> myValue = gateway.callThatApi(...)
return myValue.filter(Utils::isNotTooShabby)
.map(this::mapToCoolerType)
.map(getDecorator())
.orElseThrow(new TotallyBlewItException())
Is this perfect and beautiful? Nah. But it's better than the 20 lines of Java 7 code I'd need to do the same thing. I'm able to write simple predicates and mapper functions as class variables, dynamically if I want, and call them in order as I like. It's short, it's descriptive rather than prescriptive. It isolates what I want from how I do it.Debugging is annoying, yes, but I think there's hope that a good pattern for it will be figured out by the community.
The way I see it it's like this:
Optional<Integer> num = getSomeRiskyNumber();
if (!num.isPresent())
return ... code to bubble up a blank optional
vs Integer num = getSomeRiskyNumber();
if (num == null)
... throw exception or return null
I get that the author has adopted a more functional programming methodology for dealing with their data but for some tasks this isn't acceptable. To just return 1 value you've allocated at least one object (Optional) and make at least 2 function calls on it (isPresent() and get()).You get `if (value == null)` for free. Throwing exceptions is very heavy and I'd place Optional<> above that but there isn't any way to signify the error that you acctually got. You'd need to make an Optional<Maybe<T>> where Maybe<T> supports error/exceptions. Maybe Maybe<T> holds data and you can extend it with BadMaybe<T> who extends exception or something so you can put that in instead of your value to signal your exception.
I don't see the benifit. Maybe I'm just crazy but `if (v == null)` has all the features Optional has for less of an overhead and less cognitive load. (If you're afraid of NPEs then just document all the return states of your methods and use @nullable to show when you need to check. IIRC IntellJ catches that kind of mistake).
Null ruins that. It's a known source of runtime exceptions, and it is not a compilation error to return null or forget the null-check. It is a compilation error to pretend that an Optional<T> is just a T. The Java compiler is not smart enough to prevent you from doing something silly like calling get() on Optional<T> without checking isPresent() first, but at least it gives you something.
Also Optional has some really nice creature comforts. It gives you the ability to map to compose an Optional<T> with functions that couldn't care less about the Optionality, and it lets you use flatMap to chain successive calls that might fail together, avoiding an arbitrary number of null checks (that you might forget!) in your code.
Frankly arguing against Optional<T> is like arguing against map on lists/streams and saying that for is good enough for you. You can do it, but I think you'd be nuts to work against yourself like that.
If something is wrong, throw an exception. If it's the kind of error that must be handled, throw a checked exception. If there's no value to return, return Optional.empty.
The code receiving that Optional<Foo> is now free to do what the author is suggesting: map, filter, all without worrying if the value is actually there or not. The code is cleaner, easier to read, etc.
But no one should ever have to check for null anymore.
You're right about code that branches based on `isPresent()` is semantically the same as null checks - and if you only did that and didn't use any functional paradigm like mapping on the types, you don't really get any real benefit!
BufferedImage avatar = users.getUser(userName) .flatMap(User::getAvatarUri) .flatMap(ImgIO::read) .orElse(anonymousAvatar);
I made this up, but I hope you see the point. Optional is just a fancy null pointer if you don't use it in a functional way. :-)
Optional<Integer> num = getSomeRiskyNumber();
match num {
Some(x) => ... ;
None => return error or whatever;
}
The compiler will verify you at least pretended to look at the None result instead of just ignoring it (but you could still swallow the none).1: does java even have this?
It also provides some nice combinator syntax for handling that null beyond .isPresent, but if you were to just get .isPresent it would be a win.
Documentation is optional (hah) and therefor shouldn't be relied on.
Optional as a return type means there's no need to check for documentation, there's no need for @Nullable, and you get some pretty syntax to boot.
If you want to return a status instead of nothing, you use an Either type (as mentioned in the post). In an Either, you populate left or right, but not both. You can let left be your status and right be your template type. You could codify this as StatusOr<T>.
In a StatusOr land, you return if there's an error; otherwise you continue the computation. This gives a few advantages, but one is that you control the flow of control. (This is also done decently with exceptions in Java, iirc)
However, all of these - maybes, either - they make your functions composable.
There's a good talk... Railroad oriented programming... that dives into this.
getSomeRiskyNumber map {
case Some(i) => Map("risk" -> i)
case None => Map("error" -> "Error retrieving number.")
} map { response =>
Json.toJson(response)
}
Reading this code you know you're always going to generate a json response to the user, and depending upon what getSomeRiskyNumber() returns, the json response will either look like { "risk": 23 } or { "error": "Error retrieving number." }. No exceptions needed.https://www.infoq.com/presentations/Null-References-The-Bill...
May be nitpicking, but in your other example, you've allocated at least one object (Integer) to just return 1 value, which means that supertight performance is not important in getSomeReskyNumber
You'd typically use a sequence of map, flatMap and orElse calls. For example, this is some controller code I'm currently shipping:
return Drawing.getForLocation(id)
.map(d -> ok(Json.toJson(d)))
.orElse(notFound("No drawing for location " + id));
That construct is more readable to me than an if(drawing != null) check. The cognitive load issue is a bit of a red herring. I'm used to optionals now and they're no more of a cognitive load than an if construct. But that's eye of the beholder stuff, so let's park the cognitive load issue for now.Getting down to the meat of your argument, why is optional better than an if not null check or an exception?
I think anyone can see why it's better than throwing an exception. Optional makes it obvious to the caller that (a) there can be no response, and (b) what happens exactly when there's no response. With exceptions, either they're unchecked, in which case you don't know what exception to catch without peeking, and it's really easy to forget to add a try/catch handler at all; or it's checked, but even in that case the handling code is uglier than an equivalent optional construct.
The more interesting question is why it is better than deliberately returning null. Again, it's about signaling. A library author can signal to the caller that they should expect not to get a value sometimes. Yes, it's possible to return null from a method that returns an optional, and this will cause an NPE, but in that case the error is the library's, not the callers. In making API's, you want to encourage correct use, and optionals help do that.
Optionals in java suffer from the same problem as most API's in java do though. They're uglier than they need to be, and some common use patterns are awkward. For example, in my spark code in scala I'll use constructs like this one to convert a stream between types where the conversion might fail for some elements (and I don't care about the ones where it does):
val someIntermediate = someStream.flatMap(convertMaybe)
That works because Option can be handed to flatMap directly.But in Java streams, you can't do that, and you end up with constructs like this:
someStream.map(convertMaybe).flatMap(o -> o.isPresent() ? Stream.of(o.get()) : Stream.empty())
Which is insane. In Java 9 they're kind of sort of fixing this and it's going to be someStream.map(convertMaybe).flatMap(Optional::stream)
Which is still ugly though.Any way, I think optional has its uses in Java. It's definitely something that helps when building out robust API's. But where optionals would be most valuable, the combination with streams and futures, is where they're painful to work with. That's par for the course in Java though. Oracle's not actually all that good at designing API's. Usually the first attempt at something turns out so broken it has to be replaced wholesale later (like java.util.Date), or it's so ugly it feels bad to use it (CompletionStage? seriously?). But, to each their own, I'm sure my attempt at it would be even worse, and despite the warts I still like Java 8 overall.
ListCondi<ObjType> myValue = gateway.callThatApi(...)
foreach(tmp as Condi in myVlue)
{
if (tmp.worked) { foo(tmp.Generic);}
else {bar(tmp.Generic);}
}
PS: Sorry, have not touched java in like 10 years, but I assume you can have that foreach as a map.In the case you're presenting, one might consider:
Stream<Foo> myVals = gateway.callThatApi(...)
return myVals.forEach(val -> getFooOrBar(val).apply(val))
Not quite as nice as Scala would let me do it, but closer, simpler.Streams also let you add filters to the "list" of values, reduce them to a single value with a reducer, or use a nice set of "collectors" that reduce to Java generic collection types. (Collectors.groupBy is so handy).
Streams are also lazy, which I think is more good than bad.
The thing I tell everyone to do right now, today, is to open their Java code and search for "return null", then replace it with "return Optional.empty()". It'll break your code in lots of places, but things will be better when you fix all those things. Often you'll find lots of places that didn't handle the null possiblity at all!
The real problem is branching - when reading code I have to think through two conditional cases.
In this particular example (where you have to validate a client request), I don't see a way out of branching. However I don't think this post has produced the ideal:
JsonParser.parse(request.getBody())
.flatMap(Validator::validate)
.map(ServiceObject::businessLogic)
.flatMap(JsonGenerator::generate)
.match(l -> HttpResponse.internalServerError(l.getMessage()),
r -> HttpResponse.ok(l));
The problem with this is that I have to think through branching all the way through the data flow. However the only function that should branch is validate, to prepare the request to meet the preconditions of the rest of the data flow, all of which should be non-branching.In other words, I should be able to read this part of the data flow without thinking of branching:
(generate-json (business-logic req))
So this I believe is objectively better: (if (valid? req)
(generate-json (business-logic req))
(generate-json (errors req)))
Yes, I've used an if. (If we don't like ifs we can easily get rid of it, of course - but again our problem is branching not the if.)Why is this objectively better? Because we now have to think about branching wrt to the validation function ONLY. We've minimized where branching matters, and that's solving the core issue.
(doseq [n (range 1 101)]
(println
(match [(mod n 3) (mod n 5)]
[0 0] "FizzBuzz"
[0 _] "Fizz"
[_ 0] "Buzz"
:else n)))
To my mind, this reads much more clearly than if I wrote a bunch of if() statements.The structure is almost identical. You even have an "else" clause!
module Raindrops (convert) where
import Control.Monad (liftM2)
import Data.Foldable(fold)
import Data.Maybe (fromMaybe)
convert = build rules
build = liftM2 fromMaybe show . fold
rules = uncurry rule <$> [(3, "Pling"), (5, "Plang"), (7, "Plong")]
where rule i s j = if j `mod` i == 0 then Just s else Nothing
Short explanation: the fold function combines lists of monoids. In this case it combines three seperate monoids and does `[Int -> Maybe String] -> Int -> Maybe String`. It takes a list of functions that might create a string from a number, gives them all the same input and concats all strings that were returned. If none were returned the result stays Nothing and is then replaced by the string representation of the input via fromMaybe.I like this solution because it shows how powerful it is to abstract over these concepts but also that trying to be too clever quickly ends in impossible to follow complexities.
Three times you introduce possible failure, the original parse as well as each call to flatMap. It flattens each of these possible failures into one, though, so you don't have to think about it!
So if the programmer is sane and doesn't hide side effects in there you only have to think about branching at the pattern match - if any step failed do this, otherwise do this!
Is this a satire?
Clarity I think comes from breaking out branching into small parts. Coding just for one-liners leads to confusion in teams many times for non functional programmers. You might have programmers yak shaving just to cut down on "ifs" because they are harmful now? Same with null types. Some languages are built for it and others aren't.
The example is less lines of code, and functional clean code, but is it easier to expand, follow and use throughout your codebase? I guess that is up to the project/team, I feel like in many cases this outlook could be adding complexity where simplicity does just fine.
If a function could return either T or null, return Optional<T>, and then use Optional::flatMap to join them together.
Pretty straightforward.
Optional<String> x = Helper.functionOne()
.map((descriptiveName) => Helper.functionTwo(descriptiveName))
.flatMap((descriptiveName) => Helper.functionThree(descriptiveName));It's been a long time since I've written any Java, so I wonder, how is this better than Clojure?
But does that String contain @ character (checking for email)? Is this Date in the future or in the past (validating a credit card form)? Types say nothing about that. I'm not saying that types have zero utility, but in the vast majority of my use cases compile-time type checking doesn't go very far.
- Optional does not protect from NPE, null is still allowed
- It adds extra layer of complexity
- some libraries use it, some do not. it is not enforced
- extra typing, Java does not even have type inference and `val` declaration
- `if` expression in java does not return a value, no pattern matching... again far more typing
- no support for chained call on several nullable fields
I use Kotlin for couple of years. It has nullability baked into type system and enforced by the compiler. And it works with existing java libraries.
It feels like going back 15 years to Java 1.4, when I use Optional in Java8 or Scala.
- You can still use intermediate variables for readability
e.g. instead of
nums.filter((n) => n % 2 == 0).map((n) => n + 2).foreach((n) => print(n))
do: nums.filter((n) => n % 2 == 0)
.map((n) => n + 2)
.foreach((n) => print(n))
or: List<Integer> evens = nums.filter((n) => n % 2 == 0)
evens.map((n) => n + 2)
.foreach((n) => print(n))Java
IntPredicate even = i -> i % 2 == 0;
IntUnaryOperator add2 = i -> i + 2;
UnaryOperator<IntStream> process = s -> s.filter(even).map(add2);
for (int i in process.apply(numbers)) {
System.out.println(String.valueOf(i));
}
Haskell: printProcessed = mapM_ print . process
where process = map (+2) . filter evenBasically I want to see a "skip to next chained method" button alongside "step" and "step into" and "step over". And maybe another that lets me step to the next iteration of a given chained method as well.
When you have "if" statements, you can press "next" repeatedly in Eclipse/IntelliJ/whatever and step through the code. When you use chained method calls you're constantly going into and out of function calls, and all the local variables become return values that you don't ever necessarily get to see in a stack frame, and sometimes you have to step through loads of boilerplate to see the next interesting line of code, even when you're doing something totally simple.
x = something that might result in an exception
x = f(x)
Now f has to check whether x contains an exception, and it should return that exception in that case, and otherwise it should just apply the function to the argument.I fail to see much difference between an Optional has/doesn't have a value and null. It's just paint and there is no insight into why there is no value. Something like an Expected type that has either a value or the Error/Exception is much more explicit and may let someone do something about it. At least then the user of the method can choose what an appropriate action is with knowledge. But optional and null are the same and give you no more information than a result or that there is no result.
This is because the standard library can forego memory allocation for temporary data structures implied in expressions throughout the stream statement. Also, the Java 8 VM can apply other aggressive optimizations to the lambda functions to inline them.
When you heavily use stream constructions, you're relying on the JVM to:
• Synthesise classes for the lambdas.
• Inline the map/filter/fold calls and then inline the lambdas too. The JVM doesn't make any guarantees about inlining and may unpredictably bail out. If inlining doesn't happen then profile pollution will kill off some of the other optimisations the JVM does.
• Escape analyse any temporary objects created like iterators and then scalar replace them.
• Try and do some loop fusion, but I'm not sure to what extent the JIT compilers can do that.
This is a long list of complex and often fragile optimisations. If any of them don't get applied then you end up with virtual method calls, objects being created, poor cache utilisation etc. Sure objects that die young are cheap but they aren't entirely free. It's still best to avoid them.
The reality is that writing traditional style code is going to be more efficient or at least more reliably efficient than functional style code for the forseeable future.
Note that the JVM has a much easier time of it when using Kotlin's support for lambdas and functional programming because the inlining is guaranteed to be done by the Kotlin compiler not the JVM, and that fixes a lot of issues with profile pollution and unpredictable performance drops.
https://bitbucket.org/iopq/fizzbuzz-in-rust/src/bf4968973d73...
https://play.rust-lang.org/?gist=3fb51314d7df9249c9f774dde96...
https://github.com/zachcoyle/fizzbuzz-without-booleans/blob/...
But I like your solution better
Let's say you wanted to change the program to yield "Fizzbuzz" instead of "FizzBuzz" in the 15 case, but the rest being the same.
I would first build my string from "fizz" and "buzz" and then I'd have to do .map(capitalize) as the second to last step. If I wanted to have the correct case in the originating strings I could just run them into to_lowercase first before anything happens and capitalize correctly afterwards.
(Actually, the capitalize function itself is probably more complicated than this change)
Similarly, other changes are intuitively easy to stick into the correct place in the program once you know what this program does.
I would have really liked to hear what the argument for switching to Java is - over staying on Clojure or switching to a language other than Java.
It sounds like a ternary operator to me.
a != null ? fn(a) : null
it's more like (a, fn) => a != null ? fn(a) : nullIf you want to chain calls, you could do that easily by passing "possibly null" returned values to methods with parameters that are marked @NotNull and handling null checks as exceptions down the line instead of inventing the optional type.
We should use exception all the time. We should adopt exceptional programming.
null is the type to return, just in case we haven't thrown an exception yet.
Everyone know the exception API, how to throw them, how to catch them, we have to use it all the time. It's great.
We definitely don't need a strong type system, we need an exceptional programming language, everything is an exception.
But, hey, if it helps make Java code cleaner and safer (and Optional and Either definitely help a lot with that), I'm all for it.
Who did it first can be interesting, but it's rarely much use.
java.util.Optional<int> opt = some_null_returning_function( );
Database theory tells us that having multiple null values is useful ("eh, I don't know" vs "your question has no meaning").
Seriously though, it would be great if they just added a "non null reference" type to the language. C++ has this, and it is useful (even though the compiler doesn't enforce the non-null bit).
The thing that always irked me about Optional<T> is that they are synonymous with bare Java references (which also can be null, at least according to the language spec / compiler). It is like a Java version of
#define THIS_PROGRAM_IS_WRITTEN_IN_C *
to my eyes.
To each their own.
Not sure how far it can go down the rabbit hole though. This is really where value types/regular types are really nice.
I think it would have been better for Java to allow the flatmap like stuff for any reference instead of creating an optional. But I am not a fan of either for when there is no value due to error conditions. There are optional values and there are errors that prevent the fulfillment of the contract. So either throw so something can be done or return a class that can either be an error or the value. But returning null does not allow the caller to act on it.
You are so right though about non-null references. I would go further and say nullable should not have been the default but an added keyword. That is the trait of a c++ reference I like. Just use it and don't check for null. With that Optional can have some differentiation and is meaningful for situations where an error hasn't occurred.
Very out of context, and very off topic, but this is a profoundly deep statement... its veracity is questionable and unknown.
More details: http://www.furidamu.org/blog/2017/01/28/error-handling-with-...
The potential for JSON parsing to fail is encoded in its
type, not in the potential for a variable to null, or
false, or for an exception to have been thrown. You’re
leaning on the compiler to tell you if you’ve handled the
failure cases properly, as the code won’t compile
otherwise. Now instead of testing for runtime exceptions
you only test to make sure that your business logic is
correct.
Last time I did Java (it like 7 years ago) the compiler did enforce exceptions types as part of the signature. Has this changed in between? Otherwise this does not seem like a valid argument to me.The OP does discuss checked expcetions a little bit:
Checked exceptions guarantee that someone will deal with
the issue, but they are extremely annoying, and might
result in disparate and different exception handlers all
over the place.
They don't explain why exceptions are annoying (because the compiler checks them, just like optional?) and technically there should be exactly the same number of try-catch handlers as match calls in equivalent Optional<> code... It seems to me that his arguments are mostly based on aesthetics. Something the author half-acknowledges by starting their discussion with "Well, first off I think it’s beautiful."There are some actual problems with exceptions though:
1. It may be hard to tell from looking at the code which particular calls inside a function produce which particular kinds of exceptions.
2. This is particularly problematic with stateful code, as to ensure exception-save stateful transactions.
3. They tend to be more expensive for the exceptional codepath -- on the other hand, they are faster than Optional for the non-exceptional path!
In my experience points 2 and 3 are the most important. Since in Java many things may throw, one has two think about exception safety anyways most of the time. This is also a good argument maybe, to just avoid statefulnes instead.
Point 3 is very important. Maybe exceptions should be relegated to truly exceptional situations, and not be used as a replacement for an if. Optional<>/Either<> is excellent in this sitiation. Still in some languages do use exceptions for this and familiarity might be something to consider there (I'm thinking of idiomatic Python signals iteration end or key presence in maps, but Python also has very different performance characteristics as Java and does not have static types for the most part anyways...).
As much as I actually love FP, and I am also trying to bring more FP to other languages (C++), I don't believe in fighting the language for the sake of it. Having a nuanced conversation about the "why" and a the "when" is important. Specially when bringing these techniques to communities that are not used to them and, at the end of the day, already have methods that "Just Work TM". Otherwise, you end up having reactions like this: https://news.ycombinator.com/item?id=13505620
Over the past two decades I've internalized the value of writing code that's very easy to understand. Otherwise 6 months later I can't figure things out myself. This Java style reminds me very much of Scala which seemed like a decent language until I saw how people actually use it in practice. Noped right out of that in a hurry.
If this API is not understood then the code feels convoluted.
Therefore such projects have to contain a document that will explain the most common usage of the API to the newcomer. I think that this would remove most of the confusion.
Naturally it would be good if we had one most widely standardized API that most of the people are familiar with (like they are with if and for).
I think there is more going on here. For the sake of eliminating an if, you are lifting everything else into what is effectively a separate language with the original code embedded in that language. Overall, that doesn't look like a win to me.
After all, what is the actual domain logic? Is it flatMap().map().flatMap().map()? Or is it validate().businessLogic().generate() ?
That doesn't mean that what is going in isn't useful, but it seems to me we need to have a way to specify the lifting without writing it down everywhere, so that the actual code can be expressed at the base level again.
There are subtle variations in naming like Java's Map is Select in C# because Microsoft modeled its functional API on SQL, but you have the same differences for iteration and selection (the things you are referring to for and if).
For example you have 'for (int i = 0; i < n; i++)' in C, but you don't have the exact same way of doing it in Python, where you have to use a range: 'for num in range(0, n):' or you can write a 'while' loop in Java but not in Go where it's a 'for' with a single expression.
I would choose a functional version over a three level nested for loop monster any day.
It's often justified to do that. For instance, we model stuff as matrices or graphs to gain access to their mathematical properties or because we have specialized hardware for matrix operations.
The big debate right now is whether the mathematical properties of functional programming are useful enough to move further away from natural language for general purpose programming.
We probably think something like this:
Parse and validate the request
If that fails then
return 400, "invalid request: " + err
Run business logic to get a result
If that fails then
return 500, "logic failed: " + err
return 200, result as JSON
So should we use a formal language that looks similar to that or are there good reasons to use a chain of map and flatMap calls to hide the branching logic?I can dream up APIs which will keep confusing you, no matter how long you use them (hello Android SDK!). Or which have difficult to memorize syntax (hello Bash!).