Learning to See in the Dark (2018) (opens in new tab)

This is completely untrue.

What you are describing is usually called automatic tone mapping. This is basically noise reduction and possibly color normalization from brightening a dark image. Them showing their black image as the starting point is silly, because jpg will make a mess of the remaining information. What they should show is the raw image brightened by a straight multiplier to show the noisy version that you would get from trying to increase brightness in a trivial way.

What you are describing is a class of filters known as edge preserving filters. You can look at bilateral filters and guided filters for examples that have been around for decades at this point.

Note (a) and (b) are separate photographs (different angles and everything), and that (c) is based on (a), not (b); comparing the glare between (b) and (c) isn't quite an even comparison.

Absolutely! We recently rebranded our AI solutions from ALLIS (Advanced Low Light Imagine Solution) to ALIIS (All Light Intelligent Imaging Solution) specifically because we are beginning to branch out to handle use cases such as this!

As a proof of concept that this task can be tackled directly, a quick search brought up "DeepFlash: Turning a Flash Selfie into a Studio Portrait"[0]

Beyond denoising, we are already running experiments with very promising results on haze, lens flare, and reflection removal; super resolution; region adaptive white balancing; single exposure HDR; and a fair bit more.

One of the other cooler things we are doing is putting together a unified SDK where our algorithms and neural nets will be able to run pretty much anywhere, on any hardware, using transparent backend switching. (e.g. CPU, GPU, TPU, NPU, DSP, other accelerator ASICs, etc..)

[0] https://arxiv.org/abs/1901.04252

The way I mistakenly initially parsed this comment gave rise to a potentially-dumb idea/question:

What would happen if you

- begin capturing video (unsure of fps) on a phone-quality sensor in a near-dark environment

- pulse the phone's flash LED(s) like you're taking a photo

- do super-resolution on the resulting video to extract a photo...

- ...while factoring in the decay in brightness/saturation in consecutive video frames produced by the flash pulse?

I vaguely recall reading somewhere that oversaturated photos have more signal in them and are easier to fix than undersaturated. Hmm.

IIRC super-resolution worked with 30fps source video for better quality; I wonder if 60fps or 120fps source video would produce better brightness decay data, or whether super-resolution could actually help extract more signal out of the decay sequence too.

On the other hand, I'm not sure if super-resolution fundamentally requires largely consistent brightness in order to work as well as it does. :/

Perhaps individual networks could be trained/tuned to specific slices/windows of the brightness gradient. I also wonder if it would be useful to factor the superresolution process into each of the brightness-specific stages or just to do it at the end.

Sure, we have a short deck[0] that gives an intro to our noise reduction, and also here is a folder[1] that shows off a calibration target we captured with a actual camera (20ms, f22) in low-light conditions: (original, 100x gain, 100x gain + ALIIS)

We also have some more raw data[2] where there is the original bayer data available as .npy files with 40db analog gain applied, however I think the calibration targets show off what we are able to accomplish more dramatically. Finally, we have a short youtube video[3] that shows off how it works when applied to video.

[0] https://www.dropbox.com/s/0bm4dpxhn35vkhe/ALLIS_Investor_Int...

[1] https://www.dropbox.com/sh/k861saentyq1cs6/AADmO7X_L49nUkEI_...

[2] https://www.dropbox.com/sh/fv8omdf4fbx59m9/AABDnf6sdvv7rtIml...

[3] https://youtu.be/99Cq1bWCmMM

I don't know that I agree with this first statement, but even if I do, everything is abstracted by import statements even outside ML. You say this is a huge amount of code abstracted, but it wouldn't be difficult to reimplement this in numpy and pandas directly without using tensorflow at all. The code would expand a bit, and you'd have to deal directly with backprop and calculating derivatives but it wouldn't expand things too much. But then you could make the same claim about numpy abstracting the linear algebra, and I could show you that I could extract that and do it without numpy but then it would be the python math library. It's turtles all the way down. My point is, your comment applies to everything.

We don't have good GPUs that is as fast until 580 (or to some extents, the first Titan). Previous generations only about 2 to 5x faster depending on what types of CPU you compare against.

IMHO, credit should always go to Alex Krizhevsky for the rapid spread of deep learning. He has shown us it was possible. Even without Tensorflow and PyTorch, we will be fine with Caffe, torch, mxnet or Julia.

Reading a license plate seems like precisely the kind of circumstances where spurious 'plausible interpretation' of limited data can cause trouble.

You show in a courtroom a CNN-enhanced low light image of a car and it's there, literally 'clear as day' - the jury will find it pretty compelling. But maybe the data really wasn't there in the original image, and the CNN just filled in some blanks based on previous images of license plates, letters, and just random noise it had seen in the past.

The worry is when these kinds of algorithms get built in to basic image capture processes, so you never even see the raw data, only data that has already been filtered through the inbuilt prejudices of the CNN enhancement suite.

The camera never lies, but now it doesn't have to, because it can convince itself it saw something that wasn't really there...

Using these processed images for evidence purposes sounds like precisely what the parent comment is concerned about.

Photographs have never been faithful records. The map never the territory. There are always judgement calls. The whole concept of JPEG is to throw away information.

This hits on an interesting point.

There is an entropy limit to the message, but the message isn't actually the only data.

One thing humans are great at is integrating existing knowledge into a messy situation and intuiting more than is available just from the raw message.

I.e. The message has an entropy limit, but the message isn't the whole dataset.

The courts & hopefully the jurors should be aware that humans are fallible & capable of lying, but may have hard time believing that cameras can lie as well.

Two 2D Images. The brain doesn't have to guess much when it can use the parallax effect created by both eyes. That's why quite a few animals have two eyes. And I believe that's why we have two ears, too.

Yes, therefore it is better to turn on the lights!

But wouldn't that level of glare be what would actually happen if you took the original image, and shot it in the amount of light required to make it look like the output image?

It's not trying to make things readable; it's trying to make things look like there was more light in the room when they were shot. In rooms with high lighting, some objects have glare. That's "correct"—it's what would appear in the training data.

> the model actually had [...] information

Wait, did it? Isn't the middle photo being shown for comparison only, rather than as an input?

You shouldn't be downvoted - with a big recession/depression looming, link rot and many sorts of repositories shutting down are a big issue that will slow down the pace of research.

And while brains are the original neural networks, they don't resemble what's going on with ML DNNs at all.

To be clear, the parent did not fail to include a citation. The parenthetical note is that rod cells are so sensitive that they react to being struck by a single photon.

That's actually a very cool fact: https://www.nature.com/news/people-can-sense-single-photons-...

Also other senses, including proprioception. In a completely dark environment, you could swear that you see your hand waving in front of your face. That's because your brain actually does know it's there, and it's trying to create a unified model.

Kind of like this well trained CNN is no longer relying entirely on the raw pixel values, but is statistically inferring a brighter image from the baseline.

I wonder what things would be like when phone cameras are as good as human eyes, the climax of consumer photography?

The problem is the amount of pictures you'd need. It's much easier to use available datasets if you know how to preprocess the data.

Longevity.

Which is extremely lossy, because any ISO other than the sensor's native level is the result of in-camera processing. Unlike film, adjusting the "ISO" in a digital camera doesn't increase sensitivity; that's physically impossible. Instead, very strong overgain processing is applied.

So in this instance they're processing lossily on top of an image already processed lossily in-camera.

Apologies dang, I’ll keep this in mind in future :)

The title needs to be changed so brains recognize it as such. It either needs a preceding adjective or letter indicating what type of convolutional network it is.

The other option is spelling it out.

Most people will read CNN as the news channel. Even those familiar with neural networks.

I think OP was being sarcastic

Still -1 second is impressive!

Well, Turner used to colorize b/w movies, I guess progress marches on.

See also Ted Chiang's What’s Expected of Us, where there's a device with a button and a LED, and the LED always lights up one second before you press the button.

Convolutional Neural Network (for example, some more info here: https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Unders...)

Convolutional neural network

Meta: this is when I like the Slashdot method, you allow the jokes, tag them as jokes, and let people use their own settings to show/hide the jokes. Basically be permissive on content, but demand proper tagging, then allow people to filter out what they don't want.

Title is confusing (?intentionally).

Would make sense to add Tensorflow to make it more specific.

I don't understand what you are saying, it is very "HN" to comment on the title and not on the article.

And it is even more "HN" to comment on details of the title or the article because you don't really know what to say about the article.

Look at my comment.