Show HN: Invertornot.com – API to enhance your images in dark-mode (opens in new tab)

(invertornot.com)

175 pointsmattismegevand2y ago38 comments

Hi HN, I built (https://invertornot.com) it's an API that can predict whether an image will look good/bad while inverted. This is particularly useful for images in dark-mode as you can now safely invert them.

The conservative solution to adapt images for dark-mode consist in dimming the image, however there is a lot of images that can be inverted (graph for example). Using deep learning we can avoid heuristics and obtain a much more reliable solution.

The API uses an EfficientNet pre-trained model fine-tuned on a custom dataset (1.1k samples). EfficientNet was chosen as it was pre-trained and offered the best performance for its size.

The trained model is very small (16MB) which means you can easily run your own instance. This problem is very simple for deep learning as it's a simple binary classification.

For this project training the model wasn't the challenge as most of the time was spent on the construction of the dataset.

For the API I'm using FastAPI, Redis and ONNX Runtime to run the model. The API can be used by posting the images to the API, using URL and using SHA-1 for already processed images.

The API is free and open-sourced (http://github.com/mattismegevand/invertornot).

38 comments

madeofpalk2y ago

Semi-related - I saw this on https://v8.dev a while back, but `filter: hue-rotate(180deg) invert();` can be a neat CSS trick to 'dark mode' some kinds of graphics, while not screwing with the colours so much. The `hue-rotate` help a bit to keep 'blues blue' etc.

It's far from perfect, but it's a neat one to have in your back pocket.

therein2y ago

Anyone who thought Retool should have a dark mode has come across this. Definitely a useful trick.

kinj282y ago

Hey This is cofounder of a retool alternative here.

We did introduce dark mode in a manner where a creator can even make it dynamic for end user to choose a different theme.

While I was loggerheads with my CTO cofounder on this - I am glad someone finds this useful in the community of users.

I am curious what kind of use case you have for retool/ alternate tools where you see dark mode as a need

2 more replies

medstrom2y ago

If you don't have a lot of images, you can go over them all manually and write the "please invert me" request directly into the filename.

CSS:

    img[src^="/images/invertable"] {
        filter: hue-rotate(180deg) invert(1);
    }

Now all image names starting with "invertable" will be inverted.

gwern1y ago

Yep, or you can just set a class on each image. It's easy even in Markdown: `![](image){.invertible}`.

It's just that often you do have a lot of images and, more importantly, they may not be known in advance. If images are dynamic, then you obviously can't go over them manually, no matter how few of them there are.

This was the principal motivation for me for InvertOrNot.com: I am willing to manually classify hundreds or thousands of images for inversion, and I did, but what I can't do is do that for all of the many Wikipedia popups on gwern.net - not just because there's 22k+ of them, but because they are constantly being added, WP articles are constantly changing, and they are fully recursive so a reader could pop up any WP article. A fully-automated API is the only possible solution. (Doesn't have to be neural nets, that's just the most obvious & easiest approach; I expect classical approaches would work too and would be easier to get running in-browser too.)

And then if you have a high-quality fully-automated API, then you might as well drop the manual classifying. It's tedious and adds friction to writing.

alanbernstein2y ago

Neat, as a reader, I've absolutely wanted this exact behavior many times. Do you have any examples of incorrectly classified images? As you say, the model seems quite simple, but I wonder if there are some pathological kinds of photographs that result in inversion.

mattismegevandOP2y ago

Some graphs with lots of colors can be difficult for models to handle. In these cases, the model often chooses not to flip the graph to avoid making it hard to read, based on past problems with similar graphs. The model generally does a good job at deciding when to flip something. Although I haven't seen it flip photos, it's possible that there are examples out there.

gwern2y ago

Some recently-added datapoints I think the current model mis-classifies:

https://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Pr... https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/US... https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Na... https://upload.wikimedia.org/wikipedia/commons/thumb/b/bb/Li... https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Pi...

rockwotj2y ago

I built dark mode for an email client’s (shortwave.com) HTML emails and this was the last major hurdle to solve. I wish that had existed then :)

Generally making emails dark mode is a non trivial problem, there is an hope source library that is a good start from Tempo: https://github.com/yourtempo/tempo-message-dark-mode

However there are plenty of edge cases that plugin doesn’t handle that we had to fix at Shortwave. Examples include legacy HTML4 attributes, better support for emails that are half dark mode, emails that have background images of #fff (looking at you google play reciepts), the list goes on. It’s honestly pretty solid now

noduerme2y ago

Huh. This is neat.

As a side, I have a great frustration balancing dark mode with other things. I use flux on my Mac, not for color temp but strictly for the "darkroom" mode which inverts everything and makes the (inverted) brightness values into red. I only do this when coding at night, to help with my sleep... but the trouble is it's made for people who use light mode during the day. I use dark mode all day. So I have to get out of dark mode and change my vscode theme to light in order for it to work. Not a huge deal, but then I open up some web page with a black background and go blind from all the red.

hunter2_2y ago

Does this distinguish white backgrounds from transparent backgrounds? For example, a graph consisting of anything-on-white should probably be inverted, while a graph consisting of pure red on transparent should probably not be inverted. For the latter, increasing the luminance (i.e. make it pink on transparent) might be best, followed by leaving it as-is, followed by inverting.

But I guess with ML reinforced by human feedback, trends like this manifest without having to explicitly handle them as special cases at all...

rob742y ago

I'm confused... has the meaning of "invert" changed since I last looked? Because the bar chart given as an example on the page isn't actually inverted (the color of the bars and as far as I can see also the text stay the same, only the background changes from light to dark). So, additionally to this API, you need another model that does the actual "inversion"? Or just good old editing?

EDIT: from another comment - actually "inverting" seems to refer to doing

    hue-rotate(180deg) invert()

...which (if I read it correctly) inverts the hue of colored areas before applying the actual inversion in order to keep the colors the same (more or less - see also https://stackoverflow.com/questions/65344006/why-does-filter...) ? Not a frontend developer, so I wasn't familiar with this trick until now. TIL...

a2code2y ago

How can you use or create this model for conventional non-API projects? Like to embed it within some C code?

lights01232y ago

They're using ONNX Runtime to run the model, which has a C API: https://onnxruntime.ai/docs/get-started/with-c.html. You could also use any other ML inference library that supports the ONNX format.

You'd need to look at this project's code to see how it converts input data to the expected matrix format, though.

mattismegevandOP2y ago

Exactly, the transformation done for inference are located here: https://github.com/mattismegevand/invertornot/blob/main/api/...

SCUSKU2y ago

Very cool! Nice job! Out of curiosity, how much CPU/GPU/RAM resources do you need to allocate to run something like this? Like you mention you're using small models. For example, could this run on a t2.micro on EC2? (1 vCPU + 1GB RAM)

mattismegevandOP2y ago

Thank you! Very little resources are needed to run it, gwern is using the API and at first I was using a 6$ Droplet (exact same specs as the t2.micro) and it was running well.

singularity20012y ago

not exactly the same but for colored pictures GPT recently suggested inverting the image and then turning the color space by 180° so that the original colors are restored... pretty genius

noisy_boy2y ago

What I am interested in is a plugin/extension for Firefox/Chrome that can invert a video running on a page (darkreader is not doing that).

gwern2y ago

Does that really make sense? Unlike a single static image, a video is not one thing. It might go rapidly between invertible and non-invertible. Would you expect it to switch strategies potentially every scene or second...?

noisy_boy1y ago

Yes it indeed has very specific use case of tutorial videos with white background; those generally stay the same until the switch to dark mode for showing code. It would be impressive if the add-on is smart enough to do what you said to maintain dark background throughout (I don't have a lot of idea of how technically challenging this might be).

Dibby0532y ago

As a one-off solution you could switch to light mode and invert the display.

pimlottc2y ago

I would be interested to see more samples. Naively it seems like you could just say "isPhoto ? REDUCE_BRIGHTNESS : INVERT"

mattismegevandOP2y ago

The two big cluster are indeed photo and charts. However chart or meme are not all invertible, charts with lots of color can for example be degraded by inversion. Also some chart are already done for dark theme, for those images the model will understand that it doesn’t need to invert them.

bagels2y ago

Is inverting the colors in a photo ever the right choice? People come out looking really strange.

juancampa2y ago

That’s the point of using a NN. If it detects people, dim it (losing contrast). If it detect bright vector graphics, invert it (not losing contrast).

klntsky2y ago

What's your accuracy?

mattismegevandOP2y ago

Close to 100%, this problem seem to be easily solved by the model. I plan to do cross validation and also to explore the use of a smaller model.

klntsky2y ago

Did you separate the test dataset from the training one?

1 more reply

hansihe2y ago

This is really cool! Do you have some more example images somewhere?

mattismegevandOP2y ago

Thank you, yes you can try it directly (https://invertornot.com/docs) or you can either upload an images or spam the button to get random images and see their prediction.

Labo3332y ago

Pretty cool! A nice addition would be an in-browser runtime!

rvnx2y ago

It used to be implemented in Chrome, it was part of the feature called "Force Dark Mode Web Contents" and it had a runtime tflite model that determines if the lightness should be inverted (and a simpler heuristics that just counts the number of colored pixels).

block_dagger2y ago

Dark Reader needs this! Nice work.

j / k navigate · click thread line to collapse

38 comments

madeofpalk2y ago

It's far from perfect, but it's a neat one to have in your back pocket.

therein2y ago

Anyone who thought Retool should have a dark mode has come across this. Definitely a useful trick.

kinj282y ago

Hey This is cofounder of a retool alternative here.

We did introduce dark mode in a manner where a creator can even make it dynamic for end user to choose a different theme.

While I was loggerheads with my CTO cofounder on this - I am glad someone finds this useful in the community of users.

I am curious what kind of use case you have for retool/ alternate tools where you see dark mode as a need

2 more replies

medstrom2y ago

If you don't have a lot of images, you can go over them all manually and write the "please invert me" request directly into the filename.

CSS:

    img[src^="/images/invertable"] {
        filter: hue-rotate(180deg) invert(1);
    }

Now all image names starting with "invertable" will be inverted.

gwern1y ago

Yep, or you can just set a class on each image. It's easy even in Markdown: `![](image){.invertible}`.

And then if you have a high-quality fully-automated API, then you might as well drop the manual classifying. It's tedious and adds friction to writing.

alanbernstein2y ago

mattismegevandOP2y ago

gwern2y ago

Some recently-added datapoints I think the current model mis-classifies:

rockwotj2y ago

I built dark mode for an email client’s (shortwave.com) HTML emails and this was the last major hurdle to solve. I wish that had existed then :)

Generally making emails dark mode is a non trivial problem, there is an hope source library that is a good start from Tempo: https://github.com/yourtempo/tempo-message-dark-mode

noduerme2y ago

Huh. This is neat.

hunter2_2y ago

But I guess with ML reinforced by human feedback, trends like this manifest without having to explicitly handle them as special cases at all...

rob742y ago

EDIT: from another comment - actually "inverting" seems to refer to doing

    hue-rotate(180deg) invert()

a2code2y ago

How can you use or create this model for conventional non-API projects? Like to embed it within some C code?

lights01232y ago

They're using ONNX Runtime to run the model, which has a C API: https://onnxruntime.ai/docs/get-started/with-c.html. You could also use any other ML inference library that supports the ONNX format.

You'd need to look at this project's code to see how it converts input data to the expected matrix format, though.

mattismegevandOP2y ago

Exactly, the transformation done for inference are located here: https://github.com/mattismegevand/invertornot/blob/main/api/...

SCUSKU2y ago

mattismegevandOP2y ago

Thank you! Very little resources are needed to run it, gwern is using the API and at first I was using a 6$ Droplet (exact same specs as the t2.micro) and it was running well.

singularity20012y ago

not exactly the same but for colored pictures GPT recently suggested inverting the image and then turning the color space by 180° so that the original colors are restored... pretty genius

noisy_boy2y ago

What I am interested in is a plugin/extension for Firefox/Chrome that can invert a video running on a page (darkreader is not doing that).

gwern2y ago

noisy_boy1y ago

Dibby0532y ago

As a one-off solution you could switch to light mode and invert the display.

pimlottc2y ago

I would be interested to see more samples. Naively it seems like you could just say "isPhoto ? REDUCE_BRIGHTNESS : INVERT"

mattismegevandOP2y ago

bagels2y ago

Is inverting the colors in a photo ever the right choice? People come out looking really strange.

juancampa2y ago

That’s the point of using a NN. If it detects people, dim it (losing contrast). If it detect bright vector graphics, invert it (not losing contrast).

klntsky2y ago

What's your accuracy?

mattismegevandOP2y ago

Close to 100%, this problem seem to be easily solved by the model. I plan to do cross validation and also to explore the use of a smaller model.

klntsky2y ago

Did you separate the test dataset from the training one?

1 more reply

hansihe2y ago

This is really cool! Do you have some more example images somewhere?

mattismegevandOP2y ago

Thank you, yes you can try it directly (https://invertornot.com/docs) or you can either upload an images or spam the button to get random images and see their prediction.

Labo3332y ago

Pretty cool! A nice addition would be an in-browser runtime!

rvnx2y ago

block_dagger2y ago

Dark Reader needs this! Nice work.

j / k navigate · click thread line to collapse