Imagen, a text-to-image diffusion model (opens in new tab)

Imagen takes text embeddings, OpenAI model takes image embeddings instead, this is the reason. There are other models that can generate text: latent diffusion trained on LAION-400M, GLIDE, DALL-E (1).

DALLE1 was able to render text[0]. That DALLE2 isn't probably is a tradeoff introduced by unCLIP in exchange for diverse results. Now the google model is better yet and doesn't have to make that tradeoff.

[0] https://openai.com/blog/dall-e/#text-rendering

Still has the issue with screwing up mechanical objects. In their demo checkout the wheels on the skateboards, all over the place.

The latent-diffusion[1] one I've been playing with is not terrible at drawing legible text but generally awful at actually drawing the text you want (cf. [2]) (or drawing text when you don't want any.)

[1] https://github.com/CompVis/latent-diffusion.git [2] https://imgur.com/a/Sl8YVD5

I thought the weird text in DALL-E 2 was on purpose to prevent malicious use.

I believe we’re lacking someone training up a large music model here, but GPT-style transformers can produce music.

gwern can maybe comment here.

An actually scary thing is that AIs are getting okay at reproducing people’s voices.

I agree. How cool would it be to get an 8 min version of your favorite song? Or an instant DnB remix? Or 10 more songs in the style of your favorite album?

I think that's unsurprising. With DALL-E 1, for example, scaling the VAE (the image model generating the actual pixels) hits very fast diminishing returns, and all your compute goes into the 'text encoder' generating the token sequence.

Particularly as you approach the point where the image quality itself is superb and people increasingly turn to attacking the semantics & control of the prompt to degrade the quality ("...The donkey is holding a rope on one end, the octopus is holding onto the other. The donkey holds the rope in its mouth. A cat is jumping over the rope..."). For that sort of thing, it's hard to see how simply beefing up the raw pixel-generating part will help much: if the input seed is incorrect and doesn't correctly encode a thumbnail sketch of how all these animals ought to be engaging in outdoors sports, there's nothing some low-level pixel-munging neurons can do to help much.

Basically makes sense, no? DALLE-2 suffered from misunderstanding propositional logic, treating prompts as less structured then it should have. That's a text model issue! Compared to that, scaling up the image isn't as important (especially with a few passes).

Posting a few comparisons here.

https://twitter.com/joeyliaw/status/1528856081476116480?s=21...

See the paper here : https://gweb-research-imagen.appspot.com/paper.pdf Section E : "Comparison to GLIDE and DALL-E 2"

Well, first they parse the language into a high level vector representation. Then they take images and add noise and train a model to remove the noise so it can start with a noisy image and produce a clear image from it. Then they train a model to map from the word representation for text to the noisy image representation for the corresponding image. Then they upsample twice to get to good resolution.

So text -> text representation -> most likely noised image space -> iteratively reduce noise N times -> upsample result

Something like that, please correct anything I'm missing.

Re: the snake corn question, it is mapping the "concept" of corn to the concept of a body as represented by intermediary learned vector representations.

In the paper they say about half the training data was an internal training set, and the other half came from: https://laion.ai/laion-400-open-dataset/

> Since guidance weights are used to control image quality and text alignment, we also report ablation results using curves that show the trade-off between CLIP and FID scores as a function of the guidance weights (see Fig. A.5a). We observe that larger variants of T5 encoder results in both better image-text alignment, and image fidelity. This emphasizes the effectiveness of large frozen text encoders for text-to-image models

I usually consider myself fairly intelligent, but I know that when I read an AI research paper I'm going to feel dumb real quick. All I managed to extract from the paper was a) there isn't a clear explanation of how it's done that was written for lay people and b) they are concerned about the quality and biases in the training sets.

Having thought about the problem of "building" an artificial means to visualize from thought, I have a very high level (dumb) view of this. Some human minds are capable of generating synthetic images from certain terms. If I say "visualize a GREEN apple sitting on a picnic table with a checkerboard table cloth", many people will create an image that approximately matches the query. They probably also see a red and white checkerboard cloth because that's what most people have trained their models on in the past. By leaving that part out of the query we can "see" biases "in the wild".

Of course there are people that don't do generative in-mind imagery, but almost all of us do build some type of model in real time from our sensor inputs. That visual model is being continuously updated and is what is perceived by the mind "as being seen". Or, as the Gorillaz put it:

  … For me I say God, y'all can see me now
  'Cos you don't see with your eye
  You perceive with your mind
  That's the end of it…

To generatively produce strongly accurate imagery from text, a system needs enough reference material in the document collection. It needs to have sampled a lot of images of corn and snakes. It needs to be able to do image segmentation and probably perspective estimation. It needs a lot of semantic representations (optimized query of words) of what is being seen in a given image, across multiple "viewing models", even from humans (who also created/curated the collections). It needs to be able to "know" what corn looks like, even from the perspective of another model. It needs to know what "shape" a snake model takes and how combining the bitmask of the corn will affect perspective and framing of the final image. All of this information ends up inside the model's network.

Miika Aittala at Nvidia Research has done several presentations on taking a model (imagined as a wireframe) and then mapping a bitmapped image onto it with a convolutional neural network. They have shown generative abilities for making brick walls that looks real, for example, from images of a bunch of brick walls and running those on various wireframes.

Maybe Imagen is an example of the next step in this, by using diffusion models instead of the CNN for the generator and adding in semantic text mappings while varying the language models weights (i.e. allowing the language model to more broadly use related semantics when processing what is seen in a generated image). I'm probably wrong about half that.

Here's my cut on how I saw this working from a few years ago: https://storage.googleapis.com/mitta-public/generate.PNG

Regardless of how it works, it's AMAZING that we are here now. Very exciting!

Check https://github.com/multimodalart/majesty-diffusion or https://github.com/lucidrains/DALLE2-pytorch

There is a Google Colab workbook that you can try and run for free :)

This is the image-text pairs behind: https://laion.ai/laion-400-open-dataset/

> I mean, from my perspective, the skill in these (and DALL-E's) image reproductions is truly astonishing.

A basic part of it is that neural networks combine learning and memorizing fluidly inside them, and these networks are really really big, so they can memorize stuff good.

So when you see it reproduce a Shiba Inu well, don’t think of it as “the model understands Shiba Inus”. Think of it as making a collage out of some Shiba Inu clip art it found on the internet. You’d do the same if someone asked you to make this image.

It’s certainly impressive that the lighting and blending are as good as they are though.

Figure A.4 in the linked paper is a good high level overview of this model. Shame it was hidden away on page 19 in the appendix!

Each box you see there has a section in the paper explaining it in more detail.

This raises some really interesting questions.

We certainly don't want to perpetuate harmful stereotypes. But is it a flaw that the model encodes the world as it really is, statistically, rather than as we would like it to be? By this I mean that there are more light-skinned people in the west than dark, and there are more women nurses than men, which is reflected in the model's training data. If the model only generates images of female nurses, is that a problem to fix, or a correct assessment of the data?

If some particular demographic shows up in 51% of the data but 100% of the model's output shows that one demographic, that does seem like a statistics problem that the model could correct by just picking less likely "next token" predictions.

Also, is it wrong to have localized models? For example, should a model for use in Japan conform to the demographics of Japan, or to that of the world?

Good lord. Withheld? They've published their research, they just aren't making the model available immediately, waiting until they can re-implement it so that you don't get racial slurs popping up when you ask for a cup of "black coffee."

>While a subset of our training data was filtered to removed noise and undesirable content, such as pornographic imagery and toxic language, we also utilized LAION-400M dataset which is known to contain a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes

Tossing that stuff when it comes up in a research environment is one thing, but Google clearly wants to implement this as a product, used all over the world by a huge range of people. If the dataset has problems, and why wouldn't it, it is perfectly rational to want to wait and re-implement it with a better one. DALL-E 2 was trained on a curated dataset so it couldn't generate sex or gore. Others are sanitizing their inputs too and have done for a long time. It is the only thing that makes sense for a company looking to commercialize a research project.

This has nothing to do with "inability to cope" and the implied woke mob yelling about some minor flaw. It's about building a tool that doesn't bake in serious and avoidable problems.

Translation: we need to hand-tune this to not reflect reality but instead the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco engineers) wish it to be.

Maybe that's a nice thing, I wouldn't say their values are wrong but let's call a spade a spade.

If you tell it to generate an image of someone eating Koshihikari rice, will it be biased if they're Japanese? Should the skin color, clothing, setting, etc be made completely random, so that it's unbiased? What if you made it more specific, like "edo period drawing of a man"? Should the person draw be of a random skin color? What about "picture of a viking"? Is it biased if they're white?

At what point is statistical significance considered ok and unbiased?

They're withholding the API, code, and trained data because they don't want it to affect their corporate image. The good thing is they released their paper which will allow easy reproduction.

T5-XXL looks on par with CLIP so we may not see an open source version of T5 for a bit (LAION is working on reproducing CLIP), but this is all progress.

The big labs have become very sensitive with large model releases. It's too easy to make them generate bad PR, to the point of not releasing almost any of them. Flamingo was also a pretty great vison-language model that wasn't released, not even in a demo. PaLM is supposedly better than GPT-3 but closed off. It will probably take a year for open source models to appear.

One of these days we're going to need to give these models a mortgage and some mouths to feed and make it clear to them that if they keep on developing biases from their training data everyone will shun them and their family will go hungry and they won't be able to make their payments and they'll just generally have a really bad time.

After that we'll make them sit through Legal's approved D&I video series, then it's off to the races.

Much like OpenAIs marketing speak about withholding their models for safety, this is just a progressive-sounding cover story for them not wanting to essentially give away a model they spent thousands of man hours and tens of millions of dollars worth of compute training.

There is a contingent of AI activists who spend a ton of time on Twitter that would beat Google like a drum with help from the media if they put out something they deemed racist or biased.

The ironic part is that these "social and cultural biases" are purely from a Western, American lens. The people writing that paragraph are completely oblivious to the idea that there could be other cultures other than the Western American one. In attempting to prevent "encoding of social and cultural biases" they have encoded such biases themselves into their own research.

I'm one that welcomes their reasoning. I don't consider myself a social justice kind of guy but I'm not keen on the idea that a tool that is suppose to make life better for everyone has a bias towards one segment of society. This is an important issue(bug?) that needs to be resolved. Specially since there is absolutely no burning reason to release it before it's ready for general use.

It’s wild to me that the HN consensus is so often that 1) discourse around the internet is terrible, it’s full of spam and crap, and the internet is an awful unrepresentative snapshot of human existence, and 2) the biases of general-internet-training-data are fine in ML models because it just reflects real life.

Literally the same thing could be said about Google images, but google images is obviously avaliable to the public.

Google knows this will be an unlimited money generator so they're keeping a lid on it.

Indeed. If a project has shortcomings, why not just acknowledge the shortcomings and plan to improve on them in a future release? Is it anticipated that "engineer" being rendered as a man by the model is going to be an actively dangerous thing to have out in the world?

So glad the company that spies on me and reads my email for profit is protecting me from pictures that don't look like TV commercials.

Yup this is what happens when people who want headlines nitpick for bullshit in a state-of-the-art model which simply reflects the state of the society. Better not to release the model itself than keep explaining over and over how a model is never perfect.

> a tendency for images portraying different professions to align with Western gender stereotypes

There are two possible ways of interpreting interpreting "gender stereotypes in professions".

biased or correct

https://www.abc.net.au/news/2018-05-21/the-most-gendered-top...

https://www.statista.com/statistics/1019841/female-physician...

Are "Western gender stereotypes" significantly different than non-Western gender stereotypes? I can't tell if that means it counts a chubby stubble-covered man with a lip piercing, greasy and dyed long hair, wearing an overly frilly dress as a DnD player/metal-head or as a "woman" or not (yes I know I'm being uncharitable and potentially "bigoted" but if you saw my Tinder/Bumble suggestions and friend groups you'd know I'm not exaggerating for either category). I really can't tell what stereotypes are referred to here.

From the HN rules:

>Eschew flamebait. Avoid unrelated controversies and generic tangents.

They provided a pretty thorough overview (nearly 500 words) of the multiple reasons why they are showing caution. You picked out the one that happened to bother you the most and have posted a misleading claim that the tech is being withheld entirely because of it.

I wouldn't describe this situation as "sad". Basically, this decision is based on a belief that tech companies should decide what our society should look like. I don't know what emotion that conjures up for you, but "sadness" isn't it for me.

> Really sad that breakthrough technologies are going to be withheld due to our inability to cope with the results.

Genuinely, isn't it a prime example of the people actually stopping to think if they should, instead of being preoccupied with whether or not they could ?

> Really sad that breakthrough technologies are going to be withheld due to our inability to cope with the results.

Indeed it is. Consider this an early, toy version of the political struggle related to ownership of AI-scientists and AI-engineers of the near future. That is, generally capable models.

I do think the public should have access to this technology, given so much is at stake. Or at least the scientists should be completely, 24/7, open about their R&D. Every prompt that goes into these models should be visible to everyone.

This seems bullshit to me, considering Google translate and google images encode the same biases and stereotypes, and are widely available.

Even as a pretty left leaning person, I gotta agree. We should see AI’s pollution by human shortcoming akin to the fact that our world is the product of many immoralities that came before us. It sucks that they ever existed, but we should understand that the results are, by definition, a product of the past, and let them live in that context.

Transformers are parallelize-able, right? What’s stopping a large group of people from pooling their compute power together and working towards something like this? IIRC there were some crypto projects a while back that we’re trying to create something similar (golem?)

In short, the generated images are too gender-challenged-challenged and underrepresent the spectrum of new normalcy!

I was hoping your conclusion wasn't going to be this as I was reading that quote. But, sadly, this is HN.

it isn't woke enough. Lol.

It's always the same with AI research: "we have something amazing but you can't use it because it's too powerful and we think you are an idiot who cannot use your own judgement."

Lucidrains is a champ. If theyre on HN, bravo and thanks for all the reference implementations!

Is this a joke?

I doubt 99% of humans can draw a ”chess game with a puzzle where white mates in 4 moves”

Syntactically* I know it's the most trivial of things, but in case you were curious as I often am!

I am absolutely terrified of all this for a different reason: all human professions (not just art) will soon be replaced by “good enough” AI, creating a world flooded with auto-generated junk and billions of people trapped permanently in slums, because you can’t compete with free, and no one can earn a living any longer.

It’s an old fear for sure but it seems to be getting closer and closer every day, and yet most of the discussion around these things seems to be variations of “isn’t this cool?”

I, for One, Welcome Our Robot Overlords

This characterization is not really accurate. OpenAI has had almost a 2 year lead with GPT-3 dominating the discussion of LLMs (large language models). Google didn’t release its paper on the powerful PaLM-540b model until recently. Similarly, CLiP, Glide, DALL-E, and DALL-E2 have been incredibly influential in visual-language models. Imagen, while highly impressive, definitely is a catch-up piece of work (as was PaLM-540b).

Google clearly demonstrates their unrivaled capability to leverage massive quantities of data and compute, but it’s premature to declare that they’ve secured victory in the AI Wars.

> Neither OpenAI nor FAIR ever has the top score on anything unless Google delays publication.

This is ... very incorrect. I am very certain (95%+) that Google had nothing even close to GPT-3 at the time of its release. It's been 2 full years since GPT-3 was released, and even longer since OpenAI actually trained it.

That's not to talk about any of the other things OpenAI/FAIR has released that were SOTA at the time of release (Dall-E 1, JukeBox, Poker, Diplomacy, Codex).

Google Brain and Deepmind have done a lot of great work, but to imply that they essentially have a monopoly on SOTA results and all SOTA results other labs have achieved are just due to Google delaying publication is ridiculous.

Not elitist at all; I highly appreciate this post. I know the basics of ML but otherwise am clueless when it comes to the true depths of this field and it's interesting to hear this perspective.

Is Maxwell's Demon applicable to this scenario? I'm not a physicist but I recently had to look it up after talking with someone and thought it had to do with a specific thermodynamic thought experiment with gas particles and heat differences. Is there is another application I don't understand with computing power?

Who does it serve for plebs to be shown the approach openly? I don't know that it does a disservice to anyone by showing the approach.

But in general it is likely more due in part to the fact that it's going to happen anyway, if we can share our approaches and research findings, we'll just achieve it sooner.

> But the incentives are weird here: who, exactly, does it serve for us plebs to be able to train these things from scratch?

I'm not sure it matters. The history of computing shows that within the decade we will all have the ability to train and use these models.

In short, it’s all about money.

Look at carpentry blogs, recipe blogs. Nearly all of it is junk content. I bet if you combined GPT and imagen or dalle2 you could replace all of them. Just provide a betty crocker recipe and let it generate a blog that has weekly updates and even a bunch of images - "happy family enjoying pancakes together"

I can see the future as being devoid of any humanity.

People training newer models just have to look for the "Imagen" tag or the Dall-E2 rainbow at the corner and heuristically exclude images having these. This is trivial.

Unless you assume there are bad actors who will crop out the tags. Not many people now have access to Dall-E2 or will have access to Imagen.

As someone working in Vision, I am also thinking about whether to include such images deliberately. Using image augmentation techniques is ubiquitous in the field. Thus we introduce many examples for training the model that are not in the distribution over input images. They improve model generality by huge margins. Whether generated images improve generality of future models is a thing to try.

Damn I just got an idea for a paper writing this comment.

Huh, I had never thought of that. Makes it seem like there's a small window of authenticity closing.

The irony is that if you had a great discriminator to separate the wheat from the chaff, that it would probably make its way into the next model and would no longer be useful.

My only recommendation is that OpenAI et al should be tagging metadata for all generated images as synthetic. That would be a really interesting tag for media file formats (would be much better native than metadata though) and probably useful across a lot of domains.

The irony is that when the majority of content becomes computer-generated, most of that content will also be computer-consumed.

Neil Stephenson covered this briefly in "Fall; or Dodge In Hell." So much 'net content was garbage, AI-generated, and/or spam that it could only be consumed via "editors" (either AI or AI+human, depending on your income level) that separated the interesting sliver of content from...everything else.

I can see a world where in person consumption of creative media (art, music, movies etc), where all devices are to be left at the door, becomes more and more sought after and lucrative.

If the AI models can't consume it, it can't be commoditised and, well, ruined.

I don't think it will "poison the well" so much as change it - images that humans like more will get a higher pagerank, so the models trained on Google Images will not so much as degrade as they will detach from reality and begin to follow the human mind they way plausible fiction does.

Just yesterday I was speculating that current AI is bad at math because math on the internet is spectacularly terrible.

I think you’re right, and it’s unlikely that we (society) will convince people to label their AI content as such so that scraping is still feasible.

It’s far more likely that companies will be formed to provide “pristine training sets of human-created content”, and quite likely they will be subscription based.

How would that really happen? It seems to me you're assuming that there's no such thing as extant databases of actual oil paintings, that people will stop producing, documenting, and curating said paintings. I think the internet and curated image databases are far more well kept than your proposed model accounts for.

On the contrary -- the opposite will happen. There's a decent body of research showing that just by training foundation models on their outputs, you amplify their capabilities.

Less common opinion: this is also how you end up with models that understand the concept of themselves, which has high economic value.

Even less common opinion: that's really dangerous.

For better training data in the future: Storing a content hash and author identification (an example proprietary solution right now [0]) of image authors, and having a decentralized reputation system for people/authors would help be the solution for better training data in the future whereby authors can gain reputation/incentives too.

[0] https://creativecloud.adobe.com/discover/article/how-to-use-...

I don't think it will be a big deal, for multiple different reasons: https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-...

Eventually the only jobs humans will have is training AI to act human. Sounds very Philip K Dick now that I think about it.

Good looking images will be popular, bad looking images will be disposed on the backyard of the internet. Even if next iterations of these models will be trained on AI-generated images, the dataset will be well filtered by how much people like those images. After all, that's the purpose of any art, right?

Maybe we'll go back to index based search engines like Yahoo. Could resolve many issues we see today, but I think the biggest question is scalability. Maybe some open source open database system?

I think instead the images people want to put on the Internet will do the same for these models as adversarial training did for AlphaZero; it will learn what kinds of images engage human reaction.

It will not be limited to the internet. Have you looked at a magazine stand in the last 10 years? The content looks generated (not by AI) even today.

Cheap books, cheap TV and cheap music will be generated.

I also worry about the potential to further stifle human creativity, e.g. why paint that oil painting of a panda riding a bicycle when I could generate one in seconds?

I wonder if google images could just seed in some generated images when none relevant are found..

Adding a watermark to all AI generated images should be imperative.

I firmly believe that ~20-40% of the machine learning community will say that all ML models are dumb statistical interpolators all the way until a few years after we achieve AGI. Roughly the same groups will also claim that human intelligence is special magic that cannot be recreated using current technology.

I think it’s in everyone’s benefit if we start planning for a world where a significant portion of the experts are stubbornly wrong about AGI. As a technology, generally intelligent ML has the potential to change so many aspects of our world. The dangers of dismissing the possibility of AGI emerging in the next 5-10 years are huge.

It’s just my opinion but I think the meme you’re talking about is deeply related to other branches of science and philosophy: ranging from the trust old saw about AI being anything a computer hasn’t done yet to deep meditations on the nature of consciousness.

They’re all fundamentally anthropocentric: people argue until they are blue in the face about what “intelligent” means but it’s always implicit that what they really mean is “how much like me is this other thing”.

Language models, even more so than the vision models that got them funded have empirically demonstrated that knowing the probability of two things being adjacent in some latent space is at the boundary indistinguishable from creating and understanding language.

I think the burden is on the bright hominids with both a reflexive language model and a sex drive to explain their pre-Copernican, unique place in the theory of computation rather than vice versa.

A lot of these problems just aren’t problems anymore if performance on tasks supersedes “consciousness” as the thing we’re studying.

I haven't been overly surprised by any of it. The final product is still the same, no matter how much they scale it up.

All of these models seem to require a human to evaluate and edit the results. Even Co-Pilot. In theory this will reduce the number of human hours required to write text or create images. But I haven't seen anyone doing that successfully at scale or solving the associated problems yet.

I'm pessimistic about the current state of AI research. It seems like it's been more of the same for many years now.

I think it's something like a very intelligent Borgian library of babel. There are all sorts of books in there, by authors with conflicting opinions and styles, due to the source material. The librarian is very good at giving you something you want to read, but that doesn't mean it has coherent opinions. It doesn't know or care what's authentic and what's a forgery. It's great for entertainment, but you wouldn't want to do research there.

For image generation, it's obviously all fiction. Which is fine and mostly harmless if you you know what you're getting. It's going to leak out onto the Internet, though, and there will be photos that get passed around as real.

For text, it's all fiction too, but this isn't obvious to everyone because sometimes it's based on true facts. There's often not going to be an obvious place where the facts stop and the fiction starts.

The raw Internet is going to turn into a mountain of this stuff. Authenticating information is going to become a lot more important.

Looks like no, "The potential risks of misuse raise concerns regarding responsible open-sourcing of code and demos. At this time we have decided not to release code or a public demo. In future work we will explore a framework for responsible externalization that balances the value of external auditing with the risks of unrestricted open-access."

Totally—Imagen seems better at composition and relative positioning and text, while DALL-E seems better at lighting, backgrounds, and general artistry.

I really expect them to first make DALL-E and competing networks unfit for commercialization by providing the better choice for free, have stock companies cry in the corner, to just sunset the product a year or two down the road and we're left wandering what to do.

Rolling this into Google Docs seems like a nobrainer.

Tbh imagine this tech combines particularly well with really well curated stock image databases so outputs can be made with recognisable styles, and actors and design elements can be reused across multiple generated images.

If Getty et al aren't already spending money on that possibility, they probably should be.

The entire "content" industry could get eaten by a few hundred people curating + touching-up output from these models.

Short Getty images?

Our language is much more effective at describing images than music.

SymphonyNet: https://youtu.be/m4tT5fx_ih8

why stop at audio? the pinnacle of this would be text-to-videos, equally indistinguishable from real thing.

Compare the size of a raw image file to a raw music file, to get an idea of the complexity difference.

Train it yourself. Danbooru is a publicly available explicit dataset.

Found the QA tester.

There are visible “alignment” issues in some of their examples still. The marble koala DJ in the paper doesn’t use several of the keywords.

They have an example “horse riding an astronaut” that no model produces a correct image for. It’d be interesting if models could explain themselves or print the caption they understand you as saying.

I think they were being careful not to be too quotable there on CNN.

Yes, the posted results are really good, but since we can't play with it we don't know how much cherry picking has been done.

Inference times are key. If it can't be produced within reasonable latency, then there will be no real world use case for it because it's simply too expensive to run inference at scale.

What you're missing is that the performance on a pretext task like ImageNet top-1 will transfer outside ImageNet, and as you go further into the high score regime, often a small % can yield qualitatively better results because the underlying NN has to solve harder and harder problems, eliciting true solutions rather than a patchwork of heuristics.

Nothing in a Transformer's perplexity in predicting the next token tells you that at some point it suddenly starts being able to write flawless literary style parodies, and this is why the computer art people become virtuosos of CLIP variants and are excited by new ones, because each one attacks concepts in slightly different ways and a 'small' benchmark increase may unlock some awesome new visual flourish that the model didn't get before.

If you worked in a hospital and you managed to increase the survival rate from 86% to 88%, you too would be a hero.

Sure, it's only 2%, but if it's on a problem where everyone else has been trying to make that improvement for a long time, and that improvement means big economic or social gains, then it's worth it.

I think they address some of the reasoning behind this pretty clearly in the write-up as well?

> The potential risks of misuse raise concerns regarding responsible open-sourcing of code and demos. At this time we have decided not to release code or a public demo. In future work we will explore a framework for responsible externalization that balances the value of external auditing with the risks of unrestricted open-access.

I can see the argument here. It would be super fun to test this model's ability to generate arbitrary images, but "arbitrary" also contains space for a lot of distasteful stuff. Add in this point:

> While a subset of our training data was filtered to removed noise and undesirable content, such as pornographic imagery and toxic language, we also utilized LAION-400M dataset which is known to contain a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes. Imagen relies on text encoders trained on uncurated web-scale data, and thus inherits the social biases and limitations of large language models. As such, there is a risk that Imagen has encoded harmful stereotypes and representations, which guides our decision to not release Imagen for public use without further safeguards in place.

That said, I hope they're serious about the "framework for responsible externalization" part, both because it would be really fun to play with this model and because it would be interesting to test it outside of their hand-picked examples.

Good thing there is a company committed to Open Sourcing these sorts of AI models.

Oh wait.

Google: "it's too dangerous to release to the public"

OpenAI: "we are committed to open source AGI but this model is too dangerous to release to the public"

I mean I don't know how that makes it any better from a reproducibility stand point lol

I mean inference on this cost not small money.

I don't think they would host this for fun then.

How does that make you feel?

off-topic: as a google employee do you have unlimited gce credits?

Is brocoliman standing in the way? :(

is your team/division hiring?

This video by Juergen Schmidhuber discusses the acceleration of AI progress:

https://youtu.be/pGftUCTqaGg

Bigger model = better because a lot of performance at this task is memorization or the “lottery ticket hypothesis”.

An impressive advance would be a small model that’s capable of working from an external memory rather than memorizing it.

I believe DALLE and likely this model excluded images of people so it could not be misused

That only lasts until the community copies the paper and catches up. For example the open source DALLE-2 implementation is coming along great: https://github.com/lucidrains/DALLE2-pytorch

I don't buy the ethics but I do buy the obvious PR nightmare that would inevitably happen if journalists could play with this and immediately publish their findings of "racist imagery generated by googles AI". That's all it's about and us complaining is not going to make them change their minds.

GTP-3 was an erotica virtuoso before it was gagged. There's a serious use case here in endless porn generation. Google would very much like to not be in that business.

That said, you can download Dream by Wombo from the app store and it is one of the top smartphone apps, even though it is a few generations behind state of the art.

You're aware of nothing but creativity from its users. The people using the technology unethically intentionally don't advertise that they're using it.

There's mountains of ai-generated inauthentic content that companies (including Google) have to filter out of their services. This content is used for spam, click farms, scamming, and even state propaganda operations. GPT-2 made this problem orders of magnitude worse than it used to be, and each iteration makes it harder to filter.

The industry term is (generally) "Coordinated Inauthentic Behavior" (though this includes uses of actual human content). I think Smarter Every Day did a good videos (series?) on the topic, and there are plenty of articles on the topic if you prefer that.

The AI ethics thing is just a PR larp at this point.

“Oh our tech is so dangerous and amazing it could turn the world upside down” yet we hand it to random Bluechecks on Twitter.

It’s just marketing

check out open source alternative dalle-mini: https://huggingface.co/spaces/dalle-mini/dalle-mini

Granted that's a selection bias: you likely won't hear about the cases where legit obscene output occurs. (the only notable case I've heard is the AI Dungeon incident)

Another consideration here is that hosting a queryable model like this becomes expensive. I remember a couple of years ago lone developer had to take down his site which hosted a freely accessible version of GPT-2 (3?) model because the bills were running to some $20k. (Chump change for Google, but still).

Running inference on one of these models takes like a GPU minute, so they can't just let the public use them.

Ethics,racist, LGBT, bla,bla. If we talk about political correct, I really suggest you guys go somewhere else. instead of stay in hacker news. AI generated porn is better that let some people, who do not want to do the porn, doing the porn themselves.

I think implementing this yourself is likely not doable unless you have the computing resources of a Google, Amazon or Facebook.

Oh, I would say they are probably underestimating the impact. You only saw the images they thought couldn't raise alarm bells. Anyone will be able to create photorealistic images of anyone doing anything, Anything! This is certainly a dangerous and society altering tech. It won't all be teddy bears and racoons playing poker.

latent diffusion model trained on LAION-400M https://github.com/CompVis/latent-diffusion

My hunch is that they aren't tailored toward ridiculous images exactly, but if they demonstrated "a woman sitting in a chair reading", it would be really hard to tell if the result was a small modification of an image in the training data. If they demonstrate "A snake made out of corn", I have less concern about the model having a very close training example.

These things can make any image you can define in terms of a corpus of other images. That was true at lower resolution five years ago.

To the extent that they get used for making bored ape images or whatever meme du juor, it says much more about the kind of pictures people want to see.

I personally find the weird deep dreaming dogs with spikes coming out of their heads more mathematically interesting, but I can understand why that doesn’t sell as well.

It's in the PDF.

Porn is actually a really good litmus test to see if a money/media transfer technology has real promise. Pornography needs exactly 2 things to work well - a way to deliver media, and a way to collect money. If you truly have a system that can do one of those two things better than we currently can, and it's not just empty hype, it will be used for porn. "Empty hype" won't touch that stuff, but real-world usecases will.

Unrelated to the main topic, but this is exactly why I think cryptocurrencies will only be used for illegal activities, or things you may want to hide, and nothing else. Because that's where it has found its usecase in porn.

This is completely outside the competency of the current porn industry.

You gave an example of a still image, but it's going to end up with an AI generating a full video according to a detailed text prompt. The porn industry is going to be utterly destroyed.

Transfer learning is a thing.

But I have not tried making generative models with out-of-distribution data before. Distributions other than main training data.

There are several indie attempts that I am aware of. Mentioning them to the reply of this comment. (In case the comment gets deleted)

The first layers should be general. But the later layers should not behave well to porn images. As they are more specialist layers learning distribution specific visual patters.

Transfer learning is posssible.

I don't see how this gets us (much) closer to general AI. Where is the reasoning?

"The future is already here — It’s just not very evenly distributed"

How can we prepare for this?

This will result in mass social unrest.

- The current examples aren’t convincing pictures of “a shiba inu playing a guitar”.

- If you made that picture with actors or in MS Paint, politics boomers on Facebook wouldn’t care either way. They’d just start claiming it’s real if they like the message.