Code Llama, a state-of-the-art large language model for coding (opens in new tab)

Simple, concise, more efficient:

  def primes_upto(limit: int):
    """Generate prime numbers < *limit*.""" 
    # Sieve of Eratosthene
    is_prime = [True] * limit 
    for n in range(2, limit):
        if is_prime[n]:
            yield n  # found prime number
            for c in range(n*n, limit, n):  # start with square, less values are marked already
                is_prime[c] = False # mark composites

  if __name__ == "__main__":                
    from itertools import islice
    
    print(*islice(primes_upto(100), 10)) # -> 2 3 5 7 11 13 17 19 23 29

For printing the first 10 prime numbers, there's a one line solution to this problem:

print("1, 2, 3, 5, 7, 11... and so on!

Funny watching HN be nerd sniped by a machine :-)

How did you get access to the model?

I have been waiting for weeks, and am still waiting, to get access to Llama2 (released a month+ ago), and access to this model goes through the same form, so I'm not very hopeful. Are you getting it from other methods?

Interesting here and in some of the other comments in this thread is that 1 is not a prime number ¹)!

Now granted, that's more or less by definition and I don't doubt there's communities and fields where it is considered one, but still shows some of the subtleties at play when using language models.

¹) https://www.google.com/search?q=is+1+a+prime+number

how much VRAM do you need to run quantised 7b model?

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

Looks like they aren't releasing a pretty interesting model too. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane.

Meta says later on that they aren't releasing it and give no explanation. I wonder why given how incredible it seems to be.

Did Meta add scalable rope to the official implementation?

> The Code Llama models provide stable generations with up to 100,000 tokens of context.

what is the trick to achieve 100k context? They can't just use 100k wide transformer layer, it is cost prohibitive, right?..

I'm not sure copilot is using codex anymore[0]. They've also been talking about a shift towards GPT-4 with "Copilot X" a few times now[1][2].

[0] https://github.blog/2023-07-28-smarter-more-efficient-coding...

[1] https://github.com/features/preview/copilot-x

[2] https://github.blog/2023-07-20-github-copilot-chat-beta-now-...

>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot

It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.

I think this is called "mixture of experts" and also there's a lot of speculation that it's how GPT-4 works, although probably with just a few large models rather than many small ones.

If you can find a large body of good, permissively licensed example code, you can finetune an LLM on it!

There was a similar attempt for Godot script trained a few months ago, and its reportedly pretty good:

https://github.com/minosvasilias/godot-dodo

I think more attempts havent been made because base llama is not that great at coding in general, relative to its other strengths, and stuff like Starcoder has flown under the radar.

Mark my words: you’ve caught a glimpse of the near future :). Google “Society of Mind” if you’re not yet familiar

Start with a CodeLlama for C, and start treating these systems as natural language compilers. C is low level enough and still readable for those rare moments

The good thing is that if it's only finetuned on 15k instructions, we should see a community made model like that very soon.

Ollama supports it already:

`ollama run codellama:7b-instruct`

https://ollama.ai/blog/run-code-llama-locally

More models uploaded as we speak:

https://ollama.ai/library/codellama

What kind of cpu/gpu power do you need for quantization or these new gguf formats ?

If I don't want to run this locally is it runnable somewhere on huggingface?

can it be quantised further so it can run locally on a normal laptop of a developer?

That seems daft.

You can, I suppose, contract your code so that it’s context free and uses less tokens, but that makes it more confusing for humans and language models.

Taken to the extreme, you can see obviously with one letter functions and variables like i, j, k the model will be able to infer literally nothing and, thus, produce arbitrary nonsense.

Clearly the solution is to do what we already do to manage complexity which is to decompose large tasks into smaller black box modules with an api where the (large number of tokens) implementation is hidden and not known or relevant to using it.

If you give an LLM a function signature and good description, maybe some usage examples, it doesn’t need the implementation to use it.

Terseness decreases the ability of LLMs to process code; it doesn’t solve context length, and even at best it doesn’t scale.

100k tokens is plenty.

You don’t need to do anything like that.

Your developer tool already maps out the entire code base in useful ways, such as knowing all the symbols available in the current context and the structure of classes. This information can be distilled for presentation to the LLM. For instance, if you’re wanting to generate a method implementation inside a C++ class, the LLM can be given a condensed version of the header files that the compiler would have access to on compiling that specific class. Removing white space and comments and boiling macros down saves a lot of tokens.

You can also probably skip including standard library headers since those will be well known to the LLM through its fine tuning.

Either way, consider that a typical preprocessed C++ file would push against the 100K limit even with some optimizations. You will definitely want to have some middleware doing additional refinement before presenting that file to the LLM.

I’ve found the utility of the coding LLMs gets a lot higher when you’ve got code comments and descriptive variable and function names - the LLM makes better inferences and suggestions. We’ve seen similar on data - properly tagged data and descriptive field names helps the LLM to produce much more useful responses. I’m secretly hoping the spread of these tools will finally lead my fellow developers to comment their code and stop using three character variable names.

I built a VS code extension a while back that I still use that wraps GPT-4 and writes code directly in my editor.

The method I used to choose which files to feed GPT-4 was embeddings-based. I got an embedding for each file and then an embedding from the instruction + some simple processing to pick the files more likely to be relevant. It isn't perfect but good enough most of the time in medium-sized codebases (not very large ones).

The one thing I started doing because of how I implemented this is make files shorter and move stuff into different files. Having a 1k+ LOC file is prohibitive because it eats up all the context window (although with 100k context window maybe less so). I think it's a good idea to keep files short anyways.

There's other smarter things that can be done (like embed and pass individual functions/classes instead of entire files) so I have no doubt someone will build something smarter soon. You'll likely not have to change your coding patterns at all to make use of AI.

This sounds like a job for middleware. Condensing split code into a single huge file, shortening comments, removing whitespace and such can be done by a preprocessor for the llm.

A good practice is to have a prompt file where you keep the information you want the model to have at its disposal. Then you put it in the start of your conversations with GPT-4. It's also good documentation for people.

You start a project by defining the task. Then as you iterate, you can add new information to the prompt. But it can be also partially automated - the model can have a view of the file structure, classes, routes, assets and latest errors.

I was really hoping that the one year update of Codex would be that - a LLM that can see deep into the project, not just code, but runtime execution, debugging, inspecting and monitoring. Something that can iterate like autoGPT. Unfortunately it didn't improve much and has weird conflicts with the native code completion in VSCode, you get freezes or doubled brackets.

I’m working on a project related to that: https://github.com/0a-io/Arch-GPT

I think hypergraph is an overlooked concept in programming language theory

Solutions exist that feed LLMS ctags, and seem to work well. The function signatures and symbols names for a code base are much smaller than the actual code.

> Not a bad context

A little understated, this is state of the art. GPT-4 only offers 32k.

Give Cody a try! (Cody.dev)

With Cody you can create embeddings for your entire repo, so Cody will have much greater context about your code base and the problems you're trying to solve.

Disclaimer: I just joined Sourcegraph a few weeks ago.

SourceGraph Cody is going in that direction, as is Copilot Chat. But it's still early days. I don't think there's anything robust here yet.

You can use Continue for all of this, as easy as highlighting code and making the request. We also support using Code Llama: https://continue.dev/docs/walkthroughs/codellama

Neither of those tasks require AI. IntelliJ IDEA will happily suggest both for you today, locally. It can find large chunks of duplicated code and automatically refactor them out to functions for you. And it has many inspections that suggest refactorings to make code clearer.

Copilot calls these "Code Brushes" https://githubnext.com/projects/code-brushes/

Last I heard they are in beta and don't work very well (even on the examples page: the "add types" brush is too strict, since `a` and `b` are checked for `null`, and the "fix simple bug" is a typo)

I've been using Cursor (https://www.cursor.so/) and it can do embeddings of the entire codebase, refactoring entire classes, etc. I had it rewrite a UI to add state to show one item at a time and have a selection list to the left and it executed it perfectly in MUI controls, first try.

https://docs.github.com/en/copilot/github-copilot-chat/using... can basically do that if you're in the beta.

There's an instruct model in there, you can definitely use it for this, that's one of the objectives.

An instruct model means that you can ask it to do what you want, including asking it to give you refactoring ideas from the code you will give it.

I let mine generate whatever it likes, then add a comment below such as "# Refactor the above to foo.." Works fairly well at times.

at refact.ai we have different functions for code refactoring, making it shorter, simplifying complexity, etc

I've used Ollama to run Llama 2 (all variants) on my 2020 Intel MacBook Pro - it's incredibly easy. You just install the app and run a couple of shell commands. I'm guessing soon-ish this model will be available too and then you'd be able to use it with the Continue VS Code extension.

Edited to add: Though somewhat slow, swap seems to have been a good enough replacement for not having the loads of RAM required. Ollama says "32 GB to run the 13B models", but I'm running the llama2:13b model on a 16 GB MBP.

34B should be able to run on 24GiB consumer graphics card, or 32GiB Mac (M1 / M2 chips) with quantization (5~6bit) (and 7B should be able to run on your smart toaster).

If you want to run them fast, a 12GB GPU (e.g 3060) for the 13B and a 24GB GPU for the 34B (e.g 3090). Otherwise llama.cpp CPU inference would work on most machines.

You can use Continue as a drop-in replacement for Copilot Chat with Code Llama. We've released a short tutorial here: https://continue.dev/docs/walkthroughs/codellama. It should save you a lot of time context-switching; you can just highlight code and ask questions or make edits, all with keyboard shortcuts

For in-editor like copilot you can try this locally - https://github.com/smallcloudai/refact

This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure.

For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/

http://cursor.sh integrates GPT-4 into vscode in a sensible way. Just swapping this in place of GPT-4 would likely work perfectly. Has anyone cloned the OpenAI HTTP API yet?

That’s just one perspective… Another perspective is that LLMs enable programmers to skip a lot of the routine and boring aspects of coding - looking up stuff, essentially - so they can focus on the fun parts that engage creativity.

This should be the only goal of mankind so we can smell the flowers instead of wasting our years in some cubicle. Some people will always want to work, but it shouldn't be the norm. What's the point really unless we're doing something we're passionate about? The economy?

Is automation not what every engineer strives for when possible? Especially software developers.

From my experience with github copilot and GPT4 - developers are NOT going anywhere anytime soon. You'll certainly be faster though.

The best interpretation of this is you mean eventually ML/AI will put programmers out of a job, and not Code LLama specifically.

However it is hard to tell how that might pan out. Can such an ML/AI do all the parts of the job effectively? A lot of non-coding skill bleed into the coder's job. For example talking to people who need an input to the task and finding out what they are really asking for, and beyond that, what the best solution is that solves the underlying problem of what they ask for, while meeting nonfunctional requirements such as performance, reliability, code complexity, and is a good fit for the business.

On the other hand eventually the end users of a lot of services might be bots. You are more likely to have a pricing.json than a pricing.html page, and bots discover the services they need from searches, negotiate deals, read contracts and sue each other etc.

Once the programming job (which is really a "technical problem solver" job) is replaced either it will just be same-but-different (like how most programmers use high level languages not C) or we have invented AGI that will take many other jobs.

In which case the "job" aspect of it is almost moot. Since we will be living in post-scarcity and you would need to figure out the "power" aspect and what it means to even be sentient/human.

Do you really want to spend you days writing REDUX accumulators?

I understand the fear of losing your job or becoming less relevant, but many of us love this work because we're passionate about technology, programming, science, and the whole world of possibilities that this makes... possible.

That's why we're so excited to see these extraordinary advances that I personally didn't think I'd see in my lifetime.

The fear is legitimate and I respect the opinions of those who oppose these advances because they have children to provide for and have worked a lifetime to get where they are. But at least in my case, the curiosity and excitement to see what will happen is far greater than my little personal garden. Damn, we are living what we used to read in the most entertaining sci-fi literature!

(And that's not to say that I don't see the risks in all of this... in fact, I think there will be consequences far more serious than just "losing a job," but I could be wrong)

When mechanized textile machinery was invented, the weavers that had jobs after their introduction were those that learned how to use them.

If we get to the point where these large language models can create complete applications and software solutions from design specs alone, then there's no reason to believe that this would be limited to merely replacing software devs.

It would likely impact a far larger swath of the engineering / design industry.

You can't get promoted unless you put yourself out of a job.

We're not looking at a product that's putting anyone out of a job though, we're looking at a product that frees up a lot of time, and time is great.

Well, since Brexit anyway.

Someone "grafted" llama 33B onto llama v2 13B to make "llama 22B"

https://huggingface.co/chargoddard/llama2-22b

Theoretically this is an even better size, as it would fit on a 20GB-24GB GPU with more relaxed quantization and much longer context.

Metrics are slightly below 13B, but the theory is that the higher parameter count is more amenable to finetuning. If you search for 22B on huggingface, you can see that frankenllama experiments are ongoing:

https://huggingface.co/models?sort=modified&search=22b

Looks like they left out another model though. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane.

Meta says later on that they aren't releasing it and give no explanation. I wonder why given how incredible it seems to be.

I can't imagine it being better than Llama1 33B, after all this code finetuning.

SDXL and DeepFloyd can spell. It's more or less just a matter of having a good enough text encoder.

I tried Ideogram yesterday and it felt too much like existing generators (base SD and Midjourney). DALLE2 actually has some interestingly different outputs, the problem is they never update it or fix the bad image quality.

Did ideogram release a checkpoint?

Editor plugins are fantastic about completing based on a pattern. That's the main thing you're missing out on imo - it's worth it to hit tab, but not to copy/paste and say "finish this line for me, it looks almost like the one above."

There's also the real-time aspect where you can see that it's wrong via the virtual text, type a few characters, then it gets what you're doing and you can tab complete the rest of the line.

It's faster to converse with when you don't have to actually have a conversation, if that makes sense? The feedback loop is much shorter and doesn't require natural language, or nearly as much context switching.

This runs locally on a MacBook.

Seriously, I was expecting to read the article and them be on a level on-par with GPT-4 or higher. For all this chat of how long Google/Facebook have been in the AI space longer than OpenAI, their products don't speak to that..

https://github.com/abetlen/llama-cpp-python has a web server mode that replicates openai's API iirc and the readme shows it has docker builds already.

hallucinations can be resuces by incorporating 'retrieval automated generation' , RAG, on the front end. likely function library defs could be automagically entered as prompt/memory inputs.

Probably the researchers at meta are only interested in research, and productionizing this would be up to other teams.

They show the benchmarks in the original post, a few pages down

This is 404ing now. (Not your fault, the email's link is similarly broken.)

Partner integrations will follow. For now we just have the weights available.

But don't worry, this community moves fast!

It’s less of a concern if you are in mid career. But someone should warn all these college kids that are going into comp sci. I don’t think this will be the kind of lucrative field they think it’s going to be over the course of a 40 year career.

The days of getting paid well for making crud are numbered (which most of us do, even in the most interesting problem spaces).

# need a front end boilerplate that hits a backend with the following end points. REST api for movies catalogue, and a corresponding ui. Oh, unit tests please. Go with a responsive design and also make a React Native version (matter of fact provision it to my iPhone). Decide between Heroku or AWS, set up deploy with git hooks.

# scrape IMDb for initial population of the db

# I think a Reddit like comment system would be good to add, so add it. No upvote/downvote though

# handle user login with google/fb/email

# also make an admin page to manage all this

I guess the technical product designer will be the new unicorn.

You can say this about every major invention. The loom destroyed jobs! The engine destroyed jobs! So on and so forth.

This view is critically flawed in two major ways:

1) AI is not anywhere near being able to replace the majority of what developers do on a product team. We are decades away from a PM at Facebook being able to type "make a twitter clone that uses instagram login and can scale to 1 billion" and have an AI just do it.

2) Programming and product work is not zero sum. The more we can do means the more product we can make. It means more products can be made overall. After the loom came out, we simply made more clothes than ever before and in the process created a ton of jobs. We are not at some peak software point where we've completely saturated all humanity's need for software or profitable software and thus tools that increase efficiency don't put us out of work.

And frankly, if we develop the kind of general AI that accept a query like "make a facebook competitor capable of scaling to 10 billion" and simply do it, inventing whatever languages, frameworks, hardware, processors, patterns, methodologies, global datacenters handling global politics and law, etc, etc necessary to accomplish such a task, then so be it. I welcome the overlords!

Improve productivity, cheapen goods and services. Nature of technological advancement

Any improvement in tooling benefits capital owners the most, since the productivity gains mostly end up in their pockets.

But the answer to that is to deal with concentration of capital, not to eschew better tools.

"If software engineering becomes a solved problem"

It will simply move to a higher level of abstraction.

Remind me, how many programmers today are writing in assembly?

I agree. This will ultimately become another way to extract more value straight into the pockets of the owners.

Unfortunately, I don't believe there's a way to stop (or even slow down) this train. We can't defeat it, so the only logical answer is to join it.

It's the classical issue with progress removing jobs. In today's world, since mostly everyone (aside from the capitalists themselves) relies on jobs to survive, barring a complete switch from capitalism (which will not happen in our lifetimes), we're fucked.

Next best thing we can do is to try and democratize it enough so that not only the rich have access to it.

I'm not worried because it solves a problem for the semi-competent.

We have three options, IMO:

1. As a species decide to never build another LLM, ever.

2. Change the path of society from the unequal, capitalist one it’s taken the last 2-300 years.

3. Give up

I know which I believe in :). Do you disagree?

not quite self-hosted but gradient.ai gives you access to llama2 via CLI

Have you tried Code Llama? How do you know it can't do it already?

In my applications, GPT-4 connected to a VM or SQL engine can and does debug code when given error messages. "Reliably" is very subjective. The main problem I have seen is that it can be stubborn about trying to use outdated APIs and it's not easy to give it a search result with the correct API. But with a good web search and up to date APIs, it can do it.

I'm interested to see general coding benchmarks for Code Llama versus GPT-4.

That sounds like an interesting finetuning dataset.

Imagine a database of "Here is the console error, here is the fix in the code"

Maybe one could scrape git issues with console output and tagged commits.

I'd be surprised if GPT-4 couldn't already do that with the caveat that piping in so much code might cost you billionaire money at scale.

What’s Europe using for machine learning?

try refact.ai they have plugin for JetBrains IDEs and support for local LLMs https://github.com/smallcloudai/refact/

Also curious about this. There must be a better way to scrape using LLM.

Maybe we'll merge that one hither to split the karma.

There are several different ways, but the easiest way in my (clearly biased) opinion is just got to ollama.ai, download it, and start playing around. It works out of the box w/ newer Macs, but there are versions for Linux and Windows in the works.

As if by magic... https://huggingface.co/TheBloke/CodeLlama-13B-fp16. Empty so still uploading right now, at a guess.

Every time a new model hits I'm waiting for his ggmls

12Gb of VRAM lets you run 13B models (4-bit quantized) with reasonable speed, and can be had for under $300 if you go for previous-generation NVidia hardware. Plenty of developers around with M1 and M2 Macs, as well.

... "seeing as running a heavy language model on local hardware is kinda unlikely for most developers"

for now it is :) but with quantization advances etc. it is not hard to see the trajectory.

As we all know, computers stay the same and rarely improve.

It would be interesting to understand if a ~30B Llama-2 model would be interesting and for what reasons.

Rumour has it that the 30b ran into safety issues and thus was not released

In the past, LLAMA access was granted nearly immediately. For HuggingFace downloads, it took a full day.

The bloke on hugging face usually has quantised versions minus the legal form

To a certain extent, I think it's just IBM disease. A company the size of Meta is expected to have an AI research department like Microsoft or Google, even if their core business (social media) derives relatively less benefit from the technology.

Pretend you're an uncreative PM on an AI team; what part of Facebook or VR could you feasibly improve by iterating on LLMs? Perhaps the content moderation system... but that would require wrangling with the company ethics comittee and someone else at the company probably already took ownership that idea. You've gotta do something compelling or else your ML engineers are going to run off somewhere else.

If I were to ask my ML engineers about what they wanted to work on, they're going to avoid areas where their model is outgunned (i.e.: chat) and instead prefer lower hanging fruit which generalizes well on a resume (i.e.: "Pioneered and published key innovations in LLM code-generation").

Of course, the alternative answer is that Meta wants to replace all of their jr. developers with GPUs, but I think their leadership is a little too preoccupied with VR to be actively pushing for such a transformative initiative in anything more than a very uninvested capacity (e.g.: "Sure I'll greenlight this. Even if it doesn't pay off I don't have any better ideas")

The ability to work less historically has always came as a byproduct of individuals earning more per hour through productivity increases.

The end goal of AI isn't to make your labour more productive, but to not need your labour at all.

As your labour becomes less useful if anything you'll find you need to work more. At some point you may be as useful to the labour market as someone with 60 IQ today. At this point most of the world will become entirely financially dependent on the wealth redistribution of the few who own the AI companies producing all the wealth – assuming they take pity on you or there's something governments can actually do to force them to pay 90%+ tax rates, of course.

A few folks and I have been working on an open-source tool that does some of this (and hopefully more soon!) https://github.com/jmorganca/ollama

There's a "PrivateGPT" example in there that is similar to your third point above: https://github.com/jmorganca/ollama/tree/main/examples/priva...

Would love to know your thoughts

I work for a Fortune 100 company with 80,000+ employees. All of us are explicitly forbidden from using any sort of AI/LLM tool without written permission from the head of legal AND the CEO. In other words, nobody is going to get permission.

The concerns are 2 fold - 1. We might inadvertently use someone else’s intellectual property. 2. Someone else might gain access to our intellectual property.

What you are describing would help alleviate the concern about issue 2, but I’m not sure if it would help alleviate the concerns with issue 1.

GPT4All satisfies these requirements, except for (AFAIK) running a web server

https://gpt4all.io/index.html

I have been using refact.ai on my laptop, it has been quite good.

https://github.com/smallcloudai/refact/blob/main/README.md

Agreed. After several rounds of setting up various python environments and tinkering with directory structures and debugging glitches and quantizing models just to end up playing around for a few minutes and getting bored, it would be nice to have the experience just be seamless. I wouldn't try to set up a workflow around seriously using what's out there to run on localhost now.

That said, non-subscription is essential, and that's probably going to be a heavy lift considering how quickly things are evolving.

Did you try Oobabooga (https://github.com/oobabooga/text-generation-webui) yet?

What I want is even simpler: just an API that you make requests to and receive answers back. Surprisingly hard to find, outside OpenAI that is.

We are doing this with https://Faraday.dev - would love to chat more - my email is on my profile!

You want a polished product and something that can run all sorts of different models. I don't know that both are possible at the same time.

we're going in this direction for code models with Refact https://github.com/smallcloudai/refact/ - right now you self-host code models, fine-tune them on local files, get the model running locally inside your IDE

I’ve used llama.cpp easily for some local things, but it does lack a good ui.

They are behind commercially, very behind.

They also don't have the same economic setup and DNA as MS/OpenAI. Large corporate customers don't pay for access to the FB cloud, nor are they likely to -- Ellison has spent years building out Oracle Cloud, and he's on the FB board, for example. And I bet you didn't think of using Oracle's Cloud for your last project.

So, your company DNA is free-to-all social based on ad monetization, with a large bet on metaverse / AR / experiential social compute being next. You aren't a trusted corporate partner for anything but gatekeeping your immense community through ad sales.

And, it's clear you a) have some of the most interesting private social data in the world, including photos and DMs and texts, and b) this AI thing is huge.

A play that doesn't f with your existing corporate structure too much is to build this stuff, give it away, keep publishing, build your AI team internally, and see where it takes you.

This isn't the only play, but I think it's reasonable. It's pretty clear large enterprises are going to need their own, internally built / owned, Foundation models to be competitive in a bunch of arenas in the next decade. In this case, if FB can get a little mindshare, keep the conversation going, and as a sidenote, be a disruptor by lowering Azure/OpenAI revs with open releases at-the-edge, that's probably a strategy win.

If I were in charge of AI strategy at FB, I'd probably double down more on generative AI, and I'd be working hard on realtime multimodal stuff -- their recent very large multimodal speech to text in multiple languages work is good. If a team could eyeball realtime-ish video chat with translations, that would be something the platform has a natural advantage in pushing out. Generative hits existing customers, and metaverse asset creation, which is going to experience radical changes in costs and productivity over the next few years, and impact Oculus 100% no matter what anybody wishes were true.

Clearly the research team at Meta knows the domain as well anybody, has access to a data trove as large as anybody and their distribution capability is as large scale as anyone's.

If their choice right now is not to try to overtly monetize these capabilities but instead commoditize and "democratize" what others are offering it suggests they think that a proprietary monetization route is not available to them. In other words they do not leave money on the table. They think that (at least right now) there is no money on the table that they can get to.

Rather than remaining quiet and isolated, the best alternative - their conjectured thinking goes - is to show up as they do, buying up good will with various stakeholders, maintaining mindshare internally and externally etc.

Assuming that the above reading is correct it still leaves various options as to why they may have come to that conclusion: For example reasoning about the future of this sector they might be thinking that there is no real technical moat and they simply accelerate that reality to gain some brownie points.

It may be also idiosyncratic reasons specific to their own business model (data privacy challenges and how any AI monetization will mesh with all that). The drawback of being the elephant in the room is that there is not much room to move.

The nature of their long game depends on which of the decision branches carries more weight. Maybe it is wait-and-see until others clear up the regulatory hurdles. Or keep the engines running until the real and irreducible added value of LLM algos and the like becomes clear.

Retention project to keep their top ML/AI staff engaged and not straying away?

Working towards NLU that can solve content moderation once and for all? Contrast with tiktok which is clearly using word filters that are easily worked around with phrases like "un-alived" or "corn".

They want to replace influencers and your friends with chatbots and keep you scrolling through an infinite feed of ads and AI generated content?

As a business strategy I would see it as preventing themselves from being hemmed in from the market leaders. By open sourcing and raising the bar for commodity AI, they get to crowd source improvement to their models and techniques to get ahead in their own uses by co-opting open source improvement. I would say to sage this is working amazingly well - the amount of interest around open source models from meta is immense. I also think, as do I, the majority of uses in the future will be from fine tuned RAG capable models embedded in devices, not pangalactic planet sized computers running septillion parameter models. Llamacpp is a perfect illustration of where that’s working.

We followed a similar model under more duress at Netscape. When you use Firefox that’s the fruit of that effort. It didn’t save Netscape, but Meta has a better and more diversified revenue base.

Zuckerberg talked about it on Lex Fridman podcast https://youtu.be/Ff4fRgnuFgQ?t=1113

On Lex Fridman, Mark said the strategy is to attract talent while keeping the playing field level (not a fan of big tech moating this up).

I watched a good talk from Yann LeCun who is Chief AI Scientist at Meta, and he explained that the thinking is that open source AI models will be the long-term winner, so it's best for them to work in that arena.

https://www.youtube.com/watch?v=vyqXLJsmsrk

If you watch the Connect talks, I'll be speaking about this..

Probably just talent acquisition. As Google and OpenAI start sharing and publishing less, they become less attractive to scientists. No scientist wants to fall into a black hole and not publish for 8 years.

Meta has a clear channel to leverage generative AI in profitable ways in their ads. At some point in the probably not so far future, everybody's going to have custom ads generated for them that are optimized to get that particular person to click/buy/etc. Those will convert well, and the better ads convert, the more businesses will be willing to pay Meta for a given ad.

This compares favorably with Google, which is as likely to cannibalize its search business with generative AI as to create new value for itself.

Thus, for all the gen AI stuff like this, for which Meta doesn't have an obvious path to commercialization, it makes sense to release it publicly. They get plenty of benefits from this - for one, engineers (and smart people generally) who are working on really complex problems like to be able to talk about the work they're doing. If you're picking between jobs at Meta and Google, the fact that Meta's going to release your stuff publicly might well be the deciding factor.

I would also argue that there's an economic incentive. Right now, being seen as an AI company is definitely a positive for your multiple. I think the movement of Meta's stock price over the last 12 months relative to their change in profit and revenue is certainly driven in part by the perception that they're a leader in AI.

It makes sense to me that Facebook is releasing these models similarly to the way that Google releases Android OS. Google's advertising model benefits from as many people being online as possible and their mobile operating system furthers that aim. Similarly, Facebook's advertising model benefits from having loads of content being generated to then be posted in their various products' feeds.

> Does anyone have a good explanation for Meta's strategy with AI?

Yes. I said it many times. Meta is already at the finish line in the AI race to zero. All the other cloud-based AI models cannot increase their prices given that a $0 free AI model is available to be self-hosted or used on-device for private / compliance reasons.

Cloud-based AI models cannot afford to compete with free or close to free. It costs Meta close to nothing to release a readily available $0 AI model which is good enough for most use-cases that ChatGPT has already done.

> The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?

Mostly benefits the PyTorch ecosystem which Meta has an active community around it.

vessenes and rvz kind of sum the idea I think they're going for to me.

AI has no moat, but many players are in denial about this still. Microsoft and other might have tight enough control they can use a product dumping strategy to get people dependent upon their implementation such they can start charging, but that isn't a delusion Meta can have.

That max revenue license they used with the models seemed fairly clever to me. It will seed the environment with players that base their product on Meta tech in return for them being born with a poison pill preventing their use by big players (other than Meta) buying them. This is a long term play that may not really work but it creates the potential for big opportunities. And even if it doesn't work out, denying easy wins for their powerful competitors might be worth the price on its own.

I posit it is similar to how Adobe lets students pirate Photoshop, because when they join the workforce that is what they know and need their employers to buy Adobe services, which for corporate is very expensive.

Meta by democratizing AI access is generating more capable developers which will make the Metaverse a reality, where FB leads. They have already realized they have a losing gambit with Google, Apple, Microsoft (also X?) having an antagonistic monopoly against Meta product advancement

the only beneficiary of this are the hardware vendors.. nvidia and amd. and startups which get these foundation models for free.

because language models are a complementary product, and the complement must be commoditized as a strategy.

I see AMD as a bigger beneficiary, since, very soon, amd will equal nvidia for inference and fine-tuning, but amd has a long way to go to equal in foundation model training.

Maybe Meta is waking up to the endgame of humanity and has decided to stop playing the old game? Who knows :)

> Microsoft

But they're a partner in Llama too. Why is Microsoft in this space too, how do they benefit?