I'm admittedly a skeptic on all this so take what I am about to say with a grain of salt: You should trust that voice. We're in a hype cycle. It was VR before and crypto before that. Big tech is trying _very_ hard to convince you that you need this. They need you to need this tech because they are lighting billions on fire right now trying to make it smart enough to do anything useful. Short of them coming up with a truly miraculous breakthrough in the next 12 to 24 months (very unlikely but theres always a chance) investors are gonna get fed up and turn off the money fountain.
It's always a good idea to learn and grow your skillset. I am just not sure this is an investment that will pay off.
I will second this. Even if you think localghost is wrong about AI, it is important to always trust that voice of skepticism (to a limit).
But I will say that we are in a hype cycle and as a researcher I'm specifically worried about this. I get that we have to bootstrap because you can't say "we want to spend money on research" (why?), but if you make a bubble the goal is to fill that bubble before it pops. The more hype you make, the more money you get, but the quicker that bubble pops. My concern here is that too much hype makes it difficult to distinguish charlatans form honest people. Charlatans will jump from cool topic to the next (don't trust someone who was a VR founder, then a crypto founder, and now a ML founder. Trust people who have experience and can stick with a topic for longer than a hype cycle).
The big danger, is if charlatans dominate the space, the hype disappears, and then there is no money for everyone. So if you do believe in the possibility of AGI and that AI/ML can make the world better (I truly do), make sure that we don't over hype. There's already growing discontent for products pushed too early with too big promises. If you really believe (like I do), you have to get rid of the bad apples before they spoil the whole barrel.
The other day I had an idea for a Chrome plugin. I'm a senior dev, but I've never made a Chrome plugin. I asked ChatGPT 4o if my idea was possible (it was) and then I asked it to create an MVP of the plugin. In 10 seconds I had a full skeleton of my plugin. I then had it iterate and incrementally add capability until it was fully developed.
I had to do some stylesheet tweaking and it asked for a permission that we didn't need, but otherwise it completely nailed it. Easily provided 95% of the work for my extension.
I was able to do in 60 minutes what would have probably taken several days of reading specs and deciphering APIs.
Is my Chrome plugin derivative? Yes. Is most of what we all do every single day derivative? Also yes.
How are people still skeptical of the value that LLMs are already delivering?
I'm part of a team building developer tools for real-time AI use cases (voice and video). I feel like I have three overlapping perspectives and goals re this new stuff:
1. To figure out what we should build I need to have a good understanding of what's possible and useful right now.
2. I talk to our customers a lot. Helping them understand what's possible and useful today (and what that might look like six months or a year from now) is part of my job.
3. I think this is a step-function change in what computers are good at, and that's really exciting and intellectually interesting.
My AI information diet right now is a few podcasts, twitter, and email newsletters. A few links:
- Latent space podcast and newsletter: https://www.latent.space/podcast
- Ben's Bites newsletter: https://news.bensbites.com/
- Ethan Mollick newsletter: https://www.oneusefulthing.org/
- Zvi Mowshowitz newsletter: https://thezvi.substack.com/
- Rohan Paul twitter: https://x.com/rohanpaul_aialways looking for ideas on how to serve this audience better. feel like there could be more I should be doing.
I've been trying to find the best next step and what seems fruitful from my vantage point are:
1. Cohere's LLM University - Seems to go more in depth into terms like embeddings that are still pretty unclear to me. 2. promptingguide.ai - For similar reasons, that it covers terms and concepts I see a lot but don't know much about. 3. Reading survey-level papers.
I'm including this info just in case it's useful to you, as I've really appreciated all the content you've put together.
One specific thing you or someone else could do that is simple yet high value is to create a list of "the first 20 LLM papers you should read". I've looked for this to build out more base knowledge, but have yet to find it. Suspect it would be helpful to others as well.
Try writing single page web app or command line python app using the Claude 3.5 chat. Interact with it like you might in a pair programming session where you don’t have the keyboard. When you’ve got something interesting, have it rewrite it in another language. Complain about the bugs. Ask it what new features might are it better. Ask it to write tests. Ask it to write bash scripts to manage running it. Ask it how to deploy and monitor it. Run llama 3.1 on your laptop with ollama. Run phi3-mini on your phone.
The problem is that everyone says they aren’t going to get better, but no one has any data to back that up. If you listen carefully it's almost always based on a lack of imagination. Data is what matters, and we have been inventing new benchmarking problems because they're too good at the old ones. Ignore the hype, both for and against: none of that matters. Spend some time using them and decide for yourself. This time is different.
The point isn't that LLMs are useless, or that they aren't interesting technology in the abstract. The point is that aside from the very real entertainment value of being able to conjure artwork apparently out of thin air, when it comes to solving practical problems in the tech space, it's not clear that they are achieving significantly more - faster or cheaper - than existing tools and methods already did.
You're right that it's probably too early to have data to prove their utility either way, but given how much time, money and energy many companies have already sunk into this - precisely without any evidence to prove it's worthwhile - it does come across rather more like a hype cycle at the moment.
Three years ago an LLM would conversationally describe what the code would look like.
Two years ago it might crib common examples with minor typos.
Last year it could do something that isn't on StackOverflow at the level of an intern.
Earlier this year it could do something that isn't on StackOverflow at the level of a junior engineer.
Last week I had a conversation with Claude 3.5 that went something like this:
Write an interactive command-line battleship game
Write a mouse interactive TUI for it
Add a cli flag to connect to `ollama` and ask it to make guesses
There's a bug: write the AI conversation to a file so I can show you
Try some other models: make options for calling OpenAI and Anthropic
GPT and Anthropic are throwing this error (it needed to switch APIs)
The models aren't doing as well as they can: engage them more conversationally
What will LLMs do next year?
You can use them on top of those frameworks. The point is, you + LLM is generally a way faster you no matter what tech you're using.
It was okay, but kind of annoying. I understand js well enough to just debug the code myself, but I wanted it to spit out some boilerplate that worked. I can't remember if this was chatgpt omni, I was using or if it was still 3.5. It's been a short while.
Anyways, it is cool tech, but I don't feel like it offers the same predictive abilities as class ML involving fits, validation, model selection etc for very specific feature sets.
The other thing I’ve noticed is something you alluded to: the LLM being “confidently incorrect”. It speaks so authoritatively about things and when I call it out it agrees and corrects.
The more I use these things (I try to ask against multiple LLMs) the more I am wary of the output. And it seems that companies over the past user rushed to jam chatbots into any orifice of their application where they could. I’m curious to see if the incorrectness of them will start to have a real impact.
I actually find things to be the opposite. My skepticism comes from understanding that what LLMs do is token prediction. If the output that I want can be solved by the most likely next token, then sure, that’s a good use case. I’m perfectly capable of imagining those cases. People who are all in on AI seem to not get this and go wild.
There’s a difference between imagination and magical thinking.
Don't mistake the "what" for the "how". What we ask LLMs to do is predict tokens. How they're any good at doing that is a more difficult question to answer, and how they are getting better at it, even with the same training data and model size, is even less clear. We don't program them, we have them train themselves. And there are a huge number of hidden variables that could be encoding things in weird ways.
These aren't n-gram models, and you're not going to make good predictions treating them as such.
Regarding your projects, either just brute force into an existing one, or start a new project. For the former, the purpose isn't to make the product better (exactly) but for you to learn. For the later, OpenAI and Anthropic APIs are good enough to mess around and build a lot of different things. Don't let analysis paralysis stop you, start messing around and finding out.
ML moves fast, but not as fast as you probably think. There's a difference between innovations in architectures and demonstrations of them in domains (both are useful, both are necessary research, but they are different).
Instead, keep up with what tools are relevant to you. If things are moving fast and aren't sticking, then in a way they aren't moving fast, are they? You're just chasing hype and you'll never keep up.
On the production side, I also see a common mistake of relying on benchmarks too heavily. I understand why this happens, but the truth is more nuanced than this. Just because something works well on a benchmark does not mean it will work well (or better than others) on your application. ResNet is still commonly used and still a great option for many applications. Not everything needs a 1B+ transformer. Consider your constraints: performance, compute, resource costs, inference time, and all that jazz. Right now if you have familiarity (no need for expertise) in FFNs (feed forward/linear), CNNs, ResNets, and Transformers, you're going to be fine. Though I'd encourage you to learn further about training procedures like GANs (commonly mistaken as an architecture), unsupervised pretraining (DINO), and tuning. It may be helpful to learn a high level of diffusion and LLMs, but it depends on your use cases. (And learn whatever you're interested in and you find passion in! Don't let need stop you, but if you don't find interest in this stuff, don't worry either. You won't be left behind)
If you aren't just integrating tools and need to tune models, then do spend time learning this and focusing on generalization. The major lessons learned here have not drastically changed for decades and it is likely to be that way. We do continue to learn and get better, but this doesn't happen in leaps and bounds. So it is okay if you periodically revisit instead of trying to keep up in real time. Because in real time, gamechangers are infrequent (of course everyone wants to advertise being a gamechanger, but we're not chasing every new programing language right?). Let the test of time reduce the noise for you.
> I most often can’t see any use case for AI/ML in our products
Look to solve problems and then right tool for the problem, don't look for problems to justify a tool (fine for educational purposes).
Examples:
- Build a useful bash script using ChatGPT prompts and blog about it
- Build a text summariser component for your personal blog using Xenova / Transformers.js
- Build an email reply bot generator that uses ChatGPT prompt with sentiment analysis (doesn't have to actually send email, it could just do an API call to ChatGPT and print the message to the screen).
Just a few small examples and maybe a course or two (e.g. Prompt Engineering for Developers) should look great.
However I question how many companies really care about it right now. Most interviews I've done lately didn't bring it up even once.
But that said, maybe in a few months or year or so it will become more essential for most engineers.
So I don't want you to think my question was being sarcastic... I'm genuinely curious if you think this sort of thing would be a useful or interesting thing to blog about or only in the cases of a resume building thing?
A while back I wrote a prompt to build a script that runs git-reflog to get a the list of distinct authors. After a few small tweaks I got it roughly working. This took about 1 hour. Writing it myself would have definitely taken multiple hours, especially having to learn the details of git-reflog.
But that said I think it's mainly resume-building. ChatGPT isn't going to overall transform our productivity.
Run the following models:
- Speech-to-text - Text-to-text - Text-to-speech - Text-to-image - Image-to-text - Text-to-video - Video-to-text
Start by integrating third-party APIs, and later switch to open-source models.
Implement everything using your preferred backend language. After that, connect it to a frontend framework of your choice to create interactive interfaces.
You want use your own data? Put it in a database and connect it to your backend, and run these models on your database.
Once you’ve done this, you’ll have completed your full stack development training.
I am in a similar position to you… I have a job they the application of AI to that job isn’t readily apparent. My posture during all this is to use AI products as they become available that are appropriate and help me, but ultimately I’m waiting for the market to mature so I can see if and how I should move forward once the bubble pops and directions are more clear. I have little interest in running on the hamster wheel that is bleeding edge AI development, especially when I already have a job that doesn’t need it.
But in my experience the problem this in turn has a problem that you do not see the real problems you could solve with a piece of technology if you don't understand the technology.
So, sometimes, it makes sense to just start doing something with it. You will soon see potential uses, apply it, learn more, and overcome this hurdle.
Just do it without expecting any returns besides learning something new.
Hacker News is a good source for news.
As far as learning, you have to build something.
I suggest you just start with example code from the OpenAI or Anthropic website for using the chat completion API. They have Node.js code.
r/locallama on reddit is interesting.
On Youtube, see Matt Wolfe, Matthew Berman, David Shapiro. Not really developer-focused but will mention developments.
You can search for terms like 'AI Engineer' or "agentic" or "LangChain" on youtube also.
To get motivated, maybe play around with the replicate.com API. It has cut and paste examples and many interesting models.
More ideas: search for "Crew AI" on X/Twitter.
Would your product benefit from recommender systems, natural language input/output, image detection, summarization, pattern matching and/or analyzing large datasets?
If so, then maybe ML can help you out.
I'm of the opinion that if you need ML, you'll eventually realize it because the solutions to your problem you find will be served by applications of ML.
That's to say, while doing research, due diligence, etc, you will inevitably stumble upon approaches that use ML successfully or unsuccessfully.
It's definitely worth trying them out as a user just to see what they're capable (and incapable of). There are also some pretty interesting use cases for them for tasks that would be ridiculously complicated to develop from scratch and "it just works" (ignoring prompt poisoning). Think parsing and summarizing. If you're an app developer, look into edge models and what they can do.
Otherwise dip your toes in other model types - image classification and object recognition are also still getting better. Mobile image processing is driven by ML models at this point. This is my research domain and ResNet and UNet are still ubiquitous architectures.
If you want to be sceptical, ignore AI and read ML instead, and understand these algorithms are just another tool you can reach for. They're not "intelligent".
I would suggest following / reading people who talk about using Claude 3.5 sonnet.
Lots of people developing whole apps using 3.5 sonnet and sometimes cursor or another editor integration. The models are getting quite good now at writing code once you learn how to use them right and don't use the incorrect LLMs (a problem I often see in places other than twitter unfortunately.) They seem to get better almost weekly now too. Just yesterday Anthropic released an update where you can now store your entire codebase to call as part of the prompt at 90% token discount. Should make an already very good model much better.
Gumroad's CEO has also made some good YouTube content describing a lot of these techniques, but they're livestreams so there is a lot of dead air.
This said, I have started some experiments and like in all hyped technologies, there is some useful substance there as well.
My recommendation is to start with some very simple and real, repetitive need, and do it with the assistants API.
I started by turning a semi-structured word document of 150 entries into a structured database table. I would have probably done it more quickly by hand, but I would not have learned anything that way.
I think the sweet spot for generative AI right now is not in creative jobs (creating code, creating business communication , content creation etc.) but in mundane, repetivive things, where using generative AI seems like overkill at first.
I'd suggestion learning about pgvector and text embedding models. It seems overwhelming at first but in reality the basic concepts are pretty easy to grok.
pgvector is a Postgres extension, so you get to work with a good traditional database plus vector database capabilities.
Text embeddings are easy to work with. Lots of models if you want to do it locally or adhoc, or just use OpenAI or GCP's api if you don't want to worry about it.
This combo is also compatible with multiple vendors, so it's a good onboarding experience to scale in to.
Offtopic, but today I encountered my first AI-might-be-running-the-business moment. I had a helpdesk ticket open with IT for an issue with my laptop. It got assigned to a real person. After a few days of back-and-forth, the issue was resolved. I updated the ticket to the effect of, "Yup, I guess we can close this ticket and I will open a new one if it crops up again. Thank you for your patience and working with me on this." A few seconds later, I get an email saying that an AI agent decided to close my ticket based on the wording of my update.
Which, you know, is fine I guess. The business wants to close tickets because We Have Metrics, Dammit. But if the roles were reversed and I was the help desk agent, seeing the note of gratitude and clicking that Resolved button would very likely be the only little endorphin hit that kept me plugging away on tickets. Letting AI do ONLY the easy and fun parts of my job would just be straight-up demoralizing to me.
there is enough space for creating user experiences built on top of existing work instead of learning how the sausage is made. but i pray to you to not just stick to text/LLM
For the former side, Copilot-type implementations are pretty intuitively useful when used this way. I find it most useful as an autocomplete, but the chatbot functionality can also be a nice, slightly-better alternative to "talk to a duck when you're stuck." That said, I'll focus on the latter side (using AI/ML in your actual work) from here.
Generalized AI/ML/LLMs are really just a "black box API" like any of the others in our toolbelt already, be they Postgres, Redis, SSE, CUDA, Rails, hell, even things like the filesystem and C atop assembly. We don't need to know all the inner workings of these things, just enough to see how to use the abstraction. You probably take when to use a lot of these things for granted at this point, but the reason we use any of these things is that they're good for the specific problem at hand. And LLMs are no different!
What's important to recognize is the types of problems that LLMs are good for, and where to integrate them into your apps. And, well, a pretty obvious class of this is parsing plain text into structured data to be used in your app. This is pretty easy to prompt an LLM to do. OpenAI and WebLLM provide a pretty straightforward common set of APIs in their NPM libraries (and other language bindings are pretty similar). It's far from a "standard," but it's definitely worthwhile to familiarize yourself with how both of these work.
For an example, I've made use of both OpenAI and WebLLM in an "Event AI" offshoot to my social media app [1], parsing social media events from plaintext (like email list content, etc.); feel free to test it and view the (AGPL) source for reference as to how I'm using both those APIs to do this.
For projects where you actually have money to spend on your LLM boxes, you'll probably do this work on the BE rather than the FE as demoed there, but the concepts should transfer pretty straightforwardly.
If you're interested in really understanding the inner workings of LLMs, I don't want to discourage you from that! But it does seem like really getting into that will ultimately mean a career change from full-stack software engineering into data science, just because both have such a broad base of underlying skills we need to have. I'm happy to be wrong about this, though!
[1] Source: https://github.com/JonLatane/jonline/blob/main/frontends/tam... | Implementation: https://jonline.io/event_ai
Yes, absolutely. The most effective we I know to develop that sort of intuition (not just in AI/ML, but most subjects) is to try _and fail_ many times. You need to learn the boundaries of what works, what doesn't, and why. Pick a framework (or, when learning, you'd ideally start with one and develop the rest of your intuition by building those parts yourself), pick a project, and try to make it work. Focus on getting the ML bits solid rather than completing products if you want to get that experience faster (unless you also have no "product" experience and might benefit from seeing a few things through end-to-end).
> stay relevant in the long run
Outside of the mild uncertainty in AI replacing/changing the act of programming itself (and, for that, I haven't seen a lot of great options other than learning how to leverage those tools for yourself (keep in mind, most tasks will be slower if you do so, so you'll have a learning curve before you're as productive as before again; you can't replace everything with current-gen AI), and we might be screwed anyway), I wouldn't worry about that in the slightest unless you explicitly want to go into AI/ML for some reason. Even in AI-heavy companies, only something like 10% of developers tangentially touch AI stuff (outside of smallish startups where small employee counts admit more variance). Those other 90% of jobs are the same as ever.
> keep up my learning in these areas
In addition to the general concept of trying things and failing, which is extremely important (also a good way to learn math, programming, and linguistics), I'd advise against actively pursuing the latest trends until you have a good enough mentor or good enough intuition to have a feel for which ones are important. There are too many things happening, there's a lot of money on the line, and there are a lot of people selling rusty pickaxes for this gold rush (many intentionally, many because they don't know any better). It'll take way too much time, and you'll not have a good enough signal-to-noise ratio for it to be worth it.
As one concrete recommendation, start following Yannic Kilcher on YouTube. He covers most of the more important latest models, papers, and ideas. Most of his opinions in the space are decent. I don't think he produces more than an hour per day or so of content (and with relatively slow speaking rates (the thing the normal YT audience wants), so you might get away with 2x frame rate if you want to go a bit faster). Or find any good list of "foundational" papers to internalize (something like 5-20). Posting those is fairly common on HN; find somebody who looks like they've been studying the space for awhile. Avoid advice from big-name AI celebrities. Find a mentor. The details don't matter too much, but as much as possible you'd like to find somebody moderately trustworthy to borrow their expert knowledge to separate the wheat from the chaff, and you'll get better results if their incentive structure is to produce good information rather than a lot of information.
Once you have some sort of background in what's possible, how it works, performance characteristics, ..., it's pretty easy to look at a new idea, new service, new business, ..., and tell if it's definitely viable, maybe viable, or full of crap. Your choice of libraries, frameworks, network topologies, ..., then becomes fairly easy.
>> other people saying to build something simple with LLMs and brag about it
Maybe. Playing with a thing is a great way to build intuition. That's not too dissimilar from what I recommended above. When it comes to what you're telling the world about yourself though, you want to make sure to build the right impression. If you have some evidence that you can lightly productize LLMs, that's in-demand right this second. If you publish the code to do so, that also serves as an artifact proving that you can code with some degree of competency. If you heavily advertise LLMs on your resume, that's also a signal that you don't have "real" ML experience. It'll, ideally, be weighed against the other signals, but you're painting a picture of yourself, and you want that picture to show the things you want shown.
> can't see any use case for AI/ML
As a rule of thumb (not universal, but assuming you don't build up a lot of intuition first), AI/ML is a great solution when:
(1) You're doing a lot of _something_ with complicated rules
(2) You have a lot of data pertaining to that _something_
(3) There exists some reason why you're tolerant of errors
I won't expand that into all the possible things that might mean, but I'll highlight a couple to hopefully help start building a bit of intuition right away:
(a) Modern ML stuff is often written in dynamic languages and uses big models. That gives people weird impressions of what it's capable of. At $WORK we do millions of inferences per second. At home, I used ML inside a mouse driver to solve something libinput struggled with and locked up handling. If you have lot of data (mouse drivers generate bajillions of events), and there's some reasonable failure strategy (the mouse driver problem is just filtering out phantom events; if you reject a few real events per millisecond then your mouse just moves 0.1% slower or something, which you can adjust in your settings if you care), you can absolutely replace hysterisis and all that nonsense with a basic ML model perfectly representing your system. I've done tons of things beyond that, and the space of opportunities dwarfs anything I've written. Low-latency ML is impactful.
(b) Even complicated, error-prone computer-vision tasks can have some mechanism by which they're tolerant of errors. Suppose you're trying to trap an entire family of wild hogs at once (otherwise they'll tend to go into hiding, produce a litter of problems, and never enter your trap again since they lost half their family in the process). You'd like a cheap way to monitor the trap over a period of time and determine which hogs are part of the family. Suppose you don't close the trap when you should have. What happens? You try again another day; no harm, no foul. Suppose you did close it when you shouldn't have? You're no worse off than without the automation, and if it's even 50-80% accurate (in practice you can do much, much better) then it saves you countless man-hours getting rid of the hogs, potentially taking a couple tries.
(c) Look at something like plant identification apps. They're usually right, they crowd-source photos to go alongside predictions, they highlight poisonous lookalikes, the prediction gives lists and confidences for each prediction, and the result is something easy for a person to go investigate via more reliable sources (genus, species, descriptions of physical characteristics, ...). I'm sure there exists _somebody_ who will ignore all the warnings, never look something up, and eat poison hemlock thinking it's a particularly un-tasty carrot, but that person probably would have been screwed by a plant identification book or a particularly helpful friend showing them what wild carrots look like, and IMO the app is much easier to use and more reliable for everyone else in the world, given that they have mechanisms in place to handle the inevitable failures.