undefined | Better HN

0 pointsrudedogg2y ago0 comments

I’m a mac user at home, and I don’t get their AI story/path now that they’re not supporting AMD/Nvidia GPUs since the Apple Silicon transition.

Maybe they’ll manage to get LLMs running well locally with the new low-bit developments? Not my area. But for training/learning it seems like Apple is DOA. They have the same problem as AMD, no one is doing research with their hardware or software.

Intentionally shipping low RAM/unified memory quantities seems short sighted too. Maybe with a 16GB baseline they could do something special with local LLMs.

0 comments

dylan6042y ago

I think you are looking at a very narrow use case and deciding that because they do not make a system you'd be happy with for your niche use that they are DOA. Someone selling just under 6.5 million units of anything seems like the opposite of dead to me. Are there vendors selling more? Of course, but there are also vendors selling less. Not every Mac user cares about AI and training or fine tuning a local LLM.

rudedoggOP2y ago

Very true, my needs are niche for sure. But I’m more thinking about the near future. AI/LLMs are going to have some general applications that users are going to want, and will become the norm, and I think it’s clear that will shake out soon. Apple is at risk of being left behind because the only people working on that stuff for Apple, work at Apple. Hobbyists and researchers are on Linux/Windows for the most part. Software development doesn’t have such a large platform difference, lots of developers use macOS. But ML is different and I think they should care.

FridgeSeal2y ago

> But ML is different and I think they should care.

It’s totally this time I promise, just like, one more ~~lane~~ model.

I’m sure they do care. I wouldn’t be surprised if they land significant support for on-app processing of models, they’ve already got the chip, dropping in local models is a sensible next step, and if close to zero effort for them.

> LLMs are going to have some general applications that users are going to want, and will become the norm

I have yet to see anyone, in my personal or professional circles, use any LLM:

- for more than a week

- for anything more than cutesy trivial things.

I’m sure there’s people around stapling models into their toaster, but this is so far from the normal.

1 more reply

j / k navigate · click thread line to collapse