I'm hopeful for tinygrad as a piece of software, but I'm skeptical about the future of the tinybox if they keep waffling on the hardware so much.
[1] https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
Both him and Musk are quite intelligent in their respective areas, there shouldn't be any doubt about that. But there is a big issue with intelligent people where they often end up on completely wrong tracks. There are some studies that show its actually easier to convince smarter people of wrong information than ordinary people.
Generally, the way someone gets very smart in a certain area is through repeated exposure to material. The only way to have that sort of dedication is that you have intrinsic desire to do certain things because it fulfills some purpose in your life, which basically means that you have an ideology attached to that desire. That ideology can be any number of things, from wanting to help humanity, to a more selfish one of just viewing yourself as better than other people because you can accomplish stuff they cant.
The thing is, for a very smart person who has a personality like that, you can take a narrative that fits within that ideology, and they are almost guaranteed to believe it, whether its actually true or not. Because them denying it would mean that their ideology isn't correct, which means that their entire life has been wasted on wrong things - and no person would ever self destruct like that.
Its for this reason Musk believes in all the right wing conspiracy theories despite him having multiple successful startups. To him, democrats are bad (because of the exclusion of Tesla from environmental fund under Biden, which admittedly was pretty shitty), and then everything anti democrat must be correct - you can trace the start of his descent into madness right up to that event.
Same thing with Hotz. Super smart dude, but he managed to pick a fight with Sony and basically lost, which would leave a bad taste in anyone's mouth, then he read Unqualified Reservations, which perfectly explained the world in such a way where he is the good guy, and the rest follows. As such, he things literature like Atlas Shrugged is actually very good, aligns heavily with Musk because Musk is anti leftism, e.t.c. At Comma he was heavily against WFH because Musk was against WFH (for very, very stupid reasons), prided himself on being extremely selective with hiring only people that are both smart and are driven, but now with Tiny Grad, he has come to sort realize that you sort of need to hire remote people if you want to have talent, BUT instead of actually hiring people, he only has a few employees and the rest of the work is on a bounty system (being claimed as the future of employment).
One could make an argument that when looking at successful tech companies that "won" at a sector, like Apple with consumer electronics, Amazon with online shopping, none of them did it like Comma or Tiny Grad. I personally want nothing more for Hotz companies to succeed, which would truly be disruption at its core, but I don't think they will. It seems that you have to essentially engage with all people as humans and appeal to them, but when you view humanity from a viewpoint of "From each according to his ability, to each according to his needs", and tweet about it, its pretty clear that you dislike a large portion of humanity. And of course, engaging with people like that means that you have to appeal to the "professional managerial class" which goes strictly against his ideology.
For Comma Ai, that would look like playing nice with investors, getting more money, hiring a wide range of talent local and remote, which would put them on a stage where car companies take them seriously and want to work with them, at which point instead of dedicating resources to making hardware they can be actually making the software a shitload better, which is core to their mission of solving AI. And yes, there would have to be a bunch of bullshit that they would have to sift through, but thats just unavoidable fact of life.
You can view Tinycorp in the same light.
The way things go, I think the AMD consumer card experience will only get better once AMD manage to gimp consumer cards’ ML throughput or RAM.
I can certainly appreciate frustration with the AMD stack, but be blunt, I was not impressed with Hotz's YouTube rant from before.[1] It didn't give the impression of a stable framework, and this doesn't either.
Also (at least from the end user llm inference side of things) ROCm is not nearly as unusable as it used to be. We would certainly be renting MI300s over A100s (or even H100s) if we could get any, and we use a number of different inference backends.
There are some boutique hosts like Hot Aisle serving MI300s (who I really should reach out to), but for the immediate future our little startup is stuck with the big cloud providers. No MI300s for us mere mortals, not even to rent.
https://pytorch.org/ has a rocm support . This doesn’t make the outlook on this company very good …
The job of firmware is so close to the hw that it's nearly impossible to decode. You need to decode a custom CPU instructions for a IP block the microcode is running on (it's custom, no ARM/RISC/MIPS..).
After that you have to decode what firmware actually does. It writes something to this.... What does it mean? It's completely opaque number written to a opaque memory... cache control? Delay?
And you do that why? So you can ship tinybox (i.e. cheap consumer GPU). Let's says you succeed. Do the same thing Next gen will be similar challenge, except firmware will be better locked, because AMD will want consumer HW to be segregated from data center GPU, the same way NVidia does.
The task itself is basically impossible and waste of time. There is the reason why NVidia driver driver for Linux was used basically only to install official driver.
and an bunch more https://hn.algolia.com/?q=tenstorrent
I know some people don't like Nvidia but like their competition had their opportunity and what needed to be done spelt out to them and did nothing.
Even the whole OpenCL versus CUDA, they had years to ship something that was great tooling alternative, instead they did everything but that.
It's technically way more advanced. Not as outrageously priced as an H100 either.
$12k for 6 4090 for 144GB GDDR vs $20k H100 PCIe 80GB HBM2 (price likely dropping later this year when B100 is released).
And H100 has a lot of features like async and loading directly to tensor cores not present in consumer cards.
I want to root for the little guy, but it seems the AI hardware landscape will be Nvidia for the next few years. And us GPU poors accessing it via cloud (shudders).
tinygrad isn't in the realm of production ready though, AFAIK.
