GPU advancements in M3 and A17 Pro [video] (opens in new tab)

What Apple calls a GPU core seems to be roughly the same as what Nvidia calls a “stream multiprocessor”.

For example a 1080 GTX GPU has 20 stream multiprocessors (SM), each containing 128 cores, each of which supports 16 threads.

Meanwhile Apple describes the M1 GPU as having 8 cores, where “each core is split into 16 Execution Units, which each contain eight Arithmetic Logic Units (ALUs). In total, the M1 GPU contains up to 128 Execution units or 1024 ALUs, which Apple says can execute up to 24,576 threads simultaneously and which have a maximum floating point (FP32) performance of 2.6 TFLOPs.”

So one option to get a single number for a rough comparison is to count threads. The 1080 GTX supports 40,960 threads while the M1 supports 24,576 threads.

There’s obviously a lot more to a GPU — for starters, varying clock speeds, ALUs can have different capabilities, memory bandwidth, etc. But at least counting threads gives a better idea of the processing bandwidth than talking about cores.

I think a better comparison is to look at floating point performance. For example, the 10 core M2 GPU does 3.6 TFLOPS (FP32) while an RTX 4060 does 15 TFLOPS and an RTX 4090 82.58

Each Apple core (heh) has 128 FPUs so 40 cores would be akin to 5120 CUDA "cores".

> So how does one translate to an equivalent in "CUDA cores" type terminology?

I don’t really think we can, even if we knew exactly what is in a M3 GPU core, which we don’t. Both architectures are very different, and different again from AMD GPUs. We have to count Tflops.

Nvidia "cheat" by counting approximately ALUs.

It is an overview of what is new about their hardware and technical advice (but not overly technical) on how to maximize the GPUs performance.

I watched the full video and thought it was excellent. I wish other CPU/GPU manufacturers made technical overview videos like this. I've never programmed graphics targeting metal before, but I feel much more inclined after watching this so I guess it was good advertising.

I think it's important to understand that this video is aimed at a broad audience of developers. Obviously some of them are GPU experts but a lot of people are quite literally app developers who are going to watch this video as an introduction to "hmm everyone keeps telling about this GPU thing, what can I do with it?" So the video has to provide them with some context they can use to explore more.

It's one of ̶W̶W̶D̶C̶ developer sessions. Some of them at WWDC are basic 101 introduction for rookies.

There are advanced (for me) sessions like:

https://developer.apple.com/videos/play/wwdc2023/10127/

https://developer.apple.com/videos/play/wwdc2023/10042/

Although it's true that they won't discuss at hardware level.

> On the other hand, I'm looking forward to watching the Asahi folks crack this stuff open.

It will still be years before it is practical for Linux developers to target these features. Eventually, the rate of change in GPU design will slow and Linux will catch up once and for all. But it's hard to not drool over the hardware that proprietary OSs get to use today.

What's not technical about it? It tells you what the new advancements in the M3/A17 GPUs are, what you should look out for, and how you can take advantage of them. It provides enough information so you can understand what technical tradeoffs you need to make when you target M3 / A17 GPUs. E.g. Register pressure is a real concern in large shaders, and this helps explain how the behavior would be different under a dynamic allocation scheme. It explains how the ray tracing acceleration works, and how it reorders the different intersection calls and how you should avoid intersection queries.

There are some hyperbole interjected about how incredible the performance is but that's only in between the useful data. (I did chuckle at the… enthusiasm of the speaker though)

This isn't a technical document for GPU designers. Apple doesn't really need or want you to understand exactly how the implementation works because that's basically trade secret for them. This is aimed at letting app / game developers know how they should optimize for the new GPUs, since previously Apple just made some ambiguous remarks about some of these new technology ("Dynamic Caching") without explaining what they meant.

But yes, I do like how the Asahi folks tend to end up documenting a lot of how these hardware works, but they also only have public information like this to start from so these are still useful info to have for them.

it literally has code and diagrams of how to organize SIMD instructions to fill out all the shader cores

The beginning does sound like marketing but eventually it gets into technical information.

I’ve also noticed this on lots of YouTube videos, where the creator clearly meant one thing, but the subtitles substitute a more common, similarly-sounding word with a different meaning.

I suspect they have the videos transcribed externally, and don’t check the transcription (or only do so in a cursory manner).

Why is it surprising, it's not like content ownership gives you any advantage in the typical transcription algorithms

The guy introduced himself by name, I was really confused for a while if it was just a human trained to sound like an AI, or an AI trained to sound like a human.

They even created a Twitter profile for this AI persona

https://nitter.net/jhaberstro

From the variety of intonations based on context, I doubt it’s speech synthesis.

It was wild seeing Linus Tech Tips demoing resident evil village on the iPhone 15 Pro.

This is unfortunately inevitable; CPUs are just so much easier to benchmark in a broadly useful way. And the extreme leakiness of geekbench is helpful (I suspect Apple sees this as a feature; most recent Apple chip iterations have leaked on geekbench)

They should just support directx. Devs will never support two graphics api’s. it costs too much especially to grab the marginal mac os share that has powerful enough gpu’s. Id bed in 4 years apple moves to directx.

Interestingly, that's a different component than the GPU on these chips, which is the typical architecture in SoCs.

In fact, even in discrete GPUs, the display scanout engine is generally a nearly completely independent block relative to the rest of the GPU.

This issue can be fixed by switching the display to RGB [1]. So I think it’s a software bug but it‘s really annoying since the fix sometimes resets and the bug only occurs when there is a lot of black on the screen.

[1] https://gist.github.com/GetVladimir/c89a26df1806001543bef4c8...

Judging by the output/GUI of their GPU profiler, "complex" there is more like "complex instructions", think f32 (floating point) ops that aren't additions and multiplications (and FMAs), but trigonometry, square roots, that sort of thing.

FFT plus some game stuff requires complex numbers to do partial rendering (e.g. do some now and then do more next frame - I've lost the link to the talk but IIRC EA did a talk on how they made a shader that emulates lights in the background that are out of focus (not Guassian but the actual cool effect as if it was a real camera))

quaternions? for rotations

I doubt anyone will be able to answer questions this fine grained, not now (if the implementation is architecturally exposed - leaks into the ISA - and Asahi Linux group figures some of it out), or possibly not ever (if it's architecturally transparent and thus entirely micro-architectural).

> Does this mean there, effectively, are no registers?

I can only point out just for context that if by any chance you're asking whether the registers are implemented as actual hardware design "registers" - individually routed and and individually accessible small strings of flip-flops or D-latches - then the history of the question is actually "it never was registers in the first place" - architectural (ISA) registers in GPUs are implemented by a chunk of addressable ported SRAM, with an address bus, data bus, and limited number of accesses at the same time and limited b/w [1].

[1] see the diagram at https://www.renesas.com/us/en/products/memory-logic/multi-po...

A big part of Apple's "winning formula" is taking their giant piles of money and negotiating exclusive contracts for whatever is scheduled to be the most advanced semiconductor node next year.

Anyone else literally cannot compete, they don't have billions in pocket change they don't know how to spend otherwise, so they'll have to wait until the exclusivity agreement expires.

I don't know about your facts, but

"Is it that hard to just copy the winning formula?"

yes it is, thanks to IP law. And back in the day Steve Jobs already wanted thermonuclear war on Samsung, because he felt their flagship at the time was too close to the IPhone.

Google is probably not delusional, so they know the Tensor has no chance of being "good" as it is. Why don't you compare the A17 to the 8g3 and 8g2?

here have some Qualcomm Kool-Aid to go with that sweet Apple juice :-)

https://www.youtube.com/watch?v=h_vh7_n_OPs

It's not that simple though though. I have never got through an entire day with an iPhone (XR, 12, 13 Pro). I'm just hitting 2 days easily with my Pixel 7a with the same crap on it. My daughter just took an iPhone 15 back because it won't get through the day.

I don't share your sentiment about the guy, but that's OK.

Apple has a style, and he talks with that style very well, preventing the listener from drifting off.

I'm German and I don't hear anything wrong about the voice. He's probably not a native speaker, give the guy some slack.

I had a similar reaction at first. But, for whatever reason, when playing the video at 1.5x it felt completely fine.

It is pretty funny how amateur a multi trillion dollar can be, its so distracting and sounds like an 80's instructional VHS.