Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs (opens in new tab)

it's possible, but idk why you would expect that. just to pick an arbitrary example since steve ran some recent tests, a 1080 ti is more or less equal to a 4060 in raster performance, but needs more than double the power and a much more die area to do it.

https://www.youtube.com/watch?v=ghT7G_9xyDU

we do see power requirements on the high end parts every generation, but that may be to maintain the desired SKU price points. there's clearly some major perf/watt improvements if you zoom out. idk how much is arch vs node, but they have plenty of room to dissipate more power over bigger dies if needed for the high end.

Nvidia is a very innovative company. They reinvent solutions to problems while others are trying to match their old solutions. As long as they can keep doing that, they will keep improving performance. They are not solely reliant on process node shrinks for performance uplifts like Intel was.

There is one major 4 letter difference - TSMC. Nvidia will get tech process improvements until TSMC can't deliver, and if that happens we have way bigger problems... because Apple will get mad they can't reinvent iPhone again... and will have to make it fun and relatable instead by making it cheaper and plastic again.

As long as TSMC keeps improving die size it will keep getting incremental improvements. These power/thermal improvements are not really that much up to nvidia.

The intel problem was that their foundries couldn't improve the die size while the other foundries kept improving theirs. But technically nvidia can switch foundry if another one proves better than TSMC even though that doesn't seem likely (at least without a major breakthrough not capitalized by ASML).

I mean it's like 1/6 of their revenue now and will probably keep sliding in importance over the datacenter. No real competition no matter how we would wish. AMD seems to have given up on the high end and Intel is focusing on the low end (for now, unless they cancel it in the next year or so).

They already predicted this hence DLSS and other AI magic.

Huh? Nvidia does three things well: - They support the software ecosystem - Cuda isn't a moat, but it's certainly an attractive target. - They closely follow fab leaders (and tend not to screw up much on logistics). - They do introduce moderate improvements in hardware design/features, not a lot of silly ones, and tending to buttress their effort to make Cuda a moat.

None of this is magic. None of it is even particularly hard. There's no reason for any of it to get stuck. (Intel's problem was letting the beancounters delay EUV - no reason to expect there to be a similar mis-step from Nvidia.)

Rasterizing results in better graphics quality than DLSS if compute is not a limiting factor. They are trying to do an apples to oranges comparison by comparing the FPS of standard rendering to upscaled images.

I use DLSS type tech, but you lose a lot of fine details with it. Far away text looks blurry, textures aren’t as rich, and lines between individual models lose their sharpness.

Also, if you’re spending $2000 for a toy you are allowed to have high standards.

Because if two frames are fake and only one frame is based off of real movements, then you've actually lost a fair bit of latency and will have noticably laggier controls.

Making better looking individual frames and benchmarks for worse gameplay experiences is an old tradition for these GPU makers.

These are NVidia's financial results last quarter:

- Data Center: Third-quarter revenue was a record $30.8 billion

- Gaming and AI PC: Third-quarter Gaming revenue was $3.3 billion

If the gains are for only 10% of your customers, I would put this closer to the "next to no gains" rather than the "massive gains".

DLSS artifacts are pretty obvious to me. Modern games relying on temporal anti aliasing and raytracing tend to be blurry and flickery. I prefer last-gen games at this point, and would love a revival of “brute force” rasterization.

If you're doing frame generation you're getting input lag. Frame generation from low framerates is pretty far from ideal.

Fake frames, fake gains

I have 2070 Super. Latest Call of Duty runs on 4k with good quality using DLSS with 60 fps and I can't notice at all (unless I look very closely, even with my 6k ProDisplay XDR) so yeah I was thinking of building a 5090 based computer and it will probably last many more years than my 2070 super with latest AI developments.

>Do my games look good

i d like to point you to r/FuckTAA

>Do my games run well

if the internal logic is still in sub 120 hz and it is a twichy game, then no

Any frame gen gains don’t improve latency so the usefulness is reduced

They held back. Had they used 32Gbps GDDR7, they would have reached 2.0TB/sec memory bandwidth. 36Gbps GDDR7 would have let them reach 2.25TB/sec. The GB202 also reportedly has significantly more compute cores, TMUs, ROPs, tensor cores and RT cores than the 5090 uses:

https://www.techpowerup.com/gpu-specs/nvidia-gb202.g1072

Maybe there is a RTX 5090 Ti being held in reserve. They could potentially increase the compute on it by 13% and the memory bandwidth on it by 25% versus the 5090.

I wonder if anyone will try to solder 36Gbps GDDR7 chips onto a 5090 and then increase the memory clock manually.

DLSS and DLAA are terrible for any high-movement Games Like FPS, racing games, Action Games. I wouldn't exactly call it near identical quality. To shareholders this may ring true, but most gamers know that these GPS gains are not worth it and don't use it.. (They still buy it tho)

The workstation cards have MSRPs. The RTX 6000 Ada’s MSRP is $6799:

https://www.techpowerup.com/gpu-specs/rtx-6000-ada-generatio...

Nvidia also doesn't make the "core" (i.e. the actual chip). TSMC and Samsung make those. Nvidia designs the chip and (usually) creates a reference PCB to show how to make an actual working GPU using that chip you got from e.g. TSMC. Sometimes (especially in more recent years) they also sell that design as "founders" edition. But they don't sell most of their hardware directly to average consumers. Of course they also provide drivers to interface with their chips and tons of libraries for parallel computing that makes the most of their design.

Most people don't realize that Nvidia is much more of a software company than a hardware company. CUDA in particular is like 90% of the reason why they are where they are while AMD and Intel struggle to keep up.

They still make reference founders editions. They sell them at Best Buy though, not directly.

Didn't Nvidia piss of some of their board partners at some point. I think EVGA stopped making Nvidia based graphics cards because of poor behavior on Nvidia part?

Also aren't most of the business cards made by Nvidia directly... or at least Nvidia branded?

  it's not worth it.

I wonder how much "it's not worth it". Surely it should have been at all profitable? (a honest question)

The founders edition ones that I had were not great gpus. They were both under cooled and over cooled. They had one squirrel cage style blower that was quite loud and powerful and ran bascially at no speed or full blast. But being that it only had the one airpath and one fan it got overwhelmed by dust or if that blower fan had issues the gpu over heated. The consumer / 3rd party made ones usually have multiple fans at lower speeds larger diameter, multiple flow paths, and more control. TL;DR they were better designed, nvidia took the data center ram as much air as you can in there approach which isn't great for your home pc.

Imagine how sad you'd be if you were still poor.

Without thinking about it too deeply I'm wondering if GPU demand is that much higher than let's say iPhone demand. I don't think I've ever heard of iPhones being scarce and rare and out of stock.

> At least I can get a Nvidia GPU anytime I want from Amazon, BestBuy or whatever.

You can? Thought this thread was about how they're sold out everywhere.

Yet the FE versions end up cheaper than third party cards (at least by MSRP), and with fewer issues caused by the third parties cheaping out on engineering…

Not just the production of the finished boards, but also marketing, distribution to vendors and support/RMA for defective products.

There is profit in this, but it’s also a whole set of skills that doesn’t really make sense for Nvidia.

How does that help "build out a locked in ecosystem"? Again, comparing to Apple: they have a very locked-in ecosystem.

For the curious what I actually did was stop gaming and haven't bought a GPU since 2000's! GPU stuff is still interesting to me, though.

I went from 970 to 3070 and it now draws less power on average. I can even lower the max power to 50% and not notice a difference for most games that I play.

Yeah, do like me, I lower settings from "ultra hardcore" to "high" and keep living fine on a 3060 at 1440p for another few gens.

I'm not buying GPUs that expensive nor energy consuming, no chance.

In any case I think Maxwell/Pascal efficiency won't be seen anymore, with those RT cores you get more energy draw, can't get around that.

I thought opposite. My powersupply is just another component. I'll upgrade it as I need to. But keeping it all quiet and cool...

I built a gaming PC aiming to last 8-10 years. I spent $$$ on MO-RA3 radiator for water cooling loop.

My view:

1. a gaming PC is almost always plugged into a wall powerpoint

2. loudest voices in the market always want "MOAR POWA!!!"

1. + 2. = gaming PC will evolve until it takes up the max wattage a powerpoint can deliver.

For the future: "split system aircon" built into your gaming PC.

In the server space there is gen 5 cabling but not gen 5 risers.

The big grain of salt with that "the 5070 performs like a 4090" is that it is talking about having the card fake in 3 extra frames for each one it properly generates. In terms of actual performance boost a 5070 is about 10% faster than a 4070.

Why would you have your laser printer connected to your UPS?

Does a laser printer need to be connected to a UPS?

Your UPS is improperly sized. A 5kW Victron Multiplus II with one Pylontech US5000 would cost you around €1600 and should be able to carry all your house, not just your printer.

That’s because you have a Brother laser printer which charges its capacitors in the least graceful way possible.

Weirdly they're advertising "1 petaflop of AI performance at FP4 precision" [1] when they're advertising the 5090 [2] as having 3352 "AI TOPS" (presumably equivalent to "3 petaflops at FP4 precision"). The closest graphics card they're selling is the 5070 with a GPU performing at 988 "AI TOPS" [2]....

[1] https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...

[2] https://www.nvidia.com/en-us/geforce/graphics-cards/50-serie...

I’m really curious what training is going to be like on it, though. If it’s good, then absolutely! :)

But it seems more aimed at inference from what I’ve read?

That's s blind way to look at that imho. Doesn't work on me for sure.

More energy means more power consumption, more heat in my room, you can't escape thermodynamics. I have a small home office, it's 6 square meters, during summer energy draw in my room makes a gigantic difference in temperature.

I have no intention of drawing more than a total 400w top while gaming and I prefer compromising on lowering settings.

Energy consumption can't keep increasing over and over forever.

I can even understand it on flagships, they meant for enthusiasts, but all the tiers have been ballooning in energy consumption.

the closest is the L4 https://www.nvidia.com/en-us/data-center/l4/ but its a bit weird.

Innodisk EGPV-1101

I wish mining was still a thing, it was awesome to have free heating in the cold winter.

Yep. I use ~80% and barely see any perf degradation. I use 270W for my 3090 (out of 350W+).

Yes, though how you do it depends on what you're doing.

I do a lot of training of encoders, multimodal, and vision models, which are typically small enough to fit on a single GPU; multiple GPUs enables data parallelism, where the data is spread to an independent copy of each model.

Occasionally fine-tuning large models and need to use model-parallelism, where the model is split across GPUs. This is also necessary for inference of the really big models, as well.

But most tooling for training/inference of all kinds of models supports using multiple cards pretty easily.

Yes, multi-GPU on the same machine is pretty straightforward. For example ollama uses all GPUs out of the box. If you are into training, the huggingface ecosystem supports it and you can always go the manual route to put tensors on their own GPUs with toolkits like pytorch.

I just made a video on this very thing: https://youtu.be/JtbyA94gffc

Yes. Depends what software you're using. Some will use more than one (e.g. llama.cpp), some commercial software won't bother.

So you are saying that Nvidia will finally force USA to the 220V standard? :)

I wonder if they will start putting lithium batteries in desktops so they can draw higher peak power.

Project Digits... https://www.nvidia.com/en-us/project-digits/

I haven’t had great luck with the wizard as a counter point. The token generation is unbearably slow. I might have been using too large of a context window, though. It’s an interesting model for sure. I remember the output being decent. I think it’s already surpassed by other models like Qwen.

Do you have any recommendations on models to try?

You can also run the experts on separate machines with low bandwidth networking or even the internet (token rate limited by RTT)

Maybe, but if they strapped these with 64gb+ wouldn’t that be wasted on folks buying it for its intended purpose? Gaming. Though the “intended use” is changing and has been for a bit now.

If the VRAM wasn't small, the cards would all get routed to non gaming uses. Remember the state of the market when the 3000 series was new?

Saw someone else point out that potentially the culprit here isn’t nvidia but memory makers. It’s still 2gb per chip and has been since forever

So you're saying more VRAM costs more money? What a novel idea!

Conversely, this means you can pay less if you need less.

Seems like a win all around.

No gamers need such high VRAM, if you're buying Gaming cards for ML work you're doing it wrong.

Totally agree. I call this the "Apple Model". Just like the Apple Mac base configurations with skimpy RAM and Drive capacities to make the price look "reasonable". However, just like Apple, NVIDIA does make really good hardware.

Well, they are gaming cards. 32GB is plenty for that.

Makes sense. The games industry doesn't want another crypto mining-style GPU shortage.

Is there actually less VRAM on the cards or is it just disabled?

Yes there should be. We don’t want to pay literal 10x markup because the card is suddenly “enterprise”.

> There should be no expectation that consumer grade gaming cards will be optimal for ML use.

And yet it just so happens they work effectively the same. I've done research on an RTX 2070 with just 8 GB VRAM. That card consistently met or got close to the performance of a V100 albeit with less vram.

Why indicate people shouldn't use consumer cards? It's dramatically (like 10x-50x) cheaper. Is machine learning only for those who can afford 10k-50k USD workstation GPU's? That's lame and frankly comes across as gate keeping.

Honestly I can't really imagine how a person could reasonably have this stance. Just let folks buy hardware and use it however they want. Sure if may be less than optimal but it's important to remember that not everyone in the world has the money to afford an H100.

Perhaps you can explain some other better reason for why people shouldn't use consumer cards for ML? It's frankly kind of a rude suggestion in the absence of a better explanation.

From all the communication I’ve had with Nvidia, the prevailing sentiment was that the 4090 was an 8K card, that happened to be good for AI due to vram requirements from 8K gaming.

However, I’m a AAA gamedev CTO and they might have been telling me what the card means to me.

If I recall correctly, the 3090, 3090 Ti and 4090 were supposed to replace the Titan cards that had been Nvidia's top gaming cards, but were never meant for gaming.

I wonder if these will be region-locked (eg, not for HK SAR).

>Gamers not being able to find an affordable option is still a problem.

Used to think about this often because I had a side hobby of building and selling computers for friends and coworkers that wanted to get into gaming, but otherwise had no use for a powerful computer.

For the longest time I could still put together $800-$1000 PC's that could blow consoles away and provide great value for the money.

Now days I almost want to recommend they go back to console gaming. Seeing older ps5's on store shelves hit $349.99 during the holidays really cemented that idea. Its so astronomically expensive for a PC build at the moment unless you can be convinced to buy a gaming laptop on a deep sale.

Are crypto use cases still there? I thought that went away after eth switched their proof model.

The xx90 cards are really Titan cards. The 3090 was the successor to the Titan RTX, while the 3080 Ti was the successor to the 2080 Ti, which succeeded the 1080 Ti. This succession continued into the 40 series and now the 50 series. If you consider the 2080 Ti to be the "value high end card" of its day, then it would follow that the 5080 is the value high end card today, not the 5090.

Yes, but Nvidia thinks enough of them get pushed up to the 5090 to make the gap worthwhile.

Only way to fix this is for AMD to decide it likes money. I'm not holding my breath.

Gaming enthusiasts didn't beat an eye at 4090 price and won't beat one there either.

4090 was already priced for high income (in first world countries) people. Nvidia saw 4090s were being sold on second hand market way beyond 2k. They merely milking the cow.

Double the bandwidth, double the ram, double the pins, and double the power isn't cheap. I wouldn't be surprised if the profit on the 4090 was less than the 4080, especially since any R&D costs will be spread over significantly less units.

Leaks indicate that the PCB has 14 layers with a 512-bit memory bus. It also has 32GB of GDDR7 memory and the die size is expected to be huge. This is all expensive. Would you prefer that they had not made the card and instead made a lesser card that was cheaper to make to avoid the higher price? That is the AMD strategy and they have lower prices.

MSRP went from 1959,- to 2369,-. That's quite the increase.

We will not really know until memory bandwidth and compute numbers are published. However, Project Digits seems like a successor to the NVIDIA Jetson AGX Orin 64GB Developer Kit, which was based on the Ampere architecture and has 204.8GB/sec memory bandwidth:

https://www.okdo.com/wp-content/uploads/2023/03/jetson-agx-o...

The 3090 Ti had about 5 times the memory bandwidth and 5 times the compute capability. If that ratio holds for blackwell, the 5090 will run circles around it when it has enough VRAM (or you have enough 5090 cards to fit everything into VRAM).

Don't forget that you can link for example two 'Digits' together (~256 GB) if you want to run even larger models or have larger context size. That is 2x$3000 vs 8x$2000.

This will make it possible for you to run models up to 405B parameters, like Llama 3.1 405B at 4bit quant or the Grok-1 314B at 6bit quant.

Who knows, maybe some better models will be released in the future which are better optimized and won't need that much RAM, but it is easier to buy a second 'Digits' in comparison to building a rack with 8xGPUs. For example, if you look at the latest Llama models, Meta states: 'Llama 3.3 70B approaches the performance of Llama 3.1 405B'.

To interfere with Llama3.3-70B-Instruct with ~8k context length (without offloading), you'd need: - Q4 (~44GB): 2x5090; 1x 'Digits' - Q6 (~58GB): 2x5090; 1x 'Digits' - Q8 (~74GB): 3x5090; 1x 'Digits' - FP16 (~144GB): 5x5090; 2x 'Digits'

Let's wait and see which bandwidth it will have.

The 4090 indeed does have ecc support

We've looped back to the "math coprocessor" days.

https://en.wikipedia.org/wiki/Coprocessor

At what point did we stop calling them phones?

Nvidia literally markets H100 as a "GPU" (https://www.nvidia.com/en-us/data-center/h100/) even though it wasn't built for graphics and I doubt there's a single person or company using one to render any kind of graphics. GPU is just a recognizable term for the product category, and will keep being used.

It's a good question. I'll note that, even in the GPGPU days (eg BrookGPU), they were architecturally designed for graphics applications (eg shaders). The graphics hardware was being re-purposed to do something else. It was quite a stretch to do the other things compared to massively-parallel, general-purpose designs. They started adding more functionality to them, like physics. Now, tensors.

While they've come a long way, I'd imagine they're still highly specialized compared to general-purpose hardware and maybe still graphics-oriented in many ways. One could test this by comparing them to SGI-style NUMA machines, Tilera's tile-based systems, or Adapteva's 1024-core design. Maybe Ambric given it aimed for generality but Am2045's were DSP-style. They might still be GPU's if they still looked more like GPU's side by side with such architectures.

How long until a "PC" isn't CPU + GPU but just a GPU? I know CPUs are good for some things that GPUs aren't and vice versa but... it really kind of makes you wonder.

Press the power button, boot the GPU?

Surely a terrible idea, and I know system-on-a-chip makes this more confusing/complicated (like Apple Silicon, etc.)

I mean HPC people already call them accelerators

A bunch of new games are RT-only. Nvidia has aggressively marketed on the idea that RT, FG, and DLSS are "must haves" in game engines and that 'raster is the past'. Resolution is also a big jump. 4K 120Hz in HDR is rapidly becoming common and the displays are almost affordable (esp. so for TV-based gaming). In fact, as of today, Even the very fastest RTX 4090 cannot run CP2077 at max non-RT settings and 4K at 120fps.

Now, I do agree that $1000 is plenty for 95% of gamers, but for those who want the best, Nvidia is pretty clearly holding out intentionally. The gap between a 4080TI and a 4090 is GIANT. Check this great comparison from Tom's Hardware: https://cdn.mos.cms.futurecdn.net/BAGV2GBMHHE4gkb7ZzTxwK-120...

The biggest next-up offering leap on the chart is 4090.

Many people are running 4k resolution now, and a 4080 struggles to to break 100 frames in many current games maxed (never-mind future titles) - therefore there's plenty of a market with gamers and the 5x series (myself included) who are looking for closer to 4090 performance at a non obscene price.

1080ti is most definitely not powerful enough to play modern games at 4k 120hz.

> is RT really necessary to play games? Who cares about real-world reflections?

I barely play video games but I definitely do

> Any modern card under $1000 is more than enough for graphics in virtually all games

I disagree. I run a 4070 Super, Ryzen 7700 with DDR5 and I still cant run Asseto Corsa Competizione in VR at 90fps. MSFS 2024 runs at 30 something fps at medium settings. VR gaming is a different beast

> Who cares about real-world reflections?

Me. I do. I *love* raytracing; and, as has been said and seen for several of the newest AAA games, raytracing is no longer optional for the newest games. It's required, now. Those 1080s, wonderful as long as they have been (and they have been truly great cards) are definitely in need of an upgrade now.

You need as much FPS as possible for certain games for competitive play like Counter Strike.

I went from 80 FPS (highest settings) to 365 FPS (capped to my alienware 360hz monitor) when I upgraded from my old rig (i7-8700K and 1070GTX) to a new one ( 7800X3D and 3090 RTX)

> Any modern card under $1000 is more than enough for graphics in virtually all games. The gaming crisis is not in a graphics card market at all.

You will love the RTX 5080 then. It is priced at $999.

It's a leisure activity, "necessary" isn't the metric to be used here, people clearly care about RT/PT while DLSS seems to be getting better and better.