2) This is not upstreamable in its current form (nvidia admit this in their press release)
3) In an ideal world, nouveau (the open source driver for nvidia hardware) would be able to target this kernel code. Right now though there's no commitment for any sort of stable userland ABI, and that makes that difficult (moving to a new driver version may break interfaces that nouveau uses)
The press release does say that they have plans to figure out a more upstream friendly approach in the long term, which is great - and what has been released will undoubtedly help development of the existing nouveau codebase.
It is far more important to have open sources for all programs that are executed in privileged mode than to have the source for the programs that are executed in user mode.
The kernel can control what a user-mode program does and limit its access to any hardware resources or CPU time, but it cannot do anything against kernel modules that are executed in privileged mode.
So having the source for all the kernel modules is the main priority. With this new open-source kernel module from NVIDIA, the only closed-source kernel module used by a large number of Linux users is no longer so.
Now the main problems remain the BIOS/UEFI of the computers, which is free to override what the operating system does, due to the exceedingly stupid System Management Mode invented by Intel, and the firmware for the auxiliary CPUs that may be used for unauthorized remote management in most Intel- and AMD-based computers.
Take a look into the kernel-firmware repo, and you will see a tremendous amount of binary firmware blobs. For example the Intel micro-code blobs are protected by ~7 layers deep of encryption, wifi drivers may have government regulatory issues with sharing firmwares in anything but binary form. So let's please drop the evil binary blob nonesense...
The kernel does in fact accept small or large binary blobs, and some of them even come with source code...So I guess the key takeaway is to look at what separate the Nvidia driver/firmware from all those other binary firmware loading drivers.
Hint, it has more to do with hiding the program interface, and using abstractions within abstractions.
Is it licensed, or simply too too complicated to pick apart?
Graphics is a complicated field, but the techniques could still be patented - surely their market status isn’t dependent on trade secrets?
That will allow avoiding Nvidia blob except for the firmware.
Hopefully the GPU driver is next in their open-source roadmap.
In normal times that might be a fair way to get people to upgrade but in the last few years (when crypto bros and scalpers have gotten all the cards) it's been outright cruel.
True, I remember though that the semi-recent AMD Radeon drop was not immediately merge-able to mainline because they used a bespoke hardware abstraction layer. But (as alluded to by your point #1) it's a huge first step, and in AMD's case I think they eventually reworked it so that it could indeed be merged.
This open-source kernel code is currently split into OS-agnostic and kernel interface layer components. This stems from NVIDIA's proprietary driver on Linux largely being shared code across Windows / Linux / FreeBSD / Solaris. For it to be upstreamed in the Linux kernel it would likely need to be more re-factored to cater to Linux, just as AMD's DAL/DC originally had tough time upstreaming due to its numerous abstractions.What does "upstreamable" mean in this context? Is this good or bad?
As a follow-up, what would need to be different so that it is upstreamable and would that be good or bad?
"Upstream" is the kernel source tree maintained by Linus (and friends). As Linux is open source, anyone can fork it for any reason, and a lot of people do. However, Linus still maintains the common base that people use to build on top of. "Upstreaming" is the process of getting your code included in these official releases. It's significant because there are no stable interfaces inside the kernel. That is, in any release the Linux developers can change any internal system of the kernel in any way they please, with the only caveat that to get that change included they have to fix up any breakage it causes, but only inside upstream. Or, if you are building your own kernels with your own patch sets, any release of the kernel can completely break your build.
Because of this, if you want your code to be actually used by real people, upstreaming it is considered extremely good, to the point of being almost a necessity. However, it can be a significant hurdle to overcome. Partly because there are fairly strict code quality/style requirements (as any kernel dev is supposed to be able to jump into your code to fix issues they caused by modifications elsewhere), but mostly because unlike inside the kernel itself, all interfaces between kernel and userspace are stable. That is, if you start supporting some operation from userspace, you need to be able to support it in perpetuity. Also, the kernel devs are extremely reluctant to add any kind of interface to the kernel where the only thing that uses it is some specific non-free binary blob on the userspace side.
Currently the main reason the code is not upstreamable as is is that it needs a very substantial userspace program to function, which currently only exists as nonfree, and nVidia doesn't think that the interface used to connect to it is well-enough decided they want to freeze it.
So the main hurdle for upstreaming is developing that interface until they are happy with it, and then develop some free version of the userspace portion of their graphics stack that can plug into that same interface.
It's a negative that it's not upstreamable. I wouldn't go so far as to say it's bad, but it's less than good. That means you'll continue to maintain a separate kernel module that's rebuilt for each kernel.
It would need to be modified to match the kernel's coding standards and whatnot. Part of what makes linux such a strong thing is that you're just not allowed to do certain things in the kernel. They won't include ZFS in the kernel because it's got its own entire vfs -- linux filesystems are only permitted to use linux's vfs, and if you need vfs changes to make it work, you modify the 'one true' vfs. They didn't include the original AMD Radeon driver because it had its own hardware abstraction layer; you're supposed to do that in user space or not at all. (the Radeon driver was changed to remove the abstraction layer, and it was later merged into the kernel)
It's not immediately clear to me how much work it will take to get the driver into the kernel. Hopefully it won't be a lot of work, but it's possible that it would require basically rewriting it from scratch. It's at least written almost entirely in C, so that's a good start.
Regardless, this is tremendously good news.
It just means you have to download drivers separately until Nvidia give the kernel guys a pull request for the drivers to be included in the kernel sources by default.
Nvidia have merely moved their giant closed source drivers into "firmware" - they now have a 34Mb firmware image for the GPU, that would be more correctly called the real drivers. They have essentially created an open source GPL forwarding layer so that the non-GPL firmware gets access to GPL-only kernel APIs, that's it.
Holy shit.
It's even licensed as MIT.
Even OpenBSD could conceivably port this with enough manpower. Adding enough just emulation for the userspace driver would be a lot easier than maintaining a complete linux emulator.
This is one of the biggest things to happen to hardware support for open source OSes in well over a decade.
I like back-foot, underdog NVIDIA. Ascendent AMD hasn't drawn my ire yet, let's hope power corrupts slowly.
Both can/could be bypassed with some libvirtd xml magic, but still. Nvidia seem to slowly stop being assholes, AMD started already.
That "back-foot" "underdog" nVidia has the edge in the video market still... and 3x the market cap of AMD.
Wouldn't be the first time. The old 2D "nv" driver was part of X11, and maintained by Nvidia employees.
The catch, besides it being 2D only, was the "source code" was actually somewhat obfuscated.
>. Adding enough just emulation for the userspace driver would be a lot easier than maintaining a complete linux emulator.
OpenBSD it's the best BSD on support for free (Intel) and semi free drivers such as the ones from AMD, they already adapted all the src from Linux, KMS included.
But, said translation layer would have limited scope; so is a lot more feasible than maintaining a general purpose translation layer indefinitely.
IIRC these sources were already released under a permissive license along with their driver distribution. This just seems to be them putting those sources on GitHub and being more open to contributions.
On quick inspection, this is a complete, MIT-licensed kernel driver.
But this is different, it's voluntary.
Translating & simplifying the language here: sounds like GTX 10xx GPU users (Pascal architecture, e.g. 1070/1080) will stick with closed source for now, but RTX 20xx GPU users (Turing architecture, e.g. 2080) and RTX 30xx GPU users (Ampere architecture, e.g. 3080) will have the option to opt-in to the open source kernel module. Stable open source support for GTX 10xx GPU users may come later.
Nope, Turing or later gen GPU is a hard requirement.
The driver was a proprietary binary. Since a kernel module requires interfacing with the kernel API, it could be considered a derivative work and a breach of the GPL license. So, Nvidia provided a small open source shim which interfaced between the kernel and the proprietary module.
You had to compile that shim yourself with the right arcane command line incantations and if you did anything wrong, missed the right packages or had an incompatible user space, including libs and compiler, you could end up without X11 and no way to easily run a browser or google about the problem you had. You had to do it EVERY FUCKING TIME YOU UPDATED THE KERNEL!
It was still possible to edit xorg.conf or, if you were older, xf86config by hand to fix it and use the VESA driver, but it was very inconvenient. It became more reliable over the time and even fully automated with DKMS, but I hated them for it.
I used and recommended ATI and INTEL for most of the people I could for a long time because of this.
I was from a time when It was possible to use 3D acceleration on linux with 3dfx with fully open source drivers (I think), giving you a cheap UNIX-like graphical workstation with OpenGL support. When Nvidia bought 3dfx and simply killed their drivers, my hate became specially strong.
EDIT: Remember you had to recompile the shim at every kernel update and replaced "module" with "driver".
Same here but recently I somehow got a 3700X and there's no integrated GPU so I had to look for a GPU. I like my PC not just quiet but nearly silent, so a GPU with a fan was a big no-no. I couldn't find any single GPU able to drive 3840x1600 without a fan... Except for a NVidia one. Of course the proprietary Linux drivers are somehow buggy: the "sleep" doesn't work correctly, failing to reinitialize the correct video mode when waking up. It's always, always, always the same with NVidia GPUs on Linux. Thankfully I can switch to tty2, then back to graphical mode but I hate the inconvenience.
I'm thinking about selling my 3700X and getting a 12th gen Intel with an integrated GPU (I don't game and really couldn't care less about fast GPUs).
The actual name of the CPU should help you find one that would work in that regard, for example, see this link which explains the naming suffixes: https://www.androidauthority.com/amd-cpu-guide-1222438/
In particular:
X - Higher clocked desktop processor (what you got)
G - Has integrated AMD Radeon Vega Graphics (what you probably want)
GE - Has integrated AMD Radeon Vega Graphics but lower TDP (what you might want in niche use cases)
For the Zen 2 series that your 3700X belongs to, you might want to look at this table: https://en.wikipedia.org/wiki/List_of_AMD_Ryzen_processors#A...
From what i can tell, the closest option performance wise to Ryzen 7 3700X but with on board graphics would be Ryzen 7 4700G.
> I don't game
Let me help. https://www.ebay.com/itm/194948432276 that's a full DP 1.2 port in there https://www.techpowerup.com/gpu-specs/sapphire-ultimate-r7-2... so it'll drive 3840 x 1600 up to 108 Hz even without custom mode shenanigans.
Also many GPUs support the fan turning off when idle.
I’ve been through a couple 12th gens. I like them, but unless you need the machine updated right now, 13th gen is 6 months out.
Don't suppose you're in Ireland/UK/EU? Looking at building a low/mid-range gaming rig for a sibling, and a 3700X would fit fine if you're looking to sell.
The card isn't silent, but it's quieter than the high end, low wattage PSU fan, which is needed to dissipate heat from everything else.
Anyway, eliminating the video card fan would probably make my desktop louder, which was counterintuitive to me at the time.
(The setup is extremely quiet, FWIW.)
I never quite understood this logic: the same (?) binary blob is used for the FreeBSD and Solaris drivers.
* https://www.nvidia.com/en-us/drivers/unix/
So how can it be a 'derivative' of the GPL Linux if it it also used on non-GPL systems?
Note that the word "derivative" is used here as defined by the license, not in its plain English meaning.
No. Once mainlined, you had to do absolutely nothing to get the hardware working.
That's basically still what happens. Fedora automates this nicely with akmods, which automatically rebuilds these source only modules and installs them when you update the kernel. Has been working smoothly for a while, but it is fundamentally the same thing still.
I haven't bought NVIDIA since then
People has no idea on what FREEDOM do you have if you aren't bound to crappy licenses with Nvidia and CUDA for serious computing where you can be limited per core.
And it works every time, but you do need to run it every time. There is a way to automate it on new kernel installs.
> missed the right packages or had an incompatible user space, including libs and compiler, you could end up without X11 and no way to easily run a browser or google about the problem you had
I always kept the previous version of the kernel and module in case of this.
I've been recompiling my nvidia module each kernel release for over a decade and I've had no problems, you install the kernel, you install the nvidia module, and you reboot.
You don't need X11 to run a browser. But you are right that it's pretty inconvenient without.
This is great news
The closed source driver still exists but there will hopefully be a completely open source stack (Nouveau++?) For nvidia.
This blog has more details about red hats plans for this driver.
https://blogs.gnome.org/uraeus/2022/05/11/why-is-the-open-so...
I can only hope they change name to aidivn, like any sane driver should.
The latter I wish was just a general purpose feature now that things like resizable bar are seeing general support in the consumer space.
* Proprietary shader compiler that can also be used for Vulkan
* Legacy OpenGL driver optimized for closed-source workstation apps
If you don't care about that, you can run pretty much anything else and performance is great.
With the source available... it could be possible! Of course, CUDA support may never happen there, at least not using open source code.
Thank you for the work you do, I'm going through your ssd on rpi4 today!
It’s been a long time coming lol.
Haha that's awesome! I was 1 yo 23 years ago, but in the same vein, I had my middle school years of Linux. Although it was significantly simpler than what you were up to at the same age!
I wonder how much LAPSUS$ hack has to do with it.
I wonder if nvidia hardware programing interface is a mess like AMD one, just curious.
Probably zero.
1) There have been rumors about this for months
2) The hacks only happened very recently, this certainly would have taken longer to do than that.
Have a look at libdrm from the mesa project (the AMDGPU submodule), then it will give you pointers where to look into the kernel-DRM via the right IOCTLs.
Basically, the kernel code is initialization, quirks detection and restoration (firmware blobs are failing hard here), setting up of the various vram virtual address spaces (16 on latest GPUs) and the various circular buffers. The 3D/compute pipeline programing is done from userspace via those circular buffers.
If I am not too much mistaken, on lastest GPU "everything" should be in 1 PCIE 64bits bar (thx to bar size reprograming).
The challenge for AMD is to make all that dead simple and clean while keeping the extreme performance (GPU is all about performance). Heard rumors about near 0-driver hardware (namely "rdy" at power up).
I doubt this is directly because of that.
> The LAPSUS$ hacking group, which has taken credit for the breach, had an unusually populist demand: it stated that it wants Nvidia to open source its GPU drivers forever and remove its Ethereum cryptocurrency mining nerf from all Nvidia 30-series GPUs (such as newer models of the RTX 3080) rather than directly asking for cash. [1]
[1] https://www.theverge.com/2022/3/4/22962217/nvidia-hack-lapsu...
These folks look like they’ve barely touched much open source code before now.
A planned run up would surely have these folks doing more GitHub based commits even if on very private repos.
Serious, does it means we won't need Nouveau anymore? How many and which binary blobs it still needs? Are they encrypted or require signing?
> The current codebase does not conform to the Linux kernel design conventions and is not a candidate for Linux upstream. [...]
> In the meantime, published source code serves as a reference to help improve the Nouveau driver. Nouveau can leverage the same firmware used by the NVIDIA driver, exposing many GPU functionalities, such as clock management and thermal management, bringing new features to the in-tree Nouveau driver.
For now, the plan is to replicate the way AMD drivers
Work, with having shared firmware but separate user lands, one closed and the other libre(MESA for AMD, Nouveau++ for nvidia?)
More details here from Red hats Christian schaller
https://blogs.gnome.org/uraeus/2022/05/11/why-is-the-open-so...
But the goal is to create a complete open source stack like MESA for AMD.
Also its only for Turing and newer (GTX 16XX+) https://blogs.gnome.org/uraeus/2022/05/11/why-is-the-open-so...
https://www.phoronix.com/scan.php?page=article&item=nvidia-o...
Main takeaways:
- support for gaming workstation GPUs is alpha
- the user space stuff (OpenGL/Vulkan) is still closed source. (A LOT of heavy lifting is done here.)
Edit: ok, changed back
I don't even dislike Phoronix, but what was wrong with the first-party source?
Much like AMD - surely they benefit a lot from having a relatively stable non-kernel ABI they can target, though. The problem right now is that everything changes every time the kernel is updated, but if you turn it into a "here's how you dispatch PTX to the card" layer and a "here's how you turn OGL/Vulkan into PTX" blob then the dispatch layer probably isn't changing as much.
(graphics doesn't really use PTX but you get what I mean... dispatching compiled assembly and graphics API calls to the card.)
It doesn't help further the copyleft cause as much as if NVIDIA had open-sourced everything, of course, but from an end-user perspective of "I don't want my kernel updates to be tied to my driver release-cycle" it should solve almost all of the problem?
Was this an ongoing thing? Did it have to be pitched hard? What finally made the difference? I'm assuming here it's not because of LAPSUS as some speculate. This seems like it must have been in the making for quite a while.
My own motivation is to believe that NVidia isn't as broken as everyone insists it is, I guess :) and more broadly speaking it honestly seems like a Good And Interesting Idea to make the situation more clear in any case, particularly given the coverage and significant collective awareness it's attracted.
Also, I found https://www.phoronix.com/scan.php?page=news_item&px=Big-New-... in another comment - 22 May to 12 May, that's some serious stamina lol
He says that Nvidia's interest intensified in the last 3 months. Probably Steam Deck challenging their monopoly is making them uneasy.
Sounds like this is not going to be what I game on for at least a little bit?
Catastrophe is upon us!
> Showing 2,519 changed files with 1,060,036 additions and 0 deletions.
> With each new driver release, NVIDIA publishes a snapshot of the source code on GitHub
That definitely sounds like squashed commits will be the norm for now
> There will likely only be one git commit per driver release.
I've been on linux for at least a couple of decades now, and this has been a thorn in my side from the get-go. I can't overstate how huge this is!
Great work Nvidia, seriously. It does look like it's not perfect, but damn it's a great step.
https://twitter.com/marcan42/status/1524615058688724992
34MB firmware
Good, but I'll try to stick with AMD. If only AMD's opencl support was better...
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/lin...
AMD still has closed-source binary blobs in their cards too, as does everyone else. Their userland is also closed-source too, just like NVIDIA's.
Loading any out-of-tree module taints the kernel, for example the vbox modules are GPL: https://archlinux.org/packages/community/x86_64/virtualbox-h...
If nVidia open sourced all their drivers tomorrow the risk of some cheap Chinese shop making clones of their hardware and re-targeting the nVidia drivers to get lots of the features would be very high. It'd significantly reduce the value of what they'd built (to them).
Really, I don't get why so many Linux users ask questions like this. Most software is proprietary because it costs money to develop. This site we're talking on isn't open source. Windows isn't. macOS / iOS isn't. Games generally aren't. Google isn't. Azure/Bing aren't. Open source is the exception, not the norm.
[1] https://github.com/NVIDIA/open-gpu-kernel-modules/tree/main/...
"Currently, the kernel modules can be built for x86_64 or aarch64."
WE REQUEST THAT NVIDIA COMMITS TO COMPLETELY OPEN-SOURCE (AND DISTRIBUTE UNDER A FOSS LICENSE) THEIR GPU DRIVERS”
https://www.theverge.com/2022/3/1/22957212/nvidia-confirms-h...
https://www.ibtimes.com/nvidia-geforce-graphics-card-virtual...
The amount of time, I was 'sucked' with a black-screen after a reboot and the amount of time I wasted with Nvidia drivers ! I swear I would never buy NVIDIA again !
so ok, not exactly "this == CUDA" but this == decent kernel module drivers for the GPU.
But anyway, the user-space stuff is still the same. This only affects the kernel modules.
Fucking finally.
Most importantly, will they open source Optimus? Even though the drivers work, getting everything to render on the GPU, and output to the laptop LCD, has always been an inconsistent pain.
"After evaluating our position and Nvidia's, we decided to add one more requirement."
"We request that Nvidia commits to completely Open Source (an distribute under a FOSS License) ther GPU drivers for Windows, macOS and Linux, from now on and forever."
"If this request is not met, on Friday we will release the complete Silicon, Graphics and Computer Chipset Files for all recent Nvidia GPUs."
[edit] found it: https://www.phoronix.com/scan.php?page=news_item&px=Big-New-...
Remember, it isn't enough that the kernel module is FOSS. The firmware is where the crux is.
Considering this is coming well after the supposed deadline, I don’t think it’s related.
Since the deadline passed I guess Nvidia negotiated, or they were bluffing.
Shout out to nvidia for this.