I think you have a misunderstanding of the full system. Like you are thinking the GUI is much simpler than it actually is. Conversely I think you're thinking that CLIs are much more complicated. There's a lot more going on than just painting pixels on the screen. Maybe start by asking why vim is several magnitudes less resource intensive then Microsoft word. Then ask the same about neomutt vs Thunderbird vs outlook/Apple Mail

I'm also not sure why you think it's an extra layer between the gpu. Both the browser and terminal emulator are sitting in top of the OS. But also remember if you run headless then your OS doesn't need to load the desktop environment, though you can still use the CLI. At this point clearly the GPU is seeing less, right?