The VPS was a VM, but I moved from that to dedicated a long time ago.
Hetzner has nice dedicated servers for €33/mo:
https://www.hetzner.com/sb
I'm on an older one that just got bumped up to €29/mo due to increasing electricity prices; it was €21/mo until now, and I can't blame them for that one. The specs are E3-1245 V2 / 16GB / 2x3T, there's over 45 vhosts on it across ~25 wwwroots plus other random services, and CPU usage is basically nothing. The cores are really there just to handle bursty stuff. Most random side projects and small websites don't need almost any resources on modern hardware.
Previously I was on a Scaleway Dedibox, which go as low as €15/mo right now. It was €10 at one point even.
https://www.scaleway.com/en/dedibox/start/start-2-s-sata/