The most insane part here is that the AMD EPYC 4565p can beat the turin's used on the cloud providers, by as much as 2x in the single core.
Our tests took 2 minutes on GCP, 1 minute flat on the 4565p with its boost to 5.1ghz holding steady vs only 4.1ghz on the gcp ones.
GCP charges $130 a month for 8vcpus. ALSO this is for SPOT that can be killed at any moment.
My 4565p is a $500 cpu... 32 vcpus... racked in a datacenter. The machine cost under 2k.
i am trying hard to convince more people to rack themselves especially for CI actions. The cloud provider charging $130 / mo for 3x less vcpus you break even in a couple months, it doesn't matter if it dies a few months later. On top of that you're getting full dedicated and 2x the perf. Anyways... glad to see I chose the right cpu type for gcloud even though nothing comes close to the cost / perf of self racking
For €104/mo you can get a 16-core Ryzen 9 7950X3D (basically identical to your 4565p) w/ 128GB DDR5, 2x2TB PCIE Gen4 SSD.
That's not to say you're wrong about dedicated being much better value than VPS on a performance per dollar basis, but the markup that the European companies charge is much, much lower compared to what they'd charge in the US.
In this instance you're looking at a ~17 month payback period even ignoring colo fees. Assuming a ~$100 colo fee that sibling comment suggested, you're looking at closer to 8 years.
It’s fun to start thinking about building your own server and putting in a rack, but there’s always a lot of tortured math to compare it to completely different cloud hosted solutions.
One of the great things about cloud instances is that I can scale them up or down with the load without being locked into some hardware I purchased. For products I’ve worked on that have activity curves that follow day-night cycles or spike on holidays, this has been amazing. In some cases we could auto scale down at night and then auto scale back up during the day. As the user base grows we can easily switch to larger instances. We can also geographically distribute servers and provide lower latency.
There is a long list of benefits that are omitted when people make arguments based solely on monthly cost numbers. If we’re going to talk about long term dedicated server contracts we should at least price against similar options from companies like Hetzner.
> The cloud provider charging $140 / mo for 3x less vcpus you break even in a couple months, it doesn't matter if it dies a few months later
How do you calculate break even in a couple months if the machine costs $2,000 and you still have to pay colo fees?
If your colo fees were $100 month you wouldn’t break even for over 4 years. You could try to find cheaper colocation but even with free colocation your example doesn’t break even for over a year.
colo fees are cheap if you need more than just 1u. even with a 50-100 fee you easily get way more performance and come ahead within a year
Just as a rule of thumb, if your servers cost more than 1k$ per month or even 500$ maybe even, depending upon if you are okay with colocation and everything. I have found it to break even (more than even the cheapest say hetzner or similar actually so for GCP or anything which charge significantly more, maybe you should warrant a deeper analysis on what is better colocation or dedicated servers or for short burstable units maybe even vps)
Not sure where that fear comes from. Cloud challenges can be as or more complex than bare metal ones.
Big +1 to this. For what I thought was a modest sized project it feels like an np-hard problem coordinating with gcloud account reps to figure out what regions have both enough hyperdisk capacity and compute capacity. A far cry from being able to just "download more ram" with ease.
The cloud ain't magic folks, it's just someone else's servers.
(All that said... still way easier than if I needed to procure our own hardware and colocate it. The project is complete. Just delayed more than I expected.)
Probably because most developers these days have not known a world without using cloud providers, with AWS being 20 years old now.
It’s funny, bc AWS did not start this tour of business. What they did do is make it possible to pay by the hour. The ephemeral spare compute is what they started.
Yet almost nobody understood the ephemeral part.
You might even be better off running a macmini at home fiber, especially for backend processing
If you need single-threaded performance, colo is really the only way to go anyway.
We have two full racks and we're super happy with them.
Similar lower specced machines that were closer to the public internet had boot disk failures, but I had a few of them, so it wasn’t an issue. Spinning metal and all.
One of the db servers dying would have required a next day colo visit… so I never rebooted.
- sometimes you need to limit the list of available CPU features to allow live migration between different hypervisors
- even if you migrate the virtual machine to the latest state of the art CPU, /proc/cpuinfo won't reflect it (linux would go crazy if you tried to switch the CPU information on the fly) (the frequency boost would be measurable though, just not via /proc/cpuinfo )
What do you think the typical duty cycle is for a CI machine?
Raw performance is kind of meaningless if you aren't actually using the hardware. It's a lot of up front capex just to prove a point on a single metric.
Our CI run smaller PR checks during the day when devs make changes. In the “downtime” we run longer/more complex tests such as resilience and performance tests. So typically a machine is utilised 20-22/7.
A year ago I gave a talk about optimizing Cloud cost efficiency and I did a comparison of colocation vs cloud over time. You might find it interesting here, linking to the relative part: https://youtu.be/UEjMr5aUbbM?si=4QFSXKTBFJa2WrRm&t=1236
TLDR, colocation broke even in 6 to 18 months for on-demand and 3y reserve cloud respectively. But spot instances can actually be quite cheaper than colocation.
You generally don't go to the cloud for the price (except if we are talking hetzner etc).
Don't use Hetzner for anything actually important to you. :(
This proc is a hidden gem.
For most workloads it’s not just the most performant, but also the best bang-for-buck.
That is ... hard to believe for a CPU-bound task. Do you have any open benchmark which can reproduce that?
This was a really, really good write-up. I appreciated the breadth of VMs tested and the spread of benchmarks. A few random observations:
1. Turin is a beast.
2. The data on price-performance makes Hetzner look really fantastic, especially for small scale projects where region placement doesn’t matter much and big bursty scaling isn’t required.
3. I think the first ever cloud VM I ever provisioned was on DigitalOcean. I was surprised at how old their fleet was, but I guess they have some limited Emerald Rapids offerings now: https://www.digitalocean.com/blog/introducing-5th-gen-xeon-p...
They're a typical hardware maker unable to focus on software, which is why NVIDIA is now a multi-trillion dollar corporation and AMD is "just" a few hundred billion.
They've focused too much on CPUs and completely dropped the ball on AI and compute accelerators.
It's especially sad considering that the MI300 and related accelerators on paper are competitive with NVIDIA hardware, it's just that they have nowhere near the same software stack, so nobody cares.
We were stuck with Intel, its nice that we have better CPUs.
And don’t get me started on the valuation of companies riding the AI bubble.
AMD produces AI chips, and they seem to be doing quite well.[0] If they didn't, AMD wouldn't be worth anywhere near what it is.
[0] https://openai.com/index/openai-amd-strategic-partnership/
This benchmark seems to recommend Oracle Cloud, but I’ve heard that Oracle has historically used aggressive licenses and legal terms to keep customers locked-in.
Weirdly they didn’t allow me to add payment info to continue. Even weirder their sales people kept contacting me asking me to come back. When I explained the situation they all tried to fix it and then went radio silent until the next sales rep came along to try to convince me to stay.
I searched Reddit at the time and a lot of other people had the same experience. A lot of other people were bragging about abusing their free tier and trials without consequences. I still don’t know how they decided to permanently close my account (without informing the sales team)
As long as you don't use their exclusive DBaaS, moving away is easier than from other places, as egress traffic is free.
The user experience though, stuff of nightmares...
The account creation process was really confusing, and they kept turning off my instance because usage was not high enough.
It seemed quited oudated/confusing to use last time I tried it a few years ago.
> Idle Always Free compute instances may be reclaimed by Oracle. Oracle will deem virtual machine and bare metal compute instances as idle if, during a 7-day period, the following are true:
> • CPU utilization for the 95th percentile is less than 20%
> • Network utilization is less than 20%
> • Memory utilization is less than 20% (applies to A1 shapes only)
The stupid but presumably effective solution is to waste resources to keep above those limits.
Another solution is offered by the email multiple sources cite they send when they reclaim (or warn they will reclaim? not clear) an instance:
> You can keep idle compute instances from being stopped by converting your account to Pay As You Go (PAYG). With PAYG, you will not be charged as long as your usage for all OCI resources remains within the Always Free limits.
Once we did this, the move was fairly easy (the exception being having to write our own auto-scaling logic as the built-in one is very limited). Overall we reduced our cloud spend (even accounting for the additional staff) by about 40%. Bandwidth is practically free and you are not limited to specific combos of CPU/RAM (so you can easily provision something with 7 cores and 9 GB RAM). Another big factor for us was that compute costs in OCI don't vary by region.
I will not recommend using any other managed services from OCI besides the basics (we tried some and they are not very reliable). We've seen minor issues in Networking periodically (Private DNS, LBs or interconnectivity between compute instances), but overall I would say the switch has been worth it.
I wouldn't mind SAN/non-local storage performance and costs too.
VM cost isn't where AWS gets you. It's ALLLLLLLL the other nickel and diming that they kill you on, especially since outbound data transfer costs a log of money to get off of the platform.
Small nitpick, but Turin-based VM's have been available for more than a month now [1], and they are a beast.
[1] - https://techcommunity.microsoft.com/blog/azurecompute/announ...
I might do an addendum just for Azure...
7 Zip benchmark
9800X3D 130 GIPs compression, 134 GIPs decompress.
C8A 21577 MIPs (21.5GIPs) compression, 9868 MIPS decompression (9.9GIPs).
Geekbench 5
9800X3D 16975 multithread, 2474 single thread
C8A 4049 multithread, 2240 single thread
A desktop class CPU is definitely quicker single threaded and multithreaded, no surprises there most of these are dual core. The single threaded performance of the C8A is actually pretty good but its also the best of the bunch by a wide margin most of the CPUs are far behind. Memory performance appears to be attrocious all around.
I’d only add that its very common for gaming computers to be screaming fast- moreso than workstation machines or servers which are a bit more conservative with performance and emphasise correctness (slower cores, slower ram, more ECC). Its not a lot, but it can feel annoying when you sit on a company issued workstation that cost €10,000 but get worse performance than a €2,500 gaming computer.
And the pricing is laughable expensive comparr to OVH.
Every big corporate I have worked at has lower cost of capital than Amazon, and yet they want to move to AWS. I just dont understand it.
I think in practice the system administrators are still in the company now as AWS engineers, they still keep all that platform stuff running and your paying AWS for their engineers too as well as electricity. It has the advantage of being very quick to spin up another box, but also machines these days can come with 288 cores, its not a big stretch to maintain sufficient surplass and the tools to allow teams to self service.
Things are in a different place to when AWS first released, AWS ought to be charging a lot less for the compute, memory and storage, their business is wildly profitable at current rates because per core machines got cheaper.
Cloud looks expensive on sticker price, but it buys instant provisioning, autoscaling, managed databases and multi-region DR, and those benefits only pay off if you actually exploit autoscaling, reserved or savings plans, spot fleets and cost tooling like Kubecost or AWS Compute Optimizer to enforce right-sizing and kill zombie instances.
If you want cheap dev and UAT keep them on on-prem metal or cheap colo, but automate with Terraform and run reproducible runtimes like k3s or devcontainers so environments stay consistent and you do not trade lower capex for a creeping operations nightmare.
A broader thing here is -- and you may also notice this trend in software -- employees are incentivized towards complex solutions, while business owners are incentivized towards simple solutions.
(Shiny object syndrome sold separately ;)
Depends. APIs must take into account many more cases than our own specific use case, and I find we are often spending a lot of time going through unnecessary (for us) hoops. And that's leaving aside possible API changes.
"We Moved from AWS to Hetzner. Cut Costs 89%. Here’s the Catch."
https://medium.com/lets-code-future/we-moved-from-aws-to-het...
Maintaining and updating your own hardware comes with so much operational overhead compared to magically spinning up and down resources as needed. I don’t think this really needs to be said.
You're never just paying for the hardware.
Looking forward to t5g’s whenever (if ever) they release.
a CLI for managing cloud resources that lets you compare prices, apples:apples, AFAICT. I haven't tried it yet but its featureset looks pretty great.
Hetzner is pretty decent too actually but I actually think that OVH might be even cheaper while still being competitive, but in the case of OVH, I think one minor issue people can have is the setup fee at times but time to time like during black fridays or special deals, there are ways to get 0 setup fees.
If someone wants something stable, I think that OVH is pretty great as well and is comparable to Hetzner in terms of pure price but I have heard that given their scale, their support can be 50/50 but I recommend joining their discord and maybe even using (twitter oof) to message them as thsi was something which worked for someone on hackernews the last time something like this was discussed.
If you are okay with some more steal factor, use netcup.
You should also probably look at gaming type servers too. I know a person who is a one man shop who was more passionate about these stuff and did have high-end hardware (200k$) in investment in his provider.
Going more into the specifics of finance from provider side, usually the idea is that they recoup the costs in 5 years if running at decent capacity/having sold quite a bit but the first few years so as in example of the 200k$, they currently make IIRC 40k-60k$ per year, you have to somehow find your way to customers but that is being messed up because of AI ram prices as their costs to fix any broken hardware has absolutely skyrocketed eating much of their profits.
It's an extremely competitive market at times so if you are looking for something specific. You can actually ask about it in forums like lowendtalk, lowendspirit and these people/providers can respond matching what you are looking for price/performance in for and they can also provide test servers if need be and there is a unified form of benchmarking within this space called yabs (yet-another-benchmark-script) which you can ask for a provider.
But in all regards, if someone doesn't want to go through this hassle, I will just say that the same provider that I mentioned earlier had also in public mentioned how hetzner prices are pretty competitive and in all honesty I agree with that too.
If someone doesn't want to go through some/any part of this hassle, then hetzner/OVH are my go-to safe options. They are big enough that their downtime shouldn't be blamed on you for picking them while being really good in themselves. (Something which I have actually heard quite often mentioned on hackernews)
And within this space, if you don't absolutely know what you want, there is no generic winner as there are niches that are occupied. So if you ever want to find any alternative to hetzner for even more cheaper, its best that you first decide what are the things exactly that you want and then ask rather than see for pre-existing if possible.
Also if possible, make deals during blackfriday/cyber-monday. Not sure about Hetzner but in OVH/other providers, you can get recurring deals and the sever setup costs etc. removed sometimes.
Also virtualization from cloud provider is way better because they have custom hardware and software so you don't suffer from noisy neighbours for example.
