Intel's make-or-break 18A process node debuts for data center with 288-core Xeon (opens in new tab)

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process.

For those that do, your scaling example works against you. If today you can merge three services into one, then why do you need full time infrastructure staff to manage so few servers? And remember, you want 24/7 monitoring, replication for disaster recovery, etc. Most businesses do not have IT infrastructure as a core skill or differentiator, and so they want to farm it out.

That’s definitely the right call in some cases. But as soon as there’s any high-interconnect-rate system that has to be in cloud (appliances with locked in cloud billing contracts, compute that does need to elastically scale and talks to your DB’s pizza box, edge/CDN/cache services with lots of fallthrough to sources of truth on-prem), the cloud bandwidth costs start to kill you.

I’ve had success with this approach by keeping it to only the business process management stacks (CRMs, AD, and so on—examples just like the ones you listed). But as soon as there’s any need for bridging cloud/onprem for any data rate beyond “cronned sync” or “metadata only”, it starts to hurt a lot sooner than you’d expect, I’ve found.

What has surprised me about the cloud is that the price has been towards ever increasing prices for cores. Yet the market direction is the opposite, what used to be a 1/2 or a 1/4 of a box is now 1/256 and its faster and yet the price on the cloud has gone ever up for that core. I think their business plan is to wipe out all the people who used to maintain the on premise machines and then they can continue to charge similar prices for something that is only getting cheaper.

Its hard drive and SSD space prices that stagger me on the cloud. Where one of the server CPUs might only be about 2x the price of buy a CPU for a few years if you buy less in a small system (all be it with less clock speed usually on the cloud) the drive space is at least 10-100x the price of doing it locally. Its got a bit more potential redudency but for that overhead you can repeat that data a lot of times.

As time has gone on the deal of cloud has got worse as the hardware got more cores.

Do note though that AIUI these are all E-cores, have poor single-threaded performance and won't support things like AVX512. That is going to skew your performance testing a lot. Some workloads will be fine, but for many users that are actually USING the hardware they buy this is likely to be a problem.

If that's you then the GraniteRapids AP platform that launched previously to this can hit similar numbers of threads (256 for the 6980P). There are a couple of caveats to this though - firstly that there are "only" 128 physical cores and if you're using VMs you probably don't want to share a physical core across VMs, secondly that it has a 500W TDP and retails north of $17000, if you can even find one for sale.

Overall once you're really comparing like to like, especially when you start trying to have 100+GbE networking and so on, it gets a lot harder to beat cloud providers - yes they have a nice fat markup but they're also paying a lot less for the hardware than you will be.

Most of the time when I see takes like this it's because the org has all these fast, modern CPUs for applications that get barely any real load, and the machines are mostly sitting idle on networks that can never handle 1/100th of the traffic the machine is capable of delivering. Solving that is largely a non-technical problem not a "cloud is bad" problem.

I just don't know if the human capital is there.

At my job we use HyperV, and finding someone who actually knows HyperV is difficult and expensive. Throw in Cisco networking, storage appliances, etc to make it 99.99% uptime...

Also that means you have just one person, you need at least two if you don't want gaps in staffing, more likely three.

Then you still need all the cloud folks to run that.

We have a hybrid setup like this, and you do get a bit of best of both worlds, but ultimately managing onprem or colo infra is a huge pain in the ass. We only do it due to our business environment.

> These sorts of core-density increases are how I win cloud debates in an org.

AMD has had these sorts of densities available for a minute.

> Identify the workloads that haven't scaled in a year.

I have done this math recently, and you need to stop cherry picking and move everything. And build a redundant data center to boot.

Compute is NOT the major issue for this sort of move:

Switching and bandwidth will be major costs. 400gb is a minimum for interconnects and for most orgs you are going to need at least that much bandwidth top of rack.

Storage remains problematic. You might be able to amortize compute over this time scale, but not storage. 5 years would be pushing it (depending on use). And data center storage at scale was expensive before the recent price spike. Spinning rust is viable for some tasks (backup) but will not cut it for others.

Human capital: Figuring out how to support the hardware you own is going to be far more expensive than you think. You need to expect failures and staff accordingly, that means resources who are going to be, for the most part, idle.

Cloud = the right choice when just starting. It isn't about infra cost, it is about mental cost. Setting up infra is just another thing that hurts velocity. By the time you are serving a real load for the first time though you need to have the discussion about a longer term strategy and these points are valid as part of that discussion.

It seems a lot of people have forgotten how BigCorp IT used to work.

- request some HW to run $service

- the "IT dept" (really, self-interested gatekeeper) might give you something now, or in two weeks, or god help you if they need to order new hardware then its in two months, best case

- there will be various weird rules on how the on-prem HW is run, who has access etc, hindering developer productivity even further

- the hardware might get insanely oversubscribed so your service gets half a cpu core with 1GB RAM, because perverse incentives mean the "IT dept" gets rewarded for minimizing cost, while the price is paid by someone else

- and so on...

The cloud is a way around this political minefield.

Is your calculation also taking cost of energy and personnel that keeps your own infra running?

That only works if purchasers in the organisation are immune to kickbacks.

Man, how do you get box seats out of AWS, I'm missing out

Is using virtualization the only good way of taking a 288-core box and splitting it up into multiple parallel workloads? One time I rented a 384-core AMD EPYC baremetal VM in GCP and I could not for the life of me get parallelized workloads to scale just using baremetal linux. I wanted to run a bunch of CPU inference jobs in parallel (with each one getting 16 cores), but the scaling was atrocious - the more parallel jobs you tried to add, the slower all of them ran. When I checked htop the CPU was very underutilized, so my theory was that there was a memory bottleneck somewhere happening with ONNX/torch (something to do with NUMA nodes?) Anyway, I wasn't able to test using proxmox or vmware on there to split up cpu/memory resources; we decided instead to just buy a bunch of smaller-core-count AMD Ryzen 1Us instead, which scaled way better with my naive approach.

> These sorts of core-density increases are how I win cloud debates in an org.

The core density is bullshit when each core is so slow that it can't do any meaningful work. The reality is that Intel is 3 times behind AMD/TSMC on performance vs power consumption ratio.

People would be better off having a look at the high frequency models (9xx5F models like the 9575F), that was the first generation of CPU server to reach ~5 GHz and sustain it on 32+ cores.

> If we can't bill a customer for it, and it's not scaling regularly, then it shouldn't be in the public cloud. That's my take, anyway. It sucks the wind from the sails of folks gung-ho on the "fringe benefits" of public cloud spend (box seats, junkets, conference tickets, etc...), but the finance teams tend to love such clear numbers.

I agree, but.

For one, it's not just the machines themselves. You also need to budget in power, cooling, space, the cost of providing redundant connectivity and side gear (e.g. routers, firewalls, UPS).

Then, you need a second site, no matter what. At least for backups, ideally as a full failover. Either your second site is some sort of cloud, which can be a PITA to set up without introducing security risks, or a second physical site, which means double the expenses.

If you're a publicly listed company, or live in jurisdictions like Europe, or you want to have cybersecurity insurance, you have data retention, GDPR, SOX and a whole bunch of other compliance to worry about as well. Sure, you can do that on-prem, but you'll have a much harder time explaining to auditors how your system works when it's a bunch of on-prem stuff vs. "here's our AWS Backup plans covering all servers and other data sources, here is the immutability stuff, here are plans how we prevent backup expiry aka legal hold".

Then, all of that needs to be maintained, which means additional staff on payroll, if you own the stuff outright your finance team will whine about depreciation and capex, and you need to have vendors on support contracts just to get firmware updates and timely exchanges for hardware under warranty.

Long story short, as much as I prefer on-prem hardware vs the cloud, particularly given current political tensions - unless you are a 200+ employee shop, the overhead associated with on-prem infrastructure isn't worth it.

>Realize the savings to be had moving "fixed infra" back on-premises or into a colo versus sticking with a public cloud provider

As other people have pointed out: what happens when the PSU or mobo shits itself, what happens when the new version of vmware or docker (or whatever) shits itself, etc

on prem = capex

cloud = opex

The accounting dept will always win this debate.

I don't post much on HN, but this topic is near and dear to my heart, so here we go.

Context: been helpdesk, sysadmin & network admin, DevOps, Site Reliability Engineer, in that progression, starting in the 90's. Max on-prem was 40 racks, scaled up and down over years.

Many comments talking about how staffing is a key element of this equation that can't be overlooked, but I decided to reply to the root comment, which doesn't say whether/how it considers staffing.

This is a complex equation - and it is relatively easy to present an incomplete or misleading picture management to push the move into the cloud.. or out of the cloud.

Some factors, in no particular order:

1) Scaling: it is self-evident that pulling a single physical server worth out of the cloud is not worth it.. even for 288 cores. Or perhaps 1152 for 4xXeon in a single server. Still likely not worth it. Why? Because a single server is never just that. Someone has to swap components when it goes down. When it goes down.. ALL 1152 cores are down, along with everything they are doing. Is that acceptable for all applications running on all those cores? It is also appropriate supporting infrastructure - power, cooling, physical space. The "fairly obvious" minimum scaling is "enough servers that one can be entirely down for maintenance while keeping everything else running." But now you're paying for some overhead. At 2 servers, you're buying 2x what you need, half that capacity is idle all the time. And so on.

On this point - I think the other comments talking about "each SRE managing 4 (or 5, or 7)" racks missed the point entirely. SRE's should be doing scalable work, whether in the cloud, or on-prem. And they should NOT be swapping failed hard drives and power supplies. Designing a larger-than-one rack install is probably worth hiring consultants for if you don't have that expertise in-house, though the SREs that would be supporting it would need to supply lots of input. To some extent, server & network equipment vendors can also help. It is not trivial as the scale goes up. But then it should run for some years, with relatively unskilled people handling hardware failures and you can re-engage consultants if necessary to do upgrades as hardware and needs evolve.

But your SREs should be on-staff, and probably on-call to handle the software running on that hardware.. and to some extent to call the remote hands to deal with hardware failures.

2) Business needs: does the business need the tech skills that self-hosting requires for the core business? For example - if the business itself is cloud SAAS, maybe DIY-ing at least some of your infrastructure is right in your wheelhouse. If so - a modest increase in staff could mean a huge cost savings. But if not, all the cost of skilled staff to run it is simply part of the cost of in-housing this stuff.

3) Staffing: the people that swap broken hardware are not the same people that respond to pages because the business critical application crashed due to a bug. You can pay a colo facility for all this, typically by the hour - but it isn't cheap and you've got to supply all the spares etc. Is that part of your budget for on-prem?

4) On-call: maybe your self-hosted ERP system can be down every night and weekend without issues.. and even business hours can tolerate 98% uptime. But that doesn't mean you can get away without having someone on call - presumably you're hosting more than just this lowish-requirement ERP system. I'll disagree with other comments - the "no burnout" number of on-call staff you need is 6-7, not 4! Remember people take vacations too. This is well studied, and established, I'll reference Tom Limoncelli's books. This could be relatively cheap and require fewer staff with geo-distributed staff, and it would tend to overlap with staff you already use to provide on-call for anything you host on the cloud - so maybe for your situation it is close to a wash. But you can't forget to budget for it even if the line item is $0.

5) Vendor support: maybe you already have your own data center or colo and are hosting a ton of stuff. Why not move all your Atlassian stuff in house and save the hosting cost. Oh.. wups, Atlassian simply doesn't support that any more. Host it with Atlassian or GTFO. A minor point as most vendors would give you enough notice you can simply run out the lifetime of the hardware it is on and not replace it.

6) Market pricing: At one point Amazon was starting "by the minute, by the core cloud" (as opposed the older "cloudish" model of leasing only an entire physical server by the entire year) and priced a bit under market to get going. Then once they established dominance, they cranked the price WAY up for profit extraction. But now they do have some competition and they're a bit more selective about how they extract profits. In my perception they've shifted a lot of the profit taking to the value-added services rather than raw instance time, but I could be wrong. And they have HUGE costs - it is beyond naive to look at the per-hour cost of an instance and compare it to purchasing an identical physical server solely on the purchase price of that server. Corey Quinn / Duckbill group has spent a huge part of his career in this space - if you're already in the cloud 100% it is well worth optimizing those costs before you start comparing it to what on-prem might cost.

> The only thing giving me pause on that argument today is the current RAM/NAND shortage

Not a shortage - price gouging. And it would mean an increase in the 'cloud' prices because they need to refresh the HW too. So by the summer the equation would be back to it.

Yes there are scheduling issues, Numa problems , etc caused by the cluster in a box form factor.

We had a massive performance issue a few years ago that we fixed by mapping our processes to the numa zones topology . The default design of our software would otherwise effectively route all memory accesses to the same numa zone and performance went down the drain.

There definitely are bottlenecks. The one I always think of is the kernel's networking stack. There's no sense in using the kernel TCP stack when you have hundreds of independent workloads. That doesn't make any more sense than it would have made 20 years ago to have an external TCP appliance at the top of your rack. Userspace protocol stacks win.

I don't think there are any fundamental bottlenecks here. There's more scheduling overhead when you have a hundred processes on a single core than if you have a hundred processes on one hundred cores.

The bottlenecks are pretty much hardware-related - thermal, power, memory and other I/O. Because of this, you presumably never get true "288 core" performance out of this - as in, it's not going to mine Bitcoin 288 as fast as a single core. Instead, you have less context-switching overhead with 288 tasks that need to do stuff intermittently, which is how most hardware ends up being used anyway.

I think linux and co do already a decent job. Even on K8s (so like at least another layer removed from the host OS) you can specify your topology preferences: https://kubernetes.io/docs/tasks/administer-cluster/topology...

So on the OS side we might already have the needed tools for these CoC (cluster on chip ;))

I think linux can handle upto 1024 cores just fine.

I was wondering this, too. There's no mention of OS support, but I assume Intel is working with the usual suspects on it.

That's a great point. Linux has introduced io_uring, and I believe that gives us the native primitives to hide latency better?

But that's just one piece of the puzzle, I guess.

> OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

I mean....

IMO Erlang/Elixir is a not-terrible benchmark for how things should work in that state... Hell while not a runtime I'd argue Akka/Pekko on JVM Akka.Net on the .NET side would be able to do some good with it...[0] Similar for Go and channels (at least hypothetically...)

[0] - Of course, you can write good scaling code on JVM or CLR without these, but they at least give some decent guardrails for getting a good bit of the Erlang 'progress guaranteed' sauce.

> I wonder whether the next bottleneck becomes software scheduling rather than silicon

Yep, the scheduling has been a problem for a while. There was an amazing article few years ago about how the Linux kernel was accidentally hardcoded to 8 cores, you can probably google and find it.

IMO the most interesting problem right now is the cache, you get a cache miss every time a task is moving core. Problem, with thousands of threads switching between hundreds of cores every few milliseconds, we're dangerously approaching the point where all the time is spent trashing and reloading the CPU cache.

Agree entirely with your take. The packaging story is awesome, I wish there were more details on the stacking used on this one.

But I am at a loss to how Intel are really going to get any traction with IFS. How can anyone trust Intel as a long-term foundry partner. Even if they priced it more aggressively, the opportunity cost in picking a supplier who decides to quit next year would be catastrophic for many. The only way this works is if they practically give their services away to someone big, who can afford to take that risk and can also make it worth Intel's continued investment. Any ideas who that would be, I've got nothing.

> 18A is brand new, yields are probably rough.

That the CPU cores are low frequency cores probably helps with yield as well.

My Yocto build times on a 32-core AMD are negligible, <2 minutes for a full distro, IIRC. I suspect higher core counts have diminishing returns, especially since most dev builds are heavily cached.

As a fellow yocto enthusiast, I think they should call the process node 1.8e15 ym instead of the stupid legacy Angstrom unit.

"Is something like this competitive, or preferred, in certain applications?"

They cite a very specific use case in the linked story: Virtualized RAN. This is using COTS hardware and software for the control plane for a 5G+ cell network operation. A large number of fast, low power cores would indeed suit such a application, where large numbers of network nodes are coordinated in near real time.

It's entirely possible that this is the key use case for this device: 5G networks are huge money makers and integrators will pay full retail for bulk quantities of such devices fresh out of the foundry.

In HPC, like physics simulation, they are preferred. There's almost no benefit of HT. What's also preferred is high cluck frequencies. These high core count CPUs nerd their clixk frequencies though.

It all depends on your exact workload, and I’ll wait to see benchmarks before making any confident claims, but in general if you have two threads of execution which are fine on an E-core, it’s better to actually put them on two E-cores than one hyperthreaded P-core.

Without the hyperthreading (E-cores) you get more consistent performance between running tasks, and cloud providers like this because they sell "vCPUs" that should not fluctuate when someone else starts a heavy workload.

I don't know the nitty-gritty of why, but some compute intensive tasks don't benefit from hyperthreading. If the processor is destined for those tasks, you may as well use that silicon for something actually useful.

https://www.comsol.com/support/knowledgebase/1096

I think some of why is size on die. 288 E cores vs 72 P cores.

Also, there's so many hyperthreading vulnerabilities as of late they've disabled on hyperthreaded data center boards that I'd imagine this de-risks that entirely.

For an application like a build server, the only metric that really matters is total integer compute per dollar and per watt. When I compile e.g a Yocto project, I don't care whether a single core compiles a single C file in a millisecond or a minute; I care how fast the whole machine compiles what's probably hundred thousands of source files. If E-cores gives me more compute per dollar and watt than P-cores, give me E-cores.

Of course, having fewer faster cores does have the benefit that you require less RAM... Not a big deal before, you could get 512GB or 1TB of RAM fairly cheap, but these days it might actually matter? But then at the same time, if two E-cores are more powerful than one hyperthreaded P-core, maybe you actually save RAM by using E-cores? Hyperthreading is, after all, only a benefit if you spawn one compiler process per CPU thread rather than per core.

EDIT: Why in the world would someone downvote this perspective? I'm not even mad, just confused

E core vs P core is an internal power struggle between two design teams that looks on the surface like ARM’s big.LITTLE approach

I've seen scenarios where HT doesn't help, iirc very CPU-heavy things without much waiting on memory access. Which makes sense because the vcores are sharing the ALU.

Also have seen it disabled in academic settings where they want consistent performance when benchmarking stuff.

I guess it competes with the like of Ampere's ARM servers? I'm sure there are use cases for lots and lots of weak cores, in telecom especially.

All the compute providers turn that off anyway.

It's a trade off. Hyperthreading takes up space on the die and the power budget.

As to E core itself - it's ARM's playbook.

Isn't this new server CPU a drop in replacement though? So the DC could pull off the old CPU, drop in the new one and not touch the existing RAM setup, yet be able to deliver better performance within the limits of the existing RAM. Then once RAM prices drop (okay that might be a while) separately upgrade the RAM at a different time.

If you have enough cores, you could pool the L1 together for makeshift RAM!

In my experience, RAM costs will have very little impact on businesses buying servers. When we buy is pretty much set by contract and warranty cycles.

> Previously skillset was required. These days you plug Ethernet port, turn on Claude Code dangerously skip permissions “write a bash script that is idempotent that configures my Mikrotik CCR, it’s on IP $x on interface $y”. Hotspot on. Cold air blowing on face from overhead coolers. 5 minutes later run script without looking. Everything comes up.

Last time I tried to do anything networking with Claude it set up route preference in opposite order (it thought lower number means more preferred, while it was opposite), fucking it up completely, and then invented config commands that do not exist in BIRD (routing software suite).

Then I looked at 2 different AIs and they both hallucinated same BIRD config commands that were nonexistent. And by same I mean they hallucinated existence of same feature.

> If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.

The time of having to order a bunch of servers for new project is long over. We just spun k8s cluster for devs to self-service themselves and the prod clusters just have a bit of accounting shim so adding new namespace have to be assigned to a certain project so we can bill client for it.

Also you're allowed to use cloud services while you have on-prem infrastructure. You get best of both, with some cognition cost involved.

Some of the AMD offerings like this on Ebay are pretty close to affordable! It's the RAM that's killer these days...

I still regret not buying 1TB of RAM back in ~October...

Do you remember what you dreamed about 7 years ago? An Ampere Altra 80-core-CPU was sold for less than 210€ on eBay in January.

Wait long enough and these will be cheap on eBay.

By that point we'll be desiring the new 1000 core count CPUs though.

This is the 2026 version of "I need a beowulf cluster of these".

Aside from the memory cost being exorbitant, 4th/5th gen ES CPUs aren’t horribly expensive for the core count you get. 8480s and 8592s have been quite accessible.

Stuffed an 8480+ ES with 192gb of memory across 8 channels and it’s actually not too bad.

> with proportional RAM and storage

Let's not get carried away here

Just give it a few years and you'll be able to buy the thing for a fraction of the 'current' price. By that time it will be considered to be 'slow' and 'power-hungry' and people will wonder why you're intent on running older hardware but it'll still work just fine. The DL380 G7 under the stairs here also used to cost an arm and a leg while I got it for some finger nail clippings.

As soon as I read chiplets I thought about this too! Glad even intel agrees that chiplet architecture is the way forward.

During the 8th gen they made an i7-8086... Hopefully Intel hasn't fired that person.

I wonder if they can bin out ones that have a dead core or two specifically for this purpose.

At least you got the Intel® Core™ Ultra 9 Processor 386H :)

AFAIKS there are 18A laptops from Lenovo and HP for the first two I checked. Or do you mean if you click through and buy one it gets put on some indeterminate backorder?

I used to run many hosts with 28 cores per host. If performance scales, it's nicer to have a few 288 core hosts rather than a few hundred 28 core hosts.

Getting the performance to scale can be hard, of course. The less inter-core communication the better. Things that tend to work well are either stuff where a bunch of data comes in and a single thread works on it for a significant amount of time then ships the result or things where you can rely on the NIC(s) to split traffic and you can process the network queue for a connecrion on the same core that handles the userspace stuff (see Receive Side Scaling), but you need a fancy NIC to have 288 network queues.

Host it in proxxmox, run 8 different services on it each with 32 cores.

Yeah, virtualization, many (small) containers / VMs.

These almost always run many smaller virtual machines on top of a hypervisor. The target market is large enterprise or hyperscalers like the public clouds, Meta, etc...

data processing

This is really what I want to understand. Where can we see real world performance benchmarks?

Not competitive at all. It's easily visible on the laptop lines, where the same GPU manufactured on TSMC has 3 times the power/performance ratio compared to the Intel one.

Putting more cores is just another desperate move to play the benchmark. Power is roughly quadratic with frequency, every time you fall behind competition, you can double the number of cores and reduce the frequency by 1.414 to compensate.

Repeat a few times and you get CPU with hundreds of cores, but each core is so slow it can hardly do any work.

Only slightly related, but six years ago I was able to run 400 ZX Spectrum (Z80) emulator instances simultaneously on an AWS graphics workstation.

https://youtu.be/BjeVzEQW4C8?si=0I7UGU0Xz5WUT4ek

You know, a link would be great for this comment.

No, they're not selling this as an "AI accelerator":

Here is the quote:

"The company says operators deploying 5G Advanced and future 6G networks increasingly rely on server CPUs for virtualized RAN and edge AI inference, as they do not want to re-architect their data centers in a bid to accommodate AI accelerators."

Edge AI usually means very small models that run fine on CPUs.