If your headless server does complex physics and AI processing and needs to be optimized to handle many clients in real-time, possibly taking full advantage of multiple cores or distributing segments of the world between machines, it is hundreds of time more difficult than a Unity client that moves some animated models around.

Some servers are complex, I don't know why you'd paint with such broad strokes as "headless server = easy". Even if your background is completely web development, surely you've been exposed to more than CRUD.