I’ve also written a more detailed writeup here: https://imtomt.github.io/ymawky/
And super educational. Since then I've been pondering which problems require dropping down to the assembly level. E.g. implementing a JIT compiler, a coroutine runtime, etc.
I made a self documented (all the related links are in the file) single makefile (https://gist.github.com/ontouchstart/d3ad8e4d0adf63532303a90...) so that anyone can build ymawky from scratch and dig deeper by tinkering.
I can see the potential of doing some implementation of APL/J/K at levels even lower than C, like how those guys did APL\360 using assembly language. It is going to be super fun in the era of everyone using LLM to pump out verbose Python/TS/Rust code with context windows bigger than the whole operating system.
In a comparison between a similar fork-per-connection server written in C and this, I would imagine the throughput would be about the same, because the bottleneck in this model is fork() itself rather than the actual code. It probably matters more for binary size and startup time than requests/sec. Would be fun to actually benchmark, though.
Congratulations to the OP for the accomplishment.
I wanted to see how an LLM would do writing one in pure 8088 assembler for CGA and it one-shot a nice demo (I fed it the vectors for the Elite ship in the prompt):
Humbling.
One of my first assembly projects was a CGI Script 100% in x86 assembly.
A full web server is certainly more impressive! Though I'd recommend to beginners to look up CGI and mod_cgi in Apache first lol
The script has been lost to time. I wrote it 5+ computers ago and I don't even know where input that backup...
The overall gist is that CGI Bin specification sets Environmental variables, STDIN and STDOUT to various values. A minimal pure assembly that writes <h1> Hello World </h1> over stdout is your minimalist CGI Script.
A bit of research into what those STDIN/Environmental variables is needed for more. I knew this may e 20+ years ago but have long forgotten....
With access to the various input parameters offered over CGI, you can easily access form data (buttons and whatever clicked by the user). Use some smart file writing to store sessions and off you go....
-------
Maybe start with a Perl CGI tutorial. Then go backwards to C, and finally raw assembly by hand
Thanks for the tips on CGI! Definitely going to look into it more. The server-side execution of CGI scripts definitely interests me more than the CGI scripts themselves, so I’ll probably just look for some existing (simple) CGI scripts and work on building the env vars and executing them.
The absolute best projects ever posted here are the ones made for no other reason than “just because I wanted to”.
P.S.: I would love a copy of that book please!
Bookmarking this — going to read the source on my Sunday off.
Really cool project though!
Modularizing it into multiple files was easier than I expected it to be, you basically have other functions/labels in other files, and mark them as .global at the top. The Makefile compiles each file into their own .o, which you then link all together. You can "b" or "bl" to any label from any other file, as long as it's global and linked together. Same with data in .bss or .data, mark them as .global and they can be accessed from elsewhere.
In general, stable syscall numbers are just a Linux thing. Everyone else uses blessed system libraries
I had a very hard time simply using and even utilizing C++ or Java.
C and Turbo Pascal especially was easier because the compiled code was very much resembling to hand written code.
As the author described, you can do in 4.000 lines what others can do with way less pain in 100.
So you build macros, come up with your own library and in the end you kind of build a meta language build on top of assembly because some lines are so hard to grasp that you delegate working code into a library for reuse.
It is funny how much we take conventions for numbers for granted. If you happen to know assembly and its intricacies you immediately will learn to work with a sign bits which mark negative numbers. But how do you know? Maybe you use the whole addressable space only for positive numbers.
Small things that make a huge different.
Nice article, I enjoyed your adventures and would do the same.
Higher level languages are more convenient for 99% of things, but the directness of Assembly gives me a rush unlike any other. I didn't live through the C64/Amiga, but I was obsessed with old C64/ZX emulators growing up.
Once I was doing 3D I quickly started moving everything but the inner loops to Turbo C, because I'm not a total masochist :)
It would be awesome to read a blog post about the project. Your approach, lessons learned, unexpected stuff, etc.
- Dynamic libraries (e.g. for calling into the kernel, but also user space dynamic libraries) are OS-specific (.so for Linux, .dylib for macOS, .dll for Windows)
- Executable format is OS-specific (ELF for Linux, Mach-O for macOS, PE for Windows)
- Dynamic loading and linkage of both the above are also therefore OS-specific
jk. Metal as fuck. Love it.
...
The last time I did anything in assembler was x86 under DOS. Your code makes ARM64 with a modern OS less scary than I thought it would be.
Maybe it's finally time to move on from being a career programmer.
Artisanal code, or bespoke code, has always been the best paid and most satisfying work. If we no longer have a new generation of curious people who enjoy solving hard problems, it's only going to become more valuable.
> And the only people who will ever be in that position are the ones who take the time and effort, out of sheer curiosity, to learn how things work.
People learn something new all the time, AI does not learn anything, it just simulates and hallucinates. But the core question is not addressed with that. What would you do if you have to compete against AI, and AI is better? We already see these with the new generation of humanoid robots from China. Those things make Boston Dynamics robots look like tinker-toys in comparison - already as-is. Give it ten more years and we finally reached AI skynet for real.
success through cleverness and inventiveness - not yet fucked by AI
achievement through stubborn persistence - you can still dig deep holes in the garden
you still could have a character, if you were lucky
human agency? not yet fucked up, but it's gonna be
achievement earned through one's own qualities or effort? - intact somewhat
Nothing beats Go.
When you use HTMLX (goat) + sqlc (goat) + pgx (another goat) + Chi (yet another goat) and Sqlite (goat).
Most apps will not need anything more than Sqlite, i've several sqlite apps doing a couple of million visits per day.
Compiles to signal binary blazingly fast.
Deploy using systemd service, capture logs with alloy / Loki graphana setup, set up alerts and monitoring and go home.
And you can serve millions of requests on a server with 512MB RAM.
I don't think you'd ever need more speed than this.
Everything else is bloated, slow and doesn't give you enough room for optimization.
Here's the latency of one of my hobby projects (network latency not included): https://i.ibb.co/hJ6FQtyw/d3d6c9d15765.png
Request rate: https://i.ibb.co/Fq80nfJ4/67fcdbdb7491.png
It's running in US and EU (helps avoid atlantic routrip tax), in this one i am doing some 100s of checks, not simple CRUD work. With Go you can optimize a lot without complexity of Rust.
I didn't need to implement an Intel RDRAND streamer in C and assembler, but it was a ton of fun: https://github.com/ehbar/rdrand-stream
OP, I really liked this project. Kudos for publishing it!
I'm sure I'm not the only one who has fantasized about doing something like this as a self-soothing enterprise. Kudos to you for actually doing it!
Today, I just think, "how long would LLMs have taken to write this?"
I mourn the death of a human artform.
Got an idea that you'd need assembly language for - now you can do it instead of..... never doing it because it would have been impossible for you in any practical way.
Look to the positive instead of lamenting something that never would have happened.
It's unbelievably exciting that you can now program a computer virtually without the limitation of your ability to hand code it.
For me it's about making the computer do awesome things - I do not care how I get there I just want it to do whatever I can conjure in my head.
The difference is that now it is worthless: there is no learning, no person caring about the result, nothing aspirational for the public to look towards... we used to enjoy those challenges, used to be proud of solving complex problems... now? Yeah, whatever, execute execute commit push, let another LLM "review" and call it a day.
I wouldn’t be sad about defeating lower complexity challenges. There are always higher complexity challenges that arise once we start operating in a world when you can do more. The bar raises.
Writing whole software projects in assembly has been worthless and pointless for a couple of decades now. Even the projects who can put together a solid case will limit assembly to very specific components executed only in specific bits of a hot path. Perhaps the most performance-sensitive code we have today is high frequency trading and that field is dominated by C++.
Also, virtually all mainstream compiler suites have flags that output assembly,and that feature is largely ignored and unused.
Nobody actually needs a web server built in assembly language, it serves no practical purpose. And I say that as someone who learned to program 6502 assembly language in 1983 and has sporadically used assembly of various architectures since.
The absurdity of building it would have been the curiosity draw pre-LLMs, but when it existing is just a series of prompts away it really loses all of its meaning.
But yeah... hooray for AI. Can't wait until we learn to harness it to supercharge the most important and valuable thing we do as a human society in modern times: stuff increasingly intrusive ads in front of everyone at all times.
Wasn’t it used for that before anything else? Google invented transformers and had LLMs internally before chatgpt got released. Presumably they were using them for ads, because their public demos were insane things like talking to the moon.
Isn't that kind of view pathetic and sad, though? Why would anyone pick up and guitar or play a piano if they could just listen to the same song already made by someone else? I struggle to understand this view of people that pretend to not understand why being an expert of some skill is perceived as valuable by some people. This is also belies next problem with this line of thinking which is that it says "we don't need to learn X to do Y because we have AI" but misses the same AI could easily replace the need to have you think to do Y in the first place. I don't know.
Some 120 years ago recordings music was a living phenomena produced in the moment. Musicians worked at restaurants and coffee shops everywhere, being useful without being super stars.
Music didn’t disappear with recordings, but the works is certainly different.
I'm afraid it's an elite skill in the sense that juggling is also an elite skill. It's impressive for the first few seconds you gaze into it, but once the novelty factor wears off you understand that it's wasted effort that leads to a project that suffers from a massive maintainability problem, is limited in which platforms it can run, and brings no advantage whatsoever. It's an gimmick that has no practice use.
This is the software development equivalent of an amateur guitarist posting shredding videos on YouTube.
We need to stop thinking of software as carpenters where the magic is some physical skill and that is the "CRAFT WE MUST PROTECCT".
And at least your comment was grounded in reality; a lot of people I talk to (who are not coders) seem to think a good software engineer writes every line and every word with thoughtful genius and AI just spams code so one is better than the other. And they are convinced its some naunced smart take and they understand software development on a inner level or whatever.
And the base assumption still holds true (pure AI-generated code is garbage) but its mostly because its badly designed and is still a pretty poor architect. And there is a need to pushback against slop but why do we need to elevate typing code as if its some sacred acctivity? Most of the work a good coder does is in their mind with little connection to the phyiscal reality of the world.
Yes, an LLM can write it, it’ll probably work. Yet, it’ll remain meaningless slop while this is not.
As for why it wouldn't run on Linux, there are some pretty big differences in the actual assembly. One pretty superficial difference is calling conventions -- MacOS uses the x16 register for syscall numbers, Linux uses x8. Calling the kernel in Mac uses "svc #0x80", in Linux it's "svc #0". That's ~120 lines that need to be replaced, but easy enough to just use sed. Syscall numbers are all different, as are the struct layouts for sigaction(), MacOS has an "sa_tramp" field that Linux doesn't have. Enforcing max processes is done here using the MacOS-specific proc_info() syscall, which can be used to get the number of children any given process has. Linux doesn't have an equivalent, so process tracking would need to be done differently. Finally, Linux has the getdents64() syscall, rather than getdirentries64(), which uses a different struct and is called differently.
I'm sure an LLM could make all those changes, but it's a pretty large codebase, so it would probably make some mistakes or miss things.
Example: spend time looking at health/nutrition. I assure you that in 5 years you will get more satisfaction and returns than the assembly code you wrote.
One important caveat - the subject of health/nutrition is SIGNIFICANTLY more complicated than assembly code, and not may sources out there know what they are talking about. Computers are child's play compared to biology.
