There was a less common variant of this for finding some extra performance by reducing the iterations of a loop doing no-ops somewhere in the main game loop.
I used to work on the performance team at a Bay Area company. One of the things we did to keep our JavaScript bundle sizes under control was introduce a "ratchet". There was a threshold, enforced by CI, that you couldn't let the bundle size exceed without getting in touch with us first and figuring something out. [0]
This worked wonders for a while, until a few different teams were starting new feature development. At that point, the ratchet was forcing teams to pause their feature development to do cleanup work, which made the PMs very unhappy. Engineers got salty because they would remove dead code, only to find that another engineer had gobbled up the space they'd freed before they could land their own commit.
Engineers started working around the ratchet by hoarding dead code and disguising it to look "not dead" so that it could be easily removed later when a few dozen kilobytes were needed.
There were many thing that weren't good at this company, and the culture around ownership of the shared codebase was definitely one of them. I'd like to think that there are plenty of teams that don't have this problem, but I'm inclined to think that it's human nature to subvert these sorts of things by default.
[0] This was necessary because of the volume of tech debt. Teams/engineers had a bad habit of building new things, then not cleaning up the stuff the new things replaced. At one point, we estimated that over a third of the JavaScript was dead code. Some teams had gotten to the point where the codebase contained >2 versions of their product, while only one of them was physically accessible to users.
Often it would be the lead programmer on the game who put this array in and it wouldn't necessarily be known about by everyone. It wouldn't have worked well if people were constantly grabbing bits of memory from it during development. It worked because it was a 'secret' and its use was reserved for shipping.
Sounds like an example of the cobra effect[1]
Chet Haase & Romain Guy also told on their Android history keynote that the kernel team did the same. They have hidden 20 MB.
I was under the assumption that it was more of an apocryphal story that didn't actually happen. I mean how can someone hide a static array or no op loop from a team of 20+ programmers in a game code base.
As far as I'm aware, it was never confirmed and seems a bit to far fetched to be real, but then again.... :)
I actually found one of these when pulled in to help a title ship that had been put in for a previous title in the franchise and forgotten about when the lead moved on. I found it when memory profiling and after talking with a few people we figured out what it was there for and I got to be the 'hero' who found some extra memory to ship.
I'd assume that in the era where lots of stuff was global and code was messy and convoluted to fit into a console no one would take issue with yet another global variable if it was named a clever enough name as some buffer, temporary, cache, etc. and compilers didn't yet warn about unused variables or remove them (which is an easy fix in C anyway with a single totally portable line or a special compiler specific directive).
There's yet another article on dirty game hacks that I recall due to EULA self-hack: https://www.gamasutra.com/view/feature/194772/dirty_game_dev...
I have no idea why this article doesn't link to these two and just to the 2017 one because it's not like these are current generation tricks, it's just that Brandon Sheffield took to write about them again (one of the articles above is actually his and the other one is by GDM that he is editor in chief of).
Also, the Crash Bandicoot text is the most impressive one and the first in this article but it was very widely circulated for a while now on several sites (including on HN once), despite the "nobody was the wiser until now" at the start of the article.
Well, that's basically how most websites work, lol
I mean it can work and it's a viable tactic if startup time is fast enough.
New screen? Okay, lets throw everything away.
As long as performance is okay, I don't keep stuff around in memory.
I think the skills (and hacks) that used to be useful only to game developers and OEMs are now going to be needed by a much wider audience of devs.
I think you're underestimating the accretion of software bloat. You remember the days when you did exactly the same things, exactly as fast, with 1GB machines? 512MB machines?
As long as devs don't give a fuck (er, they make "professional decision" of optimizing dev time (over product quality)), the days of limited memory are not going to be behind us.
I remember coding while listening to music and reading docs somewhere using a P166MXX with 32mb of ram, and that ran quite snappily using debian and fluxbox.
I now code while listening to music and reading docs somewhere using an i7 with 6GB of ram and sometimes I'm swapping because I left Google Chrome opened for too long :(
I.e. Slack and Atom got absolutely lambasted for performance, sluggishness and resource use (while VS Code was applauded, so it's clearly not an Electron specific thing) despite being made by companies valued in billions and based in the most expensive region of the world, one of them even being a paid product.
Or a game with pixel art (I do like it and I understand that particular indie dev optimizing for time with such a niche product so I don't want to name names here) graphic and gameplay only as deep as some better Flash ones from mid 2000s requires as its minimal system requirements several GBs of RAM (for comparison, Doom 3 recommended, not even minimal, was 512 MB in 2003) and disk space, etc.
Or when a graphically simple 2D game requires a 64 bit OS (despite using no 64 bit features seemingly), a non-integrated GPU (and not because of some lack of OpenGL features but due to poor optimization) and runs at 30 FPS on an integrated Intel that has 0 problems with Mincraft with really far draw distance. And it attempts to load hundreds of files (all of the game assets for an entire 4-10 hour long VN) at boot, taking 30 seconds on an HDD. And they could be loaded incrementally (loading what is needed right now only and everything else in the background, even dumbly and fully into RAM as it does now) or packed into SQLite or a ZIP to avoid so much FS access, but no - hundreds of files are being opened at game boot and there are tons of XML assets with 0 compression or minization. But instead the solution to performance woes (in gaming especially but through things like Electron it's seeping into main stream) is apparently to "git gud", "stop being a poor pleb" and getting a new GPU (apparently GTX 950 M is a potato level GPU now and only an idiot would play games on it in 2017) or an SSD so that the developer doesn't have to bother to do the tiniest of optimization.
That 2D game loading all assets, wanting a 64 bit CPU and non-integrated GPU, all for no good reasons, was Tokyo Dark by the way and due to the way the developer carry themselves I have 0 problem name dropping them, I made an entire video about that game, the disk and GPU part is at 15:15 : https://www.youtube.com/watch?v=sCXwgPJGLIE
It feels like what was done with Crash Bandicoot is some interstellar death star level technology in comparison to what some developers do, not even bothering to pack files to reduce FS chatter or load smartly or compress textual assets, they probably had it developed on an SSD, it loaded fast enough for them, it's done and prime for shipping, duh! Just gotta write a hype text about how extensively we tested it and how much effort we put in making it!
I realize I sound like an ass that's ranting and I am writing too lengthy (I did think about writing articles instead of lengthy HN comments like this one so if someone is interested feel free in hitting me up) but some of the stuff just blows my mind in ways I didn't know existed.
It's not even optimization for dev time like Python could feasibly be but sometimes outright waste or lack of basic care, i.e. Slack was apparently launching a full blown browser per organization until recently (or something like that), completely needlessly, now that part is out. At the same time they had this crazy involved (and cute, because it's 2017 and things must be cute) error page: https://slack.com/asdsad , or that semi-notorious reply article from a guy using unix CLI instead of hip BigData(tm) tools to analyze relatively small amount of data (yes, the guy is rubbing it a bit in too badly when he brings out mawk): https://aadrake.com/command-line-tools-can-be-235x-faster-th...
That lack of care is evident in other areas too, i.e. in security it manifests as these SQL injections, IoT botnets, outdated software pwns and plaintext/unsalted+sha1 password debacles. Afterwards it gets justified by "state attack, China or Russia probably" or handwaved like "we store passwords in plaintext to send them to user via email when he forgets them" (an actual explanation I read once..) or "we innovated so fast to deliver SUPERB customer experience that we didn't focus on security" (while 'security' in that case would amount to closing an admin port on an IoT appliance for example..). In general software we get also stuff like that TP-Link repeater (recently on HN) that needlessly queries NTP every 5 seconds, squandering hundreds of megs of transfer per month and basically DDoSing these NTP servers.
It's like this entire mentality that good stuff is too hard or too complicated or too expensive to do (like that Chess guy and his "clever multi-threaded application") while Pareto is very much in effect and even as little as not opening a hundred files at once at game boot or reading the dense man/info pages and thinking for 20 minutes about the problem at hand or back of the napkin math could make a big difference. 10 or 20 minutes or hours of dev time per year is not a big enough reason to squander resources so badly. There is an expression in Polish that seems really apt for developers who "optimize" their time to that degree: korona ci z głowy nie spadnie (the crown won't fall off your head, basically meaning something along the lines that exerting a little effort towards something isn't too much to be reasonably asked/expected of you).
I recall a similar event when someone wanted to stress test something on a webserver and had a few million long file with URLs in it, he did while read line curl $line in bash, it brought his local machine to its knees, probably due to this rapid process creation and destruction. I gave him an xargs with -P and -n to launch a single curl per each 100 URLs instead and it ran no problem and this time the webserver we were testing was on its knees on my much weaker laptop (weakest in the company actually, since I wasn't a programmer and didn't need a strong one), as intended. I'm actually guilty of overengineering myself, since my first try was a Python 3 + requests + grequests script, and only when weeks after I forgot where I put the script and didn't want to rewrite it I ran that xargs version (very Taco Bell eqsue solution actually - https://news.ycombinator.com/item?id=10829512 ).. And that's an anecdote but it feels like people (actual 'professionals' making a paid product and working in $billion+ corps) ship stuff as bad as the original 1 curl per URL script as if it's not a big deal and then it gets justified with some handwaving, "focus on features and not performance and security", "no one is gonna hack a toaster for anything", "computers are fast and cheap", "optimizing for dev time", etc.
It's a typical high volume low margin situation, like Steve Jobs once said during original Mac building that improving a load time by even a few seconds saves lives of people because so many people will use the Mac so often that it will add to a few lifetimes.
Our main processor has more memory, but that goes away when we loose main power. Still, 64B should be enough for everyone.
I wrote a thing way back when, probably a TI chip or something, which had like 300ish? bytes of memory. It got commands from a serial in (which was a bluetooth device, commands were sent from a java application on a laptop), the commands were a simple self-made thing which was a command (two numbers) and an argument (like, "set speed to 10"). That then controlled a PWM for the engine and things like that. RC boat with a few hundred bytes of memory.
I'm reinventing all that now with using raspberry pi's and arduinos and such. I just got an EPC32, which is a $5 device that should have wifi and bluetooth and such built in already.
Also, if it's a PC game, you're looking at memory limits far below what the HW may offer - at least if you want it to run without annoyances. Windows limits the amount of memory it'll allow to be committed to the DRAM size + (current) pagefile size (may only be 4-8GB on SSDs); note how that doesn't include GPU memory - and allocating that counts as commit due to how residency works. Then you're looking at the OS eating 2-3GB of that, plus 1+GB if the user has a browser opened, plus anything slowly leaking memory. And a LOT of recent Windows applications leak memory or actually use insane amounts (including, amazingly, a built-in service on Win10) - if they never access it again it'll eventually get evicted from the working set, which means it only counts towards committed memory; task manager doesn't display per-process committed memory by default. That's only a fraction of all the problems.
When finishing a level, the game would reboot the console
and restart itself with a command-line argument
(the name of the level to start) ... into the next level.
Voila, the perfect(?) way to clear all memory between
levels.
At 22 years old, this is one of those moments where I'm in awe of the strange issues and workarounds that existed merely ~5-10 years ago which I'll probably never have to deal with. Very funny!
I guess wartime inventions are possibly an extreme example (google Hobart’s Funnies some time for some good clean ingenuity), but shipping console games to a deadline seems to have a similar effect.
Some of the tricks still apply today. For example, modern GPUs support all kinds of weird texture formats. Here’s a link for PC: https://msdn.microsoft.com/en-us/library/windows/desktop/hh3... Other platforms have conceptually similar stuff.
Wow. I am absolutely blown away by this.
One of the projects was heavily squeezed to make it fit, the other didn't need much squeezing, but you couldn't tell the difference by looking at the free space.
https://blogs.msdn.microsoft.com/larryosterman/2004/11/08/ho...
Too new to have a mature gamedev ecosystem. I only finally started tooling around with rust when someone tweeted they'd gotten it working on the PS4, but that still puts me in the early adopter boat.
> Wouldn't Rust's borrow checker and resource management at compilation time make most of the described memory reclaiming issues obsolete?
No. It might help you verify that the systems you're building to tackle these problems are correctly implemented, but it's not going to help you tackle the fundamental problem of e.g. "this game has more 'active' data than we actually have the RAM for".
In theory, this is the problem that virtual memory and page files solve. In practice, occasional small 50-100ms stalls which might be acceptable for a text editor are really nasty for a real time twitch and flow based game. 60fps gives you a frame budget of 16ms - if you want your animation to remain at a fluid 60fps, your absolute max frame time is 16ms.
So, games will end up writing their own memory management systems to e.g. partially load and unload textures at runtime. They'll build in knowledge of your level layout to try and prefetch textures before they're actually necessary based on where the player is moving, so they're available by the time the game needs them - and they might chose to render using a lower resolution version of a texture that was kept in memory if the high resolution version wasn't loaded in time instead of stalling the game.
http://www.pcgamer.com/new-details-and-screens-from-stardew-...
In Rust:
https://www.reddit.com/r/rust/comments/78bowa/hey_this_is_ky...
Another one is that borrow checker. Games are heavily multithreaded since X360. Game state + game assets = huge pile of data shared across threads. Borrow checker in Rust makes it harder to write parallel code that operates on that large shared state. For small state and/or plenty of RAM you can afford making copies but games can’t afford that. Sure Rust brings value here by eliminating a class of bugs, but for game clients (servers are different story) the optimal safety/resource tradeoff is at another place than e.g. for a web browser.
Finally, games often use many third-party libraries and frameworks (sometimes called middleware), and due to historic reasons majority of those are C or C++ libraries.
That is how they moved from Assembly into C, C into C++, adopted Objective-C, adopted C# in their tools, accepted external middleware like Unreal or Unity.
So until one of the big desktop or console owners releases an SDK where Rust is the language, they will hardly adopt it.
Sure a few indies might do it, but it will be a blip on the radar of the usual Game Developer Conference attendees.
- To improve game loading speed of a CD - Load level on PC from hardisk, log all filenames loaded to a txt file and then use that to order the files when writing to the final CD.
- Load all files into PS1 devkits memory, write out all memory in a binary blob to the harddisk, burn that memory image to CD to use for fast level loading (just load it in a single fread(..).
- have separate executables for different levels which had different features, to save memory.
- Write a small block allocator to make <256byte allocations quicker and more efficient.
- Find a tiny piece of memory in the PS2 IOP chip which doesn't get wiped on a devkit reboot (for some reason) and use that as 'scratch' space to write log messages to track down a hard to repro crash that rebooted the kit.
- Change the colour of the tvs border to different colours to track down a race condition that only existed on burnt disks and we had no debugger. The border colour setting code was quick enough to not affect the race condition, so choose some places in code to arbitrarily set to certain colours, burn the disk, test it, when it crashed see what colour the border was, then put some more colours in possible areas, re burn the disk and etc (so basically binary search the code areas using border colours).
- Use compiler optimisations settings for 'size' instead of 'speed' as the smaller executable code size meant you stayed in the DCache more which actually made the code quicker than compiling for 'speed' which resulted in generally larger code.
- Burn a master CD image for publisher, get the game ID code wrong, open up the disk image file in a hex editor and manually edit it rather than go through the whole build process again.
- have no build machine (Gold master got made off whatever code the leads machine had).
- Use sourcesafe (no atomic checkins....)
- Use a few batch files and a directory share for 'source control' of art assets.
- Have values in config files we gave to games designers which did nothing (this was accidental but they swore changing them made a difference to the game).
- Have a advertising deal with a company to have a special cheat code in the game to unlock some stuff, the programming code that does this has a bug that ships that means you have to alter the code incorrectly to get it to work....so tell the company that 'Your code was too easy so we made it harder'.
- Have a developer write code like this as he swore that passing a extra parameter would have slowed the game down:- (psuedo code, but original was in C)
Do stuff(int val)
{
foo * bar;
If (val<10)
{
bar = gStuff[val];
}
else
{
bar = (foo* )val;
}
}