However the enterprise IT/infosec folks at the client didn’t like our project and refused to whitelist our upstream host so it was not possible to download things into the client environment without going through a virus-checking firewall. This imposed various restrictions:
1) It prevented any executable, tarball etc from being downloaded. Basically if it was in a useful file format it was no bueno.
2) It prevented any file over a certain size from being downloaded. If it was over the size, the firewall would just cut the connection.
Time to get the unix toolset out and get to work. I realised first that I could easily get around #2 by slicing the file into chunks, downloading each chunk and then reassembling on the client side
#1 was a bit more tricky. The first thing I tried was encrypting the file. This would theoretically mean the virus scanner wouldn’t be able to find any signatures of hostile file formats, but it turned out that the encryption itself made the first bit of the file predictable and so my first chunk kept getting blocked.
Soo….. I added some random noise onto the front of the file. Once I tuned the length, it meant the virus scanner didn’t understand the encrypted file so it got through.
The two resulting shellscripts (called “shred” and “unshred”) are probably my favourite ever hack. You’d run “shred” on the far side, which would take any listed input files, put them in a tarball, encrypt it, add some random noise to the front and then cut it up into chunks small enough to get through the firewall, and then on the far side you’d download them and run “unshred”, which would reverse the process.
Once we had demonstrably got our software through the firewall a few times, IT/infosec realised their objections were futile and they relented and whitelisted our upstream so we could just do a normal install for all future releases.
