We needed to do a nightly transfer of data. We had a variable amount of data to transfer, but typically in the range of one to two TB. We had a 1GBit link between the data centres housing the two systems, but it wasn't an exclusive link - backups and other stuff would be running during the night as well, so if we hog all bandwidth we'd have to deal with unhappy people. Hard deadline for the transfer was start of the work day.
Now the data does compress easily - but the data is only available for compression at the beginning of our sync window. We definitely need to compress some of the data to sync everything in time, and keep other users of the line happy. But: If we spend too much time on the compressing we might not have enough time left to send the data, plus we're not alone on the systems - other people will be unhappy about their nightly jobs failing if we hog all the available CPU time.
So we needed to find the right balance of data compression and bandwidth utilisation, taking into account all those factors, to make things work in the amount of time we had available.
Thanks to AMD nowadays we'd just throw more CPUs at the problem, but back then the 8 CPU server we were using was already quite expensive.
Now, I am not a programmer by trade, but I have a hard time thinking anyone would find it nice to write an inliner. At least not if you want the inliner to always make things faster.
> an insanely unreliable network (~95% uptime)
Did you ever blog about this program? It sounds very interesting, and there is no job interview on HN!
I wish I would have found a 3rd party tool to do all this, but I never did
I even think that it's viable to output PDF without any libraries. I've investigated that format a bit and it doesn't seem too complicated, at least for relatively dumb output.
What I know having done a lot in this space is we aren't close!
Formatting can be tough. See Knuth's extensive bug list for TEX from 1987 at https://yurichev.com/mirrors/knuth1989.pdf to see the kind of tarpit one can get trapped in.
With dart, I felt that very often, when time I saved a file after editing (which activated the formatter), the code would jump around a lot, even for very small edits sometimes. I actually found myself saving less often, as I found the sudden reorganizing kind of jarring and disorienting.
Most of this, I felt like came from this wish of making the formatter keep lines inside a given width. While that's a goal I appreciate, I've come to think is one of those things that's better done manually. The same goes for the use of whitespace in general, other than trivial stuff like consistent indentation. There's actual, important meaning that can be conveyed in how you arrange things which I think is more important than having it always be exactly mathematically consistent.
It's one of the reasons I still prefer ESLint over Prettier in JS land, also, even for stylistic rules. The fact that Prettier always rewrites the entire file from scratch with the parsed AST often ends up mangling some deliberate way that I'd arrange the code.
I run my formatters manually, so I can’t comment on the jumps in code. That does seem jarring.
But some things are not like that. Two statements being right against each other, or having an empty line between them, encodes information.
In a big function call with many arguments, where do you add line breaks between arguments? That can convey information as well. As the posted link says, those are some of the most difficult scenarios for a formatter to try to deal with, and my point is that I think it's not worth the effort.
I really don't like formatters where changing one small part of a large expression results in the entire expression formatted very differently. It's simply not version control friendly. Especially if the language encourages large statements like the pro example. I would rather accept a little bit of code ugliness in this case. Sure this then means that the way the code is formatted is path dependent (depends on the history of the code), but I think it's a reasonable compromise.
So I thought, that there is no reason trying to solve all these problems, since it requires too much time investments. But without solving all this a semi-working formatter isn't good enough to be useful and not annoying.
* Handling of multiple newlines to break sections, or none to group related functions (e.g. a getter with a setter). Sometimes it's even best to move functions around for better grouping.
* They don't factor out an expression into a separate variable.
* They destroy `git blame`. This one is avoided if the tooling has always enforced formatting.
That's...not formatting, and there's probably no good deterministic rule for when to do that, anyhow.
How is this a problem with auto formatters? This is completely on the person/team, not the tool. Even if the repo doesn’t enforce on pre-commit or something, don’t most formatters have the option to only format lines you’ve actually changed?
* They implicitly enable people to write deeply nested code that lacks such factoring, without feeling like anything has gone wrong.
In pragmatic business environments it's not worth the fuss but I never feel great about anything I make in those kinds of environments anyways, and I always appreciate being able to shine when there's no enforced code formatting.
Compare:
#define/**/Q(x,y)r;char*q/* */=y#x","#y")",*p,x=*p%67;}
/*-IOCCC2020-*/#include/* */<stdio.h>/*-BBQlock--*/
int(y),x,i,k,Q(s[9<<9/* 12 */];float(f)[3];void(Z)
(){*f=r<0?r:-r;f[1]/* 11 1 */=42.5;f[2]=22.5;for
(k=0;++k<39;*f/=(k/* 10 2 */%2?k:-k)*6875.5/i)
y=f[1+k%2]+=*f;k=/* */f[2];p=s+k/2*86+y
;}int(main)(){p=/* 9 o-------> 3 */s;for(;i<1978;*q
>32?k=i++/86-11/* / */,y=(750>r*r+k*k
*4)*4+y/2,*p++=/* 8 L 4 */r<44?y?"+0X+0X"
"+!"[y]-1:*q++/* 7 5 */:10:*q++)r=-41
+i%86;r=20;for/* 6 */(x=13;(i=3600*
--x);*p++="XR"/* */"MOQSUWAY"[x%+
10]-9,*p+=x/10/* */*41)Z();sscanf
(__TIME__,"%d"/* \ / -------+ */":%d:%d",&k,&x
,&i);r=10;for(i/* \ / ------ | */+=(k*60+x)*60;r
+18;*p=k%2?*p%2/* \ / ------ | */?59:44:*p>39?59
:39,i=r--?i:i%(+/* \ / ------ | */3600)*12)Z();for
(p=s;*p;putchar(k/* X ------ | */%2&&k<14?q="End",
printf("%c%c",224|/* __/ \__ | | */(21554>>k&3),"gCS"
"gGMX"[k/2]+65),"E"/* / \ / \ | | */"Gh_BrG"[k/2]+64:*p
),++p)k,"#define/**/"/* \__/ \__/ +--+ */"Q(x,y)r;cha""r*q=y#"
"x\",\"#y\")\",*p,x=*p"/* */"%67;}/*-IOCCC2020-*/#"
"include<stdio.h>/*-BBQl"/* */"ock--*/int(y),x,i,k,Q(")
#define /**/ Q(x, y) \
r; \
char *q /* */ = y #x "," #y ")", *p, x = *p % 67; \
}
/*-IOCCC2020-*/ #include /* */<stdio.h> /*-BBQlock--*/
int(y),x,i,k,Q(s[9<<9/* 12 */];float(f)[3];void(Z)
(){
*f = r < 0 ? r : -r;
f[1] /* 11 1 */ = 42.5;
f[2] = 22.5;
for (k = 0; ++k < 39;
*f /= (k /* 10 2 */ % 2 ? k : -k) * 6875.5 / i)
y = f[1 + k % 2] += *f;
k = /* */ f[2];
p = s + k / 2 * 86 + y;}int(main)(){
p = /* 9 o-------> 3 */ s;
for (; i < 1978; *q > 32 ? k = i++ / 86 - 11 /* / */,
y = (750 > r * r + k * k * 4) * 4 + y / 2,
*p++ = /* 8 L 4 */ r < 44
? y ? "+0X+0X"
"+!"[y] -
1
: *q++ /* 7 5 */
: 10
: *q++)
r = -41 + i % 86;
r = 20;for/* 6 */(x=13;(i=3600*
--x);*p++="XR"/* */"MOQSUWAY"[x%+
10]-9,*p+=x/10/* */*41)
Z();
sscanf(__TIME__, "%d" /* \ / -------+ */ ":%d:%d", &k, &x, &i);
r = 10;
for (i /* \ / ------ | */ += (k * 60 + x) * 60; r + 18;
*p = k % 2 ? *p % 2 /* \ / ------ | */ ? 59 : 44
: *p > 39 ? 59
: 39,
i = r-- ? i : i % (+/* \ / ------ | */ 3600) * 12)
Z();
for (p = s; *p;
putchar(k /* X ------ | */ % 2 &&k < 14
? q = "End",
printf("%c%c",
224 | /* __/ \__ | | */ (21554 >> k & 3),
"gCS"
"gGMX"[k / 2] +
65),
"E" /* / \ / \ | | */ "Gh_BrG"[k / 2] + 64 : *p),
++p)k,"#define/**/"/* \__/ \__/ +--+ */"Q(x,y)r;cha""r*q=y#"
"x\",\"#y\")\",*p,x=*p"/* */"%67;}/*-IOCCC2020-*/#"
"include<stdio.h>/*-BBQl"/* */"ock--*/int(y),x,i,k,Q(")
Feel free to not use one in your art projects.
Both are branches of philosophy, but rather distinct from one another.
imho uniformity of what the code looks like > some single person's opinion
it's so satisfying to me when I just run "gofmt" and know the thing is formatted well.
Bike shedding for weeks might happen and is not helpful. On the other side of this argument there are not-so-very-pleasant things as well. A: We follow this-and-this coding convention and we do not like wasting time on discussing it so we follow it to the letter. B: I don't like this small thing that I would like to write differently. A: We follow this-and-this coding convention and we do not like wasting time on discussing it so we follow it to the letter. B: You actually don't follow this-and-this coding convention because you are not following this-and-that rule. A: Yes, that is a change that I liked.
One can easily end up with a very old coding convention that only the oldest developer likes so the oldest developer can now be dictator over everybody else.
The uniformity is not merely for the sake of uniformity.