But I have also been part of a team that was replaced by another because we weren't heroic enough, because we had no bugs in our software and there was no drama for the business to get what it wanted and needed. Management rarely values this type of engineering.
This is condensing a multidimensional vector into just a line, but effective enough to explain to non-engineers.
If you are a precision (slow) engineer, then you are more likely on the backend, more likely to write tests, more likely to avoid costly errors. This will be wasted on some kinds of tasks (trying out new things, for example, but as a class these tasks can generate a lot of management interest). The only kind of management interest in the precision stuff is failure, and it is usually doomsday failure.
That said, security is always present, and I am noodling ideas right now on how to eliminate whole classes of security and privacy issues while making it even easier faster for all engineer types.
To add to this, if you are one of these engineers a great career hack is to find an engineer you respect on the other end of the spectrum and partner with them.
As a "slow" engineer I make sure our key interfaces are abstractions are correct and my "fast" partner ensures that everything gets shipped and that I don't sweat the small stuff. This has lead to a lot more successful and impactful projects than I could manage on my own or with another "slow" engineer.
And, once you accounted for time spent bug-fixing and validating things, they weren't actually slow.
There is a third type: those who can adapt to what is appropriate under the circumstances.
Move carefully and fix things; vs
Move fast and break things.
I’d say that most experienced tradespeople are capable of operating in either mode or somewhere along the continuum.
Personality traits, beliefs, perspective, motivations, attitudes, age, and experiences might predispose someone to favor one side or the other.
Ultimately, there isn’t a single best approach to software development just different tradeoffs. Being able to whip up a bug ridden happy path that only works for some set of data is useful when trying to build an understanding of some new system. At the other end even if most of what you do is short lived demos creating a few rock solid building blocks can save you a great deal of pain.
The heroes who are up late, solving a page or mitigating an outage are often the ones remembered and rewarded.
Meanwhile, a dependable, resourceful, and independent IC who “picks up trash on the floor”, promotes good work habits, and is dead reliable - no praise, they are just “doing their job”
Managers and leaders like drama, most staff and senior engineers gravitate to drama and love talking about it. Their day to to day work, often subpar and often not team players
I have seen this pattern repeated a few times throughout my career. People are rewarded for putting out fires they created (metaphorically), but people who are diligent and don't create problems to solve are overlooked or seen as less than capable.
Raise the issue with higher ups, maybe create some fancy charts about lost engineering time in the future, spin up specific tickets for refactoring, turn it into a two week project and you will get recognition. Management loves a chart about improving X by N% almost more than a shiny feature.
I’m confused here. The business needed your software to have bugs for the drama? Surely this isn’t the whole story.
Drama gets noticed, just quietly ticking along, producing high-quality output really doesn't.
In some situations (probably a lot of software engineering situations) output is difficult to measure, and so the habit of tuning in to activity is adopted instead. Some may even forget the difference.
The person to makes a lot of noise, good or bad, is noticed
Everyone say they want well engineered solutions, but in practice it's not what makes an impact.
And then improving over time. This came up for me after repeatedly encountering code that no one could understand but the original author ("bus count of one", or if one person gets hit by a bus we would be in trouble), or that was needlessly complex for the enjoyment of complexity (really), or that had tests where no one knew why they tested for certain behavior (until fortunately someone else came back from vacation after I was about to commit a change to the test). Etc, all for comprehensibility and maintainability, and reliability.
Ps: overall, this is one of the most enjoyable HN discussions I have seen.
pps: I also one started encouraging our team and others to maintain a set of wiki pages, listing all projects for which we were responsible, then for each one at least one simple page of documenation listing things like who are the stakeholders, where is the source code, any odd build or deployment steps, the key (one sentence) inputs and outputs, where it runs, who does backups of what, etc. Before that it was haphazard. This is short of a real ops manual and could be done better of course, and would change as other org. practices change, but it was far better than nothing, and could be created in 10 minutes from a template. Great for bringing new teammates on board. I think an organization should have something like that for the whole org, listing teams and each team having such a page, as well as listing everything essential that a new person should know rather than relying on haphazard cultural transmission. We had rules like "no new debt" that were well-adopted and began to slip out of the culture as new people joined.
I have been in the situation where over a period of time I inherited a lot of code that was written by my boss and absolutely critical to our business. Over time I began to see that they had made a number of poor design choices that made it very difficult to work with some of these internal frameworks. For example, there was a lot of global state passed around that kind of worked in production, but made it impossible to run tests in parallel (as well as making some of them flaky even when run serially). I introduced an internal dependency injection framework that leveraged some unusual host language features to remove the global state without painfully having to add a bunch of parameters to every function/class that would have to be passed around. This allowed the tests to run in parallel and completely removed the flaky failures. At the time I introduced the changes, they reduced the time to run all the tests from about 3 minutes to 20 seconds with no flaky failures. This was a major quality of life improvement not only for me, but for everyone else working on this part of the system.
Was this work rewarded? No. What I didn't consider when undertaking the refactoring was that I was implicitly making my boss look bad by fixing the systemic problems that they'd introduced. Even worse, I used techniques that were unfamiliar to them which was a further blow to their ego, though they wouldn't have admitted this at the time. Instead, they complained that they didn't understand some of my techniques rather than seriously try to learn what I'd done and appreciate the benefits that it brought. Ultimately, it became soul crushing to realize that they were more invested in doing things the way they'd always done them then to learn how to do things better (or at least offer constructive feedback beyond "I don't superficially understand this so it must be bad"). When you are in this situation, advancement becomes almost impossible because you have now become a threat to your boss, who will never make the mistake of allowing you to be promoted to their level or beyond.
I still think we should all strive to do the best work that we can because ultimately you should feel proud of the work that you've done. But this often comes with a major cost (which may ultimately be that you are forced to leave the organization).
Not all companies are like this, and I find it common that new employees are not sufficiently taught what their company is like :
A) you have been performing well as N,promotion means "we feel / hope you're ready to work as N+1 in the future "
B) you've done great as N and have repeatedly performed N+1 tasks successfully. Promotion means "we recognize what you've already been doing "
First one is "proactive" and represents faith you'll do well at next band. In principle all you have to do is do your job well. There is risk though that you man not succeed after promotion if next band has radically different role or skillset.
Second one ensures you're prepared for your new band, but you cannot there just by doing your job well, and many people are unaware or don't have opportunity to do tasks of next band in their current role
The duties of level N and N+1 are often a little different. If they're different enough, then it can be difficult to demonstrate N+1 while also doing N work. You're option is to work two jobs at once or, as I more commonly see, do the bare minimum of N work and focusing on N+1 work. This can look like the teammate who's focusing on division-wide initiatives at the expense of implementing the things their team is actually assigned - leaving their teammates to pick up the slack.
This is certainly not inevitable, and I'm definitely not saying B is a bad way to do things - just that this is a failure case I've seen of that model. The other, of course, is "why spend a year doing N+1 work here at an N salary when it's easier to just get hired as an N+1 at a new company today?"
the idea that there's no path for ICs beyond management is insulting and degrading to the hard working creatives and engineers out there. Don't give up there's a place for you, you just need to look a LOT harder and in places you wouldn't expect. Play the numbers its a statistics game.
As others have responded, there are definitely trade offs with different models. The way I've seen (B) worked well is:
* Given a project, separate out the "level" of different axes. A project could have level N technical scope, but level N+1 collaboration scope, etc.
* Let the level-N engineer handle one level N+1 axis at a time, with supervision on other axes.
* And if they ace one axis, then back off and give them more responsibilities on the other axes, for the same project or subsequent projects.
Of course there are also scenarios where someone is just thrown into the deep end. They will either sink or swim :)
- Must achieve the highest performance rating for current responsibilities to qualify for promotion.
- But current responsibilities are actually level N+1½.
- Sometimes struggle with the responsibilities from level N+2.
- Never get promoted due to inconsistent N+2 performance.
Conversely, I find it frustrating when engineers do the absolute minimum, avoid refactors, and put the next person at a disadvantage ... all while their velocity is recognized by management as good.
But when it comes to bug fixes, the "absolute minimum" is often the best approach -- it can be explained to demonstrate that we know precisely the root cause of the problem, it can be reviewed for correctness readily, and we can feel good that we are not adding a lot of new behavior that can have downstream effects. The "defensive programming" approach.
A refactoring that makes the problem go away forever is an awesome tech debt project, but when you have a hot problem and someone puts up a 1k LOC PR to refactor the problem away it often introduces so much uncertainty that the improvements aren't worth the risk. You have to start asking questions like how long has this person been around, do they understand what they are doing here, have they really tested this fix or does it just have good "coverage", do they really understand the root cause of the problem or is this just a "refactor and pray it goes away" fix...
A small-as-possible fix relies much less on the credibility of the code author and much more on the fix being self-evidently correct.
So, in general, the instinct is (and should be) to touch the code as little as possible. However, that's not always the right answer. The right answer is to do a risk/benefit calculation and choose the approach that is most likely to lead to an overall improvement.
Sometimes, that means change as little as possible to fix a bug. Sometimes, that means refactoring a major piece. It all depends.
If such engineers don't care about other or future engineers who will have to touch their work, they should at least care about themselves. Perhaps they've never heard the maxim: work to make the life of the next engineer to touch the code easier, because the "next engineer" will probably be you.
It's especially frustrating when "cleanup the code" means "change one line in a few dozen files," because this results in a PR with a high number of changed files. Even if each change is only one line, the high number in the "top line metric" (no pun intended) of "files changed" makes the PR look unfocused, and a lazy reviewer might waste everyone's time criticizing that. So the path of least resistance is to avoid even mentioning the cleanup. Repeat for every PR for the lifetime of the codebase, and your code quality will deteriorate from death by a thousand cuts.
... but you're not the one writing the checks, remember? Management is. They don't care about quality. They can't even define it, much less recognize it. They care about delivery dates. Delivery dates are easy to measure. So that's what matters.
> put the next person at a disadvantage
The first guy was at the same disadvantage - which was that he was under an unreasonable deadline that didn't leave enough time for quality improvements. So you expect him to sacrifice his reputation so that you don't have to sacrifice yours?
And thus the current state of "civilization".
All too often, today's problems stem from yesterday's solutions. This does not mean that yesterday someone made a mistake. It just means progress moves answers. If you did make a choice that feels evergreen, it is just as likely that you are oblivious to work other people are doing.
To that end, maximal choices on how to fix things so that they don't come back are tough. And just as often, large and reaching refactor jobs cause work for the sake of the work. Which is not at all a positive. If you think you can predict what will work, good luck. But don't require good luck from people below you for survival. Celebrate good fortune without belittling the others.
Of course you don't want the next person to slip on the same stone, as it were. But realize that forcing people to walk next to you is a large portion of the reason they may be slipping. And leaving holes in the ground as an alternative is clearly off. As is not waiting for the work of a larger fix.
Like over-abstraction, adding layers of indirection, changing things to use generic code that has no clear relation to the problem domain (and complicates domain-related changes to business requirements), then making other parts of the system (or other systems) re-use those same generic abstractions, so that they're all coupled and/or have increased dependencies?
A good abstraction can be useful, and can even simplify things, but sometimes adding abstractions can be injecting second-system effect into an existing system. Whenever I think of doing something that could turn out like that, I try to get at least a couple of my teammates opinions on it before I even start.
Programmers fix the code, engineers fix the underlying issue. Engineering is being able to spot patterns and know enough about a subject to be able to research it properly and efficiently. "Is this a state machine?", "can I represent this as a tree?", "is this a regular language or do I need a more sophisticated parser?".
I recall someone from a bootcamp writing a cascade of nested if-else statement, 6 level deep in some places. Then someone with a real engineering background told him that he was basically building a finite state machine, to which the other dev responded that "he didn't need anything fancy, just for the function to work". Eye opening.
But in a company environment, the firefighters are more valued are praised because that's something you can see and quantify. where as the regular cleanup/refactoring and fine-tuning the api is invisible and boring. Just a thought.
I am the most frequent consumer and refactorer [Ed. C'mon! That's gotta be a word!] of my code. Most of the breadcrumbs I leave, are for me.
I write about my approach here[2].
[0] https://news.ycombinator.com/item?id=34602701
[1] https://news.ycombinator.com/item?id=34608516
[2] https://littlegreenviper.com/miscellany/leaving-a-legacy/
The infantry is bound to reinvent whatever you come up with badly, and slip on the same stone.
It was an interesting idea, but egos in the room weren't ready for the possibility that the "official" answers weren't actually correct.
I was very new there and my manager didn't understand the algebra when I showed them, and they were very uncomfortable when I showed how the general case manifested in production. I was sacked a day or two later.
It was obviously for the best, but still pretty funny that there are some places where this sort of "maturity" is frowned upon.
He was mocked and bullied by everybody.
ps: don't forget about herd psychology and politics, not all peers want to see your good deeds, no matter how generous or useful they are.
He was also demoted for wasting time moving stones around when he was supposed to be peeling potatoes.
What about other stones inside the encampment that the soldiers can slip on? What about the stones outside of the encampment? Wait, isn't the wall around the encampment MADE OF stones?
And pretty soon you have soldiers cleaning up stones all over the landscape, replacing the wall with solid concrete and designating a stone-cleaning platoon that will go ahead and clean the stones preemptively.
I've seen too many teams happily investing in stone-cleaning activities while forgetting there's a war out there.
You have a project that has both typescript and javascript. Someone makes a change in a JS file. They try to access a property of an object but there is a typo.
You could fix the typo and be done.
Or you could fix the typo, and convert the file to TS and make sure its typed.
Only the number of defects stays the same. Puzzled, you start investigating, and eventually you realize that all of those bugs were shipped by the same person. Turns out your whole TS crusade was a distraction. Even worse: you were optimizing for poor performers at the cost of your best performers.
But when all you have is a hammer, everything looks like a nail.
It's the metaphor that doesn't make sense.
On a product's multi-year death march, I look at the next bug assigned to me for the project-within-the-product I was on.
Over an hour of diligent debugging revealed the problem - the C++ code meant to do
X = Y;
X == Y;
I decided to search the rest of the tens-of-KLOCs project for similar assignment-turned-comparison statements.
That's mindless and tedious work perfectly designed for a computer.
Several minutes later, after weeding through the false positives, I created bug entries for any offenders.
Did I stop there? No.
I connected to the source server for the entire product and kicked off the same search.
When the search finished, I separated the buggy wheat from the chaff and created bug entries accordingly.
This happened on the weekend.
When the product triage team met the following weekday morning, they saw all the bugs entered across the several projects in the product due to the same root cause - double equals instead of a single equals sign.
Management decided to take the next step. They bought a site license for a static source code analyzer. We integrated the analyzer into our project build process and ran the analyzer on each build and triaging accordingly.
Highest compliment I got for creating all this "extra work": "F*ck you!" said with a smile.
Did I stop there? No.
I kept my eyes and ears on the lookout for more possible typos. I would read commit emails and resolutions to bugs to see if the cause/fix would fit the model of "easily-found-via-grep".
I expanded my batch file to cover new cases and created new bugs when the batch file found them.
Did I stop there? No.
Eventually I hit the wall of diminishing returns for source code analysis via regular-expression searching.
I looked for a tool that could go to the next level.
At that time, the one scripting language available in our project's build tools was perl.
Off to the bookstore and I bought O'Reilly's Pink Camel Perl book.
A few hours later, I had a rudimentary source code analyzer that looked at lines of source code for typos.
I added more cases as appropriate.
But C/C++ source code isn't rigidly formatted and programmers wouldn't play nice and limit their code to one code statement per source code line. Preprocessor macros also confused the line-at-a-time script analysis.
So I bit the bullet and expanded the perl script to "parse" C/C++ code.
I added checks to make sure memory allocations were checked for failure (no exception-handling for memory failure in our codebase).
Did I stop there? No.
I publicized the script within the company, answered questions, listened to suggestions, and offered help to other product groups in the company.
A couple of other product groups integrated the script into their build process. Obviously it'd be better if each programmer would run the script before they committed modified code.
Did I stop there? No.
I had worked on the company's new signature product before the current project's death march had started. I still had access to that signature product's source code. So I would run my perl script against the product's source code. That product is huge. A source code scan over the network (my work machine didn't have enough disk space to enlist in every project in the product) would take all night. I had access to the bug database but I didn't know which source code directory mapped to which project in that product.
So I did the next, most annoying thing - I sent a cleansed list of defects to all the developers in the product.
Did I stop there? No.
The company had bought a static source code analyzer product and proceeded to integrate it into most of the company's products' development process. They created a stripped down version of the source code analyzer suitable for developer use before committing code. My perl script was obsolete now.
But that didn't mean others couldn't benefit from it even though the company didn't need it anymore.
I noticed a "call for papers" notice for an open source conference.
I emailed the company's legal department and requested to open source my perl script. Their reply: "Permission granted."
I wrote my proposal, sent it into the conference organizers, who accepted the proposal, wrote a talk, and presented at the open source conference.
Did I stop there? No.
Talk is nice. Code is better. I published/uploaded the script on CPAN (the Comprehensive Perl Archive Network - https://www.cpan.org ).
Did I stop there? No.
Around the time Coverity started scanning open source projects, I had downloaded the source code to several prominent open source projects and scanned their code and sent emails with possible bugs where appropriate.
Posts like this hit HN pretty regularly and are always popular, but they're essentially the equivalent of the perennial LinkedIn "I was already late for a job interview, but I stopped to help a stranded motorist change a flat tire, and then when I showed up for the interview, it turned out I was interviewing with the stranded motorist". For whatever reason, a lot of people seem to believe that if you post something often enough on the internet, you'll make it come true.
We'll document it later. No we won't, tomorrow there will be another fire.
> the general asked "aren't you afraid that you'll slip on the same stone on the way back?"
That makes no sense. The general is an idiot if he thinks that the stone is the problem here. There are millions of stones in a river and they shift over time. Also, it's more likely a problem with the way he was walking; not feeling around with his foot to check that the stone is stable before shifting his weight. The message I get from this story is that bosses are often looking to invent ways to criticize their subordinates in order to bring down their self-esteem. Employees with low self-esteem will be more obedient, accept lower salaries, etc...