undefined | Better HN

0 pointsmichaelochurch12y ago0 comments

I really don't understand why you think "convexity" is a new concept.

Convexity's not new. What is new is the inability for humans to compete with machines on concave labor.

Convexity and concavity come (in theory, anyway) from the logistic form:

L(x; ...) = V + A/(1+exp(-S(x-D)))

It grows exponentially (convex) over x << D, becomes linear around x = D with an inflection point at (D, V + A/2), and then turns concave at x >> D, leveling off to meet a horizontal asymptote at x = V + A.

V is a vertical offset, A is maximum potential/saturation, S is precision, and D is difficulty. Those are constants per function; x is the input (performance). L computes the economic payoff.

Generally, values of x tend to be fairly narrowly distributed based on "the state of the art" and if the typical x << D, we have "hard" (concave) labor. If the typical x >> D, then we have concave/easy stuff. Since input is abstract here, it's typical to say the "typical" x is at 0.

What management has been doing for 200 years is increasing precision (S) and pushing D to the left... making labor concave/easier and precise. That is a good thing. That's exactly what you should do if you're managing a concave job. Incidentally, if you have a convex job, increasing S is not what you want to do because it increases your failure rate.

Now, concave work is (from an economic perspective) great because the ratio of the payoff to the risk (first derivative of L) is high, and that means that in the "knapsack problem" of trying to maximize value under risk constraints, you want to be doing as much of that as you can. Right. No argument there.

With machines, S → ∞ (step function) which you never see with humans, and not only that but the "step" level is usually quite low (in cost; cents per hour of electricity) once it's properly programmed. So what this means is that, for a given business enterprise, we can often give most or all of the concave work over to machines and there's still room in our "knapsack" for high-yield (convex) work. For convex work, value-per-risk is nearly constant and thus the traditional industrial optimization heuristic (value-per-risk) becomes irrelevant. Convex is risk-intrinsic, meaning that it is the risk (rather than unpleasantness) that explains the value.

Thus, the convex work is what's left for us. If it's concave, there's no need for humans to do. The good news? When work is convex, demand for it is effectively limitless. (Limits may exist; the S-curve may level off and turn concave. But if the work's convex, that means the state of the art isn't anywhere near there.)

Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some

Oh no, that's is not what I am saying at all.

What is egotistical about doing great work and sharing it for free?

I am very much pro-OSS and would never say that.

I find that your assumption that only superior software engineers can judge the work of other engineers to be patently false and it sets up a false dichotomy.

We're just going to disagree on this one, because I think that there are attributes of work (especially presentation) that are easy to assess, and others that are hard, and, on the whole, management is about as equipped to assess code-quality issues as I am to go into a chip factory and start spouting off theories about how I think electrical engineering should work. (I have no EE experience.)

Imagine me going into an EE shop and saying, "No more use of imaginary numbers! I don't quite understand how something not real can be useful! If I catch you taking a square root of a negative number, you're fired!" Well, that's how management often comes across on software.

What software engineers simply fail to do is provide objective basis for their work because they don't want to lose any bargaining power over management

Sorry, but that's nonsense. If anything, we lack bargaining power over management, because most of us are terrible negotiators who are socially marginal and awkward, and our inability as a group to demand what we're worth is extremely detrimental.

I contribute to open source projects because it is the right thing to do. It is moral. Not to enhance my reputation in some false gift economy or portfolio waving. Perhaps I am "old school" but this is why reading Stallman was important to my development.

Finally, one does not get satisfaction from simply seeing your cool stuff in use. For example, seeing an evil regime use your cool stuff to repress its citizenry would give me no satisfaction at all.

That's awesome, and there's no disagreement coming from this corner.

0 comments

shawn-butler12y ago

I have no response to your economic theory of software labor. It seems convoluted and abstract and divorced from actual practice of work that I see. Doesn't mean it lacks value, just that I will think about it further.

>>> Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some

Oh no, that's is not what I am saying at all. >>>

I apologize if I have misunderstood, but you argued above that one of the reasons for the explosive growth of OSS projects ("cool stuff for free") is because it is primarily a means to gain recognition to "shortcut" the monotony of apprenticeship. That seems to me not entirely but mostly an egotistic motivation ("Look at me, I'm good!").

>>> We're just going to disagree on this one [about whether the merits of the work of engineers can only be judged by superior engineers] >>>

And so you think that software which at its most fundamental level is a literary endeavor is immune from critique? That borders on the inane and almost requires a leap of faith. Nearly the same argument is made that the Bible is truth and not subject to any criteria except by those of its authors to be accepted by its reader and interpreted only by a sanctified priesthood. Do you also believe art, film, literary, etc criticism is devoid of value and entirely subjective?

Your example is also ridiculous. As if the management of any successful or functional technology company has as much relation to its product as that of a stupefied onlooker to a magic show. Most if not all managers of technical projects are previous engineers. That is one of the major criticisms of the profession by engineers. You reach a stage of ageism where career growth mandates the switch to management. Your analogy of you marching into a EE shop is I assume mirrored by a Dilbert-esque characterization of a pointy-haired boss marching into a technical meeting and arbitrarily dictating software decisions based on whether he likes the name ("Sharding sounds cool")? This may match your own experience but certainly doesn't match mine. What usually happens is after a failure is a conversation like...

Manager: You said sharding would solve our business problem.

Engineer: Yes but you didn't tell me about this feature which entails <magical incantation, abracadabra, hand wavy hocus pocus>.

Even chefs and cooks are judged based on the success of the output of their secret recipes; that it meets the requirements of the restaurant and it suits the taste level of its customers. Why is software intrinsically different? You haven't convinced me so far.

>>>> Sorry, but that's nonsense. If anything, we lack bargaining power over management, >>>>

And here I can not follow as you go off the rails. You just argued that technology managers have a complete inability to complete the work by any other means than using specific skilled labor they cannot understand. This is the very definition of bargaining leverage.

Moreover, you have spilled much ink arguing that the only reason to value skilled technology workers of high multipliers is that they are hard to replace (they have an innate value). That means second-tier and lower tech workers ability to remain relevant and increase their prospects is to emulate by whatever means their higher-value peers since they cannot hope to attain such a valuation on their innate skill. That is why average engineers seemingly go to great lengths to obscure results/defects and provide no objective means to judge the output of their software teams. It does not follow that such macroscopic categories by which to judge software quality do not exist.

Also as an off-topic aside in my humble opinion this is why we have seen such a dramatic rise of "agile", chaotic processes over more established, disciplined methods to manage software production. It is not that disciplined and established production techniques fail to predict accurately the outcome of software projects; it is that they do too good a job at illustrating how immature software engineering/craftsmanship actually is. To be fair, software manufacturing has had a relatively short go at it ;-)

michaelochurchOP12y ago

No. There is no apprenticeship anymore. That's the problem with a convex economy. People have to get through a long "learning" period before the "earning" period in which the market will pay them a living wage.

It's even more convoluted now, because the learning and earning must be intermingled.

In the past, convex work was such a small part of what human society needed done that the effect of this (of making convex labor only available to a privileged class) wasn't such a problem. Now's different because the concave labor that supports the less fortunate is going away.

I'm going to step back from the management stuff because I don't want to start a flamewar. That's not to implicate you, but me. I'm sure good technology management exists, but in my experience it's goddamn rare. The only time I saw good management in software was in the public sector (including a govt. contractor). When people have (expensive for the govt.) security clearances, employees tend to be treated well.

I have once seen a business do software well, but it started to fall apart (culturally) as soon as it needed middle management. No idea if it has solved those problems since then. I left right at the inflection point (no, this wasn't Google) and it's still got a stirling reputation so it's quite possible that they actually got their middle management house in order. Certainly, their tech was excellent.

Ageism in technology is a problem, but there are a lot of badass older programmers who keep on going. But yeah, if you stop learning at 25, then by 40 management is your only option.

j / k navigate · click thread line to collapse