So many failed efforts to make programming vastly easier and more accessible have started with almost this exact line.
I think it's possible that there is a seductive trap there. We see spreadsheets and we think, “programming that doesn't have the downsides we associate with spreadsheets could be just as accessible”. But maybe the problems are intimately tied to what makes it accessible, and wheb you try to make programming that accessible you either fail or end up with something that doesn't offer any advantage over the spreadsheets we have.
> Unfortunately application programming got trampled in the internet gold rush.
No, it didn't. Oh, sure, it's not getting chased around by venture capital, but there's plenty of it going on.
> As a result the internet age has seen an exponential increase in the complexity of programming, as well as its exclusivity.
It really hasn't; we may often be solving far more complex problems, but we do it with languages, frameworks, and platforms that allow casual programmers to do so more easily than at any time in the past. Programming is “more exclusive” only in that:
(1) in working environments, IT policies restrict programming (other than spreadsheets) to the high priesthood more completely than ever before, and
(2) lots of simple problems that would be solved by casual programming in the past now have end-user tools that allow them to be solved without anything we recognize as programming.
No, it didn't
I think if you don't rely on memory but examine the history as Jonathan Edwards has done, you would be surprised to find that the late 1990's was, indeed, a big cliff in terms of application development innovation. You can argue the reasons -- maybe it's not about the internet gold rush -- but you would have to do a small fraction of his research to start to convince me.
> As a result the internet age has seen an exponential increase in the complexity of programming, as well as its exclusivity.
It really hasn't
Frankly, I'm not sure you know what you're talking about. Before the web was popularized, designing a layout meant a using a simple layout tool. Now it's a stack that is at least HTML, CSS and JS, and often more. We're doing more, to be sure, but you can't honestly claim, without a deep ignorance of the past, that it's easier or more inclusive.
Since we're on the topic of UI, I think this is actually a great place to discuss what has happened to programming.
You used to be able to use a simple layout tool. But we traded that away for a substantially increased level of visual sophistication that wasn't practical with the predecessor technologies. For visual sophistication, you had a couple other quite complex options: render it yourself, or for about 15 years, use Flash.
However, we've also smoothed out the learning curve a lot. Even if you have to learn the HTML/CSS/JS triad, we've figured out how to teach it more incrementally.
Even UI in 3d games has gotten dramatically easier. These days you can hack something together in Unity with barely any coding knowledge. When I first started learning 3-d programming 10 years ago, your options were basically to learn C++.
Maybe it's not a 100% obvious conclusion, but I think on the balance of things it really has gotten easier and more inclusive.
The phase change was later, and it wasn't so much a cliff in innovation but a change in focus. There are plenty of programmers “using languages and tools designed for simplicity of learning and use in solving relatively pedestrian problems” still, and plenty of things being built to support them—they're the people doing CRUD apps in Ruby/Rails or building utilities for simple problems in Python that, as Edwards described being historically the case, the people doing “systems programming” continuously look down on.
(There's also a bigger huge group of people doing what is more like what was actually described as application programming historically, that is, solving relatively pedestrian problems in languages designed for industrial use rather than ease of learning, like the people building the guts of enterprise applications that don't require scale, and are largely CRUD plus business rules, using C# now, or the people doing LOB apps in any of a variety of [often proprietary] languages historically.)
> Before the web was popularized, designing a layout meant a using a simple layout tool.
One might argue that the complexity of design and the complexity of programming are different things, or one might observe that graphical UI builders exist for the Web (including as part of platforms designed specifically to support “application programming" for the Web), and that people doing the less sexy kinds of programming use them all the time.
QuarkXPress, LaTex, visual GUI builders. These are different layout tools for very different purposes but they all have in common that they’re not as accessible (with Delphi / RAD Studio in the GUI builder category as the only exception that comes to mind) as “View source” or a simple text editor.
Granted, setting up a new web app with a modern framework such as Angular or React is way more complex than that. However, you also have tools like WordPress that allow you to publish content in a decent layout with little to no experience at all.
Maybe developing web application is harder now (and i may disagree, react and node are really an improvement compared to 2013 SenchaJS imho). Also, CMS now exist for minimal programming.
But you're looking application-wise, not tooling-wise:
- Javascript is safer and easier to use with the last ES iteration (ore maybe it's just me, but my last pet project was really easy to complete with the new javascript).
- OpenBLAS now exist, and libraries were written around this fantastic piece of software. Compare a jupyter notebook to matlab, and appreciate how easy it is to do math on a computer now (and i had to use matlab in college for my math bachelor degree, so i KNOW it is less complex for young mathematicians right now)
- I've wrote stuff in C89 (gcc option, for school related work). Then i was allowed so use C99 (still for school). C99 was easier to write and to optimize. Also, my current collegues were using C in the early 90s, and for them debugging was a pain, while to me, C is one of the easiest language to debug thank to gdb (using the TUI or not).
- Sytem building automatisation is easier (not painless yet, but easier). You can argue it's more complex, but automated stuff i always more complex than manual (but faster too). And in the 90s, system automation was Perl job. I like Perl, but let's agree ansible is less complex.
- System monitoring is easier now too.
And i'm a relatively young programmer, with 2 languages i'm comfortable using for professional work (3 with bash, 4 with ansible, if it count). I'm pretty sure experienced programmers who can use other stuff than C and python have plenty of other experiences. Also, consider compiling your own kernel, writing your own filesystem and/or paquet manager. Would you rather do it in 2018 or in the early 00's? It is a second year project in my school. Second year. It use to be a master degree assessment in college.
Tools are easier to use, and information easier to get.
Building a proper house is so much more that just piling rocks mixed with mortar. You have to take into account the ground composition, moisture, light, ventilation, red tape, usability, safety, etc. In other words: it takes a huge set of skill. You can use advanced tools, like a tractor shovel for digging the base but that requires even more skill.
We just have to accept that doing complex stuff requires lots of training to do it properly. The more advanced tools you use, the more specific training you need but the more complex stuff you can build in shorter time.
As for programming. Abstract thinking and clear, explicit communication of ideas is not trivial to automate. This is problem even between two persons (just think about trying to decipher what your sales person or customer actually wants), let alone between a human and a machine. It’s a form of art, not some mechanical process you automate using an array of boxes.
Sure it might need some training, but it should be the kind that can be explained in a DIY TV show.
But the person hanging the shelf should have enough of a grasp of mechanics so it doesn't fall down. They should know enough basic physics and chemistry so they don't install something prone to corroding. If they encounter asbestos or old lead paint while installing, they should know enough to know what to do about that, and what not to do.
If some business owner finds a way to monetize such tools, nerds will be happy to implement. Just as those do-it-yourself fix-your-home books and tools have been monetized and implemented.
Saying, we can't make it simpler is a cop-out. It used to be simpler. The question isn't whether we can do better (we know we can) but how to find the will, effort and funding to accomplish it.
There are many more people capable of abstract and clear thinking than there are programmers. IMO we should be giving them the tools necessary for them to not build programs, but to solve problems in their field with computers.
edit: to add, those people already have the tools they need to automate the things they want. They just lack the will to learn to use the tools.
edit2: to continue the analogy. I can drill a hole to a wall but if I need to dig a big hole, I can either learn to drive a tractor shovel or hire a professional to do it. But I’m not complaining why tractors are so hard to drive..
Here’s an obvious example where it could be: web apps are mostly conceptually simple: a thin client which is basically a UI framework talks to a process on a remote server. So you need something to develop the UI and something for the server. But instead of the “web UI toolkit” (which could mostly be a graphical tool) we have JavaScript, CSS, HTML, jquery, a gazillion hacked front end frameworks trying to (imperfectly) fix the mistakes of everything below it, a whole system for preprocessing, packaging, running tasks, deploying - just for a UI. And all those tools need customising, rely on knowledge of a bunch of random tools and languages and file editing.
So we could definitely improve 10x in making “web apps” or whatever you would replace them with something much much simpler (even if for a subset of use cases). And that’s just one example.
I think the difficulty of making tools for abstracting web content are related to the difficulties of creating more intuitive programming tools.
But it's not as easy as replacing it with something simpler. If you do, you will discover all the things you can't do with it. So you start to improve it, add features, etc. And after a short time, it's probably even more complex as html/css and still not as flexible and powerful.
Exactly. So what do we have coming next? WebAssembly.
This is vague and insulting.
Many of us programmers spend a lot of energy to keep systems simple, to a point that one could mistakenly conclude that they were easy to write.
We don't dream of having to maintain complex machineries reminding us of Rubik's cubes. We consider maintainance nightmares and hard-to-understand systems as failures, not goals. We don't love complexity, we hate it (but the less experienced of us haven't got the chance to fully develop this hatred).
This is vague and insulting.
As someone who can solve a Rubik's cube and has programmed and played D&D since the version predating the "Advanced" ones, I think it's spot on!
Many of us programmers spend a lot of energy to keep systems simple, to a point that one could mistakenly conclude that they were easy to write.
I've seen code like that. It's the exception, not the rule!
We don't dream of having to maintain complex machineries reminding us of Rubik's cubes
Really? Because I've seen lots of 20-something programmers trot out a bit of code because it's "neat" without regard to cost/benefit within the project and within the current codebase.
We consider maintainance nightmares and hard-to-understand systems as failures, not goals.
I see lots of programmers who blame the skill and intelligence of their fellows while not demonstrating self-awareness by examining how everyone's participation in the system creates the mess in an emergent fashion -- then go on to fail to solve it.
We don't love complexity, we hate it (but the less experienced of us haven't got the chance to fully develop this hatred).
Plenty of the younguns just straight out fetishize complexity. Overtly. Enthusiastically!
Strict functional languages are a step in the right direction. As are verifiers as done in Rust.
By using that kind of tooling you are already acknowledging what the article says in some ways. By using programming languages and editors that let you figure out what the computer can do much faster (like figuring out, at edit time, the result of a bunch of if then else statements for instance), you are getting further away from simple.
And yet this is our normal and it hardly changes over time. So yeah, as someone who has programmed for over 30 years I see the D&D and Rubiks Cube comparison. We do not like it but that does not make it less true; it was easier to solve simple problems ‘fullstack’ when fullstack was 1 language and 1 simple install that had everything; in my opinion we did not evolve well here. It was easier to create most (No I am not talking about a Facebook; that is far from most) things when you did not have to worry about libraries or packaging for deployment. It always was a Rubiks Cube, but we are getting more D&D by the week imho.
And then there are the sort of things Edwards likes to experiment with; relevant live feedback during coding with enforceable prompts to repair simple things that your brain is just not (by far) as good or as fast at as a computer is.
Good. It should be. Get mad about that. Get so mad that you devote time and effort to fixing it.
Programming compared to other disciplines is what might arise from a Lord of the Flies situation. Like the end of that novel, we should stop, fall at the feet of a comparison of programming to other disciplinnes, and weep tears of shame.
Interestingly, I feel that insulting people is one of the worst possible ways to get them to change their behaviour.
There is no technical solution to a social problem caused by hiring cheap young and naive programmers.
> Programming today is exactly what you’d expect to get by paying an isolated subculture of nerdy young men to entertain themselves for fifty years. You get a cross between Dungeons & Dragons and Rubik’s Cube, elaborated a thousand-fold. I’m serious: Dungeons & Dragons requires mastery of an encyclopedia of minutiae; Rubik’s Cube is a puzzle requiring abstract mathematical thinking. That is programming in a nutshell. As a bonus you also get a toxic culture of intellectual one-up.
It's boring for a developer. I don't understand many of the problems described here (e.g. people complain about the complexity of server management, complexity of web frameworks and web in general). Actually it has become more simple than ever to create an app, a backend and scale the system.
With more servers (I use systems that can be scaled horizontally) thrown at it, good caching and some load balancing, it will work for 99% of businesses. Here [1] is a good description of a service that uses this thinking.
I believe there are many who celebrate complexity. Otherwise I can't understand most codebases - I've seen terrible codebases with big LoC's where I was able to reimplement the whole feature set in a couple of days. I believe many programmers don't want to see that their jobs become increasingly easy to do and the boilerplate can be reduced by 90%.
> We don't dream of having to maintain complex machineries reminding us of Rubik's cubes.
Many people do because their jobs depend on it. If businesses would adopt those fast, clean and scalable engineering techniques, 70% of the work force is no longer needed. It's basically the principal-agent problem [2] in action.
[1]: https://medium.com/unboxd/how-i-built-an-app-with-500-000-us...
[2]: https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble...
edit: in terms of "No Silver Bullet" - I think if we decrease accidental complexity (using good techniques and automated and tested systems) to a record low, we won't need as many people to manage the essential complexity.
In my experience the problem of complexity is another one: big enterprise makes a lot of money using their terrible code. They will keep throwing money at it to avoid major rewrites. Programmers will complain about unnecessary complexity, but complexity is not what's important for the bottom line most of time. The system just works and prints money, so there's no need to correct (until it explodes in complexity, but a major rewrite is also a risky decision for managers and can result in huge investment losses).
> Many of us programmers spend a lot of energy to keep systems simple
"Many" is not good enough (and makes me think of the cringy and problematic #notallmen thing on twitter). Ask yourself: who introduces the complexity to those systems again?
> What reason is there to believe we can improve programming technology to be vastly easier and more accessible? To start with, spreadsheets. Spreadsheets are by far the most popular programming tool, used by far more people than “real” programming. Surely it must be possible to carve out other domains like that of spreadsheets without taking on the full complexity of programming.
As someone writing a spreadsheet app with magical programming powers (https://6gu.nz) no wonder the author seemed to be sweet-talking me :-).
And yeah, I think we can do a lot better. Some low-hanging fruit: backslashes in strings, "what is Unicode", "what is a hash function and why do I care?". Etc.
There's a crap-tonne of essential complexity in just about everything we do, but most tasks for which "programming" should be the right answer are not the sorts of things programmers actually do.
Part of the problem is, we have all the best tools because we are the tool makers. There isn't good version control for Excel. Linting is mediocre. And the language itself sucks, because the programmers who work on it don't think spreadsheets are programming, and don't think the people using spreadsheets are programmers.
I do think the conclusions of the article are o ff-base, though. I have no faith in professional societies, and I don't think the answer is "institutions". The answer is business -- bringing programming to the masses is worth trillions. Programming is the biggest force-multiplier that exists. It's what the FAANGs (widely disparate companies in different industries) have in common -- they (as companies) "know how to program".
Maybe businesses are short-sighted, but the value here is so big it just has to happen. Magic of capitalism and all that.
Since you're writing a spreasheet app, I presume you are familiar with Feliene's work on this topic?
http://www.felienne.com/publications
Also, you probably will find Edwards videos about his own experiments interesting:
I'm not! Thank you!
I've been pretty heads-down writing my thing, but I'll be sure to set aside my evenings to go through both sites exhaustively, and probably try to get in contact with both to ask for feedback.
No, they don't look like spreadsheets, but THAT is the industrial strength tool being used by a professional chainsaw juggler.
I think "traditional" databases will fail to reach a general audience (or rather, much expand the definition of "technical audience") because SQL is not really easier than Python etc. Maybe the problem is environments and not languages, though.
Personally I think the relational model is just a little too limited. It's possible to build abstractions in SQL (views, temporary tables, `WITH` etc) but it's hard. And repetitive.
Basically I think functions and variables are great little building blocks, so I'm pushing them. And then, having not backed myself into a corner with by having chosen a limited computational model, I'll make relational stuff easy too :-).
Where do I think programming has stalled is in the business environment. There, as the article correctly points out, it's much harder to automate your job unless you work in an IT department.
Most business end-users do their work in Microsoft Office, and this is one area where I think the tools have stalled. Visual Basic and its object models are a painful hurdle for a smart non-programmer who is trying to get things done efficiently, or who wants to learn programming for its own sake.
There have been some attempts to wire up Excel with Python, but until Microsoft decides to bundle an easy tool with every user's MS Office, programming is going to remain an arcane art.
Depends on your business. Ecommerce can be automated without needing to code. Many use a central product repository (e.g. Amazon), and build their web frontend (CMSs, shop engines, landing page & app builders) around that, interfacing with it through easy installable plugins and other automation tools. They're already "coding", without code, but "business components".
To have successful funnels, they also need to know UX basics that they can reproduce with those tools, or pick tools with those basics already baked in. They're basically full stack developers who don't code.
The downside to that approach is that you're totally dependent on all those other businesses. But many who start out this way, employ devs as they grow, or start with programming themselves, and transition to their own infrastructure.
And we don't want to build a future where the masses should be programming! You want my mother to be programming? My sister? Why? I work as a software engineer and most of the time programming is the last thing I want to do when I get home!
Sure it can be overwhelming at times with all the new technologies coming out, but in the future technologies are going to be completely different, with AI and VR becoming more widespread, programming is going to have to change again. And good, because that is what will allow us to build the future!
I am not an electrician, but I can fix basic electric issues. Sometimes, I can fix broken furniture. I am a programmer and I can fix almost nothing.
When we can program, we are in control. I want to be in control of the objects I own, or at least to know that it is possible if I really want it. When I see all what already possible in the 90s with so simple computers, I do not think that the complexity is necessary.
But that took a lot of work on an undocumented software in machine code in 8051 and had the best tools.
The issue is that is not easily traceable from machine code representation. Even electronics are easier.
Um.... yes? I think a better question is why not? Do you also not want your mother and sister to write or do math even if they are not authors or mathematicians?
I think you may be part of the problem. Why gatekeep your loved ones if it's something they want to do? That implies crass arrogance at best, cruelty at worst.
> You want my mother to be programming? My sister?
And yes, of course we'd like your mother and sister to program, in the same way we want everyone to be able to read and write or drive a car. Programming is a wonderful tool that can do so much in many different contexts, and limiting that power to an elite few is a tragic loss of potential.
> you want my granddad to do programming?
I do understand what he's saying here, but saying that software has no evolution is just wrong. Like yes we still have C code kicking around everywhere but we also have new technologies languages (rust, go, etc etc), new philosophies on how to write code (TDD?), and new tools (jupyter notebook for interactive data munging for example).
Furthermore how many people have been through a rewrite? developers (try) to replace and renew all the time, although yes this is difficult.
also:
> Unfortunately application programming got trampled in the internet gold rush
It seems to me like application programming is alive and well, although admittedly I am younger and am probably considered an application programmer myself (I.E. I don't develop software as a profession, but I write code all the time to help with my work).
And when you see software as literacy a lot of the stuff he moans about simply goes away.
Yes it is seen as exclusive and a priesthood - imagine what illiterate serfs thought of those who could write. It took years of practise.
No it's not a good idea to make programming more accessible. We don't have easy literature, we don't think someone who reads and writes solely with fridge magnets (#) has mastered the language. It is a good idea to invest more in education and that is being done reasonably well, but as in all things more is needed.
Yes the results can be a car crash at times - my favourite analogy for this is our management structures.
Would we ever take a literate organisation like the Washington Post or Harper Collins and put a layer of senior management in place who were totally illiterate from birth? If we did do we think they would make sensible decisions, empower those organisations? No.
Yes there is a lot of shiny new thing, going on. But that's because there are a lot of people doing software - and many of them select for being good at marketing too. The real big software projects tend to select for conservatism married to pragmatism- look at Linus on a mailing list or the PEP process, or Debian.
So yeah software needs to sort it self out - and professional bodies will start to solidify (personally I feel a lawyer like body, concerned with managing the course of OSS code used in government is the most beneficial), and until then don't try and fix the world or boil the ocean - just focus on making your coding practises as good as possible, even if your boss is making crazy calls.
(#) Wanted: Better analogys - can you write pithy phrases that sizzle like ice cream on a griddle? Contact the author in complete confidence today.
You're mixing up the content of writing with the methods of writing them. Writing "a rose by any other name would smell as sweet" is still as poetic when using fridge magnets.
Because of that, you fail to see that modern writing systems were massively improved by making it more accessible than what came before it, and that this has had a major impact on literacy levels as well. We started with Scriptio Continua[0]. We had to invent spaces and punctuation. Originally we just had capital letters. And so on.
And if we look beyond the West (which we should), you will encounter Hangul, which is a phonetic alphabet that points out how arbitrary our letters really are and replaces it with something more systematic and easy to learn[1][2].
Sure, mastering writing is hard. Basic literacy and writing systems do not have to be.
[0] https://en.wikipedia.org/wiki/Scriptio_continua
[1] How Korea crafted a better alphabet - History of Writing Systems #11 https://www.youtube.com/watch?v=j9hzK0K1L4I
[2] Learn to read Korean in 15 minutes, http://www.ryanestrada.com/learntoreadkoreanin15minutes/
it does not matter if one is born in china or UK or korea - children are given huuuuge amounts of training to become literate. We don't dumb down the writing system, and we don't dumb down the literature written in the system, we spend more on training the kids.
Similarly with programming - we should not look for some uber click and go system that makes it easier - we just have to work harder.
Fridge magnets are just a placeholder for "trying to make reading and writing easier without actually learning to be literate". I mean if someone writes in korean no one suggests they are not literate because it's easier. but if someone can only communicate by drawing, like some ultra-fast pictionary competitor, it's hard to say they are literate.
There is no such thing as a programming alphabet.
In programming, there's no tool that allow non-priests to automate repetitive tasks over data sets (though there are some simple trigger-based programmable actions, like Automator or ITTT).
The closest thing allowing end-users to handle repetitive data manipulation is the Spreadsheet, but it requires manual input-output of data, and it isn't easily integrated with actions in the operating system or other applications.
Reading and Writing we have sorted. We can write on Facebook ok. what we cannot do is arrange our data so it can be iterated over by our script / agents.
Literature is accessible and as a society we value the importance of ensuring that everybody has access to the power it provides. You can either believe that programming is for the elite /or/ that it is like literature.
no it's it for the elite. see me for examples.
I think the problem is: Programming is designed for and by a small group of programmers who enjoy making it difficult.
The solution is to create industrial societies / institutions / guilds etc. who can encourage a transition away towards simpler and more accessible programming approaches.
Otherwise the idea is completely not constructive.
Programming is always a tradeoff between ability and knowledge of the programmer, time he can invest, performance the product must deliver and the unpredictable future of the product. Nothing can be perfect because nothing is perfect.
Generally speaking, complexity of a software system is inversely proportional to the collective skill and experience the development team.
If I want quality products or services, in any field, I shouldn't expect to get it from the lowest-cost option.
We started already: work is afoot to throw away C, C++ and JavaScript, the main culprits. Rust is a good stab at all three.