The Falacy of Very High Level Languages

During my career as a systems programmer, I interviewed hundreds of candidates and was intimately involved with dozens of development projects, from one man teams to teams of hundreds, writing in binary, assembler, C, and various "high" level languages, and a few very high level languages,.  Over the course of all of this one of the things I realized was that language tools, such as big libraries, extremely high levels of abstraction, object orientation, and quite a few other "tools" are at best crutches, which can actually get in the way of solving the problem.  During the 70s and 80s, a bunch of people noticed that developers writing in assembler had a much higher success rate than developers working in BASIC or several other high level languages.  Without thinking very carefully, they blamed the particular high level language--for example, lack of structured programming tools.  But that's not the problem.

I spent quite a bit of time thinking about this.  What's happening, I think, is that the various tools promote barriers to entry.

On this chart, every programmer/development environment has a point.  A poor programmer is toward (or off) the bottom, better ones toward the top.   Working in a lower level language puts you toward the left, higher level toward the right, as exemplified by a few languages listed across the top.    As everybody knows, working in a lower level language makes it harder to get a program working than in a higher level language.  I think there are actually two thresholds at work.  The lower threshold, the lower, blue line, is the "can get it limping" level.  If you're below that level, you'll never get anything to work at all.  As you can see, a lot more programmers can get something going at the higher level languages than at the lower level.

The higher, red line, is the "good program" level.  If a programmer is working below the red line, they'll never produce something good.  The red line is at a very shallow slope for a simple reason:  The things that make a good program--critical thinking and analysis, memory, good planning, deep grasp of the problem and effective use of approaches to solving it--are largely independent of programming language or style.   If your thinking is muddled, you won't do a good job, irrespective of language, tools, design approach, or whatever.

The key, and the reason the "best" programmers seemed to be working in the hardest language, is that assembler provides such a high barrier to getting anything working at all that most of the muddled thinkers can't ever get anything that even looks like it's working.  There's not much space between the red and blue lines when you're working in assembler.   The lower level languages allow more programmers to look like they're producing something useful, while not actually doing much.  90% of the productive work is done by the 5 or 10% that are above the red line.

Up until about 1990, a programmer's ability to work in assembler was a pretty good calibration of their overall competence.   A good assembler programmer working in a higher level language would simply be more productive.  But after that time, astonishingly few professional programmers really understood the machine.  By 1997, it was rare to find a entry level candidate for an operating systems team that had more than a smattering of assembly experience.  And the higher level languages allow a lot more muddled thinkers into the process.  In the 70s and 80s, we couldn't tolerate much muddled code into the final product--there was some, but memory and disk space was too precious.  Nowadays, nobody seems to care.  The muddled thinking--and remember that with today's very high level languages, which functionally consist of gluing libraries together, that's an extremely high percentage--becomes a very high percentage of the shipping project.  One consequence of this is that you must download and install 700MB to update your telephone.  If all the programmers involved were working above the red line, it would probably be about 20MB.