Recently I stumbled across a three part article: The Blacksmith and the Bookkeeper, Part 1, Part 2, Part 3. The premise is that programming as a profession is doomed to go the way of the blacksmith, that it will create the tools that eventually bring about its end. In the article, Max Goff presents a hypothesis about why blacksmiths faded away while bookkeepers thrive today: In summary, blacksmithing was easily automated, streamlined, or assembly-lined, while bookkeeping required creative application of knowledge. Goff then posits that programming (except for embedded programming) falls into the same general category as smithing: Something that can be easily described and automated. But I think there are essential skills of programmers beyond being able to decipher assembly language or Java or Perl, and that those essential skills will be necessary for quite a long time.
I can easily see how he can come to his conclusion--the IT industry seems to be shrinking at an alarming rate, H1-B's and exported jobs are displacing native workers, and the premium that the average programmer once commanded has dwindled and become much more modest. In fact his conclusions are not unique: Another article in USA Today (Endangered species: US programmers) puts forth similar conclusions. It's not the first time I've encountered this concept. I've fielded questions on this topic from various friends and family members for years, in fact. And every new programming automation tool seems to herald the end of programming; there have been enough "silver bullet" software products and methodologies touted in the last twenty years to slay an army of werewolves, and yet none has lived up to the promises of its press releases.
As someone who has been a professional programmer and software engineer since 1987, I personally do not see the demand for my skills decreasing any time in the near future. Why am I not worried? In part because, as a video game programmer, my skills are in the minority--Goff makes reference to "Embedded Java" programmers in his article as being an exception to the rule, but I would put forth that many more programming specializations are going to need programmers for the foreseeable future.
With the dot com boom, the demand for programmers went way up along with their salaries, and many new programmers were being cranked out by universities. Many more companies than the supply of programmers could satisfy "needed" to have a Web presence as soon as possible, and so the demand created as if by magic a non-sustainable number of extra IT positions. With the bust, the demand necessarily goes down--partly because of market saturation, in that most companies now have a Web presence, and partly because putting an application up on the Web is a mature, understood process, and for a vast majority of typical Web sites, is not innovation--it's often just assembling the correct puzzle pieces at this point. I would guess that even in the heyday of smithing that during, e.g., wartime many more blacksmiths were trained and highly paid to produce weapons and armor than could be supported when the demand for hammered iron was lower. So part of the problem is certainly related to this: There are simply too many programmers for the positions that are available.
So will there be enough jobs for everyone who wants to make a living as a programmer? No, probably not. Which sucks for the thousands who lined up to get CS degrees for guaranteed high pay and job security--another broken promise of the "New Economy." The secret to getting steady work in the field is to find a niche where the programming that needs to be done isn't one that isn't easily generalized. I think this is a corollary to Goff's premise, in fact: If the job you're doing is something repetitive and predictable, if there's no (or very little) adaptation necessary for each new project, then it's going to be in the greatest danger of being automated or eliminated. There's more skill in adapting to a dynamic situation than in assembling components by rote. And having a skill that's in high demand is what raises your paycheck, not something magical about the titles "Programmer" and "Software Engineer". We're not just entitled to a high paycheck due by virtue of being able to speak Java or (in my case) C++. The high paycheck comes from providing a lot of value to a business.
This argues for restricting H1-B access, since the supply is truly greater than the demand for some positions. It turns out, though, that in niche industries like video game programming, where you really do need highly specialized talent that really is hard to find, even today, the H1-B applicant will often be the only one qualified for the position. Not just the cheapest one, but the only one. So it's not a simple issue--preventing H1-B access can prevent some companies from finding employees with the right skills. Which hurts the economy, etc. For a similar reason, video game outsourcing isn't thriving: I know that it does exist, though for the most part it exists as complete game development houses or component providers.
I think that another real problem facing the army of programmers out there right now is that many are working on similar problems, and that those problems are predominantly solved. Most Web sites and custom applications use the same features, many use off-the-shelf back ends with similar collections of components organized in slightly different ways, and one or two programmers can now do in a short time what it once took a small army of programmers many months. Note though that it still does require those few programmers--for the simple reason that programmers are trained to break a problem down analytically and debug it when it doesn't work as expected. Even if the programmers aren't manipulating anything like code, they will still be needed--maybe they'll be much more efficient, but they'll be around. Of course when they're more efficient, there won't be as many jobs...see above.
When a problem domain is well understood, however, it can often be analyzed, packaged, and automated into a wizard that you don't need a programmer for--as Paul Graham describes in his essay Beating the Averages, you can automate the process of creating an entire E-Commerce site. Now his software is used in Yahoo Stores, and while most of the Yahoo stores tend to have a similar look, they look professional and, with a custom graphics layer, would rival many custom-generated sites. In one wizard on a Web site, in under an hour, a person who knows nothing about programming can assemble an entire custom store with inventory management, payments, shipment tracking, and whatever other features Graham's group was able to squeeze in--with no direct programmer interaction at all. You might think I'm undermining my own argument here. But note this is a single problem domain, and however well it solves the Generic E-Commerce Site problem, it won't help in the slightest with creating a piece of software that tracks shipping containers for an export company, or that analyzes communication traffic to optimize switching, or any number of unrelated problems that still will need programmers until those problems are well understood, at which point they could potentially be automated and the programmers move on to new problems. And the Yahoo Store generating software still needs a team of programmers--who maintain 20000+ web sites rather than just a handful.
I do believe that companies are right to not want to pay premium salaries for programmers who are not really creating anything new. So programmers should always endeavor to keep their minds sharp, and always be in the position of creating new things rather than just pushing components around. I'm not against components--use components or any other tool that gives you greater leverage by all means!--just be sure you're personally adding value and not just running wizards. If a programmer isn't flexible enough to be able to adapt to a new industry or specialization, then unfortunately that programmer has become a blacksmith of programming, a specialist fit for one job on a programming assembly line, and unfortunately that's the position that's easiest to outsource or otherwise replace. So what you need to do is either reinvent yourself as a programmer, or reinvent yourself as something else. But don't just feel sorry for yourself that the bandwagon that you jumped on is overcrowded and that people keep falling off as its carrying capacity shrinks.
There are an infinite number of potential problem domains out there--and we're nowhere near putting together a tool that can solve any large fraction of them. To be able to generate a completely new program (presuming that the program is non-trivial, and understanding that what is meant by "trivial" evolves over time), you need to understand the motivations and needs of the people who will be using that program, distilling those concepts into a specification, sub-divide the specification into whatever logical units you have at your disposal, refine the implementation to improve its usefulness, and debug it when it doesn't work quite correctly. And these are the skills that epitomize a programmer or software engineer. I don't care if the "programming" is done by writing lines of code, dragging boxes around, talking into a microphone, or projecting thoughts into a brain-scanning UI. Programming will evolve. The only way to eliminate programmers from the equation entirely is to replace them with human-like AI entities with a breadth of understand of human endeavors, which, if created, would certainly do more than just put programmers out of business. Programming won't be the major that people flock to for easy money, certainly, but its oft-reported imminent death has been greatly exaggerated.
Subscribe to:
Post Comments (Atom)
3 comments:
Mario,
A very good point--the niche of "video game programmer" is small. On one survey I saw a while back, only 2% of people who considered themselves software engineers or programmers described themselves as video game developers. And I certainly see things from my own viewpoint, that's undeniably true.
And it's true that video game jobs are not common in Iowa or Nebraska. And as I mention in my article, there are too many people seeking too few jobs in IT at large. Some people, such as yourself, find a way to leverage your skills as something other than a programmer. If I were forced to move away from the San Francisco Bay Area, I would probably also be forced to reinvent myself. I have actually contemplated doing just that, though I'm in a good position at the moment.
But articles like Goff's foresee the end of programming, whereas I see it as simply a market shift. I think that as a result of the boom, we ended up with a huge surplus of a particular category of programmer, and that, within that category, which may encompass a majority of the industry, there will be a lot of people out of work and/or changing careers. Which is often painful for those involved, but inevitable under the circumstances. Maybe it is the end of easy money flowing from every computer science degree, shouldn't surprise anyone any more than the market crash, something many people expected but barely anyone acted as if it were every going to happen.
I think that, rather than blacksmiths, the correct analogy may be longshoremen. Their history isn't as long and celebrated as that of the blacksmith, but may be more relevant. A longshoreman is someone who helps to unload and load goods from ships, and this used to be done one human-liftable box at a time. But then along came the innovation of container freight, so that one person in a crane could do the job of dozens--and a large quantity of longshoremen lost their jobs. The remaining jobs paid well, however, and longshoremen still exist. No one called longshoreman an endangered species, no one calls them extinct now. Heck, if we had a union as good as theirs, the programmers who still have jobs would be making MORE money instead of less as a result of their increased productivity, but that's another can of worms.
There probably aren't many longshoreman jobs in Nebraska or Iowa either, come to think of it. But I bet crane operators can find work there. If I were to try to find work there, it would likely be work-from-remote that was based elsewhere--industries tend to accumulate in particular areas, and neither Nebraska nor Iowa show up on my radar as software development centers. Now if you said Portland, Vancouver (Washington or BC, take your pick), Seattle, Redmond, Denver, Dallas, Austin, San Diego, or any of a dozen other cities, I would say that software development work for programmers is available. Probably in every city serviced by craigslist.org, in fact.
So my point is not that people won't be losing jobs, and that does suck. My point is that when the dust settles, there will still be plenty of people out there doing what I call programming--not just 10,000 hobbiest blacksmith-programmers--for many, many years to come. And that is fundamentally how I disagree with the original article.
I agree with your points as to why the original statement is incorrect. If the only requirement is to push modules together to build applications, then use of programmers is going to go down as end users are able to build applications themselves in the same way that the development of direct-dial phones caused the elimination of most telephone operators.
The problem is that for many problem domains the solution is not merely the slinging together of modules, it is having the knowledge and experience of how to implement a solution to the problem.
You can buy "Commercial Off The Shelf" (COTS) applications for a lot of problem solutions and perhaps many of them will solve a number of problem domains that are easily implementable and will be satisfied by a "cookie cutter" solution of a packaged application. But if either the standard applications won't work in your case or there is no standard application for your problem, your only solution is to hire a programmer to solve it.
Another thing being that also, if we have components that can be put together to build applications is that a professional can use their training and expertise to use them to reduce the amount of work they have to do.
The existence of easy to use components does not necessarily eliminate the need for people to do the same job; the existence of cake mixes hasn't stopped people from buying a cake baked at Safeway. The existence of packaged food available for sale in grocery stores doesn't cut down on the use of mass-market commercial component cooked food sellers (Burger King, Mcdonalds, Wendys), of custom developers of cooked food sellers (Eat at Joes), of custom pre-packaged food for instant consumption (Frozen Burritos which can be microwaved at 7-11), or all the other ways people buy food. Does the existence of grocery stores spell the death-knell for McDonalds, or does McDonalds mean the end is near for Safeway?
It's also forgotten that the development of component-based software and software tools such as GUI development environments do something else; they increase the productivity of a professional programmer, who can now produce more work at less cost, which means someone can have a computerized solution to something where it was otherwise unaffordable before, leading not to less demand for programmers but to more demand as new problems requiring solutions that do not exist have to be developed. And if the solution does not exist because it has to be thought of, it can't be automated.
Machine production of machine tools is basically a "low hanging fruit" that was easy to do. That which requires specialized ability and knowledge is a skill-based occupation that is not likely to be automated.
I got into programming in 1976 and I've been a professional since 1978. And while I've seen how things are done change as computers became less expensive, more powerful and more ubiquitious, the amount of work needing to be done has not gone down, it has increased and the market for computer development has increased by huge, huge leaps and bounds. If this was not the case, we should have seen an overall drop in the software purchasing market as all of the work that needed to be done was done by existing COTS applications. No such drop has occurred.
Paul Robinson <paul @paul-robinson.us>
Post a Comment