[Interesting weekend - for the first time in a while, I dropped in only long enough to post on Sunday; otherwise, I was off from Saturday morning until now. There really is life going on outside the Web.]
I have some thoughts about the essential paradoxes of the positions being taken by those in power, but I haven't worked them out thoroughly. So I'll write about something of which I was reminded in an article by Norm Matloff of UC-Davis, to which I was pointed in a comment by Citizen Carrie. Matloff writes:
The problem, of course, with any language change of this type is that it removes important and useful distinctions (I wrote a post about this months ago, referring to the words "legendary" and "historic" - I understand why the descriptivist linguists think of this as part of the natural process of language change, but what are we supposed to use for events or people that really are "legendary" or "historic"?). Once you call every entry-level position Software Engineer, you lose the ability to set the bar (and the salary) for a job that actually requires engineering skills.
To me, these terms are two elements of a spectrum to which a third term needs to be added: developer. I do not say that there was ever a time at which these terms were used the way I will propose, but I believe there was a time when we seemed to be moving toward a standard. Apparently, we are moving away now.
A programmer is someone who works at writing code for a computer to execute. They can most readily be defined as what they are not, that is, not developers or engineers. They tend to write code to the exclusion of all else unless required to do something else, their code tends to need rework to be made robust, and they enjoy the hacker mentality. They may be very productive if measured by raw lines of code, but come up well short if measured by effective lines of code. You might think that programmers are only valuable as a transitional state on the way to one of the other two categories, and that companies would have a strong incentive to help them move, but, in practice, that often doesn't happen. There are an amazing number of programmers who have 15, 20, 25 years of experience and are still just throwing code down, hoping it works.
A developer represents the next level of proficiency. This is someone who understands the interactions between various parts of a project, has some inkling of the business side and its requirements, and voluntarily does non-code activities (though reminders may be necessary). They have an idea as to what makes a professional; perhaps they've read one or more books on software development but have little interest in anything more than cherry-picking the hints they like to use. The better ones have some idea of development as an exacting discipline, and have confidence in their abilities in tackling new challenges.
An engineer represents the highest level of professionalism. They recognize that writing code, while still the most fun and satisfying, is a relatively small part of the job. Requirements need to be gathered; if the engineer isn't doing that themselves, they are reviewing them rigorously. Similarly, testing needs to be seen as an essential part of the process, and the engineer devotes a good portion of time to that. Most importantly, they see software development as more than just slapping down code to accomplish a task, but as a communication to the future, a way of explaining a business process through a computer language. No challenge is too great as long as the problem can be broken down and expressed as a sequence, and the tools used are whatever is most useful to the task at hand.
If you know anything about the current job market, you recognize that the engineer can have big trouble. There is a perception that the emphasis on quality is inimical to the goals of the business, namely, churning out product to the waiting market. Were there true engineers with any power working on Vista, it couldn't have been released in its initial state. Engineers, with their insistence on repeatable process and bulletproof standards, are seen as troublemakers (and I would grant there are times some engineers focus too much on the product and not enough on the "needs" of the company...but someone has to).
There are many engineers who believe that no one in software should call themselves "engineer," because there is no generally accepted licensure for software engineers. But, as you can see in my taxonomy, there is a fair amount of rigor in the definition. A software engineer should be someone who reads assiduously in the technical literature and the process literature. He or she should be as well-versed in The Mythical Man-Month as The Art of Computer Programming.
That we are inflating the title of a hack programmer into the title of a rigorous engineer is distasteful. The worst effect is that the person who has experience as a true engineer is competing against those who have merely possessed the title, and the latter will undercut the former on compensation every time. And the company will never know the difference, mainly due to the "evaluation crisis" (about which I wrote briefly before, and should probably write more). So not just the titles collapse, but the reward structure as well - and then there is no incentive to put in the work necessary to be an engineer, so we don't have any.
Then where do the jobs go?
I have some thoughts about the essential paradoxes of the positions being taken by those in power, but I haven't worked them out thoroughly. So I'll write about something of which I was reminded in an article by Norm Matloff of UC-Davis, to which I was pointed in a comment by Citizen Carrie. Matloff writes:
The job title Computer Programmer has for some years been in the process of being replaced by the title Software Engineer. Same job, fancier name. So don't be fooled by the fact that the BLS separates the two titles in their data.As someone who has been in this field for quite a few years, I'm disappointed to see this collapse of titles (I knew it was happening based on the ads I've been seeing, but I hadn't realized it was pervasive enough to be seen by academe).
The problem, of course, with any language change of this type is that it removes important and useful distinctions (I wrote a post about this months ago, referring to the words "legendary" and "historic" - I understand why the descriptivist linguists think of this as part of the natural process of language change, but what are we supposed to use for events or people that really are "legendary" or "historic"?). Once you call every entry-level position Software Engineer, you lose the ability to set the bar (and the salary) for a job that actually requires engineering skills.
To me, these terms are two elements of a spectrum to which a third term needs to be added: developer. I do not say that there was ever a time at which these terms were used the way I will propose, but I believe there was a time when we seemed to be moving toward a standard. Apparently, we are moving away now.
A programmer is someone who works at writing code for a computer to execute. They can most readily be defined as what they are not, that is, not developers or engineers. They tend to write code to the exclusion of all else unless required to do something else, their code tends to need rework to be made robust, and they enjoy the hacker mentality. They may be very productive if measured by raw lines of code, but come up well short if measured by effective lines of code. You might think that programmers are only valuable as a transitional state on the way to one of the other two categories, and that companies would have a strong incentive to help them move, but, in practice, that often doesn't happen. There are an amazing number of programmers who have 15, 20, 25 years of experience and are still just throwing code down, hoping it works.
A developer represents the next level of proficiency. This is someone who understands the interactions between various parts of a project, has some inkling of the business side and its requirements, and voluntarily does non-code activities (though reminders may be necessary). They have an idea as to what makes a professional; perhaps they've read one or more books on software development but have little interest in anything more than cherry-picking the hints they like to use. The better ones have some idea of development as an exacting discipline, and have confidence in their abilities in tackling new challenges.
An engineer represents the highest level of professionalism. They recognize that writing code, while still the most fun and satisfying, is a relatively small part of the job. Requirements need to be gathered; if the engineer isn't doing that themselves, they are reviewing them rigorously. Similarly, testing needs to be seen as an essential part of the process, and the engineer devotes a good portion of time to that. Most importantly, they see software development as more than just slapping down code to accomplish a task, but as a communication to the future, a way of explaining a business process through a computer language. No challenge is too great as long as the problem can be broken down and expressed as a sequence, and the tools used are whatever is most useful to the task at hand.
If you know anything about the current job market, you recognize that the engineer can have big trouble. There is a perception that the emphasis on quality is inimical to the goals of the business, namely, churning out product to the waiting market. Were there true engineers with any power working on Vista, it couldn't have been released in its initial state. Engineers, with their insistence on repeatable process and bulletproof standards, are seen as troublemakers (and I would grant there are times some engineers focus too much on the product and not enough on the "needs" of the company...but someone has to).
There are many engineers who believe that no one in software should call themselves "engineer," because there is no generally accepted licensure for software engineers. But, as you can see in my taxonomy, there is a fair amount of rigor in the definition. A software engineer should be someone who reads assiduously in the technical literature and the process literature. He or she should be as well-versed in The Mythical Man-Month as The Art of Computer Programming.
That we are inflating the title of a hack programmer into the title of a rigorous engineer is distasteful. The worst effect is that the person who has experience as a true engineer is competing against those who have merely possessed the title, and the latter will undercut the former on compensation every time. And the company will never know the difference, mainly due to the "evaluation crisis" (about which I wrote briefly before, and should probably write more). So not just the titles collapse, but the reward structure as well - and then there is no incentive to put in the work necessary to be an engineer, so we don't have any.
Then where do the jobs go?
10 comments:
This is an issue that's near and dear to my heart. Unfortunately, I seem to be lacking some basic knowledge in order to really be able to clarify my thoughts.
I think it boils down to what I perceive to be a (for lack of a better term) "de-professionalization" of the IT career fields. A lot of today's managers lack any understanding of how much value developers and software engineers can add to the processes, and tend to lump everyone together as programmers. Many IT professionals come from a variety of backgrounds and have graduated from well-rounded B.S., B.B.A., M.S. and M.B.A. programs. They seem to be horribly underutilized today, and tend to be lumped together in the same boat as people who have taken a few programming courses at a community college.
I absolutely agree with what you're saying. I have an MBA from one of the more prestigious schools (to go along with a solid if not so flashy MS in computer science), but I usually leave it off my resume because I'm tired of hearing I'm "overqualified."
One of the reasons, I think, is that good technical people tend to "cling" to their profession, that is, they see themselves as software people who happen to work for a company. Business is far more comfortable with people who believe in the primacy of the company.
I've seen this at conferences. In technical conventions, groups form around technologies; at business conventions, around companies. That's a social divide that's very hard to get by.
Furthermore, there's a fear based on history. 15-20 years ago, there were a lot of articles about how the information professionals were, because they were in the center of all the data, going to become the leaders in companies.
The finance, marketing, and sales folks, who have a lot more experience with corporate infighting, first peeled off their own fiefdoms to protect their data, then shunted the information professionals into more of a support role. That's the source of some of what you're referring to.
don't forget architects!
it has not been my experience that the remarkably able are lumped with the lunk heads, resulting in underutilization of potential. I work in "high tech" rather than IT and the difference might be major. IT is often wrote system management now. (I should know; sometimes my job is to write it.) Maybe people underutilized in IT need to change to "high tech" which is not pre-determined. I assure you that at that level the difference bewteen the idiots and the geniuses is apparent and important. So it could be a matter of IT losing its need for creative or brilliant people, and generally that's a good development for industry. You might be familiar with the Mort and Einstein software customer profiles. we talk about these a lot at microsoft when considering our market's needs. We seem to know the difference.
i do not share your observation that there is an "evaluation crisis" but i'm fortunate to work in places where engineering excellence is recognized and rewarded regardless of title. I'm surprised you choose not to mention your MBA. would you welcome a career that applied it?
mcfnord: I wasn't trying to exclude architects, I just don't feel that that term ever reached the mainstream enough to be included in my taxonomy (and I've always thought of it as somewhat functionally different).
As for "high tech" vs. "information technology," your point is well taken. I may be using a slightly outmoded terminology here; my definition of high tech has always included far more than IT, including biotechnology, materials science, and so forth. IT, to me, is more strictly computer-related. It does seem, now that you remind me of it, that IT is now becoming synonymous with systems support. I sense there may be a regional component to this as well, as "my" definitions could be hanging on a bit longer in the Midwest.
As for your good fortune to be in a place where talent is recognized and rewarded, well, that just isn't the norm, I'm afraid. I hate to mention the number of companies I've worked in or knew something about in which management has very little clue as to who can do the work and who cannot. It's a bigger issue than I can do justice to in a comment, but I stand on pretty firm ground when I tell you that your situation is a fortunate exception, not the norm. I could, and at some point probably will get around to, tell stories that would curl your hair as to the destructive nonsense that goes on, even in some very well-known and respected companies.
currently i am rather fortunate. i have worked for creeps, jerks, and crooks. we can relay anecdotes about creeps coming to power, or poisoning the well, or all that. but there's something so unreal about the thesis here. do you imply there's an enormous tax on potential due to organizational failure? perhaps organizationally there are typical traps that result in poor leadership or poor quality. but fundamentally, am i free? yes, i think so. and when i sense a creep, jerk, or crook, i run the hell away. it seems quite implausable that commerce as a rule organizes around failed ideas and people. the whole reason we leave commerce to private enterprise is so customers and employees both have a choice. and to wield that choice, i live by a simple creed: "Work with the best, starve the rest."
so i'll tell a story: my current boss has at least ten years of experience. she knows what i'm saying when i describe a problem, and she usually knows a great solution. she tells me routinely that she's glad i've chosen employment there. she insists on hiring great team members, and treating them with respect, providing individualized support. occasionally, she takes the team out to lunch. she prioritizes problems and demands solutions. she does not over-simplify. is it interesting? it's excellence in engineering management. but if negativity and blunder are on the marquee, then she deserves at least a mention.
my father used to rail on about his employer. now he has a lawsuit with them. i don't get the point. why stay in bad situations? let them implode! don't look back.
mcfnord: I don't entirely disagree with you here, but I don't think creeps, jerks, or crooks are necessary to poison the well. I've worked for some great people who were completely incapable of doing their jobs effectively, unable to tell who was productive and who was not due to their total lack of understanding of the work itself.
But, more importantly, there are some powerful institutional incentives which, added to ignorance, can get in the way of a proper outcome. When you have managers who are rewarded for cost-cutting, and they don't see a difference between the engineer with 20 years experience and the Bangalore new grad, no matter how well-meaning or personally fine that manager may be, you have a recipe for non-optimal outcomes for everyone.
And that's a critical part of this, the assumption that private enterprise is optimizing, when quite often it is satisficing (and the good-enough goals start with personal rewards for the higher-ups).
Your creed is great, but it requires a critical mass of "the best" which is, unfortunately, too rarely met.
i'm thankful that thoughtless quality-cutting kills a business, whether the manager understands this or not.
i do think private enterprise is optimizing, and often quite conscious of the kinds of risks you're describing, even if common pitfalls do reduce quality or efficiency.
i also know good-enough is an important commerical metric. so i don't see a problem with replacing an experienced individual with a new-grad (regardless of nationality) when the experience does not matter.
But that's assuming that the person making the decision has the ability to determine that experience does not matter, and that is not always the case (especially in the face of the personal incentives I referred to above).
you seem to feel the bulk of commerce is managed by an idiocracy. i suppose it's all relative. yes, people make errors. yes, there are typical pitfalls. but great spoils go to those who avoid errors and pitfalls. meanwhile you're discussing auto-correcting language software, while also asserting that last-rate grads are assigned the job. what happens after the money's spend and they've failed? we didn't get this far through idiocracy.
Now, now, not-A does not imply Z. I have gone out of my way not to say that executives are idiots, because most of them are not. What I am saying is that they have big incentives to take actions that are not necessarily positive for the long-term health of their enterprises, nor for the well-being of our nation.
We can choose either to change some of those incentives, but I'm not really comfortable with government trying to manage business at that level (though I think a re-look at the visa issue is probably warranted), or to begin to come to grips with what current trends are telling us about America's future. If you read through my blog, I think you'll see that the latter is a pretty consistent thread. This country is in for some changes, some of which will rock our very concept of ourselves and our expectations, and we need to adjust our views accordingly. (That's why I'm not a protectionist, as truly free trade is good; we just have to understand who will be affected negatively and figure out if and how we could mitigate that loss.)
Oh, and my post about error-detecting language had nothing to do with software, per se. I was exploring what a language with a certain linguistic "distance" would look like; computers didn't actually enter into it at this time.
Post a Comment