Software and Mind  by Andrei Sorin — related articles

The spread of software mechanism

This article is a copy of the section with this title (in chapter 6 of  Software and Mind), which is too short to include as a separate book extract. (The book, as well as individual chapters and sections, can be downloaded free at www.softwareandmind.com.)

The study of software mechanism reveals some disturbing trends. We note a marked shift in our preoccupations: from solving real problems, to dealing with the problems created by the tools we invent to solve these problems. We have to address more and more issues, and these issues are becoming more and more complicated, while moving further and further away from our real concerns. Each issue begets new kinds of problems, which did not exist before but which are now urgent, because they must be solved before we can return to the original problem.

Simple programming problems – which require only a human mind and experience – gave rise to theories of programming, which evolved into complicated methodologies and programming tools, and finally became the monstrous development environments we see today. What started simply as programming became a preoccupation with programming theories and languages, with tools and environments. We also note a tendency to spread this inefficiency into broader domains: from the world of programming itself to the organizations that use software, and then to the rest of society. Issues that ought to concern only programmers end up affecting the users of their applications, who must now deal with some new, software-related problems in addition to their own problems; and these new problems affect then those people with whom the software users are involved. In the end, the entire society is spending more and more of its resources solving spurious, software-related problems, instead of addressing real concerns.

These three trends – making activities more complicated than necessary, shifting the preoccupation from real to spurious issues, and spreading the inefficiency into broader domains – are related, of course. They are different aspects of the same phenomenon, different manifestations of the delusion of software mechanism. Let us briefly examine this phenomenon.

Since the mechanistic software theories are invalid, the software practitioners who follow them keep failing. Their belief in software mechanism acts then as defence, as a way to deny reality. The reality, we saw, is that their failures are due to incompetence – an incompetence fostered by the very theories they follow. For, in addition to leaving unsolved the programming problems they promise to solve, the mechanistic theories prevent practitioners from gaining the expertise that could solve them.

There is no limit to the number of trivial issues that we can contrive if we believe that any problem can be solved by breaking it down into simpler ones. By replacing the original, complex problem with a multitude of isolated and relatively simple issues, ignorant practitioners can delude themselves that they are working toward the solution of the complex problem. They have redefined, in effect, the challenge of programming: from developing software to solve real problems, and at the same time improving their skills, to searching for ways to reduce these problems to simple ones that match their limited capabilities.

The mechanistic doctrine, thus, helps the software practitioners to rationalize their failures. Instead of the difficult task of creating and maintaining applications, they can now perform activities so simple that they cannot possibly fail. The problems that the applications were supposed to address may remain unsolved, but the new problems always have solutions, so the practitioners are pleased with their accomplishments. In the end, the belief in software mechanism permits them to shift the definition of their profession, from solving real problems to solving their own, simpler problems.

What is left is to persuade those who trust them – their employers, and the rest of society – that what they are doing is the utmost that can be accomplished in the domain of programming. But this is easy in a society dominated by mechanistic beliefs. The concepts of software engineering are deemed “scientific” – because based on mechanistic principles – and are therefore readily accepted by everyone. Given our mechanistic culture, it is the truth that is hard to accept. We have enough evidence that the most important part of programming is knowledge, talent, skills, and personal experience; in other words, the traditional form of expertise. And yet, it is the fallacious theories and the programming substitutes based on them that are promoted in computer publications, taught by professors and gurus, and endorsed by institutes and associations.

Permitting the software practitioners to fool us with their mechanistic theories is no different from permitting researchers to fool us with mechanistic theories in the human sciences. We saw in previous chapters how the mechanistic doctrine has corrupted academic research in fields like sociology, psychology, and linguistics, turning in effect these disciplines into pseudosciences. Invalid mechanistic theories are being pursued for years and decades, and are then abandoned only to be replaced with other mechanistic theories. The “research” performed by these scientists is merely a preoccupation with the spurious, mechanistic problems they themselves keep creating when breaking down the complex phenomena of mind and society into simpler phenomena. They never explain the actual phenomena; but, despite their delusions and their failures, we continue to trust them, and the universities, and the mechanistic philosophy. It is not surprising, therefore, that we have come to trust in the same way the software practitioners, even as we see them preoccupied largely with the spurious, mechanistic problems they themselves keep creating, and despite their failures.

So those who depend on software learned to deny the reality of software failures just like those who create the software. Software users can see the failure of their applications as clearly as the software practitioners can see the failure of their theories. But, just as the belief in software mechanism helps the practitioners to rationalize their programming failures, this belief helps now those who depend on software to rationalize the failure of their applications.

The software practitioners, we saw, rationalize their failures by expanding the original problem into a vast and complicated array of trivial activities. The real problem – creating and maintaining software applications to address business or social issues – is a well-defined one. It is a difficult problem, however, requiring much expertise, so the software practitioners hide their incompetence by replacing it with a great number of isolated, simple problems. They can then concentrate on these problems, and delude themselves that they are making progress toward the solution of the original one. This technique has been so successful in the domain of programming that the users of software have now adopted it, in order to rationalize their own failures. Instead of addressing real issues, they are modifying their activities and lowering their expectations to match the inferior applications they depend on.

The software propaganda has succeeded in convincing us to accept a state of affairs that no one would tolerate in other domains. Failures that would be preposterous in manufacturing, or construction, or utilities are considered normal for software applications. The users of applications notice that programmers are using complicated methodologies and development tools, and are calling themselves “software engineers,” and conclude that they are like other professionals. Instead of being outraged by the inefficiency and incompetence they note in software development, the users of applications allowed the creators of those applications to persuade them that software-related problems must form an important part of their own preoccupations.

Thus, assured that the programming they see is the only kind of programming possible, software users started to expand their own problems into a vast array of trivial, software-related activities. Solving software-related problems, they are now convinced, is the only way to make progress toward the solution of their real problems. And the software elites encourage this delusion by telling them that their software preoccupations are a new form of expertise, critical for success in today’s world.

                                                                    * * *

When studying the spread of software mechanism, and the resulting incompetence and corruption, it is useful to distinguish three stages. The first stage, now complete, involves the world of programming itself: the programming profession has been almost totally destroyed by the mechanistic dogma, and the inefficiency in those activities directly related to programming exceeds 90 percent (and is sometimes as high as 99 percent). To put this differently, if we had true professionals instead of the present programmers, society would pay less than one tenth of the current cost to derive the same benefits from software as it does now. (In fact, with true professionals we would derive greater benefits at one tenth the cost, because the present programmers are far from delivering the best possible applications.)

Programming expertise has been redefined, as I have already remarked, from knowledge of programming to knowledge of ways to avoid programming. Solving a problem simply through programming is considered old-fashioned, while attempting to solve it by means of development environments, ready-made pieces of software, and other high-level concepts is seen as modern and professional, no matter how inefficient and expensive is the result. Programming expertise, in other words, means expertise in the use of substitutes for expertise. The responsibility of programmers is limited to operating software devices, or following methodologies, or being aware of the latest theories. They are not accountable for their work: what matters is not whether their applications are useful or not, but whether their activities conform to the current software ideology.

When an application fails, no one is blamed. People cannot be blamed when their responsibility is limited to operating devices, or to following rules and methods. The conclusion is typically that they did not use the latest “solutions” or the latest “technologies.” So the project is simply abandoned, and another one is started – perhaps with a different development environment and different hardware – while all the people involved continue to be trusted and respected. (We will return to this subject in “Software Irresponsibility” in chapter 8.)

A nice demonstration of the first stage is provided by the evolution of Microsoft Corporation – the most aggressive of the software elites. Microsoft started with simple and useful programming tools, but the more successful it became, the larger and more complicated became its systems. Eventually, every tool grew into a huge environment, where the actual programming issues are hidden in a maze of unnecessary features, options, rules, and standards – all a result of raising the level of abstraction in software development. The effort involved in using Microsoft development environments (and any other environment that depends on the Microsoft operating systems) is due largely to the environments themselves, not our real programming needs.

Microsoft has attained, thus, the goal of the first stage: to destroy the possibility of intelligent and responsible programming by forcing programmers to waste their time with spurious problems; to prevent programmers from improving their skills by making their work dependent, not on personal knowledge, but on development environments; and, ultimately, to reduce all programmers, regardless of experience or potential, to mere operators of software devices.

                                                                     * * *

The second stage, now well under way, involves the world of business: corporations, governments, financial institutions, and the like – those organizations that have been heavy users of computers for many years, as well as those that started more recently. Workers in these organizations are undergoing now, in their own fields, the same indoctrination as the one that led to the destruction of the programming profession. And as a result, the same incompetence, inefficiency, and irresponsibility that characterize the world of programming are increasingly affecting all business. Although we are observing now office workers rather than programmers, and the use of software rather than its development, the similarity of this stage to the earlier one is striking.

The second stage started when the users of software realized that programmers did not, in fact, provide an acceptable level of service: new applications took too long to create, or were never finished, or were inadequate; simple modifications that should have been implemented in a day or two took months, or were never done, or did not work. Frustrated by this state of affairs, and convinced that this was the only kind of programming possible, the users fell into the same trap as the programmers themselves: they accepted the solutions proposed by the software elites – the same solutions that had led to programming incompetence, and hence to their current problems, in the first place. As before, the solutions consisted, not in improving anyone’s skills, but in means to avoid the need for programming. Incredibly, the elites turned now to the users themselves, offering them software devices that would eliminate their dependence on programmers: ready-made applications and, when these proved to be inadequate, user-oriented software development tools.

Asking users to depend on generic applications, or to develop their own, was tantamount to acknowledging the incompetence of programmers, and hence the failure of the mechanistic programming theories. No one recognized this obvious fact, however. So the same elites that were preventing programming expertise were seen now by their victims – by the users of software – as saviours. The incompetence of programmers, we saw, is due to the mechanistic theories, which force them to depend on aids and substitutes instead of improving their programming skills. They are restricted to mechanistic knowledge, which is the same as staying forever at the level of novices. By invoking the same theories, the elites could now deceive and exploit the users of software, and prevent expertise also in other occupations.

The first end-user aids were simple software tools for database query, reporting, spreadsheets, and the like. These tools, needless to say, could not fulfil the promises made for them: they could not be a substitute for serious applications. So the users found themselves dependent on these aids, and on the software companies behind them, even as their needs remained unsatisfied. But, again, no one questioned the software concepts leading to these aids, nor the honesty of the elites. It was the aids that were suspected, so the same process of expansion that had occurred earlier for the programming aids started for the end-user aids. The simple tools grew into large office systems, complete with programming languages, heavy instruction manuals, and proficiency courses. Each new version made them more complicated, until they finally became the monstrous software environments we see today.

These systems, clearly, are the counterpart of the development environments employed by programmers. Also like the development environments, they are fraudulent, in that they do not eliminate the need for programming expertise: to create serious applications, their users need almost the same knowledge as programmers. They are called productivity systems, but this is a spurious term, as its true meaning is the opposite of its literal one. The purpose of these systems is to reduce productivity, by consuming everyone’s time with software-related activities. And they accomplish this by replacing the dependence on personal knowledge with a dependence on software devices. Like programmers before them, workers in various fields are now prevented from practising their profession and from improving their skills. Increasingly, the only thing they learn is how to operate software devices: how to select and combine functions from a range of alternatives.

Just as the programming environments can replace only the easy parts of software development, the office systems can replace only the easy parts of software use – the parts requiring mechanistic knowledge. No matter how elaborate they are, these systems cannot replace the parts demanding non-mechanistic knowledge, and hence human minds. The real problems, thus, remain unsolved, but everyone believes that the only answer is to acquire even more software devices. Like programmers, office workers now remain at novice levels, because, instead of practising, they waste their time with the problems generated by their software tools. Just like programming expertise, the expertise of office workers has been redefined to mean the skill of using substitutes for expertise.

In one occupation after another, the responsibility of people is being lowered, from the traditional one – solving a real problem, making a valuable contribution – to merely knowing how to deal with software-related issues. In some occupations, the simple skills needed to operate a particular software device – skills that can be acquired by almost anyone in a few days – are already considered more important than a lifetime of experience.

A demonstration of the second stage is provided, again, by the evolution of Microsoft Corporation. Having destroyed the concept of expert and responsible programming, Microsoft turned to the world of business. The purpose of its office systems is to raise the level of abstraction in office work; to enforce the perception that any task can be reduced to a combination of some built-in functions, thus making everyone dependent on software devices; to waste workers’ time with the problems generated by this dependence, thereby preventing them from gaining knowledge and experience; and, ultimately, to reduce all workers, regardless of position or skills, to mere operators of software devices.

                                                                     * * *

The third stage involves the spread of software mechanism into the rest of society – into our homes and personal affairs, in particular. This stage has just begun, and it is hard to predict how software mechanism will affect our life; that is, to predict what will be the equivalent, in our personal affairs, of the incompetence, the inefficiency, and the corruption that we see now in programming and in business. Judging by the previous stages, however, we can expect an ever-growing dependence on software environments and on the companies behind them; specifically, a dependence on systems that promise to solve our important problems while addressing in reality only the simple ones. In the guise of information, or education, or entertainment, more and more types of software devices will be invented, desktop and mobile, and we will spend more and more of our time with the trivial preoccupations engendered by their use.

Thus, we will have fewer and fewer opportunities to use or to develop non-mechanistic knowledge. Our definition of expertise, creativity, and responsibility will be degraded, in everything we do, to mean simply the skill of selecting and combining the functions provided by software devices. The goal of the elites now is to make all human activities as insignificant and inefficient as they have made programming and office work; to make us all as ignorant as they have made programmers and office workers, by forcing us to depend on their systems instead of improving our minds; and, ultimately, to reduce all human beings on earth to mere operators of software devices.

                                                                     * * *

The three stages overlap, of course, and this analysis may not be entirely accurate. The distinction is still useful, however, because it helps us to observe the progression of software mechanism: its expansion, outward, from the narrow domain of programming into all aspects of human existence. By studying the first stage, which is now complete, we can perhaps foresee the evolution of the other two stages. (This study is the subject of the next chapter, “Software Engineering.”)

We should try to imagine a society where all people are as inefficient and irresponsible in their pursuits as the software practitioners are in theirs today. And, by studying the past and current rate of expansion, we should try to estimate how long it will take the software elites to bring about this condition. The frightening conclusion would be that, within a few decades, the second and third stages will also be complete. So we should ask ourselves whether a modern society can function at all when everyone’s knowledge is at the low levels we see today in programming. Programmers can be as incompetent as they are only because the rest of us are willing to pay the cost of their inefficiency; that is, willing to support a software bureaucracy. But if we, too, were to be like that, who would support us all?

It is significant that the software elites are not content with merely exploiting society – something they have already accomplished, in the first stage. Their objective is not just to extract, through the software bureaucracy, vast amounts of money from society. In their plan of domination, the second and third stages are as important as the first, and they will not stop until every person is turned into an active member of the software movement.

Like all totalitarian ideologies, the software revolution is a mass movement. It is not founded on authoritarianism, but on mind control: people are not threatened, but indoctrinated; they must become devoted followers, and participate, on their own accord. Physical force is needed only against those who, despite the indoctrination, still fail to appreciate the benefits of the new social order.

Viewed in this light, the three stages of the software movement parallel the evolution of political totalitarian movements like Nazism and Communism. The revolution starts with a core of believers – who eventually become the elite, or the Party – and spreads outward, bringing increasingly broad segments of the population into its ranks. The revolution cannot end until every person and every event in society conforms to its ideology. But because the ideology is based on pseudoscientific, fallacious notions, it cannot actually work.

The initial phase, when only a small portion of society is involved, usually appears successful, and this gives the believers confidence in their utopian visions. (The software movement is currently in this stage.) The initial phase appears successful, not because the ideology is valid, but because the few who embrace it deceive and exploit the rest of society. As the movement spreads, however, more and more people are turned into bureaucrats who merely serve the ideology, and fewer and fewer are left who do real work and can be exploited. Society becomes increasingly corrupt and inefficient, and eventually destroys itself.