As I like to say, we are moving into a twenty-first century in which we are highly likely to spend a greater share of our collective income on:
- pensions (for our societies are aging);
- education (for the technological world in which we maneuver as adults is only getting more complicated);
- health care (what else are we going to be spending money on? Taylor Swift videos? Car elevators?); and
- information and information-like goods (for the share of things we value that are produced under conditions of roughly constant returns to scale and that are both rival and excludible looks like it is dropping).
Historical experience teaches us that whenever we try to supply any of these four needs via an un- or a lightly-regulated market, it does not go so well. This suggests that we are likely to be happy in the twenty-first century only if we shift our collective economic cognition and organization to place somewhat less emphasis on the market and more on... something.
The problem is that we have--or, at least, from where I sit I think we have--very good ideas about the success and failure modes of markets (i.e., Friedrich Hayek (1947), Individualism and Economic Order; Kenneth Arrow (1969), "The Organization of Economic Activity: Issues Pertinent to the Choice of Market versus Non-market Allocation"). We know rather less about the success and failure modes of politics (i.e., James Buchanan and Gordon Tullock, The Calculus of Consent; Mancur Olson, The Logic of Collective Action; Josiah Ober, Democracy and Knowledge: Innovation and Learning in Classical Athens).
And, IMHO at least, we know very little about the success and failure modes of bureaucracy.
So I would like to call for people to think, and think hard, so that a generation hence we know as much about the success and failure modes of bureaucracy and politics as we do about markets--and, I hope at least, have evolved some more tweaks to make all three modes of social organization and cognition less subject to failure.
Alas, I have no special insight into how to start thinking about these matters. But Cosma Shalizi has the best diving board I have found:
Review of Edwin Hutchins, Cognition in the Wild: Human beings coordinate their actions to do things which would be hard or impossible for them individually. This is not a particularly recondite fact, and the recognition of it is ancient; it is in the fifth book of Lucretius's De Rerum Natura, for instance.... The nineteenth century, and to a lesser degree this one, have witnessed a dramatic expansion in the numbers of us engaged in administration, bureaucracy, management, oversight--that is to say, in formally-organized tasks of collective cognition and control. We did not invent bureaucracy, the mainstay of the ancient empires, but we're much, much better at it... corrupt, inefficient institutions which work poorly; every election, Piffleburg [WI]'s citizens mutter something like "what do we pay taxes for anyway?" Yet to run any one of these institutions at the level of honesty, efficiency and efficacy which makes Piffleburg grumble would have demanded the full powers and attention of even the ablest Roman propraetor or Tang magistrate. That all of those institutions, plus the ones not restricted to a single city, could be run at once, and while governed by a very ordinary slice of common humanity, would have seemed to such officials flatly impossible.
The immediate question this raises, of why we are so much better at collective endeavors than the ancients, can be answered fairly simply. To a first approximation, the answer is: brute force and massive literacy. We teach nearly everyone to read and write, and to do it, by historical standards, at a high level. This lets us staff large bureaucracies (by some estimates, over 40% of the US workforce does data-handling), which lets us run an industrial economy (the trains run on time), which makes us rich enough to afford to educate everyone and keep them in bureaucratic employment, with some surplus left over to expand the system.
This would do us no good if our ideas of administration were as shabby as those of our ancestors in the dark ages, but they're not: we inherited those of the ancient empires, and have had quite a while to improve upon them (and improvements are made easier and faster by the large number of administrators and the high standard of literacy). Among the improvements are many techniques (standardized procedures, standardized parts, standardized credentials and jobs, explicit qualifications for jobs and goods, files, standardized categories) and devices (forms, punch cards, punch card tabulators, adding machines, card catalogs, and, recently, computers) for making the administration of people and things easier. (We've been over parts of this before, looking at James Beniger's book on The Control Revolution and Ernest Gellner's Nations and Nationalism.)
All this is in the realm of technique; when it comes to theory, we are quite at a loss. We can see, in a rough, common-sensical way, what makes us better at running things than the Romans were, but we don't understand how either they or us pull off the trick at all. That is to say, we don't really have a good theory about how collective action and cognition work, when and why they do, how they can be made to work better, why they fail, what they can and cannot accomplish, and so forth.
Intellectually, these are large, tempting problems; technologically, they have obvious relevance to the design of parallel and distributed computers; economically, they could mean real money, not just billions; and, in general, it'd be nice to know what it is we've gotten ourselves into.
Now, in a sense, this problem has been approached by many of the social sciences.... Much of the most interesting research on these problems has been done by economists. The great Friedrich Hayek (that is, Friedrich Hayek the profound social scientist, not to be confused with his evil twin, Friedrich Hayek the right-wing ideologue) was apparently the first to point out that markets perform a kind of collective cognition or calculation which would be beyond the scope of the individual actors in the markets. Since his time, the economists have devoted considerable thought to how the way a group is put together--its procedures, the distribution of power, resources, beliefs and preferences within it--effects the decisions it arrives at, the courses of action open to it. Some of this work, like Arrow's Social Choice and Individual Values and Olson's Logic of Collective Action--is now classical, and, under various names, it's an active, thriving area of inquiry....
[But still] we know next to nothing about how collective cognition works, or when it works, or how to make it work better; we have some ideas about it, but at best they've the status of artisanal rules of thumb...