Economics

Hayek Meets Information Theory. And Fails.

Modern economic theories of prices-as-information are seventy years out of date.

Share with your friends










Submit
More share buttons
Share on Pinterest

By Jason Smith

The inspiration for this piece came from a Vox podcast with Chris Hayes of MSNBC. One of the topics they discussed was which right-of-center ideas the left ought to engage. Hayes says:

The entirety of the corpus of [Friedrich] Hayek, [Milton] Friedman, and neoclassical economics. I think it’s an incredibly powerful intellectual tradition and a really important one to understand, these basic frameworks of neoclassical economics, the sort of ideas about market clearing prices, about the functioning of supply and demand, about thinking in marginal terms.

I think the tradition of economic thinking has been really influential. I think it’s actually a thing that people on the left really should do — take the time to understand all of that. There is a tremendous amount of incredible insight into some of the things we’re talking about, like non-zero-sum settings, and the way in which human exchange can be generative in this sort of amazing way. Understanding how capitalism works has been really, really important for me, and has been something that I feel like I’m a better thinker and an analyst because of the time and reading I put into a lot of conservative authors on that topic.

Putting aside the fact that the left has fully understood and engaged with these ideas, deeply and over decades (it may be dense writing, but it’s not exactly quantum field theory), I can hear some of you asking: Do I have to?

The answer is: No.

Why? Because you can get the same understanding while also understanding where these ideas fall apart ‒ that is to say understanding the limited scope of market-clearing prices and supply and demand – using information theory.

Prices and Hayek

Friedrich Hayek did have some insight into prices having something to do with information, but he got the details wrong and vastly understated the complexity of the system. He saw market prices aggregating information from events: a blueberry crop failure, a population boom, or speculation on crop yields. Price changes purportedly communicated knowledge about the state of the world.

However, Hayek was writing in a time before information theory. (Hayek’s The Use of Knowledge in Society was written in 1945, a just few years before Claude Shannon’s A Mathematical Theory of Communication in 1948.) Hayek thought a large amount of knowledge about biological or ecological systems, population, and social systems could be communicated by a single number: a price. Can you imagine the number of variables you’d need to describe crop failures, population booms, and market bubbles? Thousands? Millions? How many variables of information do you get from the price of blueberries? One. Hayek dreams of compressing a complex multidimensional space of possibilities that includes the state of the world and the states of mind of thousands or millions of agents into a single dimension (i.e. price), inevitably losing a great deal of information in the process.

Information theory was originally developed by Claude Shannon at Bell Labs to understand communication. His big insight was that you could understand communication over telephone wires mathematically if you focused not on what was being communicated in specific messages but rather on the complex multidimensional distributions of possible messages. A key requirement for a communication system to work in the presence of noise would be that it could faithfully transmit not just a given message, but rather any message drawn from the distribution. If you randomly generated thousands of messages from the distribution of possible messages, the distribution of generated messages would be an approximation to the actual distribution of messages. If you sent these messages over your noisy communication channel that met the requirement for faithful transmission, it would reproduce an informationally equivalent distribution of messages on the other end.

Get Evonomics in your inbox

We’ll use Shannon’s insight about matching distributions on either side of a communication channel to match distributions of supply and demand on either side of market transactions. Let’s start with a set of people who want blueberries (demand) and a supply of blueberries. These represent complex multidimensional distributions based on all the factors that go into wanting blueberries (a blueberry superfood fad, advertising, individual preferences) and all the factors that go into having blueberries (weather, productivity of blueberry farms, investment).

In place of Hayek’s aggregation function, information theory lets us re-think the price mechanism’s relationship with information. Stable prices mean a balance of crop failures and crop booms (supply), population declines and population booms (demand), speculation and risk-aversion (demand) — the distribution of demand for blueberries is equal to the distribution of the supply of blueberries. Prices represent information about the differences (or changes) in the distributions. And differences in distributions mean differences in information.

Imagine you have blueberries randomly spread over a table. If you draw a grid over that table, you could imagine deciding to place a blueberry on a square based on the flip of  a coin (a 1 or a 0). That is one bit of information. Maybe for some of the squares, you flip the coin two or more times. That’s two or more bits.

Now say you set up a distribution of buyers on an identical grid using the same process. If you flipped more coins for the buyers than the blueberries on the corresponding squares, that represents a difference in information (and likely an excess demand).

There can be an information difference even if there’s no difference between the results of the coin flips. For example, you can get one blueberry on a square because you flipped a coin once and it came up heads or you flipped a coin twice and it came up heads once and tails once. However as the number of coin flips becomes enormous in a huge market, the difference between the results of the coin flips (excess supply or demand) will approximate the difference in the information in the coin flips. This is an important point about when markets work that we will come back to later. It is also important to note that these are not just distributions in space, but can be distributions in time. The future distribution of blueberries in a functioning market matches the demand for blueberries, and we can consider the demand distribution information flowing from that future allocation of blueberries to the present through transactions.

Coming back to a stable equilibrium means information about the differences in one distribution (i.e. the number of coin flips) must have flowed (through a communication channel) to the other distribution via transactions between buyers and sellers at market prices. We can call one distribution D and the other S for supply and demand. The price is then a function of changes (Δ or “delta”) in D and changes in S:

p = f(ΔD, ΔS)

Price is a function of changes in demand and changes in supply. That’s Economics 101. But what is the function describing the relationship?  We know that an increase in S that’s bigger than an increase in D generally leads to a falling price, while an increase in D that is bigger than the increase in S generally leads to a rising price. If we think in terms of distributions of demand and supply, we can try

p = ΔD/ΔS

for our initial guess. Instead of a aggregating information into a price, which we can’t do without throwing away information, we have a price detecting the flow of information. Constant prices tell us nothing, but price changes tell us information has flowed (or been lost) between one distribution and the other. And we can think of this information flowing in either space or time if we think of the demand distribution as the future allocation of supply.

This picture also gets rid of the dimensionality problem: the distribution of demand can be as complex and multidimensional (i.e. depend on as many variables) as the distribution of supply. The single dimension represented by the price now only measures the single dimension of information flow.

Marginalism and supply and demand

Chris Hayes also mentions marginalism. It’s older than Friedman or Hayek, going back at least to William Jevons. In his 1892 thesis, Irving Fisher tried to argue (crediting Jevons and Alfred Marshall) that if you have gallons of one good A and bushels of another good B that were exchanged for each other, then the last increment (the marginal unit) was exchanged at the same rate as A and B, i.e.

ΔA/ΔB = A/B

calling both sides of the equation the price of B in terms of A. Note that the left side is our price equation above (p = ΔD/ΔS), just in terms of A and B (you could call A the demand for B). In fact, we can get a bit more out of this equation if we say

pₐ = ΔA/ΔB = A/B

We add a little subscript a to remind us that this is the price of B in terms of A. If you hold A constant and increase B (supply), the price goes down. For fixed demand, increasing supply causes prices to fall – a demand curve. Likewise if you hold B constant and increase A, the price goes up – a supply curve. However if we take tiny increments of A and B and use a bit of calculus (ΔA/ΔB becomes dA/dB) the equation becomes a differential equation that can be solved. In fact, it is one of the oldest differential equations to be solved (by Bernoulli in the late 1600s). However, the solution tells us that A is linearly proportional to B. It’s a quite limited model of the supply-demand relationship.

Fisher attempts to break out of this limitation by introducing utility functions in his thesis. However thinking in terms of information can again help us.

If we think of our distribution of A and distribution of B (like the distribution of supply and demand), each “draw” event from those distributions (like a draw of a card, a flip of one or more coins, or roll of a die) contains I₁ information (a flip of a coin contains 1 bit of information) for A and I₂ for B. If the distribution of A and B are in balance (“equilibrium”). Each draw event from each distribution (a transaction event) will match in terms of information. Now it might cost two or three gallons of A for each bushel of B, so the number of draws on either side will be different in general, but as long as the number of draws (n) is large, the total information from those draws will be the same:

n₁ · I₁ = n₂ · I₂

Rearranging, we have

n₁ · (I₁ / I₂) = n₂

We’ll call I₁/I₂ = k (for reasons we’ll get into later) so that

k · n₁ = n₂

Now say the smallest amount of A is ΔA and likewise for B. One bushel or one gallon, say. That means

n₁ = A/ΔA

n₂ = B/ΔB

i.e. the number of gallons of A is the total amount of A divided by 1 gallon of A (i.e. ΔA). Putting this together and rearranging a bit we have

ΔA/ΔB = k · A/B

This is just Fisher’s equation again except there’s our coefficient k in it expressing the information relationship, making the solution to the differential equation mentioned above a bit more interesting than being linearly proportional — now log(A) = k log(B) + b, where b is another constant. The supply and demand relationship found by holding either A or B constant and varying the other is also more complex than the one you obtain from Fisher’s equation (it depends on k). It’s essentially a more generalized marginalism where we no longer assume k = 1. But there’s a more useful bit of understanding you get from this approach that you don’t get from simple price signaling. What we have is information flowing between A and B, and we’ve assumed that information transfer is perfect. But markets aren’t perfect, and all we can really say is that the most information that can get from the distribution of A to the distribution of B is all of the information in the distribution of A. Basically

n₁ · I₁ ≥ n₂ · I₂

Following through with this insight in the derivation above, we find

p = ΔA/ΔB ≤ k · A/B

Because the information flow from A can never be greater than A’s total information, and will mostly be less than that total, the observed prices in a real economy will most likely fall below the ideal market prices. Another way to put it is that ideal markets represent a best-case scenario, one out of a huge space of possible scenarios.

There’s also another assumption in that derivation – that the number of transaction events is large, as we mentioned before. So even if the information transfer was ideal, the traditional price mechanism only applies in markets that have a large volume of trade. That means prices for rare cars or salaries for unique jobs likely do not represent accurate information about the underlying complex multidimensional distributions of market supply and demand. Those prices are in a sense arbitrary. They might represent some kind of data (about power, privilege, or negotiation skills), but not necessarily information about the supply and demand distributions or the market allocation of resources. In those cases, we can’t really know from the price alone.

Another insight we get is that supply and demand doesn’t always work in the simple way described in Marshall’s diagrams. We had to make the assumption that A or B was relatively constant while the other changed. In many real world examples we can’t make that assumption. A salient one today is the (empirically incorrect) claim that immigration lowers wages. A naive application of supply and demand (increased supply of labor lowers the price of labor) ignores the fact that more people means not just more labor, but more people to buy goods and services produced by labor. Thinking in terms of information, it is impossible to say that you’ve increased the number of labor supply events without increasing the number of labor demand events, so you must conclude A and B must both change. More immigration means a larger economy; the effect on prices or wages does not simply follow from supply and demand based on a population increase.

Instead of the simplified picture of ideal markets and forces of supply and demand, we have the picture advocates on the left (and to be fair most  economists) try to convey of not only market failures and inefficiency but more complex interactions of supply and demand. Instead of starting with the best-case scenario, we start with a huge space of possible scenarios — all but one of them less-than-best.

However, it is also possible through collective action to mend or mitigate some of these failures. We shouldn’t assume that just because a market spontaneously formed or produced a result, that it is working optimally, and we shouldn’t assume that because a price went up either demand went up or supply went down. In that case, the market might have just gotten better at detecting information flow that was already happening. We might have gone from non-ideal information transfer where n₁ · I₁ ≥ n₂ · I₂ to something closer to ideal where n₁ · I₁ ≈ n₂ · I₂, meaning the observed price got closer to the higher ideal price.

The equations above were originally derived a bit more rigorously by physicists Peter Fielitz and Guenter Borchardt in a paper published in 2011 titled “A generalized concept of information transfer” (there is also an arXiv preprint). The paper includes both the ideal information transfer (information equilibrium) and non-ideal information transfer scenarios. They call the coefficient k the information transfer index. As they state in their abstract, information theory provides shortcuts that allow one to deal with complex systems. Fielitz and Borchardt primarily had natural complex systems in mind, but as we have just seen, the extension to social complex systems — especially pointing out the assumptions necessary for markets to function — is straightforward.

The market as an algorithm

The picture above is of a functioning market as an algorithm matching distributions by raising and lowering a price until it reaches a stable price. In fact, this picture is of a specific machine learning algorithm called Generative Adversarial Networks (GAN, described in this Medium article or in the original paper) that has emerged recently. Of course, the idea of the market as an algorithm to solve a problem is not new. For example one of the best blog posts of all time (in my opinion) talks about linear programming as an algorithm, giving an argument for why planned economies will likely fail, but the same argument implies we cannot check the optimality of the market allocation of resources, therefore claims of markets as optimal are entirely faith-based. The Medium article uses a good analogy using a painting, a forger, and a detective, but I will recast it in terms of the information theory description.

Instead of the complex multidimensional distributions, here we have blueberry buyers and blueberry sellers. The “supply” (B from above) is the generator G, the demand A is the “real data” R (the information the deep learning algorithm is trying to learn). Instead of the random initial input I — coin tosses or dice throws — we have the complex, irrational, entrepreneurial, animal spirits of people. We also have the random effects of weather on blueberry production. The detector D (which is coincidentally the terminology Fieltiz and Borchardt used) is the price p. When the detector can’t tell the difference between the distribution of demand for blueberries and the distribution of the supply of blueberries (i.e. when the price reaches a relatively stable value because the distributions are the same), we’ve reached our solution (a market equilibrium).

Note that the problem the GAN algorithm tackles can be represented by the two-player minimax game from game theory. The thing is that with the wrong settings, algorithms fail and you get garbage. I know this from experience in my regular job researching machine learning, sparse reconstruction, and signal processing algorithms. Therefore depending on the input data (especially data resulting from human behavior), we shouldn’t expect to get good results all of the time. These failures are exactly the failure of information to flow from the real data to the generator through the detector – the failure of information from the demand to reach the supply via the price mechanism.

Get Evonomics in your inbox

When asked by Quora what the recent and upcoming breakthroughs in deep learning are, Yann LeCun, director of AI research at Facebook and a professor at NYU, said:

The most important one, in my opinion, is adversarial training (also called GAN for Generative Adversarial Networks). This is an idea that was originally proposed by Ian Goodfellow when he was a student with Yoshua Bengio at the University of Montreal (he since moved to Google Brain and recently to OpenAI).

This, and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion.

Research into these deep learning algorithms and information theory may provide insight into economic systems.

An interpretation of economics for the left

So again, Hayek had a fine intuition: prices and information have some relationship. But he didn’t have the conceptual or mathematical tools of information theory to understand the mechanisms of that relationship — tools that emerged with Shannon’s key paper in 1948, and that continue to be elaborated to this day to produce algorithms like generative adversarial networks.

The understanding of prices and supply and demand provided by information theory and machine learning algorithms is better equipped to explain markets than arguments reducing complex distributions of possibilities to a single dimension, and hence, necessarily, requiring assumptions like rational agents and perfect foresight. Ideas that were posited as articles of faith or created through incomplete arguments by Hayek are not even close to the whole story, and leave you with no knowledge of the ways the price mechanism, marginalism, or supply and demand can go wrong. Those arguments assume and (hence) conclude market optimality. Leaving out the failure modes effectively declares many social concerns of the left moot by fiat. The potential and actual failures of markets are a major concern of the left, and are frequently part of discussions of inequality and social justice.

The left doesn’t need to follow Chris Hayes’ advice and engage with Hayek, Friedman, and neoclassical economics. The left instead needs to engage with a real world vision of economics that recognizes the limited scope of ideal markets and begins with imperfection as the more useful default scenario. Understanding economics in terms of information flow is one way of doing that.

2017 May 18


Donating = Changing Economics. And Changing the World.

Evonomics is free, it’s a labor of love, and it's an expense. We spend hundreds of hours and lots of dollars each month creating, curating, and promoting content that drives the next evolution of economics. If you're like us — if you think there’s a key leverage point here for making the world a better place — please consider donating. We’ll use your donation to deliver even more game-changing content, and to spread the word about that content to influential thinkers far and wide.

MONTHLY DONATION
 $3 / month
 $7 / month
 $10 / month
 $25 / month

ONE-TIME DONATION
You can also become a one-time patron with a single donation in any amount.

If you liked this article, you'll also like these other Evonomics articles...




BE INVOLVED

We welcome you to take part in the next evolution of economics. Sign up now to be kept in the loop!

  • Harald Malmgren

    There is a long progression in economic thinking about the role of information, especially costs of acquiring it, consequences for structure of markets, especially the rationale for businesses or organizations of various sizes existing in a mix of decisions sytems somewhere between centralization and decentralization. This discussion progressed in the 1960s through 2000 in what was called “new institutional economics” but also could be called “organization economics”.
    Ron Coase in 1937 and I (in my work in 1959-61) are treated as foundations of what followed in economic theory, at least up to 2004. An exposition of my conception of reconciling information, expectations and the theory of the firm can be found in https://www.academia.edu/10604114/HM_on_Theory_of_the_Firm_Quarterly_Journal_of_Economics_1961
    The history, in 7 volumes by Prof Menard in Paris, is in http://www.e-elgar.com/shop/the-international-library-of-the-new-institutional-economics

    Perhaps this will be of interest to you. Harald Malmgren

    • Thank you for the references. Of course there has to be another level of complexity in real systems beyond supply and demand and therefore as quoted in that first link those “lumps of butter … in a pail of butter-milk” can exist in this framework and depending on their structure could lead to more ideal or less ideal information transfer via markets.

      However “information” is being used in the (mathematical) information theory sense here and not as a synonym for “knowledge” … in fact, the meaning in the “messages” being passed around is largely irrelevant in this context. Rather it is the space of all possible messages that is important.

  • Jamie Morgan

    This is Post Brilliant. I’ve always had a problem with the math behind Shannon. This really helps. I’m going to write a lot about this. Particularly this paragraph, “A key requirement for a communication system to work in the presence of noise would be that it could faithfully transmit not just a given message, but rather any message drawn from the distribution. If you randomly generated thousands of messages from the distribution of possible messages, the distribution of generated messages would be an approximation to the actual distribution of messages. If you sent these messages over your noisy communication channel that met the requirement for faithful transmission, it would reproduce an informationally equivalent distribution of messages on the other end.”

    I think this makes perfect sense.

    It is not about “one” message in a complex system. In complex system it is about message distribution.

    Is that right?

    I need to study the math here. But, I think this is very helpful to building a digital library.

    • Cheers, and yes you have that correct: information theory is more about all of the possible messages than a single message and communication in information theory is about being able to get any possible message to where it needs to go.

      • Jamie Morgan

        Jason, thank you for the confirmation. This is a new discovery for me. It is exciting.

        I just got out my Shannon book. It makes sense to me that Information becomes tied to “freedom of choice” and the “reduction of uncertainty.” This would mean we need a lot of data points but we need a way to “authenticate” the data points in some evaluative way.

        There is a lot of math is this post. Do you understand it enough to explain it to a non-math major like me? I think understanding the math could help build better digital libraries and search engines. Does any of this add value?

  • Mankind Global Media

    Why no date of publication, please?

  • Nkaplan

    I can’t claim to have read this entire (highly jargon laden) article, having given up in despair about half-way through. But it is clear that its author has made no effort to understand (or worse has deliberately misconstrued) Hayek’s position. This is confirmed by such laughable statements as the following:

    “Friedrich Hayek did have some insight into prices having something to do with information, but he got the details wrong and vastly understated the complexity of the system.”

    “Hayek thought a large amount of knowledge about biological or ecological systems, population, and social systems could be communicated by a single number: a price.”

    And more egregiously:

    “The understanding of prices and supply and demand provided by information theory and machine learning algorithms is better equipped to explain markets than arguments reducing complex distributions of possibilities to a single dimension, and hence, necessarily, requiring assumptions like rational agents and perfect foresight.”

    And:

    “Those arguments assume and (hence) conclude market optimality.”

    The first two quotes are perhaps understandable confusions about a position that is more subtle than the author of the above article appears to have appreciated. The latter two quotes are not only wrong, they are the precise opposite of the Hayekian position.

    In relation to the first two quotes, it is wrong to think that Hayek considered that from prices one could – as it were – ‘read off’ all sorts of information about the conditions of scarcity etc that gave rise to those prices. The mistake is understandable since Hayek does talk about prices containing knowledge or information, however he is using those terms in a technical and somewhat idiosyncratic sense. His crucial point is that market prices perform a co-ordination function, allowing people to act as if they had the relevant knowledge. Hayek thought, as the above article would seem to agree, that it was effectively impossible for any individual to actually have all (or even a small part) of the relevant knowledge. Indeed, it was precisely this that – in Hayek’s view – made socialist planning entirely unworkable. Far from underestimating the complexity of the system, it was precisely because he considered it to be so complex that he thought planning impossible.

    In relation to the latter two quotes, it is difficult to think that anyone who has actually read Hayek could think they reflect anything like his position. Indeed it is a central tenant of Austrian school economics (of which he was not quite a founding member, but is perhaps its most thoughtful member) that the efficient market hypothesis is false, that markets are never actually in equilibrium, that people are not perfectly rational agents, and that they most definitely do not have (and cannot have) perfect foresight. It was for precisely these reasons that he had such major disagreements with more mainstream schools of economic thought – and dismissed their ‘pretence of knowledge’ which had provided a (false) justification for all sorts of market interventions with which he disagreed.

    If you want to understand Hayek – rather than merely attacking straw men – I would highly recommend Mark Pennington’s excellent book ‘Robust Political Economy.’ (https://www.amazon.co.uk/Robust-Political-Economy-Classical-Liberalism/dp/1849807655)

    • Mankind Global Media
    • Stingo

      Yes. The mischaracterization of Hayek was rather jarring. It’s as if he confused Hayek’s arguments with the left and centralized planners. Very strange.

    • Guilherme

      Yes. These following quotes COMPLETELY shows the mischaracterization of Hayek in this article. People are criticizing him without reading the basic from him, and worst, without reading also the basic ABOUT him, given the absurd interpretation which have absolutely nothing to do with what he thought.

      See what Hayek says in his two MAGNUS OPUS:

      “What we must ask the reader to keep constantly in mind throughout this book, then, is the fact of the necessary and irremediable ignorance on everyone’s part of most of the particular facts which determine the actions of all the several members of human society. This may at first seem to be a fact so obvious and incontestable as hardly to deserve mention, and still less to require proof. Yet the result of not constantly stressing it is that it is only too readily forgotten.” F. A. Hayek, Law, Legislation and Liberty

      “The Socratic maxim that the recognition of our ignorance is the beginning of wisdom has profound significance for our understanding of society (…) This fundamental fact of man’s unavoidable ignorance of much on which the working of civilization rests has received little attention. Philosophers and students of society have generally glossed it over and treated this ignorance as a minor imperfection which could be more or less disregarded. But, though discussions of moral or social problems based on the assumption of perfect knowledge may occasionally be useful as a preliminary exercise in logic, they are of little use in an attempt to explain the real world. Its problems are dominated by the “practical difficulty” that our knowledge is, in fact, very far from perfect. Perhaps it is only natural that the scientists tend to stress what we do know; but in the social fi eld, where what we do not know is often so much more important, the effect of this tendency may be very misleading.” F. A. Hayek, The Constitution of Liberty

      He completely rejected the idea of OPTIMALITY, PERFECT KNOWLEDGE or that EQUILIBRIUM say something about real world in economics:

      “The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus not merely a problem of how to allocate “given” resources — if “given” is taken to mean given to a single mind which deliberately solves the problem set by these “data.” It is rather a problem of how to secure the best use of resources known to any of the members of society, for ends whose relative importance only these individuals know. Or, to put it briefly, it is a problem of the utilization of knowledge which is not given to anyone in its totality.” F. A. Hayek, The Use of Knowledge in Society

      “The statement that, if people know everything, they are in equilibrium is true simply because that is how we define equilibrium. The assumption of a perfect market in this sense is just another way of saying that equilibrium exists but does not get us any nearer an explanation of when and how such a state will come about. It is clear that, if we want to make the assertion that, under certain conditions, people will approach that state, we must explain by what process they will acquire the necessary knowledge.” F. A. Hayek, Economics and Knowledge

      “The relevance of this for my present problem will appear when it is recalled that the modern theory of competition deals almost exclusively with a state of what is called “competitive equilibrium” in which it is assumed that the data for the different individuals are fully adjusted to each other, while the problem which requires explanation is the nature of the process by which the data are thus adjusted.” F. A. Hayek, The Meaning of Competition

      And his PHILOSOPHY was ENTIRELY against the idea of hyper-rationality atributted to him in this article:

      “The rationalistic design theories were necessarily based on the assumption of the individual man’s propensity for rational action and his natural intelligence and goodness. The evolutionary theory, on the contrary, showed how certain institutional arrangements would induce man to use his intelligence to the best effect and how institutions could be framed so that bad people could do least harm. The antirationalist tradition is here closer to the Christian tradition of the fallibility and sinfulness of man, while the perfectionism of the rationalist is in irreconcilable conflict with it. Even such a celebrated figment as the “economic man” was not an original part of the British evolutionary tradition. It would be only a slight exaggeration to say that, in the view of those British philosophers, man was by nature lazy and indolent, improvident and wasteful, and that it was only by the force of circumstances that he could be made to behave economically or would learn carefully to adjust his means to his ends. The homo oeconomicus was explicitly introduced, with much else that belongs to the rationalist rather than to the evolutionary tradition, only by the younger Mill.” F. A. Hayek, The Constitution of Liberty

  • Roger Koppl

    Sorry, Jason Smith, but Nkaplan is completely right. In his famous tin example, Hayek talks about who must know what. Price changes induce adjustments without *most* of those doing the adjustments knowing why such adjustments are called for. That is not a story in which we are “compressing a complex multidimensional space of possibilities that includes the state of the world and the states of mind of thousands or millions of agents into a single dimension (i.e. price).” John Holland’s Adaptation in Natural and Artificial System is a good first approximation to Hayek if you like math. But I guess I’d better say that if you do a deep dive into Hayek, you will fairly quickly find that you must get beyond formalisms such as Holland’s. For the moment, however, that’s kind of a detail. My point here is just to chime in with Nkaplan in telling you that you are, unfortunately, misreading Hayek rather badly.

    • Mankind Global Media
      • Roger Koppl

        One could indeed read a blog devoted in part to criticizing “Austrian economics” — whatever that is. Or one could, you know, actually read Hayek.

        • Mankind Global Media

          Why would I want to read an Anti-government cultic nutcase?

          • Really? Trolling?

            If not: for two reasons:

            1a) because you want to criticze him (Hayek) or sustain others critizing him – so you should check what he has written / said to not potentially make a fool of yourself. Just in case … chinese whispers, y’know.

            1b) because your characterization of Hayek might be an interpretation that is at odds with what Hayek has said / written / done.
            If that’s the case, then it is great argument basis to attack all those (usually libertarians, anarchists) who build ‘anti-government nutcases’ by taking away their (ideo-)logical foundation.

            2) because one way to advance knowledge is by critically examining claims, thus identifying critical issues that need more (counter-)evidence, weeding out flawed claims, errors, suppositions,rumours and positively identifying those statements and the (possibly different) ranges of conditions under which statements are valid given the avialable evidence (which is certainly subject to interpretation).

          • Mankind Global Media

            Or, I could just plain not be interested, and would rather study the works of those who are looking at practical solutions in the present world. One can only have so much on his plate.

    • Mankind Global Media

      Author reads Hayek, position agreeable to 3rd party reader:

      “You’ve got it down pat~! How intelligent and Red-pill smart you must be.”

      Author reads Hayek, position unacceptable to 3rd party reader:

      “You’re not reading Hayek correctly~!!!” 😉

  • Jeff Neuman-Lee

    That Jason might misrepresent Hayek is not an issue for me; I just don’t know about all that.
    I am interested in his use of information theory, but as an uninitiated, he failed to communicate. FYI. Sort of ironic.

    • Nkaplan

      Very droll!

  • Johnson

    LOL!!! I don’t even know where to begin. The strawman fallacy is the biggest fallacy our friend uses.

    Hayek never pursued optimality in the mathematical sense. Hayek argued that the market was best placed to identify where resources would go and gave this signals via two related concepts… prices and profits.

    If you invested in what the market valued the most you’d be rewarded with higher profits and other investors would be attracted to that business. If you invested in that which the market valued the least you would be punished with losses and that field would witness an exodous of resources to more profitable investments.

    Hayek argued that in this way, a dynamic society kept reallocating its resources from time to time.

    Which brings me to another concept… capital theory. Over time there is obsolescence of capital happens from time to time and therefore he emphasized the heterodexterity of capital.

  • Joe Joe

    A physicist explaining Austrian economics; that’s rich. Can I get a landscaper to repair my dishwasher please?

  • Dylan Brewer

    I found the characterization of Hayek’s arguments to be wrong. Hayek’s works definitely do not understate the complexity of the system–he finds the market system to be so complex that he recommends avoiding attempts to intervene in it for fear of unintended consequences.

    Furthermore, stating “Hayek dreams of compressing a complex multidimensional space of possibilities that includes the state of the world and the states of mind of thousands or millions of agents into a single dimension (i.e. price), inevitably losing a great deal of information in the process” misrepresents Hayek completely. There is no loss of information. Prices reflect a lot of information at *low cost.* An economic agent may be able to gain more specific information about various supply or demand conditions in a different geographic space or time, but this information acquisition is costly. Prices inexpensively coordinate behavior.

    With regards to information theory, the major flaw (in this presentation at least–I am assuming it gets more complex) is that it assumes away human decision making. The analysis attempts to describe the outcomes of a system of humans without modelling actual human behavior. Neoclassical economics has gone to great lengths in the last century to “micro-found” its models. I would recommend reading some of Mas-Collel, Whinston, and Green’s “Microeconomic Theory” textbook to see how economists have mathematically micro-founded their models.

    Finally, the author argues that information theory does better than the current economic paradigm, stating that “supply and demand doesn’t always work in the simple way described in Marshall’s diagrams.” Economists teach the simple supply and demand diagrams to non-economists to keep things simple. Academic economics has long distinguished between perfect and imperfect markets and scenarios with a large number of economic actors and a small number of economic actors. As far as I can tell, the author is criticizing economics as it is taught to amateurs.

    I greatly enjoyed learning a little about information theory that I am not familiar with, but the article loses credibility when the author attacks works that he is not familiar with. Information theory serves as a nice complement to economic theory and I would love to see it presented as a way to think deeper rather than a reason to discard economics!

  • The problem with Hayek is probably less the subjective, psychological Hungarian-Austrian Hayek, but the interpretation of Hayek in a culture and worldview of utilitarian economic maximisation and ‘optimality’ that was and is quite strong in a Victorian ‘business’ culture.

  • ViperRum

    From the aricle,

    “This picture also gets rid of the dimensionality problem: the distribution of demand can be as complex and multidimensional (i.e. depend on as many variables) as the distribution of supply. The single dimension represented by the price now only measures the single dimension of information flow.”

    Derp. Didin’t you just get done saying this was a bad thing with regards to prices?

    Not only that you appear to basically be agreeing with Hayek, that prices convey information and what you are trying to do is put it into precise mathematical terms…so what? You can attempt to control it? You seem to have basically redone neoclassical economics (hint Hayek was NOT a neoclassical economist, that is why he was part of the Austrian school) but using the framework of information theory. Okay…maybe interesting, but I see nothing new here. At all. So where exactly did Hayek fail? If anything his article the Use of Knowledge in Society was way, way ahead of its time according to your article.

    Further, you seem to have failed to grasp a key observation of Hayek, the information necessary to use the neoclassical framework or even your information theory is so vast and huge it cannot be done. We live in a global economy. Said global economy has billions of people in it and who knows how many firms. Each of these individuals and firms engage in transactions all the time. It may not be outrageous to suggest there might be trillions of economic transactions occurring in the space of a few days or even a few hours. And many you might not even have data on to put into your machine learning algorithm. Suppose I see that the price of tomatoes is high so I start growing my own. Where are you going to get data on my production of tomatoes? You aren’t. You might see my increased consumption of potting soil or something, but will you know it is for tomatoes or pansies or something else like filling holes in my backyard my dog dug up?

    In fact, you do note this, but you assume you CAN have enough information. Hayek says, “No, you can’t.” And so we have the price mechanism which does take into account all of these things “some how” and prices emerge as a result. Does it tell us why the price of blueberries went up or down? No. But as a consumer do I really care or is all that I need to know is that the price went up? Consume less, shift consumption to say raspberries, etc. I don’t need to know that the blueberry crop in some geographic area was wiped out. I don’t need to know that there was a surge in demand from people in California, or a new fad for blueberry pies. I can make a good decision based simply off the fact that the price went up.

    So to sum up, you are rehashing neoclassical economics via information theory, you are suggesting that via this new “realistic” interpretation it can be used to control the economy–i.e. you are taking the Knowledge Problem posited not only by Hayek, but also Mises. And you ignore the point by James Buchanan in regards to the emergence of order such as prices, that that order is defined by the process of its emergence, that is you cannot analyze outside of that process. If Buchanan is right all your information theory and algorithms won’t be Jack shit (and Jack just left town). That is not only will you need to know about all the trillions of transactions that do take place but all of the trillions and trillions that could have taken place but did not. Good luck with that. Overall, an article that failed to accomplish with it set out to do and indicates the author has a shallow, at best, understanding of Hayek.

    • the9trances

      #rekt

    • “Didin’t you just get done saying this was a bad thing with regards to prices?”

      Nope. I said it was a bad thing to say that single-dimensional prices reflected the multidimensional distributions of supply or demand. Single dimensional prices can fully reflect the single dimensional distribution of information flow between supply and demand.

      ” … suggesting that via this new ‘realistic’ interpretation it can be used to control the economy”

      I am explicitly *not* suggesting this “solves” the calculation problem (I cite a reference that says it is impossible with modern computational resources).

  • Guilherme

    “The understanding of prices and supply and demand provided by information theory and machine learning algorithms is better equipped to explain markets than arguments reducing complex distributions of possibilities to a single dimension, and hence, necessarily, requiring assumptions like rational agents and perfect foresight. Ideas that were posited as articles of faith or created through incomplete arguments by Hayek are not even close to the whole story, and leave you with no knowledge of the ways the price mechanism, marginalism, or supply and demand can go wrong. Those arguments assume and (hence) conclude market optimality.”

    Wtf, you know nothing about Hayek. Nothing! I doubt even if you read the Wikipedia article about him

    • There are lots of claims I know nothing about Hayek, but they rarely cite specific errors.

      I get the impression that I may well have read (or understood) more of Hayek than his fans.

      • Guilherme

        Seriously, man? You’re saying Hayek assumes market optimality, you’re saying that he doesn’t consider falibiliy or limitations of knowledge. But it’s exactly the opposite of Hayek’s thinking. My harsh criticism is just because Hayek is one of the most known thinkers to be against the idea of market optimality, one of that who most emphazized human falibility and the limitation of knowledge.

        • Limitations of knowledge and fallibility are not directly related to optimality. A system where agents make mistakes can still have an optimum — a best case. In fact, Gary Becker showed that random “irrational” agents can still show optimal behavior in a market (“Irrational Behavior and Economic Theory”, 1962).

          And Hayek does claim exactly the kind of optimality required when you have fallible agents and “[price] adjustments are probably never ‘perfect'”. As Hayek says: “tens of thousands of people … move in the right direction”.

          In a system with noise, error, etc, saying that agents or prices move in the right direction is an optimality claim. It assumes markets work as a kind of information pump with bad information being pumped out and good information being kept in. The information theory approach tells us that this is not guaranteed:

          http://informationtransfereconomics.blogspot.com/2015/01/is-market-intelligent.html

      • Guilherme

        I read a lot of Hayek and I can provide tou some quotes fron hin so that you can see what I’m saying:

        See what Hayek says in his two MAGNUS OPUS:

        “What we must ask the reader to keep constantly in mind throughout this book, then, is the fact of the necessary and irremediable ignorance on everyone’s part of most of the particular facts which determine the actions of all the several members of human society. This may at first seem to be a fact so obvious and incontestable as hardly to deserve mention, and still less to require proof. Yet the result of not constantly stressing it is that it is only too readily forgotten.” F. A. Hayek, Law, Legislation and Liberty

        “The Socratic maxim that the recognition of our ignorance is the beginning of wisdom has profound significance for our understanding of society (…) This fundamental fact of man’s unavoidable ignorance of much on which the working of civilization rests has received little attention. Philosophers and students of society have generally glossed it over and treated this ignorance as a minor imperfection which could be more or less disregarded. But, though discussions of moral or social problems based on the assumption of perfect knowledge may occasionally be useful as a preliminary exercise in logic, they are of little use in an attempt to explain the real world. Its problems are dominated by the “practical difficulty” that our knowledge is, in fact, very far from perfect. Perhaps it is only natural that the scientists tend to stress what we do know; but in the social fi eld, where what we do not know is often so much more important, the effect of this tendency may be very misleading.” F. A. Hayek, The Constitution of Liberty

        He completely rejected the idea of OPTIMALITY, PERFECT KNOWLEDGE or that EQUILIBRIUM say something about real world in economics:

        “The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus not merely a problem of how to allocate “given” resources — if “given” is taken to mean given to a single mind which deliberately solves the problem set by these “data.” It is rather a problem of how to secure the best use of resources known to any of the members of society, for ends whose relative importance only these individuals know. Or, to put it briefly, it is a problem of the utilization of knowledge which is not given to anyone in its totality.” F. A. Hayek, The Use of Knowledge in Society

        “The statement that, if people know everything, they are in equilibrium is true simply because that is how we define equilibrium. The assumption of a perfect market in this sense is just another way of saying that equilibrium exists but does not get us any nearer an explanation of when and how such a state will come about. It is clear that, if we want to make the assertion that, under certain conditions, people will approach that state, we must explain by what process they will acquire the necessary knowledge.” F. A. Hayek, Economics and Knowledge

        “The relevance of this for my present problem will appear when it is recalled that the modern theory of competition deals almost exclusively with a state of what is called “competitive equilibrium” in which it is assumed that the data for the different individuals are fully adjusted to each other, while the problem which requires explanation is the nature of the process by which the data are thus adjusted.” F. A. Hayek, The Meaning of Competition

        And his PHILOSOPHY was ENTIRELY against the idea of hyper-rationality atributted to him in this article:

        “The rationalistic design theories were necessarily based on the assumption of the individual man’s propensity for rational action and his natural intelligence and goodness. The evolutionary theory, on the contrary, showed how certain institutional arrangements would induce man to use his intelligence to the best effect and how institutions could be framed so that bad people could do least harm. The antirationalist tradition is here closer to the Christian tradition of the fallibility and sinfulness of man, while the perfectionism of the rationalist is in irreconcilable conflict with it. Even such a celebrated figment as the “economic man” was not an original part of the British evolutionary tradition. It would be only a slight exaggeration to say that, in the view of those British philosophers, man was by nature lazy and indolent, improvident and wasteful, and that it was only by the force of circumstances that he could be made to behave economically or would learn carefully to adjust his means to his ends. The homo oeconomicus was explicitly introduced, with much else that belongs to the rationalist rather than to the evolutionary tradition, only by the younger Mill.” F. A. Hayek, The Constitution of Liberty

  • Mick McDick

    My
    copy of Road to Serfdom, page 66:”The magnificent motor roads in
    Germany and Italy are an instance often quoted-even though they do not
    represent a kind of planning not equally possible in a liberal
    society…Anyone who has driven along the famous German motor roads and
    found the amount of traffic on them less than on many a secondary road
    in England, can have little doubt that, so far as peace purposes are
    concerned, there was little justification for them. ”
    Yes,
    the famous economist drove the first superhighways in the world and
    could not see the point of them…”so far as peace purposes are
    concerned, there was little justification for them.” Sweet baby Jeebus,
    think about it Hayek – can you think of nothing that these “highways”
    may be useful for, in an economic sense? NO?
    Hayek could only see data that supported
    his foregone conclusions. He shares this shortened vision with many of todays libertarian ideologues.

  • Guilherme

    This article is a complete ABSURD. Or it is extreme dishonesty or the author doesn’t know ANYTHING about Hayek beyond his name, because it’s clear that he didn’t read even the Wikipedia page about him. Assumptions like ” Those arguments assume and (hence) conclude market optimality.”, or saying that Hayek’s theory pressuposes perfect foresight or a “ideal market” are a COMPLETE non-sense given Hayek’s ideas.

    See what Hayek says in his two MAGNUS OPUS:

    “What we must ask the reader to keep constantly in mind throughout this book, then, is the fact of the necessary and irremediable ignorance on everyone’s part of most of the particular facts which determine the actions of all the several members of human society. This may at first seem to be a fact so obvious and incontestable as hardly to deserve mention, and still less to require proof. Yet the result of not constantly stressing it is that it is only too readily forgotten.” F. A. Hayek, Law, Legislation and Liberty

    “The Socratic maxim that the recognition of our ignorance is the beginning of wisdom has profound significance for our understanding of society (…) This fundamental fact of man’s unavoidable ignorance of much on which the working of civilization rests has received little attention. Philosophers and students of society have generally glossed it over and treated this ignorance as a minor imperfection which could be more or less disregarded. But, though discussions of moral or social problems based on the assumption of perfect knowledge may occasionally be useful as a preliminary exercise in logic, they are of little use in an attempt to explain the real world. Its problems are dominated by the “practical difficulty” that our knowledge is, in fact, very far from perfect. Perhaps it is only natural that the scientists tend to stress what we do know; but in the social fi eld, where what we do not know is often so much more important, the effect of this tendency may be very misleading.” F. A. Hayek, The Constitution of Liberty

    He completely rejected the idea of OPTIMALITY, PERFECT KNOWLEDGE or that EQUILIBRIUM say something about real world in economics:

    “The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus not merely a problem of how to allocate “given” resources — if “given” is taken to mean given to a single mind which deliberately solves the problem set by these “data.” It is rather a problem of how to secure the best use of resources known to any of the members of society, for ends whose relative importance only these individuals know. Or, to put it briefly, it is a problem of the utilization of knowledge which is not given to anyone in its totality.” F. A. Hayek, The Use of Knowledge in Society

    “The statement that, if people know everything, they are in equilibrium is true simply because that is how we define equilibrium. The assumption of a perfect market in this sense is just another way of saying that equilibrium exists but does not get us any nearer an explanation of when and how such a state will come about. It is clear that, if we want to make the assertion that, under certain conditions, people will approach that state, we must explain by what process they will acquire the necessary knowledge.” F. A. Hayek, Economics and Knowledge

    “The relevance of this for my present problem will appear when it is recalled that the modern theory of competition deals almost exclusively with a state of what is called “competitive equilibrium” in which it is assumed that the data for the different individuals are fully adjusted to each other, while the problem which requires explanation is the nature of the process by which the data are thus adjusted.” F. A. Hayek, The Meaning of Competition

    And his PHILOSOPHY was ENTIRELY against the idea of hyper-rationality atributted to him in this article:

    “The rationalistic design theories were necessarily based on the assumption of the individual man’s propensity for rational action and his natural intelligence and goodness. The evolutionary theory, on the contrary, showed how certain institutional arrangements would induce man to use his intelligence to the best effect and how institutions could be framed so that bad people could do least harm. The antirationalist tradition is here closer to the Christian tradition of the fallibility and sinfulness of man, while the perfectionism of the rationalist is in irreconcilable conflict with it. Even such a celebrated figment as the “economic man” was not an original part of the British evolutionary tradition. It would be only a slight exaggeration to say that, in the view of those British philosophers, man was by nature lazy and indolent, improvident and wasteful, and that it was only by the force of circumstances that he could be made to behave economically or would learn carefully to adjust his means to his ends. The homo oeconomicus was explicitly introduced, with much else that belongs to the rationalist rather than to the evolutionary tradition, only by the younger Mill.” F. A. Hayek, The Constitution of Liberty

  • Christos D. Papageorgiou

    Let us assume that the proposed remarks of the article about market economy are all true, two very important questions are remaining.

    1. Why the non capitalist (non market) economies (Communistic economies) are completely unsuccessful generating poverty ?

    2. Why the capitalistic (market) economies operate successfully creating wealth (unequally shared), although various Marxist or Socialist oriented intellectuals discover many theoretical mistakes in the pro-market economy theories of HAYEK or FRIEDMAN and others ?

    Keynes gave a sincere reply in the second question “Capitalistic (market) economy is a remarkably successful social machine but we do not know how it operates”

    • stephenverchinski

      The framing of the first question give to me a false assumption that Communist economies are not market based and failures at creating wealth. The second question frames it that capitalistic structures are that market based and successful. I find both frames too simplistic. Both have individual greed as the ultimate value at the top with politbureaus and corporatist manipulation. Both create by design imperfect markets and both bypass the impacts long term on biological systems whose accounting system has been always an afterthought..

  • Barbara N Brown

    The other thing that seems to be of great concern given the “fake news” trends becoming visable in recent years is the question of deliberate disinformation. This is NOT noise in the system, but weighted coins if you start with statistics.
    So what does information theory tell us happens to economics if NONE of the information the majority of players receives can be trusted? Especially if there is collusion among a small group spreading lies?

    • One of the key aspects of information theory is that it helps you characterize a system without getting into the meaning of the messages.

      However correlation of agents in the state space would lead to the case I called “non-ideal information transfer”, which tends to cause the system to collapse. So in a sense, as long as the “lies” are uncorrelated, then there’s not problem. But if they are, beware.

      • Barbara N Brown

        But of course, in the real markets, the lies are correlated. That’s the definition of collusion. And yes, eventually this leads to a bubble and then massive collapse.
        But the whole purpose of information is meaning… or more precisely utility. So to many, the price no longer has utility, because the information the supposedly is summarized in the price cannot be trusted.
        Perhaps a better use of information theory might be to identify an objective confidence level in the information being received.

        • stephenverchinski

          Like the collusion we saw prior to the market epic fail in 2008 between Republicans and Democrats to suppress the reporting of the M3 money supply (derivatives etc.). Except that perhaps did not apply to those participating in the suppression. Damn them both to.someplace not nice.

  • Barış Canatan

    If you think that you can write some ML algo. that can decide on the value of an item better than yourself, go ahead write it. Nobody is holding you.

    If you want to represent the value of an item using an N dimensional vector, what is the basis for the valuespace? What are the orthogonal components (and according to whom)?

    • I see you’ve never encountered a problem where you don’t know what a good basis for your space is (trying to understand which basis visual imagery is sparse in is an outstanding problem in image processing).

      One of the advantages of the approach above is that the dimensionality problem is dealt with as an effective theory: at leading order the details of the basis are subsumed into the coefficient k (information transfer index).

  • Tony Hua

    While I do not necessarily agree with the author’s interpretation/summary of Hayek’s works, I do nonetheless find the author’s use of information theory to derive economic relationships interesting. Several of the commentators have criticized the author for attempting to build a better market mechanism, which I believe misinterprets the author’s intent. Jason Smith seems to be offering one potential interpretation of the market mechanism, an attempt to build an alternative derivation to standard marginal methods, thereby removing certain assumptions made in (admittedly textbook based) models. Nowhere does Jason makes the claim that machine learning or information theory can build a better marketplace. The chief argument of this article seems centered on better understanding the market mechanism through insights provided by information theory models.

    • Barış Canatan

      Nope. Both the title and the last chapter of the article refer to an argument against capitalism. And they fail.

      • It’s not an argument against capitalism.

        It’s an argument against assuming the market is optimal.

    • Thanks Tony.

      Yes, I don’t think I am building a better market mechanism, simply attempting to understand the existing one (and exploring what assumptions are implicitly made about it). And if we can understand failures, we can plan for them.

      I think e.g. machine learning may be a source of insight, but as I said in the piece: machine learning algorithms sometimes fail. My speculation is that understanding the failure modes of machine learning algorithms may help us understand the failure modes of markets. … There is an analogy here to the case where failure modes of machine learning algorithms can lead to insights into neuroscience.

      • Now, that is an interesting approach. Using the technical analogy to point out flaws in economic thinking resp. models.

  • Guilherme

    This article definitely makes a strawman of Hayek. See what Hayek says in his two MAGNUS OPUS:

    “What we must ask the reader to keep constantly in mind throughout this book, then, is the fact of the necessary and irremediable ignorance on everyone’s part of most of the particular facts which determine the actions of all the several members of human society. This may at first seem to be a fact so obvious and incontestable as hardly to deserve mention, and still less to require proof. Yet the result of not constantly stressing it is that it is only too readily forgotten.” F. A. Hayek, Law, Legislation and Liberty

    “The Socratic maxim that the recognition of our ignorance is the beginning of wisdom has profound significance for our understanding of society (…) This fundamental fact of man’s unavoidable ignorance of much on which the working of civilization rests has received little attention. Philosophers and students of society have generally glossed it over and treated this ignorance as a minor imperfection which could be more or less disregarded. But, though discussions of moral or social problems based on the assumption of perfect knowledge may occasionally be useful as a preliminary exercise in logic, they are of little use in an attempt to explain the real world. Its problems are dominated by the “practical difficulty” that our knowledge is, in fact, very far from perfect. Perhaps it is only natural that the scientists tend to stress what we do know; but in the social fi eld, where what we do not know is often so much more important, the effect of this tendency may be very misleading.” F. A. Hayek, The Constitution of Liberty

    He completely rejected the idea of OPTIMALITY, PERFECT KNOWLEDGE or that EQUILIBRIUM say something about real world in economics:

    “The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus not merely a problem of how to allocate “given” resources — if “given” is taken to mean given to a single mind which deliberately solves the problem set by these “data.” It is rather a problem of how to secure the best use of resources known to any of the members of society, for ends whose relative importance only these individuals know. Or, to put it briefly, it is a problem of the utilization of knowledge which is not given to anyone in its totality.” F. A. Hayek, The Use of Knowledge in Society

    “The statement that, if people know everything, they are in equilibrium is true simply because that is how we define equilibrium. The assumption of a perfect market in this sense is just another way of saying that equilibrium exists but does not get us any nearer an explanation of when and how such a state will come about. It is clear that, if we want to make the assertion that, under certain conditions, people will approach that state, we must explain by what process they will acquire the necessary knowledge.” F. A. Hayek, Economics and Knowledge

    “The relevance of this for my present problem will appear when it is recalled that the modern theory of competition deals almost exclusively with a state of what is called “competitive equilibrium” in which it is assumed that the data for the different individuals are fully adjusted to each other, while the problem which requires explanation is the nature of the process by which the data are thus adjusted.” F. A. Hayek, The Meaning of Competition

    And his PHILOSOPHY was ENTIRELY against the idea of hyper-rationality atributted to him in this article:

    “The rationalistic design theories were necessarily based on the assumption of the individual man’s propensity for rational action and his natural intelligence and goodness. The evolutionary theory, on the contrary, showed how certain institutional arrangements would induce man to use his intelligence to the best effect and how institutions could be framed so that bad people could do least harm. The antirationalist tradition is here closer to the Christian tradition of the fallibility and sinfulness of man, while the perfectionism of the rationalist is in irreconcilable conflict with it. Even such a celebrated figment as the “economic man” was not an original part of the British evolutionary tradition. It would be only a slight exaggeration to say that, in the view of those British philosophers, man was by nature lazy and indolent, improvident and wasteful, and that it was only by the force of circumstances that he could be made to behave economically or would learn carefully to adjust his means to his ends. The homo oeconomicus was explicitly introduced, with much else that belongs to the rationalist rather than to the evolutionary tradition, only by the younger Mill.” F. A. Hayek, The Constitution of Liberty

    • Adediran Adeyemi

      Thank you. This is the perfect response. My thoughts exactly.

    • Guido Wolf Reichert

      “two MAGNUS OPUS” really hurts – please make it “two MAGNA OPERA” next time. 🙂

  • If a machine learning algorithm always moved in the ‘right’ direction this means what? (Btw: who defines what right is – which is precisely one of Hayeks major points).

    It locally moves up a gradient (hill), so it might end up (get stuck) on any ‘local’ optimum (hilltop). What would happen in a landscape of troughs and hills that is wobbling and shifting on different scales?

    Here you’ll find some backgound information and summary of Hayek’s development of thought (and links to the writings of Hayek).

    http://hayekcenter.org/?p=1678

    In his 1936/1937 ‘Economics of Knowledge’ Hayek uses the equilibrium concept for the anticipations / plans of a *single* actor *as long as* his anticipations ‘hold’. New information = new anticipations = new plans = new *individual* equilibrium, no other equilibrium.

    In his 1945 ‘Use of Knowledge’ Hayek discusses the coordination among economic actors based on their individual,limited, subjective ‘knowledge’ of local cirumstances in the face of a constant flow of changes and adaptations they are subject to and causing themselves, i.e. in an evolutionary fashion.

    “The peculiar character of the problem of a rational economic order
    is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form, but solely as the dispersed bits of incomplete and
    frequently contradictory knowledge which all the separate individuals
    possess. … It is rather a problem of how to secure the best use of
    resources known to any of the members of society, for ends whose
    relative importance only these individuals know. ” Hayek 1945, p.524-525.

    • The phrase “right direction” is actually a direct quote from Hayek in ‘The Use of Knowledge in Society’:

      “The marvel is that in a case like that of a scarcity of one raw material, without an order being issued, without more than perhaps a handful of people knowing the cause, tens of thousands of people whose identity could not be ascertained by months of investigation, are made to use the material or its products more sparingly; i.e., they move in the right direction.”

      It is clear that Hayek means the “right direction” is towards an optimum (i.e. using a scarce commodity more sparingly).

      And again, the intertemporal equilibrium (involving expectations) in “Economics and Knowledge” is a somewhat different (but not entirely unrelated) concept than the price mechanism. There are two problems: the economic calculation problem and the coordination of future plans problem. The price mechanism is a heuristic solution to the former, but Hayek was just among the first to formulate the latter. See Glasner’s piece and his follow up:

      https://uneasymoney.com/2017/05/28/correct-foresight-perfect-foresight-and-intertemporal-equilibrium/

      • 1) Still here is the question who or what defines ‘right’?

        In a simple machine learning algo world the algorithm would climb up any hill (gradient’) where it starts, and where the hill is defined by the problem the algorithm is to solve. To move this to Hayek’s world you would have to have several algorithms competing against each other and they would have to be adaptable as in evolutionary strategies.

        So Hayek would answer: nobody or all actors through their interactions define what is right – it (‘right’) emerges iteratively based on the interactions of the actors or their plans in an evolutionary way without conscious design, as Hayek indeed says (and you would concur, I assume).

        2) There’s a problem here: What are the institutions (‘regulations’, rules) that govern this process (if any) and where does the process move consequently?

        If there is no regulation (as in libertarian approaches) that’s also a form of regulation that has the effect of strengthening those that already have resources and can compound these resources.

        3) What kind of environment do you assume?

        A static environment on which actors (or agents) adapt their plans to each other – or a dynamic one where the environment changes (e.g. because the plans of the actors affect it)?

  • Not really. Rather, the point is: the flaws in neoliberal economics Jason wants to point out are also inherent in Hayek’s view of economic interaction. The formalized process of economic interaction Jason describes is (intentionally) a rendition of Hayek’s view.

    The difference between ‘left’ and ‘right’ occurs in the interpretation of and conclusion from the Hayekian economic process. This relates to the following quote from Jason:

    “necessarily, requiring assumptions like rational agents and perfect foresight. Ideas that were posited as articles of faith or created through incomplete arguments by Hayek are not even close to the whole story, and leave you with no knowledge of the ways the price mechanism, marginalism, or supply and demand can go wrong. Those arguments assume and (hence) conclude market optimality.”

    What has Hayek really said and what is ascribed to him?
    A neoliberal, economic system ‘assumes and concludes’ inherent market optimality but does this necessarily apply to the market process, given that humans (economic actors) are fallible and error prone?

    It depends on how these interactions play out and how the ‘landscape’ constituted by economic interactions looks like.

    If there are limits to the economic process (like limited information, collusion, a subset of actors with insider information, etc.) then the outcome of the Hayekian process is ‘distorted’ as well – and that is why the Hayek defenders here rail against Jasons characterization of Hayek.

    • Mankind Global Media

      If you follow the links to the Social Democracy blog I posted all the way to the debate with him and another blogger, you’ll find both an Austrian and a Keynesian in general agreement that Hayek was all about Equilibrium. I guess it depends on who you ask.

      • Obviously, science and evidence is subejct to perspective and interpretation.

        It’s been a while since I was there: what / where is the debate you refer to?

        • Mankind Global Media

          You just have to look @the links I posted, check the comments to see who’s showed up to argue, and check where the author or the blog posted updates both for his and opponent’s blog posts.

          • Well, I can do that, did that up to a certain point and didn’t find the specific points you mentioned.

          • Mankind Global Media

            I just stumbled across them. It’s in the comments on one of the posts by the blogger he’s debating with.

      • …and it depends on when and where you ask:

        For instance was Hayek Member of the Board of the ordoliberal Walter Eucken Institute founded by the pupils of Walter Eucken, among them the liberal, but conservative CDU party member, Ludwig Erhardt, who was instrumental in setting up a liberal version of the nowadays so-called ‘third way’, that is the social market economy in Germany.

        In the UK and the US it is hardly realized what the German (or Austrian – the country) interpretation of ‘Austrian’ positions came to be. Only that it has been quite successful as an institutional arrangement.

        English:
        https://en.wikipedia.org/wiki/Walter_Eucken_Institut

        German:
        https://de.wikipedia.org/wiki/Walter_Eucken_Institut

        • Mankind Global Media

          ” free market order, classical liberal ideas and their institutional realisation, and the economic constitution of the European Union”

          Looks like standard Austrian memes to me. Nothing to see here.

        • Mankind Global Media

          ” free market order, classical liberal ideas and their institutional realisation, and the economic constitution of the European Union”

          Looks like standard Austrian memes to me.

          Upon further consideration, I have say that your verbosity is adorable but I don’t find very impressive. Hayek being a part of a group that wants the government to ensure the so-called “Free Market” runs like it should and was instrumental in the forming of the EU only tells me that Neoliberalism has it’s roots in Libertarianism more than has been thought. The EU seems to be on a tirade to outright destroy all vestiges of Democratic Socialism with it’s emphasis on Open Borders and the EBC tendency towards Predatory Lending. I think thinker like Ha-Joon Chang have superseded Hayek and his ilk by simply pointing out that there’s no such thing as a “Free Market” as long as there’s regulation [or even de-regulation] of immigration. “Our wages are politically determined” as he would say. [You can listen to the podcast of his I cited below for more reasons.]

          “Free Markets” as Libertarians describe them are not the thing we should be focusing on. We should be focusing on Full Employment & Stability of Wages & Prices, which are amenable to Compassion. These are who I think are the top thinkers in that regard:

          https://en.wikipedia.org/wiki/Michael_Hudson_(economist)
          https://en.wikipedia.org/wiki/Michael_Hudson_(economist)#Debt_in_the_Ancient_Near_East
          https://youtu.be/XzSNkNaLRc8
          http://cas2.umkc.edu/econ/economics/faculty/wray/papers/hudson.pdf

          https://en.wikipedia.org/wiki/Steve_Keen
          https://en.wikipedia.org/wiki/Steve_Keen#Debunking_Economics
          https://www.youtube.com/user/ProfSteveKeen

          https://en.wikipedia.org/wiki/Ha-Joon_Chang
          https://soundcloud.com/declarationspod/economic-rights
          https://www.theguardian.com/commentisfree/2017/jun/01/myths-money-british-voters-economy-britain-welfare

          https://en.wikipedia.org/wiki/L._Randall_Wray
          https://youtu.be/O03xa40oDPI
          https://youtu.be/4saOd32KeCA
          https://youtu.be/fayx7REPvEU

          OR.. if you want a 6 and-a-half minute primer on how a Modern Monetary system in order to get your feet wet, you can go here:

          https://youtu.be/90l0Z_tEsQk

          • You didn’t get the point about the German social market economy resting on ordoliberalism, which both support the goals you mention above? These are linked to Hayek – which is at odds with the anarchist and libertarian interpretation of Hayek in the anglosaxon world.

            Ever been to Germany and seen it operating?

            Democratic Socialism is by the way a fascist concept as evidenced e.g. in the former DDR – or Eastern Bloc – ask the Polish or Hungarians whether they liked it.

            And: How about explaining it once in your own words showing that you understand what you hurl at others?

          • Mankind Global Media

            And yet, many in the Eastern Bloc still tack pretty much left when it comes to whom they put in office. The Polish strike me as pretty well stuck on the right, possibly because of their close proximity to Russia.

            I don’t see anything in your paragraph that you quote above that convinces me that Ordoliberalism is not just another name for Neoliberalism. Ask the Greeks or Italians whether they liked it 😉

            ETA: I see the way Keynesianism is lumped together with Nazism and Communism, which as far as I’m concerned blows your credibility completely to shreds. You’re long on verbosity but short on substance.

          • If you apply ‘ordoliberalist’ / social market economy views to e.g. the EU and esp. Eurozone, these clash. (I.e. the views of how it should be done and how the concepts were implemented and evolved).

            That’s why there was before the political implementation and today is strong critique of German economists from left and right, though these are obviously not the only ones criticizing the Eurozone implementation and evolution. Among others, it would require fiscal balancing mechanisms and probably tighter institutional integration (which was and is hardly accepted by the voters in any country as far as I remember (‘We should pay for country ‘x’? ‘No way’).

            So let’s turn the question around:
            What would a non-neoliberal ‘system’ look like in your view and words?

          • Mankind Global Media

            https://youtu.be/0G6lwk3HGu0

            Steve Keen at 1:10 – “The UK is where Neoliberalism was born. Maggie Thatcher & her little handbag carrying around a copy of Hayek’s ‘Freedom’ book really set the tone.

            And at 23:02 “The reason we are talking about Brexit is because the Ordoliberal philosophy that really shaped the European Union has been a failure.”

            Color me utterly uninterested in what you’re trying to push, Bud.

  • This post is kind of useful in the way it portrays information theory reasonably well.
    This post is very poor in its presentation of von Hayek’s thoughts, as Guilherme and others accurately note.
    That it ignores evolution is unforgivable.

    Like most works, in attempting to simplify a very complex system, it over simplifies to the point of creating chaos.

    Any attempt to understand anything to do with humans that does not acknowledge the many levels of dimensions of systems that are present in us from our evolutionary history, both in the genetic sense of the structure of our bodies and brains, and in the cultural sense of our language, knowledge, practices, and domains of embodied wisdom; must fail in many important aspects.

    I recall attending a meeting of the International Institute of Fisheries Economics and Trade (IIFET) in 1984, and listening to the mathematical explanations of fisheries markets. They were all simplistic nonsense to me. At that point I had been a practicing fisherman for almost a decade, and had several years involvement in retail and wholesale fish markets. I also had a tertiary background in ecology, computing, psychology and neuro-chemistry. The fish markets I was involved in were all determined by the strength of the human relationships, involving reciprocity and trust far more than money, and money was an important factor.

    Jordan Peterson (YouTube videos abound) has a great understanding of many of the levels of evolved embodied cognition and information processing present in the human brain, and the ways in which social interaction instantiates probabilities to particular classes of relationship and action.

    Every individual human being is a constantly evolving entity, with infinite potential for the emergence of new levels of complexity.

    This morning I was trying to explain to my wife and a friend the way in which genetic and social evolution provide us with contextually relevant heuristic hacks to simplify the complexity present down to something we can consciously handle. She misunderstood what I was saying, and asked me what a “heuristic axe” was. The idea of someone taking a heuristic axe to a forest of complexity was actually quite appealing to me.

    Evolution, both genetic and cultural, has certainly supplied us with many levels of such “axes”, that allow us to make the sort of sense that we do of the forests of complexity within which we find ourselves (which are just mind numbingly vast when you actually seriously look at them).

    Those hacks must have worked in our past, for us and or our ancestors, to get us here – that is kind of the definition of evolution – the differential survival of that which works most frequently with least cost in terms of time and materials, across all the contexts encountered over deep time.

    The deep time of reality has exposed our ancestors to a vast array of strategic environments with existential implications. As a result, we all carry deep sets of context sensitive responses to many different sorts of contexts (our demons and monsters, as well as our better angels).
    Cooperation was a fundamental part of that, at many levels.
    Strategies to identify and punish strategies that cheat or prey on the cooperative are deeply embodied at many levels (as Axelrods logic demands).
    Cheating on the cooperative is a very high risk long term strategy (however high the short term payoffs may seem, and however long one has “gotten away with it”).

    The assumption by economics that there is any sort of equilibrium present is fundamentally flawed, and of course there are approximations to equilibria in complex highly dimensional strategic landscapes (John Maynard Smith’s multiple stable state equilibria is a low dimensional approximation to something with vastly more dimensions – thousands of them).

    Every human brain is a computational system, optimised over vast time, for solving complex problems of social relationship within multiple simultaneous hierarchies while simultaneous meeting low level bodily needs for food, water, shelter, etc.

    That simple fact is why decentralised capitalism has worked better than any sort of central planning, and must always do so.

    It is our human need to seek simplicity, even where it does not exist, that has lead to may of many of the profound mistakes of economics and economic thought.

    The invention of the abstract notion of value embodied in money (a very useful myth in a very real sense), is both one of the most profoundly useful tools ever invented, and also one of the greatest sources of existential risk when taken too simply.

    For the notion of money to work, everyone has to have the ability to get enough.
    Automation is making that impossible in a free market. We can now fully automate systems faster and easier than we can train people.
    For money to retain utility, it needs to be distributed to all in a reasonable fashion. Not a demand that everyone have only the same, but a demand that everyone have a necessary sufficiency. We know what will happen to distributions thereafter, and that is fine, provided everyone can live with what they have to start with (ongoingly).
    If the money system is to survive, there must be some form of universal basic income (UBI), and freedom will produce diversity from that base.

    Regulations need to be the minimum set necessary for the survival of the complexity present, and that is unlikely to be a small number, considering the large and expanding dimensions of complexity currently existing. So necessarily ongoing tensions and needs for conversations and negotiation of agreements.

    And some things are likely to be eternal – like a respect for individual sapient life (human and non-human, biological and non-biological), and a respect for individual liberty (within the necessary minimum set of constraints required to sustain the many levels of complexity present – requiring responsibility in ecological and social contexts from all individuals).

    The hardest thing for many cultures to accept will be the expanding diversity that must result – all dimensions.

    And such freedom is not without constraints.
    Constraints are essential for complexity to exist.
    Without constraints, everything returns to amorphous goo.
    Every new level of complexity will have a necessary minimum set of constraints for its survival.
    In a social context, morality is such a necessary minimum set of constraints – which is not to say that any existing particular moral system is such a minimum necessary set of constraints (that is actually highly improbable).
    At higher levels of complexity it has an analog in the set of attendant strategies necessary to prevent the cooperative being over-run by cheating (exploitative) strategies. And that will always be something of an evolutionary arms race, all dimensions, emergent and yet to emerge.

    So currently economics still seems to be fundamentally flawed in the way in which it conceptualises value, and for its addiction to markets as a measure of value.

    And has a base from which to evolve, it has a certain utility.

    • @TedHowardNZ:disqus though I am tempted to add to or twist some details, simply great!

      @newqueuelure:disqus any comments from your side how you would incorporate that in an algorithmic approach?

  • RM1948

    I’m very naive on economics but here’s a basic question due to this article based so much on information theory. Shannon’s work defined the maximum throughput of a communication channel. Can that be applied to market theory?

    My naive understanding is that market theory is based on the transfer of complete information between buyer and seller. (I believe more advanced discussions incorporate incomplete information.) So how could you apply Shannon’s bandwidth limitation to market theory?