Monday, November 23, 2009

Lies, Damned Lies and Statistics


Foreword

Have you ever noticed how, according to newspapers, politicians or sundry pundits, everybody is wealthier, and yet, you seem to have difficulties making ends meet?

Well, I have. My first reaction was to question myself. But, in truth, although there was room for improvement, it wasn't enough to explain everything. After some reflection on the issue, I discovered that indeed something was wrong with those newspapers, politicians and sundry pundits.

And as regularly as I can, I will be publishing short articles about this.



Imagine only people...

You and I decide to go to this new chicken place, just around the corner, for lunch.

We order a whole Portuguese chicken, with peri-peri sauce, chips, salad and two beers. Altogether, $44.50.

You are about to dig into your food, when the mobile goes off. "I've gotta take this call", you offer as explanation, as you go outside.

Five minutes later you are back at the table, but where's your food? It's gone! Neither chicken, nor chips, an empty plate, a small plastic plate, next to, where you see a bill, a $20 note and a couple of coins.

Completely astonished, you barely manage: "Wh... whaaat?"

"Well, we had one chicken and we're two. So, on average, each of us had half a chicken. You pay for your half and I pay for mine", I reply sheepishly, after finishing YOUR beer.

Now, before you rush to deny that you would ever fall for that kind of scam, I've got to tell you: I'm sorry, but you're wrong. There is a good chance that you have fallen for this, many times. If this is a consolation, you're not alone.



Some Really Simple Statistics

Let's talk a bit about statistics. Don't be afraid: it won't hurt a bit.

Imagine two islands: Patagonia and Utopia. Both have equal populations (say, 10 inhabitants each) and equal total incomes ($10, each island).

So far, you can't tell one island from the other on the basis of their total incomes, as they are equal.

Now we introduce differences: in Patagonia all income goes to one person (say, the other nine are slaves); while in Utopia the aggregate income is equally divided between the 10 inhabitants.

This information is contained in the table below. The first column is simply the individuals' IDs, and the second column is their individual incomes (one column for Patagonia, another for Utopia).



If you go through the trouble, you'll find that both populations have the same average income: $1 = $ 10 (total island income)/10 people (island population). Still, while in Utopia everyone gets exactly the average, in Patagonia no one gets it: most get less (nothing, in fact) and one gets much more (does this remind you of your half chicken?).

If you have followed up to this point, you can already understand this:

(1) Patagonia and Utopia represent two extreme possible income distributions: absolute inequality (Patagonia) and absolute equality (Utopia).

(2) Real life income distributions are invariably intermediate cases between absolute equality and absolute inequality.

(3) Regardless, average measures of income (or wealth, or prices, or many other variables) are not good indexes. And they become more and more meaningless, the more a real case differs from absolute equality.

(4) There should be better measures of how your "average Joe" fares income-wise, one should hope. In fact, in most cases, there are such measures: the median income is an extremely simple one.

You might ask: what exactly is the median? In our example, is the individual income that's larger or equal than 50% of the remaining cases, and smaller or equal than 50% of the remaining cases. If we had nine individuals (instead of the 10), sorted according to their incomes, from lowest to highest, the median would be the income of the 5th individual: 4 individuals have lower or equal incomes, and 4 have higher or equal incomes.

In our case, though, as we have 10 individuals, the median is the income of the 5th individual plus the 6th, divided by 2: $0 in Patagonia and $1 in Utopia.

This little table shows together the average and median incomes in Patagonia and Utopia:


So, whenever you hear someone talking about incomes (or wages, wealth, bank account balances, taxes paid, prices) you need to know more details. A general statement (say, "incomes increase") means little, obviously. The average quantity doesn't add much more either, as you can see in the Patagonia example.

That's when you ask: "Ok, that's all good and well, but what's the median?"

If the median is lower than the mean or average, as it is in the Patagonia example, you know that a few people are earning much more than others.

If, on the other hand, no one cares to inform you about the median, even though it's reasonable to expect that such information exists, you have a good reason to believe that there is something wrong with the information provided.



Half a Chicken, on Average?

I'm sure you have solved those spot-the-differences puzzles from newspapers. Let's see if you can spot where the information provided is faulty or misleading. The quotes below come from real-life official media releases, op-eds, or news stories:

"In addition, real wages have increased by 20.8 per cent. This compares with the 1.8 per cent decline in real wages recorded over the whole 13 year term of the previous Labor Government".
Media release. Department of the Treasury (The Hon. Peter Costello, Treasurer). 30/08/2007.

"Figures released last Friday reveal another dimension of this wealth explosion in terms of financial assets like cash in the bank, shares, bonds and superannuation holdings. Every man, woman and child has, on average, almost $60,000 worth of these assets after debt is accounted for - and this has grown by almost half in the past two years".
Steve Burrell. "We're all richer. We just don't realise it". The Sydney Morning Herald, page 18, 02/10/2007.

"Bureau of Statistics figures show net household financial wealth (assets less liabilities) hit $1.2 trillion in the June quarter, up 23.3 per cent over the year. Breaking that down on a per capita basis, the financial wealth of each Australian hit a record $57,400 at the end of June".
Nassim Khadem. "Personal wealth bursts out all over". The Age, page 3, 29/09/2007.


Now, you might ask: does it matter? I believe it does. Governments need to justify their decisions; bosses need to pretend they are fair. It's our choice to believe them on blind faith.

A further resource on basic statistics is:

Statistics Every Writer Should Know. A simple guide to understanding basic statistics, for journalists and other writers who might not know math.

Friday, November 13, 2009

The Famous Efficient Market Hypothesis

A few days ago, surfing the net, I found Rortybomb. Rortybomb is a US blog, presenting some seriously interesting discussions. Although American, I would highly recommend it to Australian readers, especially those interested in economics: its posts are thoughtful and the subsequent discussions are often quite relevant.

One such discussion (Who believes market efficiency?) introduced a link to the Fama/French Forum. In it, Prof. Eugene Fama defended the Efficient Market Hypothesis, in the following terms:

"The premise of the Fox book ['The Myth of the Rational Market'] is that our current economic problems are largely due to blind acceptance of the efficient markets hypothesis (EMH)…

The book is fun reading, but its main premise is fantasy. Most investing is done by active managers who don’t believe markets are efficient. For example, despite my taunts of the last 45 years about the poor performance of active managers, about 80% of mutual fund wealth is actively managed. Hedge funds, private equity, and other alternative asset classes, which have attracted big fund inflows in recent years, are built on the proposition that markets are inefficient. The recent problems of commercial and investment banks trace mostly to their trading desks and their proprietary portfolios, and these are always built on the assumption that markets are inefficient. Indeed, if banks and investment banks took market efficiency more seriously, they might have avoided lots of their recent problems. Finally, MBA students who aspire to high paying positions in the financial industry have a tough time finding a job if they accept the EMH.

I continue to believe the EMH is a solid view of the world for almost all practical purposes. But it’s pretty clear I’m in the minority. If the EMH took over the investment world, I missed it."


Now, let me start by saying that I haven't read Fox's book. So I'll abstain to comment Prof. Fama's views on it.

But the substance of Prof. Fama's comment was his belief on the adequacy of the Efficient Market Hypothesis as a theoretical description about how markets, in general and financial markets in particular, work.

And here I believe I am as entitled a comment as the guy next door. In fact, I did leave a comment at Rortybomb.

As I am starting my blog and it takes me quite some time to write one of these pieces (which I try to research to a reasonable extent), I imagine Rortybomb will not mind if I reproduce my comment here.

----------------------------------------------------
"This statement by Prof. Fama is bizarre...

A few years back (2004) Prof. Fama tested the CAPM and found that:

"Unfortunately, the empirical record of the model is poor - poor enough to invalidate the way it is used in applications". [1]

From the same paper, two of the assumptions of the CAPM are:

"The first assumption is COMPLETE AGREEMENT: given market clearing asset prices at t-1, investors agree on the joint distribution of asset returns from t-1 to t. And this distribution is the true one, that is, the distribution from which the returns we use to test the model are drawn". (My capitulars).

For those not too familiar, this is the Efficient Market Hypothesis (EMH).

Now, if a model derived from the EMH fares poorly in empirical tests, then I would say the EMH from which it derives becomes suspicious. Thus, Prof. Fama's claim ("I continue to believe the EMH is a solid view of the world for almost all practical purposes") is surprising, to say the least.

Still following the text above, Prof. Fama also claims that "most investing is done by active managers who don't believe markets are efficient".

If financial practitioners have abandoned the EMH -as Prof. Fama claims, without providing any evidence- it is a predictable and indeed understandable consequence of this poor performance of the CAPM, as measured by Prof. Fama himself.

But this disuse of the EMH has an striking consequence: if financial practitioners have abandoned the EMH, then in the terms used by Prof. Fama to describe the EMH, "there is NO complete agreement"...

So, we move full-circle and fall into what, to me, seems like a logical contradiction.

Note also that in a paper widely acknowledged as one of the foundational texts of modern economics, Prof. Friedman states that economics is a positive science [2]. By this it is meant that economics, among other things, is purported to describe economic phenomena which are independent of human desires. According to this view, the EMH empirical validity should not depend on financial practitioners deliberately following the EMH.

In this context, Prof. Fama's statements ("Indeed, if banks and investment banks took market efficiency more seriously, they might have avoided lots of their recent problems. Finally, MBA students who aspire to high paying positions in the financial industry have a tough time finding a job if they accept the EMH") seem not only oddly irrelevant, but they also seem to imply that the EMH is valid, only if people think it's valid.

These contradictions, by the way, seem to plague many defenses of the EMH. Prof. Robert Lucas, another renowned economist, has recently defended the EMH in the following terms: If people knew a bubble was going to burst next week, they would act accordingly now, bursting the bubble now, not next week.

Unfortunately, the text containing this defense (R. Lucas. "In defense of the dismal science". From The Economist print edition. Aug 6th 2009 [3]) is currently subscribers' content only. Anyway, for those subscribers, I include the link below.

One observes in Prof. Lucas' reasoning the same circularity present in Prof. Fama's. As in Prof. Fama's reasoning, Prof. Lucas seems to acknowledge that bubbles exist, although the EMH implies that bubbles cannot exist. This in itself seems like another contradiction, to me.

But, on top, this reasoning is factually flawed: people did know the bubble was going to burst, they acted accordingly, and the bubble did not burst as a consequence. Instead, it went on growing until it was ready to burst by itself.

Dirk Bezemer, in a recent paper [4], identifies 12 economists who did see the current crisis coming.

And, no, I am not one of those economists. Unfortunately.

Cheers,

Marco

References

[1] Eugene F. Fama, Kenneth R. French. "The Capital Asset Pricing Model: Theory and Evidence". 2004.
CRSP Working Paper No. 550; Tuck Business School Working Paper No. 03-26
Working Paper Series.
http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=998#hide440920

I understand a version of this paper was published in the Journal of Economic Perspectives, Volume 18 (2004) Pages 25-46.

[2] Milton Friedman. "The Methodology of Positive Economics"; in: Friedman, Milton: Essays in Positive Economics, University of Chicago Press, p. 3-43. 1953.

[3] Robert Lucas. "Economics focus. In defense of the dismal science".
From The Economist print edition. Aug 6th 2009.
http://www.economist.com/businessfinance/economicsfocus/displaystory.cfm?story_id=14165405

[4] Dirk J. Bezemer. "No One Saw This Coming: Understanding Financial Crisis Through Accounting Models".
Groningen University. 16 June 2009.
MPRA Paper No. 15892.
http://mpra.ub.uni-muenchen.de/15892/"

----------------------------------------------------

Wednesday, November 11, 2009

About the Magpie

Well, maybe it's best to see this first post as a personal introduction and a "mission statement" of sorts.

I am a 48 yo male and I need to work for a living.

In a previous life I was considered upper middle-class and I had post-graduate education (Master's and started a PhD), but that was long ago.

Now I work manually and I could hardly be described as anything but low income earner.

Ideologically, I describe myself as an heterodox Marxist. This shows in my personal philosophical posture, which tends to be nihilistic. If you don't know much about Marxism, you'll probably be surprised to hear that Marxism is an extremely optimistic worldview. But, if you don't know much about it, you'll have to take my word for it.

I, in contrast, am deeply pessimistic. And I attribute this in part to my nihilism.

Which leads me to the next point of this post: I am writing this because I believe the world is ill. Maybe terminally so. I realize there is little I can do to make a difference. But I want to try and the only means I have to effect a difference is through my voice.

Don't get me wrong: I am not a profoundly "idealistic" character (idealistic in the sense of altruism). Marxists are often "idealistic" in this sense, although they are likely to deny it.

I am doing this because, as a good nihilist, I believe life has no intrinsic purpose. It's up to us to provide our life with one. So, as a good Magpie, I choose to fight, in the hope this may to some degree justify my own existence.

My promise to the reader is that I will try to be informative and fair.