More on AIC and slippery slopes

Marc Mazerolle, whom I criticized on p. 216 of the book for saying that Akaike weights were probabilities (rather than being "similar to" probabilities or other such waffle-words), sent me a nice e-mail to point out that (1) the relevant chapter of his thesis is published in Amphibia-Reptilia [1] and (2) David Anderson's book, Model-based inference in the life sciences [2] explicitly says (on p. 88) that "A given $w_i$ is the probability that model $i$ is the expected K-L best model". (Please excuse the bibliographic glitch, if you're looking at this on the main blog page: it's a technical issue I don't know how fix.)


Well, Mazerolle is right (that Anderson says weights are probabilities): I don't have the book (although I should probably buy it), but I was able to look the page up in Google books. Scary as it is for me to criticize such a smart guy/big shot, I think Anderson is being sloppy here (which is unusual): he starts out by saying

These weights are also Bayesian posterior model probabilities (under the assumption of savvy model priors) …

(which is correct) but then slides into the sentence quoted above,

A given $w_i$ is the probability that model $i$ is the expected K-L best model

As far as I know, this statement is only true with that particular choice of "savvy" priors (which IMO are fairly odd — you can read about them in Burnham and Anderson's work), and is only true asymptotically (i.e. when the data set is very large: I'm not sure about this, but I think so, by analogy with other criteria like the BIC).

In thinking about the bottom line, I thought of the phrase "you can't have your cake and eat it too" — which as it turns out is exactly what I said in the original footnote referring to this mistake:

Taking AIC weights as actual probabilities is trying to have one’s cake and eat it too; the only rigorous way to compute such probabilities of models is to use Bayesian inference, with its associated complexities (Link and Barker, 2006).

So there!

1. Mazerolle, Marc J. 2006. Improving data analysis in herpetology: using Akaike's Information Criterion (AIC) to assess the strength of biological hypotheses. Amphibia-Reptilia 27 (March): 169-180. doi:10.1163/156853806777239922.
2. Anderson, David Raymond. 2008. Model Based Inference in the Life Sciences: A Primer on Evidence. Springer.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License