Miscellaneous thoughts

Mistakes in the first R session

by bbolkerbbolker 13 Jan 2010 04:45: 0 comment(s)

Several things that go wrong and confuse people on the first R session (this was going to be a top ten list but I ran out of ideas).

Pronoun trouble in science

by bbolkerbbolker 07 Dec 2008 21:07: 0 comment(s)

Rabbit-seasoning-mm.jpg

Here are some quick (and hopefully, interesting) thoughts on two separate kinds of pronoun trouble that have come up in my graduate teaching recently, (1) the ever-controversial "singular they" (i.e., using "they" to denote a single person of indeterminate gender) in referring to author(s) of a scientific paper and (2) use of the first-person singular in single-authored papers.

(continue)

More on AIC and slippery slopes

by bbolkerbbolker 07 Dec 2008 17:29: 0 comment(s)

Marc Mazerolle, whom I criticized on p. 216 of the book for saying that Akaike weights were probabilities (rather than being "similar to" probabilities or other such waffle-words), sent me a nice e-mail to point out that (1) the relevant chapter of his thesis is published in Amphibia-Reptilia [Bibliography item Mazerolle2006 not found.] and (2) David Anderson's book, Model-based inference in the life sciences [Bibliography item Anderson2008 not found.] explicitly says (on p. 88) that "A given $w_i$ is the probability that model $i$ is the expected K-L best model". (Please excuse the bibliographic glitch, if you're looking at this on the main blog page: it's a technical issue I don't know how fix.)

(continue)

Dynamite plots: unmitigated evil?

by bbolkerbbolker 04 Sep 2008 15:41: 5 comment(s)

There seems to be a general opinion among statistical graphics nerds (examples here, here, and here) that the traditional way of plotting grouped continuous data (e.g. growth rates across fertilizer treatments) as a bar plot with a unidirectional "whisker" denoting the upper 95% confidence interval is bad. People who don't like them call them "dynamite plots". (Googling for 'dynamite plot' brings up web pages about statistical graphics, the Napoleon Dynamite movie, and terrorism [Basque and 19th-century].)

dotbarbox.png

I will review the criticisms of dynamite plots, which I generally agree with, but then want to put forward a couple of their advantages, and suggest that the generally favored Tukey box-and-whisker plot is not a universal solution to graphical problems.

(continue)

AIC vs BIC

by bbolkerbbolker 11 Aug 2008 13:52: 5 comment(s)

I've been stumped for quite a while trying to decide what the criteria really are for when one should use AIC vs BIC. Burnham and Anderson talk about it quite a bit, but they are such staunch AIC partisans that it took me a while to come around to their point of view. The main reason that I would have preferred BIC is that, if you look at the derivations, BIC approximates the log of the marginal likelihood for a large dataset with an uninformative prior, while AIC approximates the same thing — but with a very strong prior (see p. 212-213 of the book, or Kass and Raftery 1995). From this point of view, the BIC seems more sensible.

On the other hand, B&A make a compelling argument that BIC was developed to identify the "dimension" or "true number of parameters" of a model, and that this is rarely sensible in ecological modeling contexts because of what B&A call tapering effects. That is, if you have a large number of predictor variables some of which have non-zero effect sizes (e.g. regression coefficients) and some of which have zero coefficients, the BIC is trying to tell you how many are non-zero. B&A point out that it is more common that there will really be a few coefficients with large magnitude, more with smaller magnitude, even more with tiny magnitudes … and that all of the predictor variables really have some effect on the response, albeit very small. What we should be trying to do, they say, is identify how many parameters are useful for prediction rather than how many are non-zero. (This also agrees in general with the Bayesian argument against point null hypotheses, i.e. that parameters are never exactly zero — somewhat ironic, since it suggests that Bayesians would prefer AIC over the "Bayesian" Information Criterion.)

The bottom line: I would say the AIC is generally the right choice for ecological questions, over BIC, unless you're really trying to identify a specific number of components. (People do this in time-series analysis, to try to identify the number of time lags or interacting species, although I think they probably shouldn't — the "tapering effects" argument really applies here.)

Is AIC frequentist?

by bbolkerbbolker 07 Aug 2008 03:58: 0 comment(s)

5 August 2008

Brian Dennis commented to me at ESA today that he doesn't agree with my categorization of AIC etc.-based methods (what I call "model selection") as not being frequentist. I see what he means, but I think the distinction is a bit subtle. Here's a fuller explanation of his point and my feeling about it:

  • AIC is actually an estimate of the expected difference in the "K-L discrepancy" between two models, that is of the difference in the distances between those models and the true model (there are further deep waters here about whether we really need to assume that a "true" model exists at all, but I'm going to skim over them). Therefore, it is an estimate of the change in distance on average, across many hypothetical realizations of the real world — that is, it is really a frequentist point estimate of the change in distance.
  • However, when you actually use AIC you don't say anything about probabilities, or think about averages across many realizations — you just say "this model is estimated to be better than this other model". So it definitely doesn't feel like a frequentist method, since you aren't making statements about the fraction of the time that some specified outcome would happen in many repeated trials (i.e., probability, according to the frequentist definition). So I thought it was easier when introducing it not to call it a frequentist method.
  • The challenge with all these sorts of things is to give a definition that (1) most people can understand and (2) is technically correct even if it glosses over some details. I think I might have failed on criterion #2 here, I will try to find a better way to describe it that still gets the point across.

(Updated 6 August, BD kindly suggested some changes.)

ESA 1

by bbolkerbbolker 07 Aug 2008 03:57: 0 comment(s)

4 August 2008

I missed Aaron Ellison and Brian Dennis's talk today at ESA about "what literate ecologists should know about statistics" (I opted for Gordon Fox's talk about diversity gradients instead). It would have been at least culturally interesting since BD is a staunch frequentist and AE is a hard-core Bayesian. (Interestingly, most statisticians have got beyond having these arguments. I apologize if either of the authors disagree with my characterization of them.) I mostly agreed with what I saw in the abstract, although I don't draw as bright a line as they appear to between "model-based" and "design-based" statistics. Here's an excerpt:

we suggest that literate ecologists at a minimum should master core statistical concepts, including probability and likelihood, principles of data visualization and reduction, fundamentals of sampling and experimental design, the difference between design-based and model-based inference, model formulation and construction, and basic programming. Because mathematics is the language of statistics, familiarity with essential mathematical tools – matrix algebra and especially calculus – is a must and will facilitate collaborations between ecologists and statisticians.

Maybe I'll post more here if they would like.

[Update August 5: Aaron said something today about publishing it in Frontiers in Ecology (IIRC).]

what's so great about models?

by bbolkerbbolker 01 Aug 2008 23:34: 0 comment(s)

1 August 2008

I really enjoyed reading Breiman 2001 (Statistical Science 3:199-215), in which he says (approximately) that statisticians are too concerned with "true models" and that they should focus on descriptive, flexible, nonparametric models (random forests, generalized additive models, etc etc) instead. Really can't do these topics justice here, but it just reminds me that there's a whole other school (or something) of statistical thought. Philosophically, it ties in with Peters' Critique for Ecology and (perhaps) Jim Clark's recent claims that we should give up on "low-dimensional" models and embrace nature in its noise and complexity [ok, perhaps a bit unfair but I'm in a hurry]. Reasons I haven't focused on such topics in the book: (1) I don't know them that well, so I can't really teach them to others; (2) they are much more complicated computationally, thus leading to a "user" perspective rather than a "builder" perspective (not necessarily a bad thing but not what I was aiming for); (3) they are typically very data-hungry, unsuited to many (but certainly not all) ecological data sets; (4) I like tying models to ecological theory. John Drake at UGA is from the Breimanish (Breimanesque) school, don't know if he has any interesting perspectives written on the topic …

Further thoughts (a few hours later): one way to compare the two statistical approaches is that the "classical" approach is trying to explain or test, while the more "modern" (all in quotation marks) approach wants to describe and predict: as I comment in the book, while these questions often have similar answers, they imply different statistical approaches — and the answers are not always the same. Another related topic is semi-mechanistic models, which Steve Ellner and Simon Wood have variously championed. More later (?)

Bibliography
1. Mazerolle, Marc J. 2006. Improving data analysis in herpetology: using Akaike's Information Criterion (AIC) to assess the strength of biological hypotheses. Amphibia-Reptilia 27 (March): 169-180. doi:doi:10.1163/156853806777239922.
2. Anderson, David Raymond. 2008. Model Based Inference in the Life Sciences: A Primer on Evidence. Springer. http://www.springer.com/life+sci/ecology/book/978-0-387-74073-7.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License