Bad science (or never let the truth get in the way of a “good” story)

Senior Conservation Policy Officer, Richard Evans, has this response to a recent article on sea eagles in the Daily Telegraph.

Bad science (or never let the truth get in the way of a “good” story)

White-tailed (sea) eagle David Tipling (rspb-images.com)

Last month, we blogged [1] about a piece in the Daily Telegraph[2] that suggested that more birds of prey were killed by wind turbines than by gamekeepers.  We pointed out that the paper had fallen into the trap of selection bias[3], because birds killed by turbines were far more likely to be reported than birds killed by gamekeepers.

Well, the Telegraph is at it again, this time beavering away at the hoary (but seasonal) old chestnut of white-tailed (sea) eagle diet[4].  According to the press release[5] from SNH and partners (Forest Enterprise Scotland, National Farmers Union of Scotland and RSPB Scotland), a camera placed by FCS at a sea eagle nest on their land to record prey brought in by the adults showed:

“The sea eagles brought a total of 117 prey items to the nest between January and July. Analysis confirmed that 67 items were unidentifiable, 21 were mammals, 14 were birds, 7 were fish, and either 8 or 9 were lambs.”

The Telegraph headlined their story: “Sea eagles eat more lamb than fish, despite their name, according to research.”  “Eight or nine” is more than seven, so the Telegraph is right this time, surely?

Ummm, not when the data are subject (just like the wind turbine and illegal persecution data) to selection bias.  Let’s look at those figures a little more closely.  Firstly, well over half the items recorded carried to the nest by cameras were “unidentifiable”.  This is not particularly surprising.  In the experience of most RSPB staff with sea eagle nest watching experience, including the use of cameras, it can be very difficult to identify some prey items, particularly if they are small and nondescript in colour.  A bit like most inshore fish and many bird species known to be taken by sea eagles.  And not in the least like lambs, which (as many RSPB staff with sea eagle nest watching experience can confirm) tend to appear very obviously large and white, and on camera also woolly.

But eight (or nine) is still more than seven, right?  Actually, almost certainly wrong when the fractionally smaller category (7 items) is much more difficult to identify than the marginally larger one (8 or 9 items); and when (as here) most items were unidentified, and likely to include many from the difficult-to-identify category (here fish; and previously, illegally-killed birds of prey). 

But it gets worse.  This time the Telegraph is guilty of a special type of selection bias commonly known as hasty generalisation[6].  As any fool knows, you can’t (or at any rate shouldn’t) generalise from a sample of one.  But that is precisely what the Telegraph article does, by applying the (incorrectly interpreted) results using a single technique, from a single year and a single site, to all sea eagles everywhere.

We have known for a long time[7] that some sea eagles take some lambs, some of which are taken alive (but at least some of which in turn are in poor condition).  We also know that there’s great variation between the diets of different pairs. So while the latest camera deployment tells us something about the diet of this pair in 2014 (mainly that we don’t actually know most of what they ate), it doesn’t allow the wild extrapolation indulged in by the Telegraph in this article.

Which all goes to show that you can take a horse to water, but you can’t make him think. Or something like that.