Why Monsanto’s attempt to “disappear” tumours by using historical control data is invalid

An Earth Open Source briefing, 26 September 2012

Monsanto has invoked “historical norms” to dismiss findings of increased tumours and mortality rates in rats fed GM maize NK603, as well as in rats exposed to levels of Roundup claimed by regulators to be safe, in a 2-year study by Professor Gilles-Eric Seralini’s research team in France.

Monsanto says that the increased mortality rates and tumour incidence “fall within historical norms for this strain of laboratory rats, which is known for a high incidence of tumours”.

By “historical norms” and “within this historical range”, Monsanto means historical control data – data from various other studies that they find in the scientific literature or elsewhere.

However, the use of historical control data is an unscientific strategy used by industry and some regulators to dismiss statistically significant findings of toxicity in treated (exposed) groups of laboratory animals in toxicological studies intended to evaluate safety of pesticides, chemicals, and GMOs.

The only scientifically valid control for such experiments is the concurrent control, not historical control data. This is because scientific experiments are designed to reduce variables to a minimum. The concurrent control group achieves this because it consists of animals treated identically to the experimental group, except that they are not exposed to the substance under study. Thus, the only variable is exposure to the substance(s) being tested – in the case of Seralini’s experiments, NK603 maize and Roundup.

With this experimental design, any differences seen in the treated animals are very likely to be due to the substance being tested, rather than due to irrelevant factors, as is the case with historical control data.

Historical control data consists of a wide range of data gathered from diverse experiments performed under widely differing conditions. As a result, factors totally irrelevant to the study are responsible for the majority of differences in historical control data. Such factors may include environmental conditions; different diet for the animals; different pesticide residue exposures; different genetic background of the animals; even different years in which the experiments were performed, which is known to affect results for reasons that are poorly understood.

In contrast, using the concurrent controls reduces such variables to a minimum and enables researchers to reach evidence-based conclusions about the effects of the substance being tested.

For these reasons, toxicological studies performed by independent (non-industry) scientists and published in the peer-reviewed literature hardly ever invoke historical control data. They certainly do not use it to dismiss statistically significant findings of harm in treated groups of animals.

Those who do use historical control data in this way include industry-affiliated sources and some regulators. The practice was introduced into risk assessment by the Organisation for Economic Cooperation and Development. Even the OECD, however, advises caution in the use of such data and warns against it in evaluating findings of tumours (such as those seen in Seralini’s study). Even by weak OECD standards, Seralini’s findings are valid.

Even if we were to follow Monsanto’s recommendation and use historical control data in evaluating Seralini’s findings, the historical control data cited by Monsanto is invalid because it relates to rats of a different origin (SD rats from Charles River Labs) than Seralini’s rats (SD rats from Harlan). Seralini took historical data on the Harlan SD rat fully into account in his study – and the results still show a marked increase in tumours and other effects. The tumour incidence in the test groups in his study was overall around three times higher than the normal rate observed in the Harlan SD rat strain he used, as reported in the literature.

Finally, the “tumour-prone rat” argument used by Monsanto and others to dismiss Seralini’s findings of increased tumours is spurious. The key point about Seralini's tumour findings was that the controls got some tumours, but the treated groups got significantly more tumours, and these appeared sooner and were more aggressive than those of the control groups.

So what matters in this study is not whether the rat strain was or was not prone to develop tumours, but

  • the earlier and more rapid rate of tumour development in the treated groups of animals, compared with the concurrent control, and
  • the larger number of tumours caused by the treatment, over and above the "spontaneous" background level.

To illustrate the point by analogy: a small proportion of people who never smoke get lung cancer. If you smoke, your risk of getting lung cancer is about 12 times higher than if you don't smoke. The measurement is called a "relative risk". So even if there were an ethnic group of people with a higher rate of naturally occurring lung cancer, if people in that group smoke, their rate of lung cancer will still increase like that of everyone else.

This is a basic principle of science and it is worrying that attempts are being made by pro-GM lobbyists to override it in the interests of keeping the products of powerful multinational biotechnology companies on the market.

The responsibility now lies with Monsanto to pay for a full 2-year carcinogenicity study on its NK603 maize and the associated pesticide, Roundup. Such a study would, however, have to be carried out, not by industry or its contracted labs, but by independent scientists commissioned by an impartial publicly funded body consisting of a wide range of stakeholders representing the public interest.

In the meantime, NK603 must be immediately withdrawn from the market and all GMOs must be subjected to long-term testing.

Read the full report here...


+17 #6 Shari 2012-10-10 04:20
Just listeneing to jargon and acusations here makes me want to shake my head. What is wrong with a new long term independant study people? If in doubt check it out. we are talking about the food of millions of people. No one was asked or notified about the GMO entry into the food chain, and my children will not be made the guinea pigs of wealthy corporations thank you. Long term studies, unbiased and independant will prove its safety
+9 #5 Claire bleakley 2012-10-05 23:37
Richard, you say "Another major criticism is that the Seralini group preceded their paper's publication by creating a media storm, ensuring that their invalid results would be remembered by activists regardless of the subsequent scientific conversation in the literature".
I believe that the "Daisy the BLG -cow" was subject to the same media storm -and there was no prior notification to others to give their views on this. Please could you provide some concrete evidence of "invalid results" and what you mean by this?
"The sample sizes used in the study are too small for his results to be statistically significant - by using the correct statistical treatment you see that the differences in tumour rates between the treatments were *not* significantly different".

Please could you provide the correct statistical treatment that you beleive should have been conducted on this trial?
+11 #4 Lise 2012-09-27 09:56
There's a simple answer to the "sample sizes too small" argument. Monsanto must pay for a group of independent scientists to do a 50 animal in each group carcinogenicity study on every GMO including NK603. Monsanto can't carry out the trial though because experience shows industry studies are biased. So the money should go to an independent body which then commissions the independent scientists.
But Monsanto must not have market access for NK603 in the meantime.
+11 #3 Chandra Pegler 2012-09-27 08:52
Richard, some of us are looking forward to the "subsequent scientific conversation in the literature": we wonder how many of the criticisms of the Seralini study will make it through peer review, especially as whatever criticism is levelled at it, the study comes out as much more extensive and careful than anything Monsanto has done on its own inadequately tested products.
It's interesting how the peer-reviewed literature tends to sort out the wheat from the chaff, in the end. With that in mind it is interesting to see that Pusztai's results on GM potatoes from the 1990s are still cited as perfectly valid in the literature (and indeed as one of the most informative papers on the toxic effects of that GM food), in spite of the hysterical yelling that the pro-GM lobby did against his study.
+13 #2 Claire Robinson1 2012-09-27 08:27
@Richard Smith
Thanks for your comment. We disagree that we have "missed the point of the criticism of the Seralini study". Monsanto is lobbying against the study by invoking historical control data, which is why we were forced to address it, albeit we would prefer never to mention this deeply unscientific topic.
"Statistical cheating" is a serious and potentially libellous allegation and one that Seralini's statistician would have to answer. The statistical approach used is, however, accepted as valid by other researchers and we've had feedback from statisticians and cancer researchers that they consider Seralini's results valid.
As we have pointed out, Seralini controlled for the historical rate of tumours in the rat strain/origin of rat he used: SD rat from Harlan labs. Still the results were statistically significant.
Re 'too small sample size': Seralini had as many rats in his groups as Monsanto analysed in their 90-day trial, so do take this argument up with Monsanto. The sample size of 10 male, 10 female rats is not the OECD carcinogenicity protocol sample size (Seralini's study was not set up as carc study) but it is the same as the chronic tox bit of the OECD 453 chronic tox/carc combined. This protocol includes tumours as a valid endpoint. If you think OECD should include more animals in its 453 chronic tox protocol, please tell OECD.
Interestingly many independent cancer studies on animals use sample groups of only 4-5 animals and these are considered valid. It depends on how tightly you control for variables. If suddenly, since Seralini's study, these studies are not valid, we'd better tell all the cancer researchers immediately!
-12 #1 Richard Smith 2012-09-27 07:21
I agree about historical controls not always being appropriate, but you've completely missed the point of the criticism of the Seralini study.

The sample sizes used in the study are too small for his results to be statistically significant - by using the correct statistical treatment you see that the differences in tumour rates between the treatments were *not* significantly different.

The Seralini group have fudged the results by statistical cheating - they used a complex statistical approach to hide the fact that it's the wrong one. That's the main criticism being levelled at the paper.

In this context, it *does*matter that they used this rat strain, and if you can find well normalised historical data to make a comparison, there is a valid one to be made, as you can demonstrate that, for the sample sizes used, you would expect the variation that was recorded to happen due to chance.

Another major criticism is that the Seralini group preceded their paper's publication by creating a media storm, ensuring that their invalid results would be remembered by activists regardless of the subsequent scientific conversation in the literature.

Add comment

Security code