By Daniel Tarade

Scientists are losing a PR battle. Vaccination rates are falling in Canada and abroad. A large number of people continue to deny the existence of climate change. Increasingly, scientific discussions have become politicized and subjected to conspiratorial thinking. What I find surprising is how often a scientific article can be found that supports a controversial scientific opinion. When confronted with conflicting scientific evidence, some people chose either to believe what feels right or instead adopt a nihilistic mindset as clearly science hasn’t the foggiest notion about anything. Both of these solutions reflect diminished trust in the scientific process. What is frustrating to me then is that much of these conflicting opinions that murky the water are published by scientists in peer-reviewed journals. If we want to tout the success of the scientific method, we need to take responsibility for those who game the system. We need to promote scientific literacy and adopt principals of scientific publishing that do not allow for dissemination of self-serving articles written in bad faith. This all seems quite abstract so let us dissect a specific example; “Sucralose administered in feed, beginning prenatally through lifespan, induces hematopoietic neoplasias in male swiss mice.”[i]

I first noticed this article in my sidebar on Facebook, when it was trending in 2016. In fact, the article currently has an incredibly high Altmetric of 735. Altmetric refers to the traction an article receives online, serving as a proxy for impact. The article in question has been re-tweeted 67 times, shared on Facebook 26 times, the subject of three reddit threads, and covered by 81 news outlets. A quick perusal reveals that most social media mentions spread the main conclusion, that Splenda (a sucralose based product) increases the incidence of leukaemia in mice. Of course, I doubt most of these individuals actually read the study, as it is behind a paywall. For $50, a person can read the study that purports an incredibly common food additive causes blood cancer. Another strike against capitalist models of scientific publishing. But, here we are. Thousands of people have been exposed to an article that reinforces that scary notion that artificial sweeteners are cancerous. And look, it’s peer-reviewed. So let us play the role of expert peer reviewers and examine the claim that sucralose increased the incidence of blood cancer in male mice.

First, it is important to dissect the study design. The researchers ordered Swiss mice, allowed co-habitation of male and female mice for five days, and then removed the males. Twelve days following impregnation, the feed of the female mice was supplemented with increasing doses of sucralose (0, 500, 2,000, 8,000, and 16,000 ppm). Now parts-per-million is a strange way to present the amount of sucralose being consumed. Emily Willingham, writing for Forbes, already took it upon themselves to email the authors to clarify the dosing. The lowest dose of 500 ppm corresponds to approx. 60 mg of sucralose per kg of mouse. If the average human is 60 kg, then the lowest dose corresponds to 3.6 grams of sucralose per day, or roughly 60 cans of diet coke. Per day. According to the same article written by Emily Willingham, this amount is also 12x higher than the highest recommended daily intake. This dosing regiment is already clearly flawed if one wants to make a conclusion on the safety of sucralose at levels regularly consumed by humans. This is gap #1. Another gap is that the study is conducted with mice. Although studies in non-primates can be useful for toxicology, any conclusion made about sucralose and humans will be abstracted from the actual observations. This is gap #2. Now let’s get to the results.

Mice were fed sucralose from when they were a fetus to when they died (roughly two year life cycle). Any toxicological studies conducted over a lifetime will make interpretations challenging as older mice have higher rates of cancer. Indeed, background rates of cancer in this study were 61.6%. That much noise makes it difficult to find a genuine effect of sucralose on carcinogenesis. This is gap #3. Upon death, organs were harvested and preserved. Apparently one pathologist examined the tissues (over 50 tissues per mouse) from the roughly 800 mice studied. Considering the flaws in study design, I am not sure if I pity them more or less. From the over 50 tissue samples collected per mouse, they offered up no less than 75 unique diagnoses of cancer ranging from islet cell adenoma of the pancreas to squamous cell carcinoma of the ear duct. Of the multitude of possible cancers being affected by unreasonably high levels of sucralose consumption, only leukaemia were found to be more common in mice being fed sucralose. Further, this increase was found only in male mice (from 1.7% of male mice fed no sucralose to 15.7% of male mice fed 16, 000 ppm sucralose) . However, the overall rate of cancer (of all types) in mice not fed sucralose was unchanged (61.6% of all mice) when compared to those fed 16, 000 ppm sucralose daily AKA 1920 cans of diet coke (61.2% of all mice). Other conflicting data was also present. The malignant cancer rate in female mice actually decreased from 67.6% in mice not fed sucralose to 59.4% at the highest dose of sucralose. This result was not discussed. Despite a decrease in cancer rates for some mice and an overall lack of change in cancer, when looking at all cancer types and all mice, the authors conclusion is that sucralose increases rates of leukaemia in male mice. Based on this observation, they question the overall safety profile of sucralose. This may seem like a fair conclusion. They did observe an increase in rates of leukaemia from 1.7% to 15.7% . However, other errors in reasoning abound.

One reason why the researchers concluded that sucralose increases the incidence of leukaemia in male swiss mice is that a statistical test suggested that there was strong evidence to reject the the null hypothesis, that sucralose does not increase leukaemia incidence. You might wonder why statistical testing is necessary when we can all see that rate of leukaemia is higher in male mice fed sucralose. However, any study will have a limited number of test subjects from which conclusions will be applied to the general group from which the test subjects are derived (i.e. all male swiss mice). This is gap #4 and applies to just about every study. It is always possible that, through random chance, the mice sorted into the sucralose arm of the study randomly developed leukaemia. Weird things happen in small sample sizes. For example, if you flip a coin three times, each time it landing on heads, you could try and make an argument that 100% of all coin flips will result in heads but, in reality, you would need to sample a larger number of coin flips to be confident in that answer. Statistical tests help to predict how often the observed outcome would occur if the null hypothesis were in fact true (i.e. that there is no difference between the control group and the experimental group). In this study, the researchers report that the p-value for the difference between male mice not fed sucralose (control group) and mice fed 16, 000 ppm sucralose (experimental group) is less than or equal to 0.01. This number means that if sucralose had no effect on leukaemia incidence rate in male mice, then in fewer than 1% of experiments would you obtain as large a discrepancy between control and experimental groups as was actually observed (i.e. 1.7% and 15.7%). Thus, the statistical test seems to indicate that it is unlikely for randomness to explain their result, resulting in there conclusion that there is a bona fide effect of sucralose on male swiss mice. However, even this is a statistically dubious interpretation. Consider how many comparisons the researchers were making. They looked at over 75 types of cancer and two sexes. That means the researchers had over 150 opportunities to “discover” a biological effect of sucralose on mice. They found one. To make sense of why their interpretation of the date is improper, imagine you were an epidemiologist tasked with finding clothing items associated with cancer in a sample of 500 humans. You look at size of clothing, type of clothing, color of clothing, etc all tested against the incidence of 75 types of cancer. You run your statistical tests and find that large red T-shirts are associated with clear cell renal cell carcinoma. But why, you ask? How can specifically red T-shirts of a large size be associated with specifically that type of cancer? This is the problem of data dredging. When you analyze hundreds of possible associations, you are increasingly likely to find a statistically significant effect, even if no effect exists. Bayesian statistics are an important tool to assess your confidence in a certain conclusion. In accordance with the scientific method, a hypothesis is generated before a study based on prior observation or inductive reasoning. If your results do not align with your hypothesis, than one has to question your original hypothesis or, depending on how much prior research has been conducted, your own results. A general guideline is that the bigger a claim one makes, stronger and stronger data is required. As there have been prior studies in model organisms (rats, dogs) and epidemiological studies in humans, all showing no correlation between sucralose intake and cancer, it would take reproducible, rock-solid data or a probable mechanism of sucralose-induced carcinogenesis to overturn the consensus that sucralose does not cause cancer.[ii] This study does not provide such evidence.

There are several pervasive issues with this study, ranging from design and dosing regiments to interpretation of the data. Ultimately, the complete lack of difference in cancer incidence between mice not fed sucralose and those which were fed high amounts is completely consistent with the public literature. The conclusion that sucralose increases incidence of leukaemia in male mice not only is poorly predictive of the effect of sucralose in humans (owing to gaps listed above) but is a dubious claim in the absence of a well-reasoned a priori hypothesis about the effect of sucralose on the white blood cell progenitors in the circulatory system of male swiss mice. So why were these conclusions pushed and why was the study published? I can only speculate. However, most researchers are under pressure to publish or perish and it is easier to publish a manuscript on sucralose causing cancer than one that confirms previous studies. As I discussed before, this is because studies reproduction studies confirming previous results are cited less often and for-profit journals are in competition to appear prestigious. The research was lead by Dr. Fiorella Belpoggi, the Head of Research at the Ramazzini Institute and Director of the Cesare Mal-toni Cancer Research Centre. Most of their career has been dedicated to studying toxicology of food additives and other common environmental exposures. Dr. Belpoggi has previously conducted research on aspartame and, most recently, on radiofrequency radiation, with conclusions linking both to various types of cancer. I have not read those studies in detail but, at least for their study concerning sucralose, pushing a positive finding (i.e. a connection between sucralose and cancer) may have been a case of seeing everything as a nail because all you have is a hammer. When you expect a certain result, a scientist can be blinded to simpler, alternative explanations. As written by others on their aspartame study, a number of damning confounding factors were identified by a third party, such as elevated rates of chronic lung respiratory disease in their lab rats, which is causative of the same cancers that the study concluded were linked to aspartame. The study on sucralose was published in the International Journal of Occupational and Environmental Health, which may have been motivated to publish the study, with its poorly-supported conclusions, because of the scientific controversy it would court. Since 2016, the article has been cited 13 times, which is great for a journal with an impact factor of 1.2. Of course, a considerable number of these studies were negative towards the conclusions. Under my proposed socialist model for scientific publishing, this study would be still be published but with tempered conclusions (including a revised title) considering the numerous gaps between the observations and conclusions and other observations that contradict their main take-away, from their own study and others previously published.

The PR problem plaguing science is difficult. On the one hand, skepticism is an important aspect of science. Dogma should be questioned. However, blindly accepting the conclusions of a controversial study is not skepticism. Environmental exposures should be tested thoroughly and by independent research groups as history has taught that for-profit companies do not labour under any delusion of ethics, instead beholden to shareholders and their bottom line. Tobacco and oil companies come to mind. However, a capitalist impulse can also underlie poorly conducted studies that challenge accepted scientific models. Such controversial claims garner more attention and for-profit journals willing to publish dubious studies exist en masse. Scientific literacy needs to be taught in schools. Any news outlet reporting on science should not simply accept the conclusions of a study, even if peer-reviewed. To do so undermines public trust in the scientific process. Lastly, journals have a responsibility to not publish reports with conclusions so at odds with the observations. This point goes double for the title of scientific articles, as paywalls mean that the first word often remains the final word. I have argued that this is only possible within a socialist publishing model. Further, journals ought to publish brief summaries written in lay language, particularly when the studies regard public health. Scientists have a responsibility to maintain the credibility of the scientific process and it is not possible to do so without reaching out to the public as partners.

[i] Soffritti, M., Padovani, M., Tibaldi, E., Falcioni, L., Manservisi, F., Lauriola, M., ... & Belpoggi, F. (2016). Sucralose administered in feed, beginning prenatally through lifespan, induces hematopoietic neoplasias in male swiss mice. International journal of occupational and environmental health, 22(1), 7.

[ii] Berry, C., Brusick, D., Cohen, S. M., Hardisty, J. F., Grotz, V. L., & Williams, G. M. (2016). Sucralose non-carcinogenicity: a review of the scientific and regulatory rationale. Nutrition and cancer, 68(8), 1247-1261.