The Therapeutic Goods Administration (TGA) has recently called out several medical cannabis companies for failing a product quality compliance audit.
All medical cannabis products supplied in Australia must meet Therapeutic Goods Order 93 (TGO93) requiring companies to test their products to ensure potency is within the permitted range.
The TGA audit – conducted by the National Measurement Institute (NMI) – showed some of these products were significantly weaker or stronger than they should have been. But ChemCentre, which conducts pre-market testing for Little Green Pharma, has rejected these findings and stands by their initial tests which showed the products in questions did, in fact, comply with TGO93.
But how can two equally accredited labs test the same batch of the same product and get different results? The answer, of course, is error.
When we think of errors, we usually think of big, blame-worthy mistakes. But in science, error does not always mean that someone has done something wrong.
Some degree of error is unavoidable, so much so that the entire scientific process is built around minimising – but not always eliminating – error. Even in the most highly accredited labs.
But it is important to distinguish between random errors (which are normal and can be managed) and systematic errors (which can cause serious trouble).
For example, when conducting an analytical test on biological material such as cannabis, it is standard practice, based on US Food and Drug Administration (FDA) or European Medicine Agency (EMA) guidelines, to limit error to within 15%. So, if the cannabis being tested actually contains 100mg CBD, you might measure anywhere between 85 and 115mg CBD. But some analytical tests for final medicinal products such as cannabis oils may be validated to a higher level of accuracy.
Error isn’t necessarily because someone did something wrong. Maybe some tiny amount of CBD got stuck to a vial. Maybe there was a light breeze that slightly evaporated a sample. Maybe the science gods were grumpy that day. But tiny, random errors can add up over time. You can manage random errors by repeating a test multiple times so that, over the long run, random over and under-estimations cancel each other out and your measurements hopefully converge on the true value.
However, repetition does not address systematic errors – errors which consistently (as opposed to randomly) lead to an over or under-estimation of the true value. These errors are often subtle, or at least non-obvious.
Labs that are properly accredited by the National Association of Testing Authorities, Australia (NATA) are much less likely to suffer from systematic errors. In this case, both NMI and ChemCentre are NATA accredited. But there are still several systemic errors which could be contributing to their divergent results.
For example, if you are testing the potency of a CBD product, you need some chemically pure CBD to compare your results against. These are ‘reference standards’ and can vary slightly depending where you get them from. These labs might be using different reference standards, or even the same reference standards but one has degraded slightly.
They might be using two different but equally appropriate techniques to conduct their analysis. Or the products in question might have slightly changed over time, due to issues with product stability or packaging, or how the products were stored.
Lab errors – be they systematic or random – have very important real-world impacts. And there are things we can do to reduce these errors in the future.
First, develop rigorous, peer-reviewed methods for conducting these tests that can be used across all labs. This peer-review process must be extremely thorough as any problems with the method would affect everyone. Second, standardise the reference material used by all labs. Although, once again, any issues with the reference material would have wide-ranging consequences. Third, conduct these tests across multiple labs, although this can become prohibitively expensive. And finally, report the level of uncertainty (error) for a given test or final product – for example, label a product as 100±10mg CBD.
It is currently hard to say whether the discrepancy between lab results is due to systematic error or random error. ChemCentre has now provided product samples to several other national and international testing facilities, so we will likely know more when those results come out in six to eight weeks.
In the meantime, this incident has shown once again that even the most credentialled and experienced organisations are not infallible.
- Dr Richard Kevin is a postdoctoral researcher and analytical chemist at the Lambert Initiative for Cannabinoid Therapeutics at The University of Sydney @RichardKevinPhD.