On 2016 Jul 28, Michael Bunce commented:

Thank you for your comments from the 5th June 2016 we welcome the chance to respond.

The main point to be taken from this paper is that there is an urgent need for an accurate testing method including DNA, toxicological and heavy metal screening of complementary medicines. It is clear from the results of this paper that some products that are available for over the counter sale to the general public, whether or not they have been correctly listed with the TGA, could pose a health risk to consumers or contain ingredients that are not declared on the packaging. The authors agree with the comment that the ‘actions of a few have the potential to tarnish the reputation of the majority whose behaviour is professional’, and this is precisely why we believe that stronger regulation and pre and post-market auditing of products should occur. Consumers of TCM and other complementary medicines should be made aware that some products do not conform to TGA regulations, and consumers are encouraged to contact the TGA for more information about particular products if they have a concern.

A few more replies to your specific queries:

1) These 26 TCM were purchased across a number of retailers as a consumer would purchase them (they were not sold as ‘food’). The fact that they are not randomised does not compromise the study as we simply report on the results of the 26 TCM tested here – noting the results warrant concern. We welcome future efforts to expand sampling to determine a more holistic overview of compliance or lack thereof. The salient point here is perhaps auditing should occur before market?

2) We fully concede (and refer to it in the paper) that salicylic acid can be naturally derived from many plant species.

3) The blue/red colour coding in Figure 1 refers to TCM that are listed and unlisted – as such the data is clearly portrayed.

4) With regard to ‘measurement error’ and ‘false positives’ – we document fully the clean room precautions, replication and controls implemented in the DNA workflows. Our lab specialises in trace DNA analyses on a variety of biological substrates. As noted in the paper assignments based on DNA metabarcoding data (publically available – see link in paper) are conservative.

5) In response to your query regarding Supplemental Table 1, the allocation of TCM8 and TCM11 to the TGA unlisted category of products was an error in the Supplemental Table and we can confirm that these two products do have AUSTL numbers and are formally listed with the TGA, as correctly reported in the published manuscript (Tables 1, 2 and 3). We will correct this minor error in the supplemental table with the publishers – many thanks for making us aware of it.

6) Finally, we would have been happy to write a reply to the ‘Journal of Chinese Medicine and Science’ but the journal has no content online and is not formally listed as a journal, the link at www.fcma.org.au (as of July 2016) lists no content. Can you please provide a formal link to the Journal so readers at PubMed can assess the legitimacy of the publication.

On 2016 Jun 05, Sherman Gu commented:

We are writing to express our concerns about some reporting errors and therefore conclusions reached in this paper. Firstly, there is a discrepancy in the reporting of which group, the Therapeutic Goods Administration (TGA) unlisted or listed products, a few of the products belong to. For example, according to the Supplementary Table (S1) of the report, TCM8 and TCM11 belong to the TGA unlisted product group, yet the results for these products appear in the TGA listed group in Tables 1, 2 and 3. It would thus appear from Table S1 that there are nine listed TCM products and 17 unlisted products, not 12 listed products as is stated in the Discussion Section of the report. Secondly, with respect to Table 2 which reports adulterants or undeclared products, trace amounts of salicylic acid were found in five listed TCMs including TCM22. TCM22 contains Panax ginseng, and one of components of this herb is known to be salicylic acid. This might explain the trace amounts found in this TCM sample. Nonetheless, it is of concern that three (excluding TCM8 and TCM11 which are apparently unlisted TCMs according to Table S1) of the nine listed TCM products were adulterated with either pseudo/ephedrine, methylephedrine (TCM2, TCM7) or sildenafil (TCM23). Thirdly, it should also be noted that five of the TCM samples were herbal tea bags (TCM11, TCM13, TCM24, TCM25, TCM26) and one was ‘flakes’ (TCM10), all of which are unlisted, according to Table S1. In our opinion, these would not be considered conventional TCM herbal medicines that a registered Chinese medicine practitioner would prescribe. Without any information as to where these six products were purchased, it is difficult to form any conclusions as to whether these are potentially prescribed by any practitioner or whether they were sold as food in a shop. These findings potentially skew the summary set out in Figure 1 (i.e. if TCM8, TCM11 do not belong in the listed group and if the salicylic acid found in TCM22 were to be consistent with that expected in a TCM containing Panax ginseng, and if the six herbal tea bags and flake samples were deleted). The percentages of contaminated/adulterated products may well be different between the unlisted and listed products. Fourth, by reporting the results of all the products sampled together (92%) in the Abstract, rather than separating them out into percentages relating to the listed group of products and the unlisted group of products, important information and an important distinction is missed. Those TCMs that are considered safe for use are those that are listed (or registered) under the TGA. Those that have not been listed (or registered) on the Australian Register of Therapeutic Goods have not been assessed by the TGA for safety, quality and in the case of registered products, efficacy. The use of unlisted (or unregistered) proprietary TCMs by a registered Chinese medicine practitioner is against the Chinese Medicine Board of Australia (CMBA)’s Code of conduct and Guidelines for safe Chinese herbal medicine practice. Fifth, the sample collection method is somewhat vague and there is no evidence of any randomisation of the process of selecting the products. There is no information about whether the Chinese medicine practitioners were registered with the CMBA and how many practices the products were purchased from, nor how many nor what kind of retail stores were targeted for the selection of products. These factors are important to consider in any discussion of the ramifications of findings, in particular in relation to the unlisted products. Finally most measurement instruments have a level of error. The possibility of false positives with respect to, for example, the DNA testing has not been mentioned. These details are important in the analysis of findings and also constitute some of the limitations of the study. Unfortunately limitations of the study do not appear to have been discussed at all, as would be expected of a scientific study. Whilst there may be valid reasons for not divulging the names of the TCM products (i.e. the potential for litigation), non-disclosure makes this investigation impossible to replicate by other researchers and therefore verify the results. The results will be of concern to responsible Chinese medicine practitioners who need to know which products should be avoided. These criticisms of the paper are not meant to detract from some of the findings of the paper. The fact that unlisted products are available, albeit possibly from only a small number of sources, is unfortunate since the actions of a few have the potential to tarnish the reputation of the majority whose behavior is professional. It is commendable that the authors of the above-named paper have conducted an independent analysis of several proprietary TCMs. However, greater care and tighter reporting of results would have gone a long way to improving the quality of this report. The popular press will always pick up what makes for a sensational headline. The consequences of reporting the combined result of 92%, the figure itself open to debate, has been to cause alarm. The general public is unlikely to read the research paper nor be able to interpret the research findings and may be left feeling concerned. We look forward to reading the results of future research which investigates such an important issue, conducted in a large and more representative sample of TCM products.

Re-printed with permission from “Gu, S., O' Brien, K., & Cheung, T. (2016). Reporting Errors in the Coghlan et al report 2015. Australian Journal of Chinese Medicine and Science, 3(1), 66-67.”

On 2016 Jun 05, Sherman Gu commented:

We are writing to express our concerns about some reporting errors and therefore conclusions reached in this paper. Firstly, there is a discrepancy in the reporting of which group, the Therapeutic Goods Administration (TGA) unlisted or listed products, a few of the products belong to. For example, according to the Supplementary Table (S1) of the report, TCM8 and TCM11 belong to the TGA unlisted product group, yet the results for these products appear in the TGA listed group in Tables 1, 2 and 3. It would thus appear from Table S1 that there are nine listed TCM products and 17 unlisted products, not 12 listed products as is stated in the Discussion Section of the report. Secondly, with respect to Table 2 which reports adulterants or undeclared products, trace amounts of salicylic acid were found in five listed TCMs including TCM22. TCM22 contains Panax ginseng, and one of components of this herb is known to be salicylic acid. This might explain the trace amounts found in this TCM sample. Nonetheless, it is of concern that three (excluding TCM8 and TCM11 which are apparently unlisted TCMs according to Table S1) of the nine listed TCM products were adulterated with either pseudo/ephedrine, methylephedrine (TCM2, TCM7) or sildenafil (TCM23). Thirdly, it should also be noted that five of the TCM samples were herbal tea bags (TCM11, TCM13, TCM24, TCM25, TCM26) and one was ‘flakes’ (TCM10), all of which are unlisted, according to Table S1. In our opinion, these would not be considered conventional TCM herbal medicines that a registered Chinese medicine practitioner would prescribe. Without any information as to where these six products were purchased, it is difficult to form any conclusions as to whether these are potentially prescribed by any practitioner or whether they were sold as food in a shop. These findings potentially skew the summary set out in Figure 1 (i.e. if TCM8, TCM11 do not belong in the listed group and if the salicylic acid found in TCM22 were to be consistent with that expected in a TCM containing Panax ginseng, and if the six herbal tea bags and flake samples were deleted). The percentages of contaminated/adulterated products may well be different between the unlisted and listed products. Fourth, by reporting the results of all the products sampled together (92%) in the Abstract, rather than separating them out into percentages relating to the listed group of products and the unlisted group of products, important information and an important distinction is missed. Those TCMs that are considered safe for use are those that are listed (or registered) under the TGA. Those that have not been listed (or registered) on the Australian Register of Therapeutic Goods have not been assessed by the TGA for safety, quality and in the case of registered products, efficacy. The use of unlisted (or unregistered) proprietary TCMs by a registered Chinese medicine practitioner is against the Chinese Medicine Board of Australia (CMBA)’s Code of conduct and Guidelines for safe Chinese herbal medicine practice. Fifth, the sample collection method is somewhat vague and there is no evidence of any randomisation of the process of selecting the products. There is no information about whether the Chinese medicine practitioners were registered with the CMBA and how many practices the products were purchased from, nor how many nor what kind of retail stores were targeted for the selection of products. These factors are important to consider in any discussion of the ramifications of findings, in particular in relation to the unlisted products. Finally most measurement instruments have a level of error. The possibility of false positives with respect to, for example, the DNA testing has not been mentioned. These details are important in the analysis of findings and also constitute some of the limitations of the study. Unfortunately limitations of the study do not appear to have been discussed at all, as would be expected of a scientific study. Whilst there may be valid reasons for not divulging the names of the TCM products (i.e. the potential for litigation), non-disclosure makes this investigation impossible to replicate by other researchers and therefore verify the results. The results will be of concern to responsible Chinese medicine practitioners who need to know which products should be avoided. These criticisms of the paper are not meant to detract from some of the findings of the paper. The fact that unlisted products are available, albeit possibly from only a small number of sources, is unfortunate since the actions of a few have the potential to tarnish the reputation of the majority whose behavior is professional. It is commendable that the authors of the above-named paper have conducted an independent analysis of several proprietary TCMs. However, greater care and tighter reporting of results would have gone a long way to improving the quality of this report. The popular press will always pick up what makes for a sensational headline. The consequences of reporting the combined result of 92%, the figure itself open to debate, has been to cause alarm. The general public is unlikely to read the research paper nor be able to interpret the research findings and may be left feeling concerned. We look forward to reading the results of future research which investigates such an important issue, conducted in a large and more representative sample of TCM products.

Re-printed with permission from “Gu, S., O' Brien, K., & Cheung, T. (2016). Reporting Errors in the Coghlan et al report 2015. Australian Journal of Chinese Medicine and Science, 3(1), 66-67.”