On 2015 Nov 02, R Andrew Moore commented:

Peter raises so many points that it is difficult to respond in detail to all of them. But they are all misdirected, and some of his contentions are plain wrong. As an example, yes, we do use examine randomisation and blinding in assessing quality, because being randomised and double blind is a minimum entry requirement in pain studies.

There are deficiencies in trial reporting – and that on adverse events in nothing new, as we reported some time ago (J Pain Symptom Manage 1999;18:427-37). Large observational studies have examined serious rare adverse events with topical NSAIDs, perhaps more appropriate given the modest numbers in RCTs. We have to work with what is available and comment on deficiencies.

Indirect comparisons have been shown to be as good if not better than direct, given available data. The fact that Peter found no effect of topical NSAIDs in his Clinical Evidence review is in direct contrast to our reviews, updated several times since our original review of topical analgesics in 1998, most recently in 2015. We are unaware of other reviews that agree with Peter’s analysis, and the results of recent reviews have been incorporated into a number of guidelines.

Peter’s attitude to the pharmaceutical industry is well known. The research group behind this review uncovered duplicate publication by industry (BMJ. 1997;315:635-40) and what some would see as the scandal of inappropriate statistical handling of missing data (Pain. 2012;153:265-8), as well as highlighting academic fraud, inter alia (Anaesthesia. 2010;65:327-30). We have tested the impact of possible industry bias in pain trials and found none in studies that compared like with like (Pain. 2006;121:207-18). We have worked with industry to obtain clinical trial reports, previously unpublished data, and individual patient level data. We have a simple rule – only do it if there is no bar to publication. Peter’s suggestion that we are in some way beholden to industry is beneath him.

On 2015 Oct 24, Peter Gøtzsche commented:

This meta-analysis is not reliable. It is not appropriate to pool all trials when small trials yield a considerably larger effect than large trials (P=0.001). The authors used rating scales for scoring the quality of trials and trial validity, but such scales have important weaknesses and should not be used (see JAMA 1999; 282: 1054-60 and the Cochrane Reviewers’ Handbook (www.cochrane.org). By using such scales, the authors rated far too many trials as being of acceptable quality. The authors are not clear about whether the trials that compared topical with oral NSAIDs compared the same or different drugs, and their finding that the old drug indomethacin was not significantly better than placebo is irrelevant since the P value depends on the sample size, and in fact, the effect obtained with indomethacin was similar to those of several of the other drugs that had significant effects. The trials reported insuffiently on harms; 56 of a total of 70 systemic adverse events occurred in a single trial, which raises the question whether this trial was more reliable than the others. The authors have consulted for various pharmaceutical companies but do not say which ones, which is an important omission, as they somewhat surprisingly report that one NSAID was better than the others. Their comparisons are questionable, however, as they were indirect, based on studies comparing active drug with placebo. In contrast to the authors, I did not find that any topical NSAID has a true effect (see London: BMJ Publications, Clinical Evidence, 2005; 14: 1498-1505). In contrast to the authors, I have no conflicts of interest and I believe that authors with financial conflicts of interest should not be authors on systematic reviews.

On 2015 Oct 24, Peter Gøtzsche commented:

This meta-analysis is not reliable. It is not appropriate to pool all trials when small trials yield a considerably larger effect than large trials (P=0.001). The authors used rating scales for scoring the quality of trials and trial validity, but such scales have important weaknesses and should not be used (see JAMA 1999; 282: 1054-60 and the Cochrane Reviewers’ Handbook (www.cochrane.org). By using such scales, the authors rated far too many trials as being of acceptable quality. The authors are not clear about whether the trials that compared topical with oral NSAIDs compared the same or different drugs, and their finding that the old drug indomethacin was not significantly better than placebo is irrelevant since the P value depends on the sample size, and in fact, the effect obtained with indomethacin was similar to those of several of the other drugs that had significant effects. The trials reported insuffiently on harms; 56 of a total of 70 systemic adverse events occurred in a single trial, which raises the question whether this trial was more reliable than the others. The authors have consulted for various pharmaceutical companies but do not say which ones, which is an important omission, as they somewhat surprisingly report that one NSAID was better than the others. Their comparisons are questionable, however, as they were indirect, based on studies comparing active drug with placebo. In contrast to the authors, I did not find that any topical NSAID has a true effect (see London: BMJ Publications, Clinical Evidence, 2005; 14: 1498-1505). In contrast to the authors, I have no conflicts of interest and I believe that authors with financial conflicts of interest should not be authors on systematic reviews.