On 2016 Jun 07, Arthur Yin Fan commented:

Acupuncture is Effective for Chronic Knee Pain: A Reanalysis of the Australian Acupuncture Trial. Altern Ther Health Med. 2016 Mar;22(3):32-6. Yin Fan A, Zhou K, Gu S, Ming Li Y. Abstract Context • In the October 2014 issue of the Journal of the American Medical Association (JAMA), Hinman et al published the results of an Australian clinical trial on acupuncture in a paper entitled "Acupuncture for Chronic Knee Pain: A Randomized Clinical Trial" (JAMA report), in which they concluded that neither acupuncture nor laser acupuncture had any greater effects than sham laser acupuncture for pain or function for patients aged 50 y and older with moderate-to-severe knee pain. That study has been criticized extensively by international scholars for its validity because serious methodological flaws existed throughout the study's design, implementation, and conclusions. Objective • The current study intended to re-examine the prior study's conclusions about the efficacy of acupuncture for chronic knee pain. Design • The current research team performed a reanalysis of relevant data from the JAMA report. Intervention • The original study included 4 groups: (1) an acupuncture group, which received needle acupuncture, inferred by the current authors to have been set up to be a positive control in the original study; (2) a laser acupuncture group, which received laser acupuncture; (3) a sham laser acupuncture group, which received sham laser acupuncture and acted as the negative controls for the laser acupuncture intervention; and (4) a control group, which received conventional care but no acupuncture or laser treatments. The study lasted 12 wk. Outcome Measures • The measures included evaluations in the following areas: (1) poststudy modifications-an evaluation of the consistency of the JAMA report with the study's intentions as identified for a grant that was originally approved and funded by the Australian National Health and Medical Research Council (NHMRC) in 2009, as indicated in the study's trial registration, and as compared with the published protocols and to the study's originally stated objectives; (2) high heterogeneity-an assessment of the heterogeneity among the 4 groups for the overall outcome related to pain; (3) ineffectiveness of laser acupuncture-an analysis of laser acupuncture's efficacy for chronic knee pain as stated in the JAMA report, using effect size (ES); (4) effectiveness of acupuncture-a reanalysis of acupuncture's efficacy for chronic knee pain in comparison with the original analysis in the JAMA report, using ES; and (5) acupuncture after data adjustment-a new analysis of acupuncture's efficacy for chronic knee pain using data from the original study that was discussed in the JAMA report, using ES, with an estimation after data adjustment and elimination of the dilution effect of the Zelen design. Results • Contrary to a general impression that acupuncture was the focus, laser acupuncture was the primary intervention tested in the actual study, "Laser Acupuncture in Patients With Chronic Knee Pain: A Randomized, Placebo Controlled Trial." The study discussed in the JAMA report was neither a truly randomized, controlled trial (RCT) for acupuncture nor was it an appropriately designed, randomized study in general. High heterogeneity was found among its groups in the evaluation of overall pain in patients. Both the ES of 0.60 that had been set by Hinman et al for the minimal clinically important difference (MCID) and the resulting interpretation of results in the JAMA report were not appropriate. Using the original study's criteria of efficacy, the reanalysis has confirmed that the laser acupuncture was not effective, whereas the acupuncture was found to be moderately effective for chronic knee pain (P < .05) for both overall pain and function at 12 wk, with an ES of 0.58, or after the adjustment of the data, with an ES of 0.67. Conclusions • The JAMA study was neither a conventional RCT nor an appropriately randomized trial, and its results are probably invalid. The ES of 0.60 for the MCID that was used in the JAMA study and the resulting explanation were not appropriate. Even with an ES of 0.60 for the MCID, acupuncture remained effective after data adjustment. Consequently, compared with conventional care, acupuncture treatment was found to be moderately effective for chronic knee pain in patients aged 50 y and older. PMID: 27228270 [PubMed - in process]

On 2015 May 30, Arthur Yin Fan commented:

The sample size calculation is inaccurate in this study. http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60184-4.pdf

It should be 773, instead of 282. Hinman described she considered the factors of multiple groups' comparison, intra-therapies,however, she did not.

See the step by step sample size calculation detail in: The methodology flaws in Hinman’s acupuncture clinical trial, Part III: Sample size calculation. http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60184-4.pdf

On 2015 May 29, Hongjian He commented:

One our main criticisms of this publication was that there was no detailed reporting of number of needles used per treatment, acupuncture sites targeted per treatment, and specific treatment duration and electric stimulation for each patient. Dr. Hinman responded by stating:

"Dr. He suggests lack of acupuncture standardization, treatment infrequency, and no electrical stimulation may explain our findings. However, when comparing acupuncture with sham treatment, a meta-analysis1 found no evidence that needle number or placement; use of electrical stimulation; or number, frequency, or duration of treatments influence acupuncture outcomes"

We looked at the article Dr. Hinman cited (MacPherson et al., PLoS 2013) and she did not do a good job of reading it. They showed that number of needles used per treatment was statistically significantly correlated with effect size. The electrical stimulation had a significantly stronger effect. She incorrectly summarized the study results. Therefore, Dr. Hinman’s statement that needle numbers, length of treatment and electric stimulation have no effect on acupuncture outcomes is incorrect. We still await a sufficient explanation for why these specifics were not reported in her study.

On 2016 Apr 09, Leigh Jackson commented:

Wit et al. conducted a pragmatic trial whose design had glaring weaknesses.

There was no sham control. There was no blinding. Control was usual care inviting negative bias. The non-randomized self-selecting cohort invited positive bias. Consort was not followed. Patients had individualised treatment creating a chaotic heterogeneity of data.

Wit et al. to Hinman et al. is as chalk to cheese. The difference in their results might be due to low dosage levels in Hinman et al. It might be due to high levels of bias in Wit et al.

On 2015 Apr 27, Qin-hong Zhang commented:

Acupuncture treatment for chronic knee pain: study by Hinman et al underestimates acupuncture efficacy

http://aim.bmj.com/content/33/2/170.full.pdf+html

Dr Hinman and colleagues [1] completed a Zelen-design clinical trial for acupuncture for chronic knee pain patients and concluded that neither laser nor needle acupuncture conferred benefit over sham for pain or function in patients older than 50 years with moderate or severe chronic knee pain. We disagree with authors because they failed to use the most effective acupuncture regimen in their trial.

We consider that their treatment regimen is inferior for the following reasons. First, the dosage of acupuncture is far from adequate. The protocol specified the acupuncture intervention as a twenty minute treatment once or twice weekly for 12 weeks, with 8 to 12 sessions in total permitted [1]. Treatment compliance was not reported. Even with an assumption of full compliance, the study participants received only 0.67 to 1.0 session weekly with a total 160 to 240 minutes in 12 weeks. This was below the treatment regimen in the trial by Witt, et al. [2], in which participants received 12 sessions of 30 min duration, administered over 8 weeks, i.e., average 1.5 sessions weekly, and 360 minutes in total for only 8 weeks. It is worthwhile to point out that Witt, et al. had opposite conclusions. Second, the study protocol did not require deqi (a renowned acupuncture sensation), which is profoundly regarded as a prerequisite of a preferable acupuncture treatment efficacy [3]. Third, the paper did not follow the CONSORT statement of acupuncture [4] that requires details of needling, such as needle manipulation, depth of needle insertion, and points selected unilateral, bilateral or both. Fourth, the dose of laser acupuncture was 0.2J per acupuncture point, which was considered too low to achieve a clinical effect [5]. Previous study suggested that the minimum dose should probably be 0.5 J/point [5] . Thus it is difficult to evaluate if effective treatment regimen was compared against the Sham.

Because the efficacy of acupuncture therapy depends on the dose and deqi of acupuncture intervention, it is premature to us to reach the conclusion that acupuncture is not effective to treatment osteoarthritis pain of the knee.

REFERENCES

1. Hinman RS, McCrory P, Pirotta M, et al. Acupuncture for Chronic Knee Pain A Randomized Clinical Trial. JAMA. 2014;312(13):1313-1322.

2. Witt C, Brinkhaus B, Jena S, et al. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet. 2005;366(9480):136-143.

3. Shi GX, Yang XM, Liu CZ, et al. Factors contributing to therapeutic effects evaluated in acupuncture clinical trials. Trials. 2012; 13:42.

4. MacPherson H, Altman DG, Hammerschlag R, et al; STRICTA Revision Group. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. PLoS Med. 2010;7(6):e1000261.

5. Baxter GD. Laser acupuncture: effectiveness depends upon dosage. Acupunct Med. 2009;27(3): 92.

On 2015 Apr 16, Arthur Yin Fan commented:

The methodology flaws in Hinman’s acupuncture clinical trial, Part II: Zelen design and effectiveness dilutions: http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60172-8.pdf

Hinman and colleagues concluded that “in patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients.” As pointed out in my former article (The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med. 2015; 13(2): 65–68.), there were serious flaws in the trial design and statistics, as well as in the interpretation of the results. In here I address problems in the Zelen design used by them -Using Zelen design in this study does cause biases.

1 High drop-out rate:

The drop-out rates were 2.82% (2/71) in the control group; 22.86% (16/70) in the acupuncture group; 18.31% (13/71) in the laser acupuncture group; and 22.86% (16/70) for the sham laser acupuncture group. According to the acceptable standards for an RCT, dropout rates less than 10% are acceptable, drop-out rates between 10% and 20% mean that the resulting data quality is poor, and drop-out rates of more than 20% mean that the data quality is considered very poor and should not be used in analysis. In this trial analysis, the data quality in the acupuncture and sham laser acupuncture groups are very poor as the drop-out rates are over 20%; the authors should not have directly used them in any statistical analysis, unless they had re-adjusted and re-balanced the sample among the groups during the study. As outlined by the National Institutes of Health, if there is a differential drop-out rate of 15% or higher between study arms, such as between the control group and the treatment group in this clinical trial, then there is a very high potential for bias. This is a flaw that can decrease the quality of the study results.

2 The effectiveness in intervention groups was diluted by various factors

The dilution rates should then be 21.87% in the laser acupuncture group, 13.80% in the sham laser acupuncture group, and 31.27% in the acupuncture group (the dilution rate calculations were shown in Tables 1–3). The dilution rate was very significant in the acupuncture group, which causes the effectiveness to be undervalued in the acupuncture group, by almost 1/3.

The effective significance was masked by limited sample size due to the Zelen design of this study.

3.The sample size calculation in this study is questionable.Too small.

4 Conclusion

The effectiveness of the acupuncture group was diluted 31.27%, and its drop-out rate was 22.86%, much higher than that of the other groups in Hinman’s clinical trial, which constitutes major flaws in how this study is analyzed and interpreted[8]. Based on the bias of Zelen design used in the study, and incorrect sample size calculation, the conclusions drawn from this study are of poor quality, inaccurate, and invalid.

http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60172-8.pdf

On 2015 Apr 11, Arthur Yin Fan commented:

The sham laser acupuncture is not a valid negative control for acupuncture

I agree acupuncture should have a real sham control in a vigorous RCT; however, in Hinman's acupuncture RCT, the sham laser acupuncture is only fit to the laser acupuncture, not to real acupuncture. Because Acupuncture and Sham laser acupuncture, these two interventions do not have comparability in both characteristics and form (i.e., not matched). Furthermore, there was no blinding method performed between these two groups-both the patients and the administrators who performed the interventions knew the difference between the groups, such as needling acupuncture and sham laser acupuncture.

On 2015 Apr 11, Arthur Yin Fan commented:

There is a crucial mistake in interpreting the Hypothesis testing - What means? if P>0.05.

Hinman said :"in......chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function (Dr. Fan notes: Her statement was based on P>0.05). Our findings do not support acupuncture for these patients”

From the perspective of hypothesis testing in Statistics, if acupuncture has better results and with significant difference over the primary control (no-treatment group), p<0.05, we can conclude that “acupuncture is effective”- no matter what the result get from the comparing to the secondary control, such as “sham laser acupuncture”, but Hinman intentionally does not report this effectiveness in her conclusion; if acupuncture has better results over “laser acupuncture” and “sham laser acupuncture”, without significant in statistics, p>0.05, we can conclude that “acupuncture is better than the laser acupuncture, and sham laser acupuncture, but need more studies to confirm”. We can’t conclude that “acupuncture is not effective” because that there are no significant difference in statistics between acupuncture and “laser acupuncture”, or between acupuncture and “sham acupuncture” does not mean there is no difference between these treatments clinically. Hinman et al mis-interprets the results and violates the basic principle of Statistics.

On 2015 Apr 11, Arthur Yin Fan commented:

Hinman's acupuncture RCT has too many methodology flaws and misleading

As an independent researcher and practitioner in Acupuncture and Chinese medicine for thirty years, I strongly disagrees with Hinman's conclusion, as there were serious flaws in the trial design, the statistical analysis of the data and in the interpretation of the results of this study. I published a commentary recently [Fan A. The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med. 2015; 13(2): 65–68.] http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60170-4.pdf

The major concerns are:

(1)There is a major mistake in the primary testing factor in this RCT: the laser acupuncture should be the primary testing factor, not the needle acupuncture;

(2)The interpretation of the results was misleading;

(3)The “under-dosed” acupuncture treatments diluted the potential real effectiveness of acupuncture;

(4)Laser acupuncture and acupuncture would be effective in Hinman’s RCT, if the statistics were re-analyzed after re-adjusting the data.

(5)It is improper to test two different testing factors in one RCT with so small sample size;

(6)Laser acupuncture is not one kind of acupuncture, the author intentionally mixes it with acupuncture;

(7)Acupuncture did have significant effectiveness (p<0.05 in week 12), compared to the control (this is a primary control). However, the author intentionally does not interpreter this important result into the conclusion, instead, she concludes acupuncture is not effective and says her findings do not support acupuncture for patients.

I feel the author, somehow, intentionally misleads readers by testing acupuncture as a major intervention in this RCT-There was no significance between the positive control and the naïve control (i.e., acupuncture and control groups). Therefore, we can only conclude that the positive control, acupuncture was under-dosed or the study was otherwise flawed. That the positive control shows significance is a basic sign of the success of a clinical trial. From this perspective, Hinman’s trial was a failed clinical trial for laser acupuncture. As it would be unethical to publish an astonishing article, with a group of almost scrapped data and confusing logic, that misleads the readers, including the general public, medical society and policy makers, the researchers should have re-adjusted or re-designed their study instead of publishing it.

On 2015 Jun 14, Arthur Yin Fan commented:

Meridians could be tested by physics method, they are low electric resistance lines.

On 2015 Jun 11, David Keller commented:

It is no longer accurate to label evidence-based science as "Western"

The scientific method is the most powerful tool ever developed for the advancement of humanity, and science has, in turn, been advanced by humans from all cultural backgrounds.

Ancient societies all developed superstition-based pseudo-sciences, which are incapable of predicting natural phenomena or truly understanding them. Astrology was a vain attempt to rationalize the movements of the planets, utterly useless until it was replaced by the science of astronomy. Similarly, alchemy led nowhere until it was transmuted by the scientific method into chemistry. Homeopathy is based on false concepts involving dilution, and yields "medicines" which are nearly pure placebo. Chiropractic medicine, in addition to being based on disproved concepts, can be downright dangerous. PubMed has documented many cases of carotid and vertebral artery dissections caused by chiropractic neck manipulations (search on "artery dissection chiropractor").

The National Institutes of Health were founded to advance the scientific method in pursuing solutions to the health problems which plague mankind. Superstition-based folk remedies should be investigated when there is a chance that ancient peoples may have stumbled on a real medicine - such as aspirin from willow bark or digitalis from the foxglove plant. The active ingredient should be isolated, synthesized, tested and meta-analyzed until it is thoroughly understood, using the scientific method.

Where in the human body can we find anatomical evidence of a meridian, or "chee"? Acupuncture is based on complex and detailed theories, but theories must be tested and proved in the material world, in a reproducible fashion. If acupuncture is incapable of demonstrating significant pain relief in a carefully conducted clinical study, perhaps its adherents should reexamine their beliefs rather than invoking ever more arcane and obscure objections to the study design.

On 2015 May 30, Arthur Yin Fan commented:

Western medicine more focuses on structure, and Eastern medicine more on function. So, when explaining a specific issue, for example, a disc hernia induced lower back pain, even sciatica, western medicine will say the nerve gets pinched, however, eastern medicine will says the Bladder meridian Qi stagnant. Both are correct. Just because using different perspective, and using different language.

What is the meridian? just a functional manifestation of the never. Nerve is the structure, the meridian is the function.

Quite simple.

On 2014 Nov 13, David Keller commented:

Should medical doctors recommend acupuncture to patients with chronic knee pain?

Here is a brief description of acupuncture: "In traditional Chinese medicine, certain body systems, called meridians, are thought to regulate body function through the normal flow of energy [this energy is spelled "qi" but pronounced "chee"] from one part to another. Disturbances in this flow are thought to cause disease. Acupressure and acupuncture techniques are believed by practitioners of traditional Chinese medicine to cure disease by restoring this flow."(1) Without physiological or anatomical evidence of the existence of meridians or qi, acupuncture should be viewed as similar to chiropractic or homeopathic therapies, which lack scientifically credible theoretical mechanisms of action. This study found no significant clinical benefit of acupuncture for chronic knee pain. Therefore, I will not begin referring patients with chronic knee pain for acupuncture. I would like to see this study repeated for chronic pain syndromes of other body regions. If the null result is confirmed in all such regions, acupuncture will have to be classified as an expensive and complex placebo. Funding for research on debunked alternative therapies should be diverted to investigators conducting basic or clinical research of a more scientific nature.

Reference

1: Wang W and Wu S. Treating Pain With Acupuncture. JAMA 2014;312(13):1365.

On 2014 Nov 07, Qin-hong Zhang commented:

None

On 2016 Apr 03, Leigh Jackson commented:

Laser acupuncture provides a means for double blinding. This is a useful methodology for acupuncture trials where double blinding has always been the weakest link.

On 2015 May 30, Arthur Yin Fan commented:

This study used a wrong sham for acupuncture, so the conclusion is not reliable.

On 2014 Oct 08, David Keller commented:

Acupuncture trials have difficulty addressing the placebo effect and blinding

Acupuncture has been challenging to test in a true double-blinded and placebo-controlled fashion. Skilled practitioners of acupuncture have invested years in their training and licensing, and may unintentionally transmit signals of their own strong belief in its purported beneficial effects, augmenting the placebo effect. Simply poking needles into a patient at randomly chosen points could have a strong placebo effect if done with an air of compassion and clinical authority.

It has been argued that sham acupuncture performed as placebo therapy in clinical trials may accidentally stimulate a true acupuncture point, and thereby reduce the apparent benefit of acupuncture in the intervention subjects compared to controls. The use of sham laser acupuncture seems to address that objection.

The arguments and criticisms of acupuncture trials will continue. More good evidence, such as the findings of this study, will be required before we can conclude that acupuncture is of no more benefit for relieving pain than placebo. Better understanding of the biological basis of pain is needed, to develop new and truly effective analgesic therapies.

On 2014 Oct 08, David Keller commented:

Acupuncture trials have difficulty addressing the placebo effect and blinding

Acupuncture has been challenging to test in a true double-blinded and placebo-controlled fashion. Skilled practitioners of acupuncture have invested years in their training and licensing, and may unintentionally transmit signals of their own strong belief in its purported beneficial effects, augmenting the placebo effect. Simply poking needles into a patient at randomly chosen points could have a strong placebo effect if done with an air of compassion and clinical authority.

It has been argued that sham acupuncture performed as placebo therapy in clinical trials may accidentally stimulate a true acupuncture point, and thereby reduce the apparent benefit of acupuncture in the intervention subjects compared to controls. The use of sham laser acupuncture seems to address that objection.

The arguments and criticisms of acupuncture trials will continue. More good evidence, such as the findings of this study, will be required before we can conclude that acupuncture is of no more benefit for relieving pain than placebo. Better understanding of the biological basis of pain is needed, to develop new and truly effective analgesic therapies.

On 2014 Nov 07, Qin-hong Zhang commented:

None

On 2014 Nov 13, David Keller commented:

Should medical doctors recommend acupuncture to patients with chronic knee pain?

Here is a brief description of acupuncture: "In traditional Chinese medicine, certain body systems, called meridians, are thought to regulate body function through the normal flow of energy [this energy is spelled "qi" but pronounced "chee"] from one part to another. Disturbances in this flow are thought to cause disease. Acupressure and acupuncture techniques are believed by practitioners of traditional Chinese medicine to cure disease by restoring this flow."(1) Without physiological or anatomical evidence of the existence of meridians or qi, acupuncture should be viewed as similar to chiropractic or homeopathic therapies, which lack scientifically credible theoretical mechanisms of action. This study found no significant clinical benefit of acupuncture for chronic knee pain. Therefore, I will not begin referring patients with chronic knee pain for acupuncture. I would like to see this study repeated for chronic pain syndromes of other body regions. If the null result is confirmed in all such regions, acupuncture will have to be classified as an expensive and complex placebo. Funding for research on debunked alternative therapies should be diverted to investigators conducting basic or clinical research of a more scientific nature.

Reference

1: Wang W and Wu S. Treating Pain With Acupuncture. JAMA 2014;312(13):1365.

On 2015 Apr 11, Arthur Yin Fan commented:

Hinman's acupuncture RCT has too many methodology flaws and misleading

As an independent researcher and practitioner in Acupuncture and Chinese medicine for thirty years, I strongly disagrees with Hinman's conclusion, as there were serious flaws in the trial design, the statistical analysis of the data and in the interpretation of the results of this study. I published a commentary recently [Fan A. The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med. 2015; 13(2): 65–68.] http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60170-4.pdf

The major concerns are:

(1)There is a major mistake in the primary testing factor in this RCT: the laser acupuncture should be the primary testing factor, not the needle acupuncture;

(2)The interpretation of the results was misleading;

(3)The “under-dosed” acupuncture treatments diluted the potential real effectiveness of acupuncture;

(4)Laser acupuncture and acupuncture would be effective in Hinman’s RCT, if the statistics were re-analyzed after re-adjusting the data.

(5)It is improper to test two different testing factors in one RCT with so small sample size;

(6)Laser acupuncture is not one kind of acupuncture, the author intentionally mixes it with acupuncture;

(7)Acupuncture did have significant effectiveness (p<0.05 in week 12), compared to the control (this is a primary control). However, the author intentionally does not interpreter this important result into the conclusion, instead, she concludes acupuncture is not effective and says her findings do not support acupuncture for patients.

I feel the author, somehow, intentionally misleads readers by testing acupuncture as a major intervention in this RCT-There was no significance between the positive control and the naïve control (i.e., acupuncture and control groups). Therefore, we can only conclude that the positive control, acupuncture was under-dosed or the study was otherwise flawed. That the positive control shows significance is a basic sign of the success of a clinical trial. From this perspective, Hinman’s trial was a failed clinical trial for laser acupuncture. As it would be unethical to publish an astonishing article, with a group of almost scrapped data and confusing logic, that misleads the readers, including the general public, medical society and policy makers, the researchers should have re-adjusted or re-designed their study instead of publishing it.

On 2015 Apr 11, Arthur Yin Fan commented:

There is a crucial mistake in interpreting the Hypothesis testing - What means? if P>0.05.

Hinman said :"in......chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function (Dr. Fan notes: Her statement was based on P>0.05). Our findings do not support acupuncture for these patients”

From the perspective of hypothesis testing in Statistics, if acupuncture has better results and with significant difference over the primary control (no-treatment group), p<0.05, we can conclude that “acupuncture is effective”- no matter what the result get from the comparing to the secondary control, such as “sham laser acupuncture”, but Hinman intentionally does not report this effectiveness in her conclusion; if acupuncture has better results over “laser acupuncture” and “sham laser acupuncture”, without significant in statistics, p>0.05, we can conclude that “acupuncture is better than the laser acupuncture, and sham laser acupuncture, but need more studies to confirm”. We can’t conclude that “acupuncture is not effective” because that there are no significant difference in statistics between acupuncture and “laser acupuncture”, or between acupuncture and “sham acupuncture” does not mean there is no difference between these treatments clinically. Hinman et al mis-interprets the results and violates the basic principle of Statistics.

On 2015 Apr 11, Arthur Yin Fan commented:

The sham laser acupuncture is not a valid negative control for acupuncture

I agree acupuncture should have a real sham control in a vigorous RCT; however, in Hinman's acupuncture RCT, the sham laser acupuncture is only fit to the laser acupuncture, not to real acupuncture. Because Acupuncture and Sham laser acupuncture, these two interventions do not have comparability in both characteristics and form (i.e., not matched). Furthermore, there was no blinding method performed between these two groups-both the patients and the administrators who performed the interventions knew the difference between the groups, such as needling acupuncture and sham laser acupuncture.

On 2015 Apr 16, Arthur Yin Fan commented:

The methodology flaws in Hinman’s acupuncture clinical trial, Part II: Zelen design and effectiveness dilutions: http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60172-8.pdf

Hinman and colleagues concluded that “in patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients.” As pointed out in my former article (The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med. 2015; 13(2): 65–68.), there were serious flaws in the trial design and statistics, as well as in the interpretation of the results. In here I address problems in the Zelen design used by them -Using Zelen design in this study does cause biases.

1 High drop-out rate:

The drop-out rates were 2.82% (2/71) in the control group; 22.86% (16/70) in the acupuncture group; 18.31% (13/71) in the laser acupuncture group; and 22.86% (16/70) for the sham laser acupuncture group. According to the acceptable standards for an RCT, dropout rates less than 10% are acceptable, drop-out rates between 10% and 20% mean that the resulting data quality is poor, and drop-out rates of more than 20% mean that the data quality is considered very poor and should not be used in analysis. In this trial analysis, the data quality in the acupuncture and sham laser acupuncture groups are very poor as the drop-out rates are over 20%; the authors should not have directly used them in any statistical analysis, unless they had re-adjusted and re-balanced the sample among the groups during the study. As outlined by the National Institutes of Health, if there is a differential drop-out rate of 15% or higher between study arms, such as between the control group and the treatment group in this clinical trial, then there is a very high potential for bias. This is a flaw that can decrease the quality of the study results.

2 The effectiveness in intervention groups was diluted by various factors

The dilution rates should then be 21.87% in the laser acupuncture group, 13.80% in the sham laser acupuncture group, and 31.27% in the acupuncture group (the dilution rate calculations were shown in Tables 1–3). The dilution rate was very significant in the acupuncture group, which causes the effectiveness to be undervalued in the acupuncture group, by almost 1/3.

The effective significance was masked by limited sample size due to the Zelen design of this study.

3.The sample size calculation in this study is questionable.Too small.

4 Conclusion

The effectiveness of the acupuncture group was diluted 31.27%, and its drop-out rate was 22.86%, much higher than that of the other groups in Hinman’s clinical trial, which constitutes major flaws in how this study is analyzed and interpreted[8]. Based on the bias of Zelen design used in the study, and incorrect sample size calculation, the conclusions drawn from this study are of poor quality, inaccurate, and invalid.

http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60172-8.pdf

On 2015 Apr 27, Qin-hong Zhang commented:

Acupuncture treatment for chronic knee pain: study by Hinman et al underestimates acupuncture efficacy

http://aim.bmj.com/content/33/2/170.full.pdf+html

Dr Hinman and colleagues [1] completed a Zelen-design clinical trial for acupuncture for chronic knee pain patients and concluded that neither laser nor needle acupuncture conferred benefit over sham for pain or function in patients older than 50 years with moderate or severe chronic knee pain. We disagree with authors because they failed to use the most effective acupuncture regimen in their trial.

We consider that their treatment regimen is inferior for the following reasons. First, the dosage of acupuncture is far from adequate. The protocol specified the acupuncture intervention as a twenty minute treatment once or twice weekly for 12 weeks, with 8 to 12 sessions in total permitted [1]. Treatment compliance was not reported. Even with an assumption of full compliance, the study participants received only 0.67 to 1.0 session weekly with a total 160 to 240 minutes in 12 weeks. This was below the treatment regimen in the trial by Witt, et al. [2], in which participants received 12 sessions of 30 min duration, administered over 8 weeks, i.e., average 1.5 sessions weekly, and 360 minutes in total for only 8 weeks. It is worthwhile to point out that Witt, et al. had opposite conclusions. Second, the study protocol did not require deqi (a renowned acupuncture sensation), which is profoundly regarded as a prerequisite of a preferable acupuncture treatment efficacy [3]. Third, the paper did not follow the CONSORT statement of acupuncture [4] that requires details of needling, such as needle manipulation, depth of needle insertion, and points selected unilateral, bilateral or both. Fourth, the dose of laser acupuncture was 0.2J per acupuncture point, which was considered too low to achieve a clinical effect [5]. Previous study suggested that the minimum dose should probably be 0.5 J/point [5] . Thus it is difficult to evaluate if effective treatment regimen was compared against the Sham.

Because the efficacy of acupuncture therapy depends on the dose and deqi of acupuncture intervention, it is premature to us to reach the conclusion that acupuncture is not effective to treatment osteoarthritis pain of the knee.

REFERENCES

1. Hinman RS, McCrory P, Pirotta M, et al. Acupuncture for Chronic Knee Pain A Randomized Clinical Trial. JAMA. 2014;312(13):1313-1322.

2. Witt C, Brinkhaus B, Jena S, et al. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet. 2005;366(9480):136-143.

3. Shi GX, Yang XM, Liu CZ, et al. Factors contributing to therapeutic effects evaluated in acupuncture clinical trials. Trials. 2012; 13:42.

4. MacPherson H, Altman DG, Hammerschlag R, et al; STRICTA Revision Group. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. PLoS Med. 2010;7(6):e1000261.

5. Baxter GD. Laser acupuncture: effectiveness depends upon dosage. Acupunct Med. 2009;27(3): 92.

On 2015 May 29, Hongjian He commented:

One our main criticisms of this publication was that there was no detailed reporting of number of needles used per treatment, acupuncture sites targeted per treatment, and specific treatment duration and electric stimulation for each patient. Dr. Hinman responded by stating:

"Dr. He suggests lack of acupuncture standardization, treatment infrequency, and no electrical stimulation may explain our findings. However, when comparing acupuncture with sham treatment, a meta-analysis1 found no evidence that needle number or placement; use of electrical stimulation; or number, frequency, or duration of treatments influence acupuncture outcomes"

We looked at the article Dr. Hinman cited (MacPherson et al., PLoS 2013) and she did not do a good job of reading it. They showed that number of needles used per treatment was statistically significantly correlated with effect size. The electrical stimulation had a significantly stronger effect. She incorrectly summarized the study results. Therefore, Dr. Hinman’s statement that needle numbers, length of treatment and electric stimulation have no effect on acupuncture outcomes is incorrect. We still await a sufficient explanation for why these specifics were not reported in her study.

On 2015 May 30, Arthur Yin Fan commented:

The sample size calculation is inaccurate in this study. http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60184-4.pdf

It should be 773, instead of 282. Hinman described she considered the factors of multiple groups' comparison, intra-therapies,however, she did not.

See the step by step sample size calculation detail in: The methodology flaws in Hinman’s acupuncture clinical trial, Part III: Sample size calculation. http://www.jcimjournal.com/articles/publishArticles/pdf/S2095-4964(15)60184-4.pdf

On 2016 Jun 07, Arthur Yin Fan commented:

Acupuncture is Effective for Chronic Knee Pain: A Reanalysis of the Australian Acupuncture Trial. Altern Ther Health Med. 2016 Mar;22(3):32-6. Yin Fan A, Zhou K, Gu S, Ming Li Y. Abstract Context • In the October 2014 issue of the Journal of the American Medical Association (JAMA), Hinman et al published the results of an Australian clinical trial on acupuncture in a paper entitled "Acupuncture for Chronic Knee Pain: A Randomized Clinical Trial" (JAMA report), in which they concluded that neither acupuncture nor laser acupuncture had any greater effects than sham laser acupuncture for pain or function for patients aged 50 y and older with moderate-to-severe knee pain. That study has been criticized extensively by international scholars for its validity because serious methodological flaws existed throughout the study's design, implementation, and conclusions. Objective • The current study intended to re-examine the prior study's conclusions about the efficacy of acupuncture for chronic knee pain. Design • The current research team performed a reanalysis of relevant data from the JAMA report. Intervention • The original study included 4 groups: (1) an acupuncture group, which received needle acupuncture, inferred by the current authors to have been set up to be a positive control in the original study; (2) a laser acupuncture group, which received laser acupuncture; (3) a sham laser acupuncture group, which received sham laser acupuncture and acted as the negative controls for the laser acupuncture intervention; and (4) a control group, which received conventional care but no acupuncture or laser treatments. The study lasted 12 wk. Outcome Measures • The measures included evaluations in the following areas: (1) poststudy modifications-an evaluation of the consistency of the JAMA report with the study's intentions as identified for a grant that was originally approved and funded by the Australian National Health and Medical Research Council (NHMRC) in 2009, as indicated in the study's trial registration, and as compared with the published protocols and to the study's originally stated objectives; (2) high heterogeneity-an assessment of the heterogeneity among the 4 groups for the overall outcome related to pain; (3) ineffectiveness of laser acupuncture-an analysis of laser acupuncture's efficacy for chronic knee pain as stated in the JAMA report, using effect size (ES); (4) effectiveness of acupuncture-a reanalysis of acupuncture's efficacy for chronic knee pain in comparison with the original analysis in the JAMA report, using ES; and (5) acupuncture after data adjustment-a new analysis of acupuncture's efficacy for chronic knee pain using data from the original study that was discussed in the JAMA report, using ES, with an estimation after data adjustment and elimination of the dilution effect of the Zelen design. Results • Contrary to a general impression that acupuncture was the focus, laser acupuncture was the primary intervention tested in the actual study, "Laser Acupuncture in Patients With Chronic Knee Pain: A Randomized, Placebo Controlled Trial." The study discussed in the JAMA report was neither a truly randomized, controlled trial (RCT) for acupuncture nor was it an appropriately designed, randomized study in general. High heterogeneity was found among its groups in the evaluation of overall pain in patients. Both the ES of 0.60 that had been set by Hinman et al for the minimal clinically important difference (MCID) and the resulting interpretation of results in the JAMA report were not appropriate. Using the original study's criteria of efficacy, the reanalysis has confirmed that the laser acupuncture was not effective, whereas the acupuncture was found to be moderately effective for chronic knee pain (P < .05) for both overall pain and function at 12 wk, with an ES of 0.58, or after the adjustment of the data, with an ES of 0.67. Conclusions • The JAMA study was neither a conventional RCT nor an appropriately randomized trial, and its results are probably invalid. The ES of 0.60 for the MCID that was used in the JAMA study and the resulting explanation were not appropriate. Even with an ES of 0.60 for the MCID, acupuncture remained effective after data adjustment. Consequently, compared with conventional care, acupuncture treatment was found to be moderately effective for chronic knee pain in patients aged 50 y and older. PMID: 27228270 [PubMed - in process]