On 2014 Feb 27, James Carroll commented:

Kadhim-Saleh et al make significant errors and subsequently draw the wrong conclusion. A comprehensive rebuttal by Bjordal et al is published here Bjordal JM, 2014 but as as most readers are unlikely to pay $39.95 to read the letter I have listed some highlights here to help restore some balance.

The authors claim they used “A very strict study selection criterion” but they excluded Chow 2004 because it “most likely” included the same cohort as Chow 2006. It it is clearly stated in the Lancet review they cited Kimler WC IV, 1996 that one cohort was recruited in 1998 – 1999, and the other cohort recruited in 2002–2003.

Rather than contacting the trial authors to resolve uncertainty about the cohorts, which is normal practice for systematic reviewers Kadhim-Saleh et al. decided to base their decision on guess work, not stringent reviewing practice.

Kadhim-Saleh et al. had criticised the lancet review because “… investigators included trials that used different and more subjective tools for assessing the primary pain outcome measure”. Bjordal et al point out they only included studies that used well-recognized and validated scales for pain assessment including “weighted mean difference for continuous data from visual analogues scale (VAS) scores for pain intensity, relative risk for dichotomized data for global improvement, and standardized mean difference to combine different validated scales of disability including the Neck Disability Index, Neck Disability Scale and the Northwick Park Questionnaire”.

Bjordal et al said they enabled a comprehensive analysis of the available evidence and the finding was that the results were consistent across the different measurement tools actually strengthened the robustness of the Lancet conclusion. Kadhim-Saleh et al. only used a single outcome measure at a single time-point as their outcome measure of success.

Kadhim-Saleh et al. also added a study by Konstantinovic 2010 that they say “failed to detect a statistically significance difference between LLLT and placebo” based on only on VAS, but Konstantinovic had measured seven other outcomes and all were in favour of LLLT [but not reported by Kadhim-Saleh]. The study was on acute radiculopathy, which is a specific diagnosis for neck pain, and this would have increased heterogeneity not reduced it which which was their criticism of the Lancet review.

Kadhim-Saleh et al. criticized the Lancet review for having heterogeneity but Bjordal et al noted that Kadhim-Saleh et al was actually worse in their analysis (I2 = 94 % compared with the Lancet analysis I2 = 91 %). This revealed that the so called “stringent” criteria used by Kadhim-Saleh et al. had failed

Bjordal et al explained that in the Lancet they had included a 650-word paragraph that described a sensitivity analyses that found that most heterogeneity was attributed to interventions and that Kadhim-Saleh et al. gave no serious consideration to the appropriateness of LLLT technique including dosage, a priori in selection criteria or analysis protocol.

He pointed out that an example of how LLLT dose explains heterogeneity was provided by examining the two acute group trials included in the Lancet review. The negative trial using LLLT on acupuncture points by Aigner et al. was under-dosed (632 nm wavelength, 0.075 J) with an ineffective dose of only 0.02 % when compared with the positive trial by Soriano et al. (904 nm wavelength, 4 J). The additional acute radiculopathy trial by Konstantinovic et al. included in the review by Kadhim-Saleh et al. had similar doses and wavelength (904 nm wavelength, 2 J) to the positive trial by Soriano et al., and both were complying with the dosage recommendations from World Association for Laser Therapy (WALT). By conducting a meta-analysis that excludes the trial by Aigner et al. and substituting it with the new acute radiculopathy trial by Konstantinovic et al. Bjordal et al have found that heterogeneity disappears completely(from I2 =91% to I2 =0%). This results in a significant and clinically relevant RR for global improvement at 2.63 (95 % CI 1.73, 4.01).

Kadhim-Saleh et al also made an unsubstantiated claim about a re-analysis of the Lancet meta-analysis by Shiri and Viikari- Juntura 2010 “After applying a random-effects model Shiri and Viikari-Juntura found no significant difference between laser-treated and placebo-treated groups in pain reduction”. Bjordal et al emphatically state “This is simply untrue, as Shiri and Viika-Junturi 2010 confirmed that pain reduction on VAS with a weighted mean difference of 19.41 (95 % CI 9.67, 29.15) in a random-effects model. The inaccuracies in our original analysis brought to our attention by Shiri and Viikari-Juntura weakened slightly the size of effect on recalculation, yet the overall result remained that LLLT gave significant and clinically relevant relief for 6 out of 8 outcomes and lasting up to 22 weeks”.

Bjordal et al also criticised the Kadhim-Saleh paper because it “cited meta-analyses published over 20 years ago to demonstrate the consistency of their claim with previous reviews that found no effect from LLLT despite 80–90 % of RCTs on LLLT being published after these citations. The rate of publication of RCTs in recent years means that the survival period for systematic reviews is typically less than 5 years. In addition, they used literature published between 14 and 30 years ago to support descriptions of LLLT mechanisms. We stand by our original findings that LLLT gives clinically relevant neck pain relief and disability improvement after treatment and possibly follow-ups up to 5 months.”

DISCLOSURE

James Carroll

CEO

THOR Photomedicine Ltd

My publications

On 2014 Feb 27, James Carroll commented:

Kadhim-Saleh et al make significant errors and subsequently draw the wrong conclusion. A comprehensive rebuttal by Bjordal et al is published here Bjordal JM, 2014 but as as most readers are unlikely to pay $39.95 to read the letter I have listed some highlights here to help restore some balance.

The authors claim they used “A very strict study selection criterion” but they excluded Chow 2004 because it “most likely” included the same cohort as Chow 2006. It it is clearly stated in the Lancet review they cited Kimler WC IV, 1996 that one cohort was recruited in 1998 – 1999, and the other cohort recruited in 2002–2003.

Rather than contacting the trial authors to resolve uncertainty about the cohorts, which is normal practice for systematic reviewers Kadhim-Saleh et al. decided to base their decision on guess work, not stringent reviewing practice.

Kadhim-Saleh et al. had criticised the lancet review because “… investigators included trials that used different and more subjective tools for assessing the primary pain outcome measure”. Bjordal et al point out they only included studies that used well-recognized and validated scales for pain assessment including “weighted mean difference for continuous data from visual analogues scale (VAS) scores for pain intensity, relative risk for dichotomized data for global improvement, and standardized mean difference to combine different validated scales of disability including the Neck Disability Index, Neck Disability Scale and the Northwick Park Questionnaire”.

Bjordal et al said they enabled a comprehensive analysis of the available evidence and the finding was that the results were consistent across the different measurement tools actually strengthened the robustness of the Lancet conclusion. Kadhim-Saleh et al. only used a single outcome measure at a single time-point as their outcome measure of success.

Kadhim-Saleh et al. also added a study by Konstantinovic 2010 that they say “failed to detect a statistically significance difference between LLLT and placebo” based on only on VAS, but Konstantinovic had measured seven other outcomes and all were in favour of LLLT [but not reported by Kadhim-Saleh]. The study was on acute radiculopathy, which is a specific diagnosis for neck pain, and this would have increased heterogeneity not reduced it which which was their criticism of the Lancet review.

Kadhim-Saleh et al. criticized the Lancet review for having heterogeneity but Bjordal et al noted that Kadhim-Saleh et al was actually worse in their analysis (I2 = 94 % compared with the Lancet analysis I2 = 91 %). This revealed that the so called “stringent” criteria used by Kadhim-Saleh et al. had failed

Bjordal et al explained that in the Lancet they had included a 650-word paragraph that described a sensitivity analyses that found that most heterogeneity was attributed to interventions and that Kadhim-Saleh et al. gave no serious consideration to the appropriateness of LLLT technique including dosage, a priori in selection criteria or analysis protocol.

He pointed out that an example of how LLLT dose explains heterogeneity was provided by examining the two acute group trials included in the Lancet review. The negative trial using LLLT on acupuncture points by Aigner et al. was under-dosed (632 nm wavelength, 0.075 J) with an ineffective dose of only 0.02 % when compared with the positive trial by Soriano et al. (904 nm wavelength, 4 J). The additional acute radiculopathy trial by Konstantinovic et al. included in the review by Kadhim-Saleh et al. had similar doses and wavelength (904 nm wavelength, 2 J) to the positive trial by Soriano et al., and both were complying with the dosage recommendations from World Association for Laser Therapy (WALT). By conducting a meta-analysis that excludes the trial by Aigner et al. and substituting it with the new acute radiculopathy trial by Konstantinovic et al. Bjordal et al have found that heterogeneity disappears completely(from I2 =91% to I2 =0%). This results in a significant and clinically relevant RR for global improvement at 2.63 (95 % CI 1.73, 4.01).

Kadhim-Saleh et al also made an unsubstantiated claim about a re-analysis of the Lancet meta-analysis by Shiri and Viikari- Juntura 2010 “After applying a random-effects model Shiri and Viikari-Juntura found no significant difference between laser-treated and placebo-treated groups in pain reduction”. Bjordal et al emphatically state “This is simply untrue, as Shiri and Viika-Junturi 2010 confirmed that pain reduction on VAS with a weighted mean difference of 19.41 (95 % CI 9.67, 29.15) in a random-effects model. The inaccuracies in our original analysis brought to our attention by Shiri and Viikari-Juntura weakened slightly the size of effect on recalculation, yet the overall result remained that LLLT gave significant and clinically relevant relief for 6 out of 8 outcomes and lasting up to 22 weeks”.

Bjordal et al also criticised the Kadhim-Saleh paper because it “cited meta-analyses published over 20 years ago to demonstrate the consistency of their claim with previous reviews that found no effect from LLLT despite 80–90 % of RCTs on LLLT being published after these citations. The rate of publication of RCTs in recent years means that the survival period for systematic reviews is typically less than 5 years. In addition, they used literature published between 14 and 30 years ago to support descriptions of LLLT mechanisms. We stand by our original findings that LLLT gives clinically relevant neck pain relief and disability improvement after treatment and possibly follow-ups up to 5 months.”

DISCLOSURE

James Carroll

CEO

THOR Photomedicine Ltd

My publications