On 2015 Mar 20, DAVID ALLISON commented:

Apparent Statistical and Reporting Errors in “Effect of transcutaneous auricular vagus nerve stimulation on impaired glucose tolerance: a pilot randomized study”

1. Huang et al. compared the efficacy of transcutaneous auricular vagus nerve stimulation (taVNS) and sham taVNS on patients with impaired glucose tolerance (IGT) in a pilot randomized clinical trial. The authors state in the Abstract, specifically in the Results section, that “100 participants completed the study and were included in data analysis." Thereafter, Huang et al. state in the Methods section of the article that “After screening, 72 IGT participants were randomly assigned to either the transcutaneous auricular vagus nerve stimulation group (taVNS) or sham taVNS group (randomization sequence created using SAS System 7.0)”. In the Results section, Huang et al. state that “A total of 102 participants completed the study (72 females, 54.4 ± 7.4 years (mean ± SD), range (33–68).” (4) The number of participants reported by the authors is inconsistent. It is not clear as to whether the number of participants is 100, 72, or 102.

2. Though described as a randomized controlled trial, this is misleading. Patients were not randomized to control. There was non-uniformity of recruitment for study participants in different groups (vagus nerve stimulation subjects were recruited from an acupuncture hospital, while control subjects were recruited from free clinics). Although the authors acknowledge the control group is inappropriate to compare to the other groups as it was recruited from another population, they cite the comparison to the control at the beginning of the discussion as one of the core findings of the study.

3. The authors report on many test statistics with 2 numerator degrees of freedom in their analysis, which appears to be accurate for this data set for a comparison between three groups (df=3-1) or of a profile analysis with two groups and three time points (df=[2-1][3-1]). However, the authors also report tests with 1 numerator degree of freedom for certain outcomes in the exact same context without explanation, which would be incorrect and makes it unclear what tests were performed in these cases. For example, the rmANOVA for a treatment-by-time interaction on 2hPG used two numerator degrees of freedom but the test for the same effect on HbAlc used only one (p.4-5).

4. Tables 2, 4, and 5 report on pairwise differences over time between the three groups. They also contain confidence intervals for the difference between groups that do not correspond to any such difference over time as given in Table 3. For instance, comparing taVNS and sham taVNS for 2hPG the CI is centered around 8.271, but the baseline-to-12-week between-group difference is only 1.1. It is therefore unclear where these values came from. It also is inappropriate to report a single confidence interval for a test with multiple numerator degrees of freedom

Note: This comment has been posted on both PubMed Commons and the Journal’s website.

Lisa H. Antoine, University of Alabama at Birmingham, School of Engineering, Birmingham, Alabama 35294, antoinel@uab.edu

Brandon J. George, Office of Energetics, University of Alabama at Birmingham, Birmingham, Alabama 35294, brgeorge@uab.edu

David B. Allison, Nutrition Obesity Research Center and Department of Biostatistics, University of Alabama at Birmingham, Ryals Public Health Building, Room 140J, Birmingham, Alabama 35294, dallison@uab.edu

On 2015 Mar 20, DAVID ALLISON commented:

Apparent Statistical and Reporting Errors in “Effect of transcutaneous auricular vagus nerve stimulation on impaired glucose tolerance: a pilot randomized study”

1. Huang et al. compared the efficacy of transcutaneous auricular vagus nerve stimulation (taVNS) and sham taVNS on patients with impaired glucose tolerance (IGT) in a pilot randomized clinical trial. The authors state in the Abstract, specifically in the Results section, that “100 participants completed the study and were included in data analysis." Thereafter, Huang et al. state in the Methods section of the article that “After screening, 72 IGT participants were randomly assigned to either the transcutaneous auricular vagus nerve stimulation group (taVNS) or sham taVNS group (randomization sequence created using SAS System 7.0)”. In the Results section, Huang et al. state that “A total of 102 participants completed the study (72 females, 54.4 ± 7.4 years (mean ± SD), range (33–68).” (4) The number of participants reported by the authors is inconsistent. It is not clear as to whether the number of participants is 100, 72, or 102.

2. Though described as a randomized controlled trial, this is misleading. Patients were not randomized to control. There was non-uniformity of recruitment for study participants in different groups (vagus nerve stimulation subjects were recruited from an acupuncture hospital, while control subjects were recruited from free clinics). Although the authors acknowledge the control group is inappropriate to compare to the other groups as it was recruited from another population, they cite the comparison to the control at the beginning of the discussion as one of the core findings of the study.

3. The authors report on many test statistics with 2 numerator degrees of freedom in their analysis, which appears to be accurate for this data set for a comparison between three groups (df=3-1) or of a profile analysis with two groups and three time points (df=[2-1][3-1]). However, the authors also report tests with 1 numerator degree of freedom for certain outcomes in the exact same context without explanation, which would be incorrect and makes it unclear what tests were performed in these cases. For example, the rmANOVA for a treatment-by-time interaction on 2hPG used two numerator degrees of freedom but the test for the same effect on HbAlc used only one (p.4-5).

4. Tables 2, 4, and 5 report on pairwise differences over time between the three groups. They also contain confidence intervals for the difference between groups that do not correspond to any such difference over time as given in Table 3. For instance, comparing taVNS and sham taVNS for 2hPG the CI is centered around 8.271, but the baseline-to-12-week between-group difference is only 1.1. It is therefore unclear where these values came from. It also is inappropriate to report a single confidence interval for a test with multiple numerator degrees of freedom

Note: This comment has been posted on both PubMed Commons and the Journal’s website.

Lisa H. Antoine, University of Alabama at Birmingham, School of Engineering, Birmingham, Alabama 35294, antoinel@uab.edu

Brandon J. George, Office of Energetics, University of Alabama at Birmingham, Birmingham, Alabama 35294, brgeorge@uab.edu

David B. Allison, Nutrition Obesity Research Center and Department of Biostatistics, University of Alabama at Birmingham, Ryals Public Health Building, Room 140J, Birmingham, Alabama 35294, dallison@uab.edu