On 2014 Sep 09, Vahid Rakhshan commented:

Reply to the author’s response: Statistical errors in a recent article: Sella Turcica in patients with Type 1 diabetes

1. The authors kindly stated in the authors’ reply that it was clearly written in the 4th paragraph of Methods that how the subjects were matched: "This was clearly defined in the 4th paragraph of Material and Methods section. In addition, the need to match the groups according to bone age, not the chronological age was discussed briefly in Discussion (paragraph 3, page 183) with supporting references." (A) I re-checked the 4th paragraph and there was no indication of criteria for matching. The authors had indeed talked about grouping the patients according to the bone age, in the 4th paragraph ("that were equally distributed into subgroups according to bone age and sex."), but that was not something about "matching" the patients. (B) Moreover, there was no mention of the chronological age in that 4th paragraph of the Methods section. However, this factor is used for correlating the two groups, according to the Results and Discussion. (C) Furthermore, in the same Methods sentence that had stated that bone age was used for grouping ("that were equally distributed into subgroups according to bone age and sex."), sex was mentioned as a grouping factor as well. So if that quoted sentence implies that bone age was a matching factor, should we consider sex as a matching criterion as well? The above-quoted sentence from the 4th paragraph of the Methods is actually the criteria for dividing the sample into subgroups. However, the authors are referring to it (in the authors’ reply) as something about matching. (D) Besides, I had noticed the comparisons and correlations of groups regarding bone ages, in the pages 182 and 183, but since those factors were not pre-defined as criteria for matching, those comparisons and the authors’ accompanying correlations could not indicate a matching between the groups to the reader.

2. I appreciate the authors’ point #2. They kindly clarified the authors’ method.

3. The authors have stated in the authors’ reply that they had tested both the whole sample and each subgroup regarding the normality... First that the authors’ article stated n > 50 for the normality test. However, the authors’ subgroups were much smaller than this. So n > 50 could not relate to subgroups of n = 19. Therefore, either the statement "n > 50" was incorrect, or they had not tested for normality in the subgroups. Moreover, normality should not be tested in the whole sample. It is completely incorrect. Besides, they did not state why they had used two different normality tests?

4. There is nothing wrong with doing a pairwise comparison, and I did not say pairwise comparisons are bad, in my original letter. However doing a pairwise comparison without an ANOVA is statistical malpractice (when the ANOVA is necessary) and bad. It might have the following problems: (1) The absence of ANOVA disallows to assess the interactions. (2) Without an ANOVA design first, we do not know if any post hoc (or pairwise) comparisons are necessary or not, in the first place. If the ANOVA is non-significant, then performing the pairwise comparisons is simply moot (if we disregard the family-wise error introduced by the unnecessary statistical analyses). So this was why I was emphasizing on performing an ANOVA before doing the pairwise comparisons.

5. The authors kindly stated " In our manuscript, "matched" means "constructing two groups whose data would be comparable with each other"." ... "The data of these groups are independent from each other and the presence of one group cannot alter the data of the other. " (A) Please note that "matched" has its own globally accepted definition that is to select groups with similarities in certain aspects. So using a personal definition for this word is not a healthy practice. (B) Moreover, the statement "constructing two groups whose data would be comparable with each other" implies (by the word "comparable") that there were actually similarities between the two groups, and thus some sort of matching had been actually performed. This again reveals that the two groups were not independent. (C) Furthermore, the Results and the Discussion (pages 182 and 183) clearly indicate that the two groups were correlated with each other according to the chronological and bone age: Page 182: "For both groups, the chronologic and bone ages of the subjects were correlated to each other" Page 183: "The control group, on the other hand, was created from skeletal Class I patients with no systemic diseases and whose chronologic and bone ages were correlated." (D) This evidence completely suffices to invalidate the use of ANY independent-samples test. Correlated groups CANNOT be independent. How "the presence of one group cannot alter the data of the other" when the authors had themselves selected the control group in a way that they were correlated to diabetic patients in terms of chronological and bone age? The control group was already altered in a way that it could be correlated to the diabetic patients.

6. The authors have kindly re-stated that they had used the Bonferroni correction method. I had already seen the authors’ similar statement in the authors’ paper, but had not found any evidence supporting the authors’ statement. Please note that the Bonferroni correction is simply dividing the alpha by the number of pairwise comparisons in each family of tests. So we would expect to see alphas below 0.05 (for example 0.013 etc. depending on the number of pairwise comparisons). In that case, if the alpha had been corrected using the Bonferroni method to something like 0.008, and the P value had become P = 0.04, the P was still Non-Significant. However, throughout the text, and in the tables, only P values above the alpha = 0.05 had been considered as non-significant. This indicated that no Bonferroni had been used to adjust the alpha. (the alpha had not been adjusted in the first place).

7. I much appreciate the authors’ correction and clarification.

Many thanks for the readers' and authors' time.

With kindest regards,

Vahid Rakhshan

On 2014 Sep 09, Vahid Rakhshan commented:

None

On 2014 Sep 09, Vahid Rakhshan commented:

None

On 2014 Sep 09, Vahid Rakhshan commented:

Reply to the author’s response: Statistical errors in a recent article: Sella Turcica in patients with Type 1 diabetes

1. The authors kindly stated in the authors’ reply that it was clearly written in the 4th paragraph of Methods that how the subjects were matched: "This was clearly defined in the 4th paragraph of Material and Methods section. In addition, the need to match the groups according to bone age, not the chronological age was discussed briefly in Discussion (paragraph 3, page 183) with supporting references." (A) I re-checked the 4th paragraph and there was no indication of criteria for matching. The authors had indeed talked about grouping the patients according to the bone age, in the 4th paragraph ("that were equally distributed into subgroups according to bone age and sex."), but that was not something about "matching" the patients. (B) Moreover, there was no mention of the chronological age in that 4th paragraph of the Methods section. However, this factor is used for correlating the two groups, according to the Results and Discussion. (C) Furthermore, in the same Methods sentence that had stated that bone age was used for grouping ("that were equally distributed into subgroups according to bone age and sex."), sex was mentioned as a grouping factor as well. So if that quoted sentence implies that bone age was a matching factor, should we consider sex as a matching criterion as well? The above-quoted sentence from the 4th paragraph of the Methods is actually the criteria for dividing the sample into subgroups. However, the authors are referring to it (in the authors’ reply) as something about matching. (D) Besides, I had noticed the comparisons and correlations of groups regarding bone ages, in the pages 182 and 183, but since those factors were not pre-defined as criteria for matching, those comparisons and the authors’ accompanying correlations could not indicate a matching between the groups to the reader.

2. I appreciate the authors’ point #2. They kindly clarified the authors’ method.

3. The authors have stated in the authors’ reply that they had tested both the whole sample and each subgroup regarding the normality... First that the authors’ article stated n > 50 for the normality test. However, the authors’ subgroups were much smaller than this. So n > 50 could not relate to subgroups of n = 19. Therefore, either the statement "n > 50" was incorrect, or they had not tested for normality in the subgroups. Moreover, normality should not be tested in the whole sample. It is completely incorrect. Besides, they did not state why they had used two different normality tests?

4. There is nothing wrong with doing a pairwise comparison, and I did not say pairwise comparisons are bad, in my original letter. However doing a pairwise comparison without an ANOVA is statistical malpractice (when the ANOVA is necessary) and bad. It might have the following problems: (1) The absence of ANOVA disallows to assess the interactions. (2) Without an ANOVA design first, we do not know if any post hoc (or pairwise) comparisons are necessary or not, in the first place. If the ANOVA is non-significant, then performing the pairwise comparisons is simply moot (if we disregard the family-wise error introduced by the unnecessary statistical analyses). So this was why I was emphasizing on performing an ANOVA before doing the pairwise comparisons.

5. The authors kindly stated " In our manuscript, "matched" means "constructing two groups whose data would be comparable with each other"." ... "The data of these groups are independent from each other and the presence of one group cannot alter the data of the other. " (A) Please note that "matched" has its own globally accepted definition that is to select groups with similarities in certain aspects. So using a personal definition for this word is not a healthy practice. (B) Moreover, the statement "constructing two groups whose data would be comparable with each other" implies (by the word "comparable") that there were actually similarities between the two groups, and thus some sort of matching had been actually performed. This again reveals that the two groups were not independent. (C) Furthermore, the Results and the Discussion (pages 182 and 183) clearly indicate that the two groups were correlated with each other according to the chronological and bone age: Page 182: "For both groups, the chronologic and bone ages of the subjects were correlated to each other" Page 183: "The control group, on the other hand, was created from skeletal Class I patients with no systemic diseases and whose chronologic and bone ages were correlated." (D) This evidence completely suffices to invalidate the use of ANY independent-samples test. Correlated groups CANNOT be independent. How "the presence of one group cannot alter the data of the other" when the authors had themselves selected the control group in a way that they were correlated to diabetic patients in terms of chronological and bone age? The control group was already altered in a way that it could be correlated to the diabetic patients.

6. The authors have kindly re-stated that they had used the Bonferroni correction method. I had already seen the authors’ similar statement in the authors’ paper, but had not found any evidence supporting the authors’ statement. Please note that the Bonferroni correction is simply dividing the alpha by the number of pairwise comparisons in each family of tests. So we would expect to see alphas below 0.05 (for example 0.013 etc. depending on the number of pairwise comparisons). In that case, if the alpha had been corrected using the Bonferroni method to something like 0.008, and the P value had become P = 0.04, the P was still Non-Significant. However, throughout the text, and in the tables, only P values above the alpha = 0.05 had been considered as non-significant. This indicated that no Bonferroni had been used to adjust the alpha. (the alpha had not been adjusted in the first place).

7. I much appreciate the authors’ correction and clarification.

Many thanks for the readers' and authors' time.

With kindest regards,

Vahid Rakhshan