On 2014 Sep 13, Vahid Rakhshan commented:

The formatted version of this unpublished letter to the editor can be found at this address (LINK).

I read with interest the valuable article of Ferreira et al. [PMID: 24182586]. Although it was a very good research, few points were unclear to me.

1. Gold standard preparation was extremely vague (P.699). The numbers of examiners, their calibration/training/inter-rater agreements, and the number of assessments per each skull were not mentioned. Measuring the bone defects might first appear as an easy task and not needing detailed clarification. However (A) the gingival bone margin does not exist in dehiscence (Figure 1) and evaluator needs to guess from where to measure the distance. (B) If there were irregularities in the bone defect, which distance would be measured? The longest one? Or the shortest one? Or the vertical one? What about horizontally expanded defects? (C) The distances do not seem perfectly vertical and similar in Figure 1. The angle of the gauge can affect the measured length as well. Therefore calibration and inter-rater agreement are critical.

2. I wonder how the evaluators could certainly identify the bone “marrow” absence by the naked eye for fenestration diagnosis (P. 699).

3. It is stated that “the level of agreement proved that…” and also “the assessments do reflect reality” (P.701). These two strong conclusions were made based on agreements between 0.47 and 0.76.

4. Surprisingly, there was absolutely No P value reported for any test/correlation/kappa, which disallows any generalizations/validations. So how statistically unsubstantiated kappa values can prove anything?

5. There is mention of a chi-square used on sagittal and axial reconstructions (P.701). However, there are no P values or results for chi-square reported throughout the study.

6. In the last paragraph of the Results, the sensitivity / specificity / accuracy of sagittal reconstructions were stated but none was stated for axial reconstructions. Yet immediately after that part (after a semicolon), the authors concluded that the sagittal reconstructions performed better than axial. How did they do so without any comparisons?

7. Perhaps the authors had used the previous paragraph that stated agreements with the gold standard. The agreements for sagittal and axial reconstructions were 78.5% and 73.3%, respectively. I would much appreciate to know how the authors compared these two without (A) first identifying whether or not these two agreements were significant; and (B) whether or not there was a statistically significant difference between these two correlation coefficients? The authors needed to run a t-test between the two agreements to see if the 78.5% was significantly greater than 73.3% or the difference was not reliable (statistically non-significant).

8. K=0.4509 and K=0.3131 are considered 78.43% and 73.33% agreement, respectively (P.701). I think it is not correct and they respectively mean 45.09% and 31.31% agreements.

9. Making decisive conclusions from a report without any P values or confidence intervals should be avoided. The findings were not substantiated statistically, and thus should be approached cautiously.

10. The conclusions of text and abstract contradict completely. The abstract concludes “There was no difference in the performance of the axial and sagittal reconstructions.” The text concludes “The sagittal section is more reliable than the axial section, mainly in the middle third.” Which conclusion to take home?

On 2014 Sep 13, Vahid Rakhshan commented:

The formatted version of this unpublished letter to the editor can be found at this address (LINK).

I read with interest the valuable article of Ferreira et al. [PMID: 24182586]. Although it was a very good research, few points were unclear to me.

1. Gold standard preparation was extremely vague (P.699). The numbers of examiners, their calibration/training/inter-rater agreements, and the number of assessments per each skull were not mentioned. Measuring the bone defects might first appear as an easy task and not needing detailed clarification. However (A) the gingival bone margin does not exist in dehiscence (Figure 1) and evaluator needs to guess from where to measure the distance. (B) If there were irregularities in the bone defect, which distance would be measured? The longest one? Or the shortest one? Or the vertical one? What about horizontally expanded defects? (C) The distances do not seem perfectly vertical and similar in Figure 1. The angle of the gauge can affect the measured length as well. Therefore calibration and inter-rater agreement are critical.

2. I wonder how the evaluators could certainly identify the bone “marrow” absence by the naked eye for fenestration diagnosis (P. 699).

3. It is stated that “the level of agreement proved that…” and also “the assessments do reflect reality” (P.701). These two strong conclusions were made based on agreements between 0.47 and 0.76.

4. Surprisingly, there was absolutely No P value reported for any test/correlation/kappa, which disallows any generalizations/validations. So how statistically unsubstantiated kappa values can prove anything?

5. There is mention of a chi-square used on sagittal and axial reconstructions (P.701). However, there are no P values or results for chi-square reported throughout the study.

6. In the last paragraph of the Results, the sensitivity / specificity / accuracy of sagittal reconstructions were stated but none was stated for axial reconstructions. Yet immediately after that part (after a semicolon), the authors concluded that the sagittal reconstructions performed better than axial. How did they do so without any comparisons?

7. Perhaps the authors had used the previous paragraph that stated agreements with the gold standard. The agreements for sagittal and axial reconstructions were 78.5% and 73.3%, respectively. I would much appreciate to know how the authors compared these two without (A) first identifying whether or not these two agreements were significant; and (B) whether or not there was a statistically significant difference between these two correlation coefficients? The authors needed to run a t-test between the two agreements to see if the 78.5% was significantly greater than 73.3% or the difference was not reliable (statistically non-significant).

8. K=0.4509 and K=0.3131 are considered 78.43% and 73.33% agreement, respectively (P.701). I think it is not correct and they respectively mean 45.09% and 31.31% agreements.

9. Making decisive conclusions from a report without any P values or confidence intervals should be avoided. The findings were not substantiated statistically, and thus should be approached cautiously.

10. The conclusions of text and abstract contradict completely. The abstract concludes “There was no difference in the performance of the axial and sagittal reconstructions.” The text concludes “The sagittal section is more reliable than the axial section, mainly in the middle third.” Which conclusion to take home?