On 2015 Oct 08, Elie Akl commented:

Dear Hilda Thank you for the excellent and critical review. I agree that this review is due for updating, and its conclusions are not current. We are not aware of more recent studies besides the ones you pointed to, but there are likely few of them. We will consider in the update the points you make below about how to include studies of medical students. thank you again Elie

On 2015 Mar 09, Hilda Bastian commented:

This is an excellent and important review, and its conclusions are likely to generally still be valid. The authors' reply to a 2011 comment about missing trials by Steven Woloshin and Lisa Schwartz indicates a particularly key trial subsequent to the 2007 search is missing (Waters EA, 2006; Cuite CL, 2008; Woloshin S, 2011). There are likely to be many more. (Several are included in a post of mine on using NNTs, at PLOS Blogs.)

The relatively small amount of data in this review on some comparisons is, I believe, becoming a problem as the conclusions of the authors are being too readily dismissed. If the update of this review is not likely to be soon, it may be useful to add a comment about the currency of the review, and highlight studies awaiting assessment to counteract the current impression.

It would also be useful if the authors could clarify in their update the status of the "additional results" in Appendix 4. As these trials are also listed as excluded from the review, it is a little confusing. Indeed, at least some of those studies do seem to be eligible: the time-to-event measure, for example.

Although I can understand the reasoning for including medical students as lay people rather than health professionals, I think that is potentially problematic, and they require separate analysis as the quantity grows.

I look forward to the update of this important review. Thanks!

On 2015 Mar 09, Hilda Bastian commented:

This is an excellent and important review, and its conclusions are likely to generally still be valid. The authors' reply to a 2011 comment about missing trials by Steven Woloshin and Lisa Schwartz indicates a particularly key trial subsequent to the 2007 search is missing (Waters EA, 2006; Cuite CL, 2008; Woloshin S, 2011). There are likely to be many more. (Several are included in a post of mine on using NNTs, at PLOS Blogs.)

The relatively small amount of data in this review on some comparisons is, I believe, becoming a problem as the conclusions of the authors are being too readily dismissed. If the update of this review is not likely to be soon, it may be useful to add a comment about the currency of the review, and highlight studies awaiting assessment to counteract the current impression.

It would also be useful if the authors could clarify in their update the status of the "additional results" in Appendix 4. As these trials are also listed as excluded from the review, it is a little confusing. Indeed, at least some of those studies do seem to be eligible: the time-to-event measure, for example.

Although I can understand the reasoning for including medical students as lay people rather than health professionals, I think that is potentially problematic, and they require separate analysis as the quantity grows.

I look forward to the update of this important review. Thanks!

On 2015 Oct 08, Elie Akl commented:

Dear Hilda Thank you for the excellent and critical review. I agree that this review is due for updating, and its conclusions are not current. We are not aware of more recent studies besides the ones you pointed to, but there are likely few of them. We will consider in the update the points you make below about how to include studies of medical students. thank you again Elie