On 2016 Jul 05, Holger Schunemann commented:

The comment by Messori and colleagues puzzles me. Why would one search using the term "GRADE method"? GRADE has rarely, if ever, been referred to as a "method". It has typically been described as a system, approach, framework etc. Thus, what if Messory and colleagues would use a more appropriate (more sensitive) search including GRADE system, approach, GRADE framework or just GRADE? Without appropriate searching for information this comment seems not useful. Also, perhaps, when the purpose of the comment is clear, a citation based search would be helpful. GRADE articles have been cited over 20,000 times and there will probably be useful information from 100's of guidance documents that have been developed with GRADE and the cited GRADE publications (beginning in 2003).

On 2016 Jul 01, Andrea Messori commented:

Structured methods of analysis and reporting: a quick literature analysis

Sabrina Trippoli and Andrea Messori

HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

In the evaluation of therapeutic interventions, structured methods of analysis and reporting are increasingly being proposed (1-6). Among these, the GRADE method (4-6), firstly described in 2004, is the most widely recognised at international level. For example, Alonso-Coello and colleagues (7) have pointed out that the “more than 100 organisations globally, including the World Health Organization, the Cochrane Collaboration, and the National Institute for Health and Care Excellence now use or have adopted the principles of the GRADE system”. In the present comment, we report the results of a preliminary literature analysis that we conducted to explore the contents of the published articles dealing with the GRADE method. Our analysis was restricted to journals indexed by PubMed (www.pubmed.gov).

a) Search engine: PubMed

b) Search term: “GRADE method”

c) Date of the search: 1 July 2016

d) Extracted citations: N=71

e) No. of citations not pertinent with the search term (false positive): N=17

f) No. of citations pertinent with the search term: N=54

g) No. of citations the contents of which were restricted to a description of the GRADE method and/or the intention to use this method for future activities: N=12

h) No. of citations the contents of which were represented by an experience of application of the method along with the results of this application: N=42. Of these 42 papers, 32 used the GRADE method with the only purpose of assessing the quality of evidence of the clinical material included in a systematic review or a meta-analysis; the remaining 10 of these 42 papers applied the GRADE method to the development of clinical guidelines or recommendations.

References

1. Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563-2577.

2. Schnipper LE, Davidson NE, Wollins DS, et al: Updating the ASCO value framework: Revisions and reflections in response to comments received. J Clin Oncol 2016, doi: 10.1200/JCO.2016.68.251

3. Cherny NI, Sullivan R, Dafni U, et al: A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-1573.

4. Atkins D, Best D, Briss PA, et al. GRADE Working Group. Grading quality of evidence and strength of recommendations.BMJ 2004;328:1490.

5. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6.

6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol2011;64:383-94.

7. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: asystematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016 Jun 28;353:i2016.

Address correspondence to: Dr.Andrea Messori, HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

On 2016 Jul 01, Andrea Messori commented:

Structured methods of analysis and reporting: a quick literature analysis

Sabrina Trippoli and Andrea Messori

HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

In the evaluation of therapeutic interventions, structured methods of analysis and reporting are increasingly being proposed (1-6). Among these, the GRADE method (4-6), firstly described in 2004, is the most widely recognised at international level. For example, Alonso-Coello and colleagues (7) have pointed out that the “more than 100 organisations globally, including the World Health Organization, the Cochrane Collaboration, and the National Institute for Health and Care Excellence now use or have adopted the principles of the GRADE system”. In the present comment, we report the results of a preliminary literature analysis that we conducted to explore the contents of the published articles dealing with the GRADE method. Our analysis was restricted to journals indexed by PubMed (www.pubmed.gov).

a) Search engine: PubMed

b) Search term: “GRADE method”

c) Date of the search: 1 July 2016

d) Extracted citations: N=71

e) No. of citations not pertinent with the search term (false positive): N=17

f) No. of citations pertinent with the search term: N=54

g) No. of citations the contents of which were restricted to a description of the GRADE method and/or the intention to use this method for future activities: N=12

h) No. of citations the contents of which were represented by an experience of application of the method along with the results of this application: N=42. Of these 42 papers, 32 used the GRADE method with the only purpose of assessing the quality of evidence of the clinical material included in a systematic review or a meta-analysis; the remaining 10 of these 42 papers applied the GRADE method to the development of clinical guidelines or recommendations.

References

1. Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563-2577.

2. Schnipper LE, Davidson NE, Wollins DS, et al: Updating the ASCO value framework: Revisions and reflections in response to comments received. J Clin Oncol 2016, doi: 10.1200/JCO.2016.68.251

3. Cherny NI, Sullivan R, Dafni U, et al: A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-1573.

4. Atkins D, Best D, Briss PA, et al. GRADE Working Group. Grading quality of evidence and strength of recommendations.BMJ 2004;328:1490.

5. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6.

6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol2011;64:383-94.

7. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: asystematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016 Jun 28;353:i2016.

Address correspondence to: Dr.Andrea Messori, HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

On 2016 Jul 05, Holger Schunemann commented:

The comment by Messori and colleagues puzzles me. Why would one search using the term "GRADE method"? GRADE has rarely, if ever, been referred to as a "method". It has typically been described as a system, approach, framework etc. Thus, what if Messory and colleagues would use a more appropriate (more sensitive) search including GRADE system, approach, GRADE framework or just GRADE? Without appropriate searching for information this comment seems not useful. Also, perhaps, when the purpose of the comment is clear, a citation based search would be helpful. GRADE articles have been cited over 20,000 times and there will probably be useful information from 100's of guidance documents that have been developed with GRADE and the cited GRADE publications (beginning in 2003).