On 2017 Jan 05, Melissa Rethlefsen commented:

The authors have wisely searched in multiple sources to inform this systematic review. Even better, they have listed the number of references found per information source prior to deduplication in their flow diagram. Upon closer examination of the flow diagram, I noted that after deduplication, the authors were left with 395 citations. This seems highly unusual, since some of their individual information sources (e.g., Scopus and "ISI Web of Science") individually contain more than 395 citations.

Though it is possible that there is some duplication in Scopus, it is highly unlikely that there were 76 duplicates in that database. Because the search strategies are not included in full (without interpretation required) and would not work in a database other than MEDLINE (due to the use of the "major medical subject heading 'pulmonary arterial hypertension'", which is MEDLINE specific), it is difficult to ascertain whether Scopus may have been searched multiple times, perhaps with multiple searches, without deduplication prior to exporting the results. This is likely the answer, but it remains unclear.

For the ISI Web of Science, it is markedly more challenging to guess what the difference between the 680 results and the 395 total deduplicated results may have been. This is primarily because the reason for the gap of 285 articles could theoretically be attributed to several reasons. The first and foremost reason is that ISI Web of Science is not a database; it is a platform that hosts many databases, depending on the subscriptions held by the institution. For example, at my institution, our ISI Web of Science platform hosts: Science Citation Index Expanded (SCI-EXPANDED) --1900-present Social Sciences Citation Index (SSCI) --1900-present; Arts & Humanities Citation Index (A&HCI) --1975-present; Emerging Sources Citation Index (ESCI) --2015-present; BIOSIS Previews; MEDLINE; Russian Science Citation Index; SciELO Citation Index; and KCI: Korean Journal Database. It is unclear which ISI Web of Science database(s) were searched in this instance. If multiple databases on the platform were searched individually (perhaps Science Citation Index and MEDLINE, e.g.), that could lead to the deduplicated results being lower than the individual platform's results. It could also be, however, that multiple searches were performed without deduplication before exporting results. Since the search strategy loosely described in the text would not work in ISI Web of Science, except in the MEDLINE database within, it is unclear what searches may have been performed on that platform.

Though the authors make a good attempt to make their methods transparent, the limited search strategy reporting and incomplete reporting as to database usage makes their data flow hard to track and reproduce. Having a librarian or information specialist on the team may have helped to improve their search process and reporting.

On 2017 Jan 05, Melissa Rethlefsen commented:

The authors have wisely searched in multiple sources to inform this systematic review. Even better, they have listed the number of references found per information source prior to deduplication in their flow diagram. Upon closer examination of the flow diagram, I noted that after deduplication, the authors were left with 395 citations. This seems highly unusual, since some of their individual information sources (e.g., Scopus and "ISI Web of Science") individually contain more than 395 citations.

Though it is possible that there is some duplication in Scopus, it is highly unlikely that there were 76 duplicates in that database. Because the search strategies are not included in full (without interpretation required) and would not work in a database other than MEDLINE (due to the use of the "major medical subject heading 'pulmonary arterial hypertension'", which is MEDLINE specific), it is difficult to ascertain whether Scopus may have been searched multiple times, perhaps with multiple searches, without deduplication prior to exporting the results. This is likely the answer, but it remains unclear.

For the ISI Web of Science, it is markedly more challenging to guess what the difference between the 680 results and the 395 total deduplicated results may have been. This is primarily because the reason for the gap of 285 articles could theoretically be attributed to several reasons. The first and foremost reason is that ISI Web of Science is not a database; it is a platform that hosts many databases, depending on the subscriptions held by the institution. For example, at my institution, our ISI Web of Science platform hosts: Science Citation Index Expanded (SCI-EXPANDED) --1900-present Social Sciences Citation Index (SSCI) --1900-present; Arts & Humanities Citation Index (A&HCI) --1975-present; Emerging Sources Citation Index (ESCI) --2015-present; BIOSIS Previews; MEDLINE; Russian Science Citation Index; SciELO Citation Index; and KCI: Korean Journal Database. It is unclear which ISI Web of Science database(s) were searched in this instance. If multiple databases on the platform were searched individually (perhaps Science Citation Index and MEDLINE, e.g.), that could lead to the deduplicated results being lower than the individual platform's results. It could also be, however, that multiple searches were performed without deduplication before exporting results. Since the search strategy loosely described in the text would not work in ISI Web of Science, except in the MEDLINE database within, it is unclear what searches may have been performed on that platform.

Though the authors make a good attempt to make their methods transparent, the limited search strategy reporting and incomplete reporting as to database usage makes their data flow hard to track and reproduce. Having a librarian or information specialist on the team may have helped to improve their search process and reporting.