On 2016 Oct 04, Atanas G. Atanasov commented:

None

On 2016 Jun 06, Thomy Tonia commented:

We would like to thank Melissa Vaught for her interesting and stimulating thoughts on our paper. She states that the title and description of our trial’s objective is misleading. However, it is clearly stated in the abstract and in the introduction that we randomised IJPH papers into two arms one of which involved IJPH social media exposure at 3 time points; moreover, our Methods section describes this intervention in detail.

Melissa Vaught makes a very valid point stating that we did not include any measures of reach/exposure such as re-tweets, shares or other users’ dissemination of our papers in social media etc. We have touched upon this issue when we discussed the limitations of our study.

She also refers to potential confounding factors. Given the randomised design of our study, however, systematic confounding cannot be an issue; our analysis is what one would call “intention to treat” in clinical trials. There are some interfering factors that we could not control, which we tried to outline in our paper. This is why we standardised our intervention as much as possible, randomising only original articles (and not systematic reviews that generally tend to receive more attention) and stratifying for open access status. We have not issued any press releases for any of the randomised papers but we could not control for any press or other exposure coming from other sources such as authors’ institutions.

Although we agree that SM strategies could influence outcomes, the studies by Fox CS, 2016 and Adams CE, 2016 cited by Melissa Vaught did not directly compare different intensities of interventions such as multiple postings per day versus less frequent postings. Additionally, as we mention in our limitations, we could not have the data on paper views which could be different from the download data.

Melissa Vaught mentions that only the study by Sorenson 2014 looked at SM promotion by the journal. However, the studies by Fox CS, 2015 and Fox CS, 2016 which were randomised and bigger in size also assessed the promotion of papers from Circulation journal on their own social media and found no effect on article views.

We fully agree with the last remark of Melissa Vaught. Indeed, having social media presence as a journal can have benefits that lie beyond downloads and citations and this is one of the reasons our Journal is active on social media.

Indeed, social media exposure and its effects on scientific is a complex and multi-layered subject. Our trial looked at only one specific aspect of this, namely the effects of exposure of our own papers on our own social media channels. Nevertheless, we believe that it adds useful information to the existing body of literature. We hope that future RCTs will assess the effect of other measures of reach and exposure in order to give a fuller picture of the effect.

On 2016 May 24, Melissa Vaught commented:

This interesting article presents the impact of a journal's social media activity on article-level metrics - one of very few randomized trials on the topic. Notably it uses a traditional (citation counts) and an alternative metric (article downloads) as outcomes.

However, the title and descriptions of the trial's objective and intervention are a bit misleading. The authors state, "We sought to investigate whether exposing scientific papers to social media (SM) has an effect on article downloads and citations." But the study does not evaluate social media exposure. Rather, it measures the effect of a journal promoting its content via its own social media channels. The study did not include any measures of reach or exposure, such as retweets, shares, or likes of their posts. It also did not examine whether other users disseminated articles in the trial via social media.

Still, this study addresses a pertinent question: Does publisher promotion of publications via social media influence reading or citation? For IJPH, there is no apparent association. Of course, there are potential confounding factors, some beyond the journal's control. Did IJPH or authors' institutions issue press releases for any publications in the trial? Did any articles garner attention in mainstream media or on social media? Would easier access to publications affect outcomes? The authors report no difference in outcomes for social media promotion vs. control, when stratified for open access; however, there were few open access articles in the trial—10% in control, and 5% in intervention group.

Using citations as an outcome is a valuable contribution of this trial. Other randomized trials have evaluated effects of social media promotion by publishers on article views, which may moderately correlate with citations. In two separate trials (Fox CS, 2015, cited by the authors, and Fox CS, 2016), Circulation found that Facebook and Twitter posts did not significantly affect views at 7 or 30 days. By contrast, the Cochrane Schizophrenia Group (Adams CE, 2016) found a significant increase in 7-day page views for systematic reviews shared on Twitter and Weibo (a popular social media site in China).

Social media strategies could influence outcomes. By using Weibo, Adams CE, 2016 extended reach to a distinct audience. Post frequency and schedules likely affect audience reach as well. For instance, tweets posted at the same time weeks apart (as in the IJPH trial) might not be as effective as multiple posting over one or two days (as in Fox CS, 2016 and Adams CE, 2016). Overall social media engagement and post volume could also come into play.

The authors note that many observational studies show positive correlation between social media exposure and article views, downloads, or citations. They suggest the difference may be due to design (randomized vs. observational). Contexts and sizes of the studies could also contribute to differences in results. Only Sorenson M, 2014 looked at social media promotion by the journal; this was only for a small number of publications (3 promoted and 3 non-promoted). Other cited studies looked at third-party or global social media activity, for a handful to thousands of publications.

Ultimately this trial evaluates whether a journal's social media activity affects reading or citation of its own publications. So who follows journals on social media, and why? Social media uptake in scholarly communities is growing. But it's far from replacing other methods for discovering articles (Ware M, 2015, pp. 52-5, 134-5). Researchers are still likely to read articles recommended by colleagues (Tenopir C, 2015), and we might expect this trend to be reflected in social media. In other words, individuals on social media may have a degree of influence that publishers do not. The benefits of journal social media accounts may lie with other outcomes.

On 2016 May 24, Melissa Vaught commented:

This interesting article presents the impact of a journal's social media activity on article-level metrics - one of very few randomized trials on the topic. Notably it uses a traditional (citation counts) and an alternative metric (article downloads) as outcomes.

However, the title and descriptions of the trial's objective and intervention are a bit misleading. The authors state, "We sought to investigate whether exposing scientific papers to social media (SM) has an effect on article downloads and citations." But the study does not evaluate social media exposure. Rather, it measures the effect of a journal promoting its content via its own social media channels. The study did not include any measures of reach or exposure, such as retweets, shares, or likes of their posts. It also did not examine whether other users disseminated articles in the trial via social media.

Still, this study addresses a pertinent question: Does publisher promotion of publications via social media influence reading or citation? For IJPH, there is no apparent association. Of course, there are potential confounding factors, some beyond the journal's control. Did IJPH or authors' institutions issue press releases for any publications in the trial? Did any articles garner attention in mainstream media or on social media? Would easier access to publications affect outcomes? The authors report no difference in outcomes for social media promotion vs. control, when stratified for open access; however, there were few open access articles in the trial—10% in control, and 5% in intervention group.

Using citations as an outcome is a valuable contribution of this trial. Other randomized trials have evaluated effects of social media promotion by publishers on article views, which may moderately correlate with citations. In two separate trials (Fox CS, 2015, cited by the authors, and Fox CS, 2016), Circulation found that Facebook and Twitter posts did not significantly affect views at 7 or 30 days. By contrast, the Cochrane Schizophrenia Group (Adams CE, 2016) found a significant increase in 7-day page views for systematic reviews shared on Twitter and Weibo (a popular social media site in China).

Social media strategies could influence outcomes. By using Weibo, Adams CE, 2016 extended reach to a distinct audience. Post frequency and schedules likely affect audience reach as well. For instance, tweets posted at the same time weeks apart (as in the IJPH trial) might not be as effective as multiple posting over one or two days (as in Fox CS, 2016 and Adams CE, 2016). Overall social media engagement and post volume could also come into play.

The authors note that many observational studies show positive correlation between social media exposure and article views, downloads, or citations. They suggest the difference may be due to design (randomized vs. observational). Contexts and sizes of the studies could also contribute to differences in results. Only Sorenson M, 2014 looked at social media promotion by the journal; this was only for a small number of publications (3 promoted and 3 non-promoted). Other cited studies looked at third-party or global social media activity, for a handful to thousands of publications.

Ultimately this trial evaluates whether a journal's social media activity affects reading or citation of its own publications. So who follows journals on social media, and why? Social media uptake in scholarly communities is growing. But it's far from replacing other methods for discovering articles (Ware M, 2015, pp. 52-5, 134-5). Researchers are still likely to read articles recommended by colleagues (Tenopir C, 2015), and we might expect this trend to be reflected in social media. In other words, individuals on social media may have a degree of influence that publishers do not. The benefits of journal social media accounts may lie with other outcomes.

On 2016 Jun 06, Thomy Tonia commented:

We would like to thank Melissa Vaught for her interesting and stimulating thoughts on our paper. She states that the title and description of our trial’s objective is misleading. However, it is clearly stated in the abstract and in the introduction that we randomised IJPH papers into two arms one of which involved IJPH social media exposure at 3 time points; moreover, our Methods section describes this intervention in detail.

Melissa Vaught makes a very valid point stating that we did not include any measures of reach/exposure such as re-tweets, shares or other users’ dissemination of our papers in social media etc. We have touched upon this issue when we discussed the limitations of our study.

She also refers to potential confounding factors. Given the randomised design of our study, however, systematic confounding cannot be an issue; our analysis is what one would call “intention to treat” in clinical trials. There are some interfering factors that we could not control, which we tried to outline in our paper. This is why we standardised our intervention as much as possible, randomising only original articles (and not systematic reviews that generally tend to receive more attention) and stratifying for open access status. We have not issued any press releases for any of the randomised papers but we could not control for any press or other exposure coming from other sources such as authors’ institutions.

Although we agree that SM strategies could influence outcomes, the studies by Fox CS, 2016 and Adams CE, 2016 cited by Melissa Vaught did not directly compare different intensities of interventions such as multiple postings per day versus less frequent postings. Additionally, as we mention in our limitations, we could not have the data on paper views which could be different from the download data.

Melissa Vaught mentions that only the study by Sorenson 2014 looked at SM promotion by the journal. However, the studies by Fox CS, 2015 and Fox CS, 2016 which were randomised and bigger in size also assessed the promotion of papers from Circulation journal on their own social media and found no effect on article views.

We fully agree with the last remark of Melissa Vaught. Indeed, having social media presence as a journal can have benefits that lie beyond downloads and citations and this is one of the reasons our Journal is active on social media.

Indeed, social media exposure and its effects on scientific is a complex and multi-layered subject. Our trial looked at only one specific aspect of this, namely the effects of exposure of our own papers on our own social media channels. Nevertheless, we believe that it adds useful information to the existing body of literature. We hope that future RCTs will assess the effect of other measures of reach and exposure in order to give a fuller picture of the effect.

On 2016 Oct 04, Atanas G. Atanasov commented:

None