Publishing industry disruption - commentary. Includes a clear simple vision for what science publishing could do.

Schmidt Futures funding long-term software engineers to maintain scientific software.

Maybe this is what some kinds do through reputation / networking that comes from being funded, irrespective of the project outcome.

what if instead of looking at the output (discoveries) you look at the input (the people, the ideas)

For example: * What's the mix of applications to do the legwork to add/build evidence for a known question or theory where it is needed, versus pursuing shiny new ideas (hypotheses with no/low evidence as yet) * Who's applying for funding - and who is not applying. Is any scientist well-served somewhere within the funding ecosystem? If not, whose ideas and thinking are we (society) missing out on with the current funding scheme designs?

There's value in understanding the diversity of funding schemes.

In a world with only one type of funding pattern, it's change within that structure or don't change.

In a world with NIH grants and HHMI efforts and FROs and VCs funding individuals and scientist-entrepreneurs self-funding research from their own spin-outs, etc, etc, there's more opportunity to experiment with changes without requiring all funding to change.

chasing the new, the disruptor

Facebook does not exist in a vacuum

I think reducing the model to only evaluate for direct monetary profit is overly reductive model, even if VC's primary aim is monetary profit. For example, one company may not return the highest profit but may contribute some kind of technological or social advance that enables/augments profit (and further advance) from other companies. (i.e. how to include "enabling advances" in the evaluative model for VCs? Which previous smaller companies/contributions were necessary for Facebook to happen?)

This multi-dimension model is closer to science too - there may be direct immediate gain from a discovery, but not recognising the enabling gain from other discoveries may lead to missing the opportunity to enable the next big discovery.

May be worth comparing UK REF to other models by which performance is evaluated in order to prescribe the next tranche of government (or other) funding.

broken link

Alternatives to GScholar - Semantic Scholar might be interesting for snowballing.

See other sources for videos, podcast, grey lit

use GScholar library to edit then download citation data

Save interesting papers to check out later using Google Scholar (signed in my library) - click the star to save a listing, add tags to help with sorting / retrieving later.

save time by setting up GScholar as needed in Settings

GScholar search will help build search terms lists

Chapters to explore

Tagging for ASAPbio list

If you (like me) don't have access to this paper, you're not going to see the note about additional materials in the paper.

To help, you can find accessible materials about this research from the lab's website: https://www.crowtherlab.com/our-research/ which links to https://crowtherlab.pageflow.io/trees-against-climate-change-the-global-restoration-and-carbon-storage-potential#215055 . There are also lots of materials linked from their twitter profile (you need to login to twitter to find these, sorry): https://twitter.com/CrowtherLab.

Such important work should not be published behind a paywall.

Data for bioRxiv preprints from start to November 2018 (covering metadata and traction on social media)

Continues to be updated, viewable from rxivist.org and associated Zenodo data records (see link from rxivist)

Preprint available at https://www.biorxiv.org/content/10.1101/515643v2

Data for bioRxiv preprints from start to November 2018 (covering metadata and traction on social media)

Continues to be updated, viewable from rxivist.org and associated Zenodo data records (see link from rxivist)

Study of the impact of preprints on attention (citations, altmetrics) over time

Preprinted work gets more attention as a publication later than non-preprinted articles.

Data for submission growth and co-authorship networks from the 9 largest OSF preprint servers (all started in the past few years)

Preprint of this work is available at: https://osf.io/5fk6c/

Data for first year of activity on EarthArXiv (submissions, downloads, etc)

Content not found on Prolifiko anymore. Content seems to be available as PDF from ResearchGate: https://www.researchgate.net/publication/265037737_Becoming_a_productive_academic_writer (I don't have an RG account so not checked & can't be sure this is the original content linked to.)

How to spot when groupthink is an issue.

Advice to avoid groupthink

Pingback:

Q: Where can I post a preprint?

There are a number of preprint servers available, ranging from discipline specific servers to more broad servers, like SAGE’s own preprint server, Advance, that covers all research in the humanities and social sciences. For a comprehensive list of preprint servers, check out ASAPbio’s survey of preprint servers.

https://us.sagepub.com/en-us/nam/preprintsfaq

Supplementary tables and figures -- including survey and more complete description of results

Analysis code (R) in .txt format

Full raw survey data in .xlsx format

PDF Instruction Manual of how to do the evaluation, as found on OSF: https://osf.io/4zk8x/

Rich Abdill has updated the Zenodo record with figure PDFs and made these CC0: https://zenodo.org/record/2603083#.XJUjbFP7SRs

Version log note: "22 Mar 2019: PDFs of each figure from the paper have been added to the repository. In addition, the license has been changed from CC-BY-NC to CC0."

Agree. Jessica and I have been looking at this using prepubmed data and Pubmed numbers: from these, preprints are currently ~2% all biomedical literature each year and growing (slowly)

On the point re timing of downloads (before/after publication), Bianca Kramer has explored this question: https://twitter.com/MsPhelps/status/1089836238482403328

Data & analysis at https://tinyurl.com/Rxivist-further-analysis (https://docs.google.com/spreadsheets/d/18-zIlfgrQaGo6e4SmyfzMTY7AN1dUYiI5l6PyX5pWtg/edit#gid=1455314569)

OA status of journal does not seem to be a factor in the # or % of preprint downloads in the first year after journal publication --> suggests the correlation between IF & downloads is not related to the link between high IF and low OA.

This has now been added, see https://twitter.com/richabdill/status/1101568939849596928?s=21

There is some debate as to whether preprints should be covered in the general media with some recommending that journalists do not report on a preprint: http://thenode.biologists.com/preprints-and-science-news-how-can-they-co-exist-a-meeting-summary/events/

Also an interesting suggestion.

This is an interesting use case. I wonder if it might be worth repurposing on ECRcentral somehow?

Absolutely, we plan to make use of these data and build on them to understand advocacy and outreach needs and opportunities. We welcome collaborations: please feel free to contact me on naomi dot penfold at asapbio dot org.

This is valuable information, thank you

Thank you for pointing this out. This is informative for ASAPbio outreach strategy.

Agree and willing to help with this.

Disclosure: My annotations are provides from both professional and personal perspectives: I am Associate Director at ASAPbio, which works to promote the productive use of preprints in the life sciences. I am not affiliated with bioRxiv. The preprint first author, Rich Abdill, has joined the ASAPbio ambassador group since posting this preprint.

I agree (personal opinion).

We’d (ASAPbio) like to produce similar and extended analysis that covers more preprint servers. Now advertising for a Vsiitng Scholar to work with Juan Pablo Alperin and colleagues at ScholCommLab: https://www.scholcommlab.ca/join-us/vsp-preprints/

Wholeheartedly agree, would be very useful!

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/355289

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/169698

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/170720

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/192336

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/410217

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/409441

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/408880

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

An earlier version of this work is available as a preprint on bioRxiv: https://doi.org/10.1101/400036

Disclosure: I work for ASAPbio, a non-profit organisation working to improve transparency in science, including through the productive use of preprinting in the life sciences.

See Daniel's blog about the impact of licensing choice, and current choices by authors (at time of writing), which includes an interesting footnote about what happened when biorxiv presented the most open license (CC-BY) at the top of their options: https://blog.dhimmel.com/biorxiv-licenses/

The process matters, as does who gets to be involved in it.

Highlights responsibility on shoulders of people who share results/interpretations upon which recommendations are built

Example of making the process of improvement fun, removing barriers to criticism.

Allow people ownership of the method, to improve trust in the results

counter-movement: #ScienceIsPolitical

Instead of presenting observations and what they might mean, we sell stories and narratives about what they do mean. The best-connected storyteller has a network advantage and tends to out-market alternatives. Discussion about whether this is valid or not is forgotten after publishing when peer review is assumed done and dusted.

Statement from Global Young Academy about Plan S: conceiving worst case and best case scenarios, and calling for ECRs to be involved in implementation discussions.

A worst case scenario: OA journals have uncontrolled APCs (in the long term, short-term cap is insufficient) and OA excludes those who can't pay to publish, thus widens inequalities.

A best case scenario: publishers receive funding from funders to act as a service, no reader or author has to pay. Publishers are evaluated for the service they provide and thus the academic community can reward the better services, and good OA publishing thrives.

typo

typo? 'the' reads better

This writes to HTML, right? Might be worth stating that, as this follows discussion of potentially writing in JATS XML.

A larger problem may be that (as I understand it) there is currently no agreed 'common' JATS XML from which content disseminators could then specialise. So agreeing a flavour of JATS XML to write in may be a bigger issue. Same applies to the markdown conversion point.

Is it possible to create a tool that warns you when fewer than 3 peers hold any of your registered content?

open-by-design

peer-to-peer (for consistency)

Note to self: this might require tooling that ensures the content creator has checked for private info before sharing, much like authors should check before they post a preprint or dataset and developers should check for passkeys etc before they push code. Errors still occur in those cases.

Does the researcher draft the between-version contents within their local filesystem, i.e. akin to my local .md files that I only push to github once I'm ready?

UPDATE: Yes, stated in next section.

This phrase doesn't seem to follow grammatically from the rest of the sentence: it is not clear what "can also help improve...".

Do you mean "However, because... not mediated. This can also help..." ?

Typo? Alice's or her?

Are these terms compliant with schema.org where appropriate? That might help with interoperability with current systems. That said, I don't know how much ORCID or others comply with schema.org.

What impact do you foresee on the degree of trust the community at large places on a profile that is not under centralised 'perceived-to-be-authoritative' control? Is that trust required or is this a notion from a centralised authority state?

I found the double-ended arrows in this quite confusing (direction versus type not clear). Potential alternative: a shape that widens where 'more', narrows where 'less', so the type is labelled in text but the quantity is depicted pictographically by width of the line/shape.

Example of author sharing all code via jupyter notebooks in a github repo. They have archived to Zenodo and include both URLs in the text. Their analysis relies on an existing toolkit - it is not obvious from the manuscript whether this toolkit has been deposited anywhere.

Journal: PLOS ONE Subject area: plant biology, ecology (check?)

Cited reference is open access and describes the method

Protocol not fully open. Based on closed access paper. Extension to method is clearly described.

Well-described dry lab notebooks with open code, and Python analysis toolkit is open source.

Example of authors archiving their own code. The eLife process is to fork the author's repo to the eLife github repo to save a snapshot of the repo at the time of acceptance. The authors here have also chosen to archive to Zenodo (via the Github release --> Zenodo mechanism?). Both the Zenodo DOI and the eLife fork are included in the text as archive/snapshot copies of the original (also cited).

Note the author's original repo is included in the references.

Journal: eLife Subject area: ecology, evolutionary biology

In this example, the archived code repository has been listed in the eLife Key Resources Table

I think PLOS ONE would disagree: their review is based on scientific validity, just removes the branding element. It's still supposed to be rigorous.

I don't understand this definition - this sounds like transparent review to me, not to do with post-publication.

a Dutch researcher not allowed into US due to a trip to Iran to present research and meet researchers there.

This suggests they are publishing the genome itself to the dash blockchain — this is incorrect. The proposal actually says they are publishing the record of production of new strains of cannabis to the dash ledger such that they can prove they created a new strain (and when) and thus protect themselves from patent trolls. As a colleague said, they could then share the hash of the hash with the patent office. https://www.dashcentral.org/p/MedicinalGenomics

Please hyperlink:

mailto:n.penfold@elifesciences.org?subject=eLife%20Sprint%20on%20CSS%20call

Include extra sentence?

"Key learnings were that inclusivity is really hard, designing for including participants in the programming helps folks to make it work for them, and that the quality and attitude of the people in the room really mattered for how successful the event was so curation of those people was crucial."

Long-term

Smith-Unna?

please link to https://elifesciences.org/inside-elife/b4ed92e1/innovation-collaboration-and-creativity-at-the-heart-of-the-elife-innovation-sprint-2018?utm_source=CSS&utm_medium=referral&utm_campaign=sprint-IE

experience with planning

and

offline-friendly

offline-friendly

mobile-friendly

is going

separate with space

one year's or many years' ?

database

sponsored

Interesting comment on this preprint from Gaetan Burgio

"This fascinating paper shows that prestige is a major driver in #academia for 1) Faculty hiring 2) diffusion of scientific ideas leading to media release, citations, conference invitations .... In short #academia is not a place for meritocracy #ECRchat"

Source: https://twitter.com/GaetanBurgio/status/1001042437953937408

This was done at the eLife Innovation Sprint, May 10-11, 2018. The work is documented at: https://github.com/rossmounce/DOIs-in-Wikipedias

died by suicide.

See more: https://www.speakingofsuicide.com/2017/09/21/suicide-language/ (plus multiple other pieces online).

Could we do this for open science?

Q for myself: How is this different to Dat? Investigating on Thursday.

How is this different to the Allen Institute resources? From the project GitHub: "these valuable resources are underused for the analysis of polygenic brain disorders because the data is not easily accessible beyond the level of a single gene." From experience, I'd agree.

Watch this

Watch this. Any simple rules to bear in mind for new innovations?

This post was planned in public at https://github.com/jupyterhub/binder/issues/22

An example of why it is important to understand the context behind a number: https://harzing.com/publications/white-papers/the-mystery-of-the-phantom-reference

Explore these

Can't find this anywhere.

It would be good to share clear examples of these.

Where's the upfront investment to get this kind of model going? (That's what VC does; substantial amounts are needed, which is risky for the grant route.)

How would this be sustainable in the long run? (Several examples, need to examine both successful and unsuccessful.)

See https://www.crossref.org/services/event-data/ (also DataCite)

Suggest Dat present?

Would we have anything to say here?

Suggest Texture/Stencila present

See also Bik, Casadevall and Fan (2016) about the prevalence of image duplication in biomedical literature. Prevalence about 4% articles in most recent end of dataset (2014). http://mbio.asm.org/content/7/3/e00809-16.full

The idea of classifying hypotheses as supported or refuted by ongoing works, as a means to identify "strongly supported" or "strongly refuted" claims is an interesting one. I would like to see further discussion of how this could be applied.

Namely, it seems the R-factor is something that should be applied to a specific scientific claim, as opposed to a whole research article. Being able to quickly identify the evidence that supports (green), refutes (red) or relates unclearly (yellow) to a claim, directly from the claim in said literature, could aid comprehension (not to mention discoverability) of the surrounding literature, and highlight claims that are well-supported or lacking in independent replications. Do the authors feel that one paper is sufficiently related to one central claim for application of the R-factor the the paper? Alternatively, I would argue that judging the "veracity" of component evidence presented within an article could be more informative.

Further, limiting these data to the "cited by" literature from that paper could skew the perspective, depending on which article you are viewing the claim in - to understand the overall "veracity" of a claim, it seems the reader would need to navigate back to the first mention of that claim in order to find the longest chain of evidence. Instead, I would be interested to explore the feasibility of a claim-centric (as opposed to paper-centric) count, and to understand whether this is already achieved by existing practises (such as meta-analyses of the literature). Perhaps an alternative approach would be to ensure that meta-analyses that include an article are more clearly visible from that article (e.g. highlighted in a "cited-by" section), and an extension to that would be to link that more recent work to the specific assertions that it relates to in the current article.

I would also be interested in whether the authors' have any thoughts on the reporting bias towards positive results (it may be hard to judge replicability, if failed replications remain in desk drawers), as well as on more nuanced evaluations of related evidence: is some evidence stronger than others? Is it feasible to define a scientific claim, or is it dependent on context/species/other factors?

Finally, I would be concerned about applying such a metric to individual researchers. An examination of unintended consequences for such a metric would be useful to discuss.

Competing interests: None identified. Full disclosure: I am Innovation Officer at eLife. My academic background is not in meta-research of reproducibility.

Interesting.

Is the journal's role there to act as a channel - and the individual chooses their preferred channel?

Who takes on responsibility for good production (links, etc) as mentioned earlier? Surely don't want duplication of efforts there. Or is that what occurs in step before journals come in and present published papers?

Would love to dig into more detailed imagined use cases here.

It should already be?

This would make it easier to trace the context of a citation, in a way that researchers could flag inappropriate or misleading citations, as well as judge whether a citation to a retracted article is erroneous or intended.

Akin to the Crossref 'article nexus' model https://www.crossref.org/blog/the-article-nexus-linking-publications-to-associated-research-outputs/

I think a place to surface all the associated content with the article is missing right now.

What level of assessment? Is this journal-independent?

We've talked about these levels:

• quick QC before preprint posting (but minimal so as not to block)
• assessment of scientific rigour (traditional peer review)
• identifying research that peers deem to be particularly important / groundbreaking / of interest to a wider audience (a place for journals?)

Love this idea!

Are the old versions still retrievable in some way?

How does this benefit people working in areas of intermittent internet access?

Currently, they would store data they need locally. Does this mean they can share it P2P, reduce the amount they store locally, yet still be able to access it even when internet down?

An authentication system could be useful for protected data (e.g. patient, clinical, personal data, where researchers must prove identity and be granted right to access data for further research). It may be worth looking into how this is currently managed and whether Dat could make it easier for these administrators.

This shouldn't be a problem for scientific datasets as they have DOIs. Have I misunderstood?

For clarity, a diagrammatic representation of this could be useful (I'm a more visual learner)

Does this mean that a journal could point to a specific version of a dataset, pertaining to that which a peer-reviewed article is based on? This version of record is important.

An additional link could then point forward to the most up to date version of the data, with a summary of the changes made since the publication version?

Inside Higher Ed to publish series of blogs from researchers working on open science

Have you also heard about Dat and IPFS? https://datproject.org/ https://ipfs.io/

And this blog: https://chartgerink.github.io/2017future/

The publication fee is not simply a hosting fee. Actually, a third of the current cost per published article is due to the scientific editors being recompensed for the work they do in peer review - see more at https://elifesciences.org/inside-elife/a058ec77/what-it-costs-to-publish. Ideally, the scientist should be free to choose between services to disseminate their research, from simple cheap servers that just upload to the web, to more costly peer-reviewed destinations that employ staff to manage press, outreach, and so on.

Are you able to provide some key examples here? Do you think it is possible to avoid this situation?

Have you shared this with him?

Permanently? Or would the researcher be identifiable in some way somewhen later?

The reason I say this is that there may be some researchers who never want to be identifiable, particularly if their work is politically sensitive. Of course, there's a counter-argument for non-anonymity in order to have accountability.

Importantly, this decouples the act of preparing all the information for sharing from the time of being ready to share, such that the information sharing is of the highest quality (soon after the event) whilst acknowledging that research is not open-by-default and there are socioeconomic factors why it may never be (even in the new system proposed here).

If you could add emoticon reactions via hypothes.is, right now I'd be struggling between a +1, a laughing face, and any representation that means 'my brain hurts when I think about blockchain'

Any paid service that enables the payee to gain greater access to research (by efficiency of discovery) could also contribute to inequality. Those with money can produce more work faster simply because they've been able to access the right information more quickly, and thus the cycle perpetuates. What do you think about this?

Or to learn the best practises outside of the limited physical places in an expert lab group, thus limiting who can contribute to the pursuit of knowledge creation.

More moments of communication that incorporate the many steps in research, i.e. pre-registered report, micropublications, data articles, software packages, evidence reviews to bundles small steps into larger breakthroughs in understanding --> opens up credit for all types of researchers: from the idea-generators, to the expert-practitioners (wet lab, data, etc) and values all contributions to the main goal: finding the/an answer to the question.

Would you consider DataCite's re3data (searchable and filterable database of data repositories) another useful resource here? http://www.re3data.org/

Is it ok to work straight into the issue template?

Or is it best to copy the template into a local text file, edit that from the research, then submit?

jump to the template? and a quick how-to (copy and paste this template into a text editor e.g. wordpad, whatever)?

Is there an easy way to organise these as claimed / unclaimed? At the mo, I'm copying the ID then ctrl+f in the spreadsheet to cross-reference.

I'd love to see a how-to guide come out of this.

We've just tried to make one, see 'Beginner's Guide to Github' at: https://docs.google.com/document/d/1gTdPPr0ZMCKv5X-_XL9au6buwV6Fz194djjw3YXSFvs/edit#

• written by me, a github beginner, so edits and corretions very welcome!

General comment: emoji use is fantastic, bringing the page to life!

broken link. Do you mean https://help.github.com/articles/about-discussions-in-issues-and-pull-requests/ ?

an issue?

would this be said if the recipient were productive and male?

R

https://elifesciences.org/content/3/e04395