Replication--getting the same result from doing the whole study again--is logically independent from whether there even exist any code/analysis from the original paper. Recommend revising and instead talking about how they are related and often conflated.

The reproduction of the original source giving as a result the same source code, input data, software and configurations in the system might be considered as Computational reproducibility

According to Patil, P et al. (2016) state that "everyone agrees that scientific studies should be reproducible and replicable. The problem is almost no one agrees upon what those terms mean. A major initiative in psychology used the term reproducibility'' to refer to completely re­doing experiments including data collection (1). In cancer biologyreproducibility'' has been used to refer to the re calculation of results using a fixed set of data and code." (pag. 1)

A possible approach to statistical reproducibility is to re-do experiments over and over again gathering all the data available to get the findings, but emphasising in the analysis with transparency in the information to have accurate conclusions.

References

Prasad Patil, Roger D. Peng, Jeffrey T. Leek. (2016). A statistical definition for reproducibility and replicability. see on https://www.biorxiv.org/content/10.1101/066803v1.full.pdf

This is the recommended pratice to achieve reproducible and shareable artifacts, unless you want to ship your image like in the Docker meme:

In a discussion in the Cuis mailing list, many people share their different approaches when working alone and in collaboration with others. See the whole thread spanning from July to August, not just the linked mail.

Reproducibility

Reusability and reproducibility of research output

When people mistakenly raise concerns about the Web platform being fragile, point to this common meme.

This is a real problem/the real problem.

⬑ "The Code Rot Problem"

==>l’échantillon de l’étude est est extrêmement limité, l’étude n’est pas répliqué et elle ne permet pas de retirer de résultats concluants

The Work of Art in the Age of Mechanical Reproduction by Walter Benjamin

1. Benjamin is part of the Frankfurt School at Institute of Social Studies in Germany.
2. They are trying to examine the failure of Marxist revolutionary social change.
3. The idea is that ideology disseminated through mass media are making it very difficult for Marx's prognotication are making it very difficult for social change to occur.
4. 19th century modernity: mass transportation, factory work, dissemination of capitalism, movement to cities and experience of urban life

 These convergent endeavors made predictable a situation which Paul Valéry pointed up in this sentence: “Just as water, gas, and electricity are brought into our houses from far off to satisfy our needs in response to a minimal effort, so we shall be supplied with visual or auditory images, which will appear and disappear at a simple movement of the hand, hardly more than a sign.” (op. cit., p. 226) Around 1900 technical reproduction had reached a standard that not only permitted it to reproduce all transmitted works of art and thus to cause the most profound change in their impact upon the public; it also had captured a place of its own among the artistic processes. For the study of this standard nothing is more revealing than the nature of the repercussions that these two different manifestations—the reproduction of works of art and the art of the film—have had on art in its traditional form.

Q: Why does it matter that film minimizes the aura?

At the time, art reacted with the doctrine of l’art pour l’art, that is, with a theology of art. This gave rise to what might be called a negative theology in the form of the idea of ‘pure’ art, which not only denied any social function of art but also any categorizing by subject matter. (In poetry, Mallarmé was the first to take this position.) An analysis of art in the age of mechanical reproduction must do justice to these relationships, for they lead us to an all-important insight: for the first time in world history, mechanical reproduction emancipates the work of art from its parasitical dependence on ritual. To an ever greater degree the work of art reproduced becomes the work of art designed for reproducibility.7 From a photographic negative, for example, one can make any number of prints; to ask for the ‘authentic’ print makes no sense. But the instant the criterion of authenticity ceases to be applicable to artistic production, the total function of art is reversed. Instead of being based on ritual, it begins to be based on another practice—politics.<br> Ritual: pre-modern timesPolitics: Despite the political painting the art piece will be associated with the aura of original piece of art.

• Film is not auratic because in film: 1) spaces and times are constructed 2) actors performance is stitched together 3) actors do not share space with spectators 4) multiple points of view 5) appeals to a MASS AUDIENCE and a COLLECTIVE AUDIENCE 6) reveals new aspects of the thing reproduced (time alpse, slow motion).

How do institutions put the aura back into film?

In photography, exhibition value begins to displace cult value all along the line. But cult value does not give way without resistance. It retires into an ultimate retrenchment: the human countenance.

The superstar is a way to put the aura back into film

The film responds to the shriveling of the aura with an artificial build-up of the “personality” outside the studio. The cult of the movie star, fostered by the money of the film industry, preserves not the unique aura of the person but the “spell of the personality,” the phony spell of a commodity. So long as the movie-makers’ capital sets the fashion, as a rule no other revolutionary merit can be accredited to today’s film than the promotion of a revolutionary criticism of traditional concepts of art. We do not deny that in some cases today’s films can also promote revolutionary criticism of social conditions, even of the distribution of property. However, our present study is no more specifically concerned with this than is the film production of Western Europe.

*Marx says capitalism produces the seeds of its demise. We can think of that as a guiding principle in which capitalism produces the neorosis that leads Chaplins character into a destructive set of behaviors that stops the Fordist capitalist production in a factory.

Feelings of belongoing and togetherness and being overhwemed by a mass you want to be a part of , but for Benjamin this is a trynanny where people feel they are in control but they are not really in control. Thus communism replies by politicizing art. Art for art's sake is harmful when put in the service of a facist regime. Art for political progress.

The growing proletarianization of modern man and the increasing formation of masses are two aspects of the same process. Fascism attempts to organize the newly created proletarian masses without affecting the property structure which the masses strive to eliminate. Fascism sees its salvation in giving these masses not their right, but instead a chance to express themselves.21 The masses have a right to change property relations; Fascism seeks to give them an expression while preserving property. The logical result of Fascism is the introduction of aesthetics into political life. The violation of the masses, whom Fascism, with its Führer cult, forces to their knees, has its counterpart in the violation of an apparatus which is pressed into the production of ritual values.

Film is not merely a translation of an in-person thetaer performance. Rather film is performing for future audiences and for the director and for cinematrography. This supports the idea that film is a collabroative creation that brings an object into the world.

One film can be playing multiple times around the world and so this can be distributed on mass scale.

Film is not merely a recording of reality. Film reproduced new aspects of the things reproduced through slow motion and it brings to light entirely new aspects of matter but discloses quite aspects within them. If Benjamin merely interested in the epistemological possibility of the film to expand our limited perceptual appartus, yes but think about how this reinforces his claim that film moves us away from the aura,...that if we can see unknown aspects by recording it then we can't rely on film to reproduce an original we have to keep in mind that the image is qualitatively distinct from our perceptual access to the thing. So film is not merely a copy of the thing that it records.

Benjamin flips things and says that maybe film isn't art the way we see an art and this will get us away from the trappings.

What is Benjamin's definition of art which he is defining with the aura, the transcendence of individual of ritual.

1. substructure or base: factors that produce commodities and economic relations that result from these concrete aspects
2. superstructure: culture, law, media that for a Marxist thinker emerges in the way that the economic structure functions; the more media/education/political cosumption that you do the less you are going to understand the conditions of your exploitation and the more you are going to think change is possible.
3. What role does cinema play in the move from cult and aura to mechanical reproduction? See snapshots.

Replicating scientific results is tough—But essential. (2021). Nature, 600(7889), 359–360. https://doi.org/10.1038/d41586-021-03736-4

AIMOS. (2021, November 30). How can we connect #metascience to established #science fields? Find out at this afternoon’s session at #aimos2021 Remco Heesen @fallonmody Felipe Romeo will discuss. Come join us. #OpenScience #OpenData #reproducibility https://t.co/dEW2MkGNpx [Tweet]. @aimos_inc. https://twitter.com/aimos_inc/status/1465485732206850054

Serra-Garcia, M., & Gneezy, U. (2021). Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21), eabd1705. https://doi.org/10.1126/sciadv.abd1705

Logg, Jennifer M., and Charles A. Dorison. “Pre-Registration: Weighing Costs and Benefits for Researchers.” Organizational Behavior and Human Decision Processes 167 (November 1, 2021): 18–27. https://doi.org/10.1016/j.obhdp.2021.05.006.

This is what I'm aiming for with triplescripts.org. Initially, I'm mostly focused on the reproducibility the build process for software. In principle, it can encompass all kinds of use, and I actually want it to, but for practical reasons I'm trying to go for manageable sized bites instead of very large ones.

There is only one reasonable approach, and it's not even mentioned as an option here: the browser edition. (I.e., written to target the ubiquitous WHATWG/W3C hypertext system.)

Conference Details. (n.d.). AIMOS. Retrieved 5 March 2021, from https://aimos.community/2020-details

ReconfigBehSci. (2020, November 23). RT @lakens: Ongoing, an incredibly awesome talk on computational reproducibility by @AdinaKrik. Slides: Https://t.co/QslHW3gEtm. If you are… [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1331265422109396995

Armeni, Kristijan, Loek Brinkman, Rickard Carlsson, Anita Eerland, Rianne Fijten, Robin Fondberg, Vera Ellen Heininga, et al. ‘Towards Wide-Scale Adoption of Open Science Practices: The Role of Open Science Communities’. MetaArXiv, 6 October 2020. https://doi.org/10.31222/osf.io/7gct9.

Peer, L., Orr, L., & Coppock, A. (2020). Active Maintenance: A Proposal for the Long-term Computational Reproducibility of Scientific Results. SocArXiv. https://doi.org/10.31235/osf.io/8jwhk

Perkel. J. M., (2020). Challenge to scientists: does your ten-year-old code still run? Nature. Retrieved from: https://www.nature.com/articles/d41586-020-02462-7?utm_source=twt_nnc&utm_medium=social&utm_campaign=naturenews&sf237106326=1

Brian Nosek {@BrianNosek} (2020) RT @BrianNosek Preprint "conversations about reproducibility have a clear underlying structure, despite the broad scope..of the crisis. ..… Twitter. Retrieved from: https://twitter.com/SciBeh/status/1298941122912440320

Grant, S., Wendt, K., Leadbeater, B. J., Supplee, L. H., Mayo-Wilson, E., Gardner, F., & Bradshaw, C. (2020). Transparent, Open, and Reproducible Prevention Science. MetaArXiv. https://doi.org/10.31222/osf.io/d2y43

We desperately need to fix these foundations of science to focus on solid foundations and reproducibility...

Science could learn something from this. Science is too far focused on the idealized positive outcomes that it isn't paying attention to the negative outcomes and using that to better define its outline or overall shape. We need to define a scientific opportunity cost and apply it to the negative side of research to better understand and define what we're searching for.

Of course, how can we define a new scientific method (or amend/extend it) to better take into account negative results--particularly in an age when so many results aren't even reproducible?

Separating a workflow description from its execution is an important aspect of reproducibility.

Moreau, D., & Gamble, B. (2020). Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychological Methods, No Pagination Specified-No Pagination Specified. https://doi.org/10.1037/met0000351

aaronpeikert. (2020). Aaronpeikert/reproducible-research [TeX]. https://github.com/aaronpeikert/reproducible-research (Original work published 2019)

Welcome to a new ERA of reproducible publishing. (2020, August 24). ELife; eLife Sciences Publications Limited. https://elifesciences.org/labs/dc5acbde/welcome-to-a-new-era-of-reproducible-publishing

Hohn, R. E., Slaney, K. L., & Tafreshi, D. (2020). An Empirical Review of Research and Reporting Practices in Psychological Meta-Analyses. Review of General Psychology, 1089268020918844. https://doi.org/10.1177/1089268020918844

Nelson, N. C., Ichikawa, K., Chung, J., & Malik, M. (2020). Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis [Preprint]. MetaArXiv. https://doi.org/10.31222/osf.io/sbv3q

Supporting Open Science Data Curation, Preservation, and Access by Libraries. (n.d.). Retrieved 24 August 2020, from https://www.youtube.com/watch?v=SbmGWHpzAHs

Peterson, David, and Aaron Panofsky. ‘Metascience as a Scientific Social Movement’. Preprint. SocArXiv, 4 August 2020. https://doi.org/10.31235/osf.io/4dsqa.

Supporting Open Science Data Curation, Preservation, and Access by Libraries. (2020, June 25). https://www.youtube.com/watch?v=SbmGWHpzAHs&feature=youtu.be

Welcome! You are invited to join a webinar: Supporting Open Science Data Curation, Preservation, and Access by Libraries. After registering, you will receive a confirmation email about joining the webinar. (n.d.). Zoom Video. Retrieved June 28, 2020, from https://zoom.us/webinar/register/2615905946283/WN_W6dYUXQFTqGQjGAZPRB74w

Minocher, R., Atmaca, S., Bavero, C., McElreath, R., & Beheim, B. (2020, June 24). Reproducibility of social learning research declines exponentially over 63 years of publication. https://doi.org/10.31234/osf.io/4nzc7

BehSciMeta repost. (2020, June 17). "Reproducibility scores for behavioural science: what are the merits and drawbacks?" Reddit. https://www.reddit.com/r/BehSciMeta/comments/har2np/reproducibility_scores_for_behavioural_science/

UKRN position on covid 19 research. (2020 May 01). School of Psychological Science | University of Bristol. http://www.bristol.ac.uk/psychology/research/ukrn/news/2020/ukrn-position-on-covid-19-research.html

Chawla, D. S. (2020). Critiqued coronavirus simulation gets thumbs up from code-checking efforts. Nature. https://doi.org/10.1038/d41586-020-01685-y

Leplaa, H. J., Rietbergen, C., & Hoijtink, H. (2020). Bayesian evaluation of replication studies [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/49tbz

Peikert, A., & Brandmaier, A. M. (2019). A Reproducible Data Analysis Workflow with R Markdown, Git, Make, and Docker. https://doi.org/10.31234/osf.io/8xzqy

• Prinsip utama yang patut menjadi pegangan para peneliti/dosen masa depan adalah: keinginan pribadi (self-driven initiative) untuk menyebar-luaskan berbagai hasil risetnya secara luas.

• Untuk itu teknologi digital dan internet harus dimanfaatkan dengan maksimal. Seringkali dosen/peneliti melupakan hal ini, bahwa internet ditemukan untuk mempercepat dan memperluas penyebaran ide.

• Integritas riset dan reprodusibilitas adalah dua nilai yang perlu ditonjolkan agar relevan dengan kemajuan era internet, bukan hanya menonjolkan nama jurnalnya dan JIFnya.

Integritas riset dan reprodusibilitas adalah dua nilai yang perlu ditonjolkan agar relevan dengan kemajuan era internet, bukan hanya menonjolkan nama jurnalnya dan JIFnya.

Argument for the reproducibility channel

The computer may be saying "oops", but I suspect that researchers are using stronger language.

sabotaging reproducibility

Sabotaging reproducibility.

I think this is one of the most outrageous statements I have ever heard.

Was there not even an editorial expression of concern or whatever they call that?

Wow. That is a very strong indictment. Big argument for why independent channels on top of the literature are so important.

I'm not sure that ghost writing is a bad thing as long as "writer" is not confused with lead scientist. Yet another strong argument for contributor roles rather than authors. https://www.force11.org/blog/are-we-ready-define-scholarly-commons-thoughts-force2015

Example of the type of harm that lack of transparency can lead to.

Validation statement RRID:AB_1078205 Summary

Validation statement RRID:AB_1078205 No primary control

Validation statement RRID:AB_1078205 Titration curve

Omission of primary antibody

Validation statement RRID:AB_1078205

Validation claim

Validation statement for RRID:AB_1078205

Definition of reproducibility

Great quote; of course, this statement is true of all communications. We have entire industries devoted to spin.

That is rather outrageous that we've known about this since 1959 and have done nothing about it.

Another case for annotation

Nice quote

Costs of irreproducibility

Great quote for reproducibility

The importance of post-publication annotation. To me, this is the critical missing piece.

The essence of science

Another damning statement.

Reproducibility is mentioned in the title of this section, but it is not mentioned elsewhere in the document and I don't see it addressed in what's written in this section.