- Feb 2018
-
rionagtaylor.gsucreate.org rionagtaylor.gsucreate.org
-
Each panel is made of a soft fabric and sewn onto a large, 12’x 12′ piece of ivory linen fabric.
This first description of the panel gives a brief but detailed imagery. Very good
-
Block #621
Great pictures - but could be more multi modal as a whole
-
- Nov 2017
-
cis-india.github.io cis-india.github.io
-
Media Maker Spaces is an exploration for them experience a creating, editing, storing. publishing and streaming media to their peers and to their immediate context.
Seems quite related to #CollecionAndIdentity
-
-
cis-india.github.io cis-india.github.io
-
Festivals and rituals - Living archives of the memories
Seems quite related to #CollectionAndIdentity
-
-
www.the-tls.co.uk www.the-tls.co.uk
-
An excellent commentary on what ails our current peer review system and how alternative quality assurance system might work in academics.
-
- Oct 2017
-
www.hewlett.org www.hewlett.org
-
review and critique each other’s work.
This is the process of replying to annotations. But annotation can also be leveraged for peer review of student writing.
-
-
Local file Local file
-
Brianna:I had a negative experience where, in my master’s, my supervisor encouraged me to submit one of my papers to a journal for publication. I just submitted the paper to a journal as a course paper without making any changes, not even changing the title page. The journal told me to re-submit with revisions, but I thought thatit was a rejection, and I stopped the process—it was intimidating. I thought being involved in a journal where I know some of the people and they won’t just get an online e-mail response from editors would be helpful
Misunderstanding revise and resubmit; misunderstanding the difference between a student paper and an article.
-
-
wellcomeopenresearch.org wellcomeopenresearch.org
-
I left a number of annotations here as part of my review of the paper.
The review is accessible via https://doi.org/10.21956/wellcomeopenres.13272.r25804 , and the notes I took on the process can be found at https://github.com/Daniel-Mietchen/ideas/issues/494 .
-
- Sep 2017
-
blog.scienceopen.com blog.scienceopen.com
-
The problems here stem from a lack of comprehensiveness, interoperability, and critical mass uptake as the de facto platform for PPPR. The result of this is a mess of different platforms having different types of commentary on different articles, or sometimes the same ones, none of which can be viewed easily in a single, standardised way. That doesn’t seem very efficient.
This is really key.
-
- Jun 2017
-
www.nature.com www.nature.com
-
protected platform whereby many expert reviewers could read and comment on submissions, as well as on fellow reviewers’ comments
Conduct prepeer review during the manuscript development on a web platform. That is what is happening in Therapoid.net.
-
intelligent crowd reviewing
Crowdsourcing review? Prepeer review as precursor to preprint server.
-
- Apr 2017
-
books.google.com books.google.com
-
p. 1
Peer review is a mechanism, then, for quality control; It protects us from contamination by error and poor argument, and affords us truth or contributions to attaining truth.
-
Shatz, David. 2004. Peer Review: A Critical Inquiry. Issues in Academic Ethics. Lanham, Md: Rowman & Littlefield.
-
-
shvdesofpink.gsucreate.org shvdesofpink.gsucreate.org
-
The Effects of the Built Environment on Child Friendliness and Obesity: Analysis on Auburn Ave
Charmaine's claim isnt clear at this very moment but she provides alot of background information on her location that I didnt knw about. If she specifically points out her claim them this analysis would be good.
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Conclusion
Claim: By investing in communities through new businesses and innovations, the population demographic may experience a drastic change from low income residents to high income residents. This then leads to other benefits such as a lower crime rate.
Evidence: The author uses a chart to display the distribution of income when looking at ages in the Ponce de Leon area (secondary source).
Overall, I really enjoyed reading your BEA. I felt that it was professionally crafted with the rhetorical situation kept in mind. One addition I would suggest would be looking also at the negative aspects of of gentrification, such as the displacement of residents that resided in the area before the process of gentrification began. By doing this, it would add to your credibility by recognizing and arguing the opposing position.
GREAT JOB!!!!!
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Description of Ponce City Market
Like the last page, the current description of Ponce City Market provides strong evidence for the claim of how gentrification can improve an area in various ways. Be sure to actually state your ideas to reinforce your ideas. Think about once again addressing the questions as well.
The pictures may also need to be cited if not personally captured.
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Description of Krog Street Market
Claim: The introduction claim needs to be incorporated into this description, because it directly provides evidence for such ideas.
Evidence: By using a highly detailed description as well as pictures, it's clear that the Krog Street Market has experienced obvious improvements. Use the questions suggested at the beginning to further emphasis how much the area has improved for the better!
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Demographics of Ponce de Leon Area
Claim: Like previous pages, the claim was not explicitly stated, but the information implied a change in population over the years.
Evidence: The chart was utilized in this case form outside sources cited both at the bottom of the page as well as within the conclusion. As I previously mentioned, I would include more about income, education, and business statistics, because I feel that this would relate better to your claim in the introduction.
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Demographics of Krog Street Area
Once again, this is great evidence for your claim in the introduction.
Claim: Your introduction claim of gentrification could really be used here to drive the point home. (The claim was implied but not stated.)
Evidence: For evidence, secondary sources were utilized to describe the changing demographic. I would suggest to discuss in further detail the categories of income and business. These categories would provide a great deal of validity to your claim, should you include it.
BE SURE TO CITE YOUR CHART
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
(“725 Ponce” 2015)
I would personally consider putting the full citation at the bottom of each page. I feel that this would be more helpful and easier to access for your audience.
-
History of Ponce City Market
Wow! This is great information. You did a great job constructing the history of Ponce in a way that is easy for the reader to understand and enjoy. I also really like the vintage pictures!
Claim: Like the previous section on the history of Krog Street Market, there is no claim physically present in this paragraph. To me, it is easily understood to be in support of the introduction claim, but you may want to explicitly state how Ponce City Market relates to the claim for the purposes of the project.
Evidence: The evidence is taken from secondary source cited both at the bottom of the page and the conclusion. Once again, I would consider addressing the questions I suggested in the previous annotation as a way to address the claim implied.
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
History of Krog Street Market
I really enjoyed reading the history of this particular location. Surprisingly, I actually remember when this property was purchased and the media attention that followed on the local news channels.
Claim: There wasn't a claim explicitly stated in this paragraph but the claim in the introduction was obviously implied.
Evidence: This paragraph mainly consisted of second hand information sited in the conclusion. I personally believe that learning about the history of the market and the surrounding area adds to your authenticity and credibility.
It maybe helpful to restate the claim and provide further evidence such as what the area looked like before versus years after the purchase. Was it run down? Also relating back to the introduction what were the crime rates before and after at this location? What was the local economy like before the recent innovations? Overall, I feel that this is a strong paragraph that could be improved through relating more to the claims in the introduction.
-
-
iamlimitless.gsucreate.org iamlimitless.gsucreate.org
-
Introduction
I really enjoyed your introduction. I feel that discussing what a built environment is and how it came about improves the readability for your audience. Further, your introduction highlights your claims about gentrification in Atlanta! I'm excited to continue reading!
Claim: When lower income neighborhoods experience gentrification, the population demographic undergoes drastic change (i.e. higher incomes). In turn, this may lead to positive benefits for the economy as well as the safety and happiness of residents.
Evidence: There is no evidence in this specific paragraph, but I assume that the other pages on Ponce City Market and Krog Street Market will act as examples. Maybe give a brief introduction of both in order to provide evidence in this paragraph.
-
-
spleenerwiley.gsucreate.org spleenerwiley.gsucreate.org
-
Thesis: The rhetoric of the built environment of Atlanta shows that racial discrimination, white flight, car dominated transportation network, and segregation by race and class have caused Atlanta to have the highest income inequality ratio in the country, and the same factors that led to severe income inequality in Atlanta are perpetuating the problem today.
The author makes the claim that Atlanta has a preeminent car based transporation system and that race, class, and racial discrimination has been a determinant in income inequality in Atlanta, stating it to be the highest in the country. Providing data or a referenced source would help to give clarity to Atlanta's ratio compared to all other countries in the U.S.
The authors' thesis is very thought-provoking and descriptive, but can be broken up into two-three sentences.
-
Built Environment Analysis (DRAFT)
Overall, the author does a great job in providing claims and arguments, but the draft in totality is not complete and needs the inclusion of citations from sources and data, graphs, etc. to provide evidence for the claims that are being made and also for general reference when mentioning ratios and numbers. Also, the author did not incorporate multiple modes of presentation. To meet the requirement, the authour should provide photographs, videos, charts, etc. which will also help to give stated claims and arguments a varied perspective.
-
As a result of a long history of white flight and racial discrimination, Atlanta’s transportation network is predominately designed for travel by car. Consistent public transportation is present downtown and in the immediately surrounding areas. Evidence:
This section makes the claim that Atlanta has an intentional, leading car based transportation system and that it has a connection to past racial segregation.
-
Cost of living map and MARTA map side by side
Listed here seems to be a description of two photographs that are to be compared, however there are no photograpghs posted to compare anything. The author should provide those images for reference and comparison as well as a description on what is being compared and why.
-
The neighborhood one grows up in has been shown to impact their chances for upward economic mobility, therefore gentrification and neighborhoods segregated by class perpetuate income inequality.
The author makes the claim that the neighborhood of ones upbringing has a direct correlation to their future rank in the economic sphere of the world. Examples on how this statement may be accurate should be provided in this area.
-
The Fading American Dream: Trends in Absolute Income Mobility Since 1940” by Raj Chetty and Nathaniel Hendren “All Cities Are Not Created Unequal” by Alan Berube
The author provides a list of evidence based sources, but does not provide key details in how the sources benefit the claim. Author needs to provide citations and references from sources to support the claim that is being made
-
history of white flight
Listed here is a term that was unfamilar and provided research shows that there is a book entitled by this name White Flight by Kevin M. Kruse: http://press.princeton.edu/titles/8043.html
Author could provide a brief description of the term because it will enhance this specific claim amd also include the link to the book's overview and how it possibly relates to the claim.
-
While there is some public transportation for people living further away from the center of the city, the current accommodations are insufficient for people without cars.
The author makes the claim that exisitng public transportation does not benefit persons without a vehicle, but acknowledges its benefits to persons that are located in somewhat distant locations from Atlanta.
-
The trend of Atlanta’s middle and upper classes moving out to the suburbs is shifting, and these groups are beginning to move back into the city. Therefore, neighborhoods are being gentrified to meet the growing demand.
The author makes the claim that Atlanta neighborhoods are becoming more gentrified due to the incoming masses of middle class persons/families.
-
Low-income residents that have settled close to the city, along public transportation routes, are having to move further out because the gentrification of neighborhoods raises the cost of housing.
Author's claim is gentrification and higher property taxes in Atlanta neighborhoods have caused perons/families of lower incomes to leave cities allowing middle-class persons/families to move in. Specific examples of this scenario in scecific Atlanta areas should be incorparted into the author's claim and provided for reference.
-
The quality and quantity of public transportation decreases as you move further away from the center of the city. Consequently, those living in poverty who have relocated further away from the city are in a worse situation because they do not have the same amenities available to them.
This claim explains the disadvantages that lower-income person families face when leaving areas that provide abundant access to public transportation and move to areas with little to less public transportation prone to be of lower quality. An example of this case should be provided in this area to compare public tranportaion between a city area and an area in which it lacks.
-
“Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “Health Impact Assessment of the Atlanta Beltline” by Catherine Ross
This is a list of the sources that the author included to support the claim, but there are no specific details provided to explain how or why it benefits the argument being made.
The author should provide direct, specific evidence from the listed sources to support the claim.
-
“Using Vehicle Value as a Proxy for Income: A Case Study on Atlanta’s I-85 HOT Lane” by Sara Khoeini and Randall Guensler “Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “The Human Scale” by Andreas Mol Dalsgaard
This is a list of the evidence based sources for the authors claim. The author should add links to the sources and provide specific evidence and citations from the sources that pertains to the claim that is being made.
-
Photo of bench in Little Five Points
The author makes note of a photograph that is not posted so there needs to be a photograpgh in this area as well as a description on its relation to the claim.
-
“Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “How Cities Use Design to Drive Homeless People Away” by Robert Rosenberger
Here are the sources provided to accomodate the author's claim, but the evidence to back up the claim in not provided.
-
“CHANGING BOHEMIA Little Five Points, a Haven of Counterculture, Faces Gentrification and Dissension” by Melissa Turner “Health Impact Assessment of the Atlanta Beltline” by Catherine Ross
Sources for this claim focus on the Atlanta Beltline and the Little Five Points Area, however, the author does not include any major points or refernces to enhance the claim on gentrification in Atlanta areas directly. An incorporation of in-text citations is needed.
-
“Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “The Human Scale” by Andreas Mol Dalsgaard
Provided here are a list of evidence based sources to reiterate the authors claim.
Again, the author should add links to the sources and provide specific evidence and citations from the sources that pertains to the claim that is being made.
-
-
hemisphere-gauche.blogs.liberation.fr hemisphere-gauche.blogs.liberation.fr
-
Présidentielles et bulles de filtres: le référencement politique de Facebook passé au crible
Hypothes.is est un outil permettant d'annoter à n'importe qui d'annoter des pages web et de commenter au fil du texte.
-
- Mar 2017
-
www.chronicle.com www.chronicle.com
-
ittle direct indication that the Trump administration or congressional leaders known for attacking scientific research on climate change and human health are looking to exploit reproducibility campaigns as a political opportunity.
Easy to connect these dots though.
-
-
-
Eve Marder, a neurobiologist at Brandeis University and a deputy editor at eLife, says that around one third of reviewers under her purview sign their reviews.
Perhaps these could routinely become page notes?
-
If Kriegeskorte is invited by a journal to write a review, first he decides whether he’s interested enough to review it. If so, he checks whether there’s a preprint available—basically a final draft of the manuscript posted publicly online on one of several preprint servers like arxiv and biorxiv. This is crucial. Writing about a manuscript that he’s received in confidence from a journal editor would break confidentiality—talking about a paper before the authors are ready. If there’s a preprint, great. He reviews the paper, posts to his blog, and also sends the review to the journal editor.
Interesting workflow and within his rights.
-
The tweet linked to the blog of a neuroscientist named Niko Kriegeskorte, a cognitive neuroscientist at the Medical Research Council in the UK who, since December 2015, has performed all of his peer review openly.
Interesting...
-
- Feb 2017
-
dev.upenndigitalscholarship.org dev.upenndigitalscholarship.org
-
object you are photographing b
"Maybe give some examples..."
-
I hope to show with this tutorial, however, that the Dino-Lite Premier AM-311S has the potential to create useful models at an affordable price.
"Good introduction of scope..."
-
capture images for processing
"Maybe be more specific..." (generic commentary)
-
-
www.nature.com www.nature.com
-
The struggle between Whewell and Lubbock represented two distinct visions of what a referee might be. Whewell was the authoritative generalist, glancing down on the landscape of knowledge. He was unconcerned with — and probably not in a position to critique — the details. Such referees were, according to the Royal Society's president, “Elevated by their character and reputation above the influence of personal feelings of rivalry or petty jealousy”4. Lubbock was a younger specialist, Airy's equal. This allowed him to take a fine-tooth comb to Airy's arguments; it also put him in the position of reviewing a direct competitor.
Two versions of what a review is.
Tags
Annotators
URL
-
-
www.wired.com www.wired.com
-
by introducing editorial management software systems.
Manuscript submission systems.
-
-
journals.plos.org journals.plos.org
-
Pivotal roles are played by three enzymes, (phospho-fructokinase (PFK), pyruvate kinase (PK) and phosphofructoki-nase/fructose-2,6-bisphosphatase (PFKFB)) through their inhibi-tion or activation by three reaction intermediates (fructose-1,6-bisphosphate (F16BP), fructose-2,6-bisphosphate (F26BP), andphosphoenolpyruvate (PEP)) in glycolysis. These enzymes havemultiple isoforms (PFKL/M/P, PKM1/M2/L/R and PFKFB1-4)which are subjected to contrasting allosteric regulations [9–11].Each isoform, therefore, affects the glycolytic activity in a distinctmanner.All three isoforms of PFK are activated by F6P and F26BP [12],but only PFKM and PFKL are activated by F16BP [13–15].PFKFB is a bifunctional enzyme whose kinase and bisphosphatasedomains catalyze the formation and hydrolysis reaction of F26BP,respectively [9,16]. Isozymes of PFKFB differ in their kinase andphosphatase activities as well as in their sensitivity to feedbackinhibition by phosphoenolpyruvate (PEP) [17–19]. Thus, eachisozyme of PFKFB has a profoundly distinct capacity inmodulating PFK activity. Pyruvate kinase (PK) in mammaliansystems is encoded by two genes that can produce two isoformseach. Except for the PKM1 isoform, the other three isoformsof PK, PKM2, PKL and PKR, are activated by F16BP to varyingextents [11]. The M2 isoform of PK, in addition to activation byF16BP, is also under the control of a host of allosteric modulatorsincluding serine, succinylaminoimidazolecarboxamide ribose-5-phosphate (SAICAR) and phenylalanine among others [
Need a figure presenting the regulation network.
-
- Jan 2017
-
www.insidehighered.com www.insidehighered.com
-
mcpress.media-commons.org mcpress.media-commons.org
- Oct 2016
-
www.timeshighereducation.com www.timeshighereducation.com
-
Examples of bad peer review and why it is damaging to researchers
-
- Aug 2016
-
-
Clarity on what qualifies as a respected preprint
Why should we respect a preprint? I'm not sure that we should respect anonymously peer reviewed journal articles as much as we do. It's important to remain critical, and I worry that trying to put a veneer of 'respectability' over preprints is not as helpful as expecting people to read them to judge content.
-
- Jul 2016
-
books.google.ca books.google.ca
-
Page 62
Borgman discussing the purpose of peer review
Pre-publication mechanisms serve as expert filters on what becomes part of the scholarly record, when doing out there researchers reading list.
-
Page 60
The use of a publication form as a proxy measure for the quality of research productivity has distorted the peer-review system so severely that some consider it broken. Peer reviewing is an expensive process, requiring considerable time and attention of editors comma editorial board members, and other reviewers. Top journals in the sciences and medicine they put fewer than half of the submitted papers through a full P review process, rejecting the remainder on an initial editorial review, and ultimately publish 6 to 10% of the total submissions. Particularly in The Sciences, researchers are under so much pressure to place papers and talk to your journals that they submit them to the same journals, whether or not the content is appropriate.
-
- Jun 2016
-
-
No Bias, No Merit: The Case against Blind Submission
Fish, Stanley. 1988. “Guest Column: No Bias, No Merit: The Case against Blind Submission.” PMLA 103 (5): 739–48. http://www.jstor.org/stable/462513.
An interesting essay in the context I'm reading it (alongside Foucault's What is an author in preparation for a discussion of scientific authorship.
Among the interesting things about it are the way it encapsulates a distinction between the humanities and sciences in method (though Fish doesn't see it and it comes back to bite him in the Sokol affair). What Frye thinks is important because he is an author-function in Foucault's terms, I.e. a discourse initiator to whom we return for new insight.
Fish cites Peters and Ceci 1982 on peer review, and sides with those who argue that ethos should count in review of science as well.
Also interesting for an illustration of how much the field changed, from new criticism in the 1970s (when the first draft was written) until "now" i.e. 1989 when political criticism is the norm.
-
Nevertheless, there were a few who questioned that definition of fairness and challenged the assumption that it was wrong for reviewers to take institutional affiliation and history into consider- ation. "We consider a result from a scientist who has never before been wrong much more seriously than a similar report from a scientist who has never before been right. . . . It is neither unnatural nor wrong that the work of scientists who have achieved eminence through a long record of important and suc- cessful research is accepted with fewer reservations than the work of less eminent scientists" (196). "A reviewer may be justified in assuming at the outset that [well-known] people know what they are do- ing" (211). "Those of us who publish establish some kind of track record. If our papers stand the test of time . . . it can be expected that we have acquired expertise in scientific methodology" (244). (This last respondent is a woman and a Nobel laureate.)
Fish reporting on the minority in response to Peters and Ceci who argued that track records should count in peer review of science
-
A similar point is made by some of the participants in a discussion of peer review published in the Behavioral and Brain Sciences: An International Journal of Current Research and Theory with Open Peer Commentary (5 [1982]: 187-255). The occasion was the report of research conducted by D. P. Peters and S. J. Ceci. Peters and Ceci had taken twelve articles published in twelve different journals, altered the titles, substituted for the names of the authors fictitious names identified as researchers at institu- tions no one had ever heard of (because they were, made up), and resubmitted the articles to the jour- nals that had originally accepted them. Three of the articles were recognized as resubmissions, and of the remaining nine eight were rejected. The response to these results ranged from horror ("It puts at risk the whole conceptual framework within which we are accustomed to make observations and con- struct theories" [245]) to "so what else is new."
Peters & Ceci 1982 comes up!
-
. Predict- ably, Schaefer's statement provoked a lively exchange in which the lines of battle were firmly, and, as I will argue, narrowly, drawn. On the one hand those who agreed with Schaefer feared that a policy of anonymous review would involve a surrender "to the spurious notions about objectivity and absolute value that . . . scientists and social scientists banter about"; on the other hand those whose primary concern was with the fairness of the procedure believed that "[jiustice should be blind" ("Correspon- dence" 4). Each side concedes the force of the opposing argument-the proponents of anonymous re- view admit that impersonality brings its dangers, and the defenders of the status quo acknowledge that it is important to prevent "extraneous considerations" from interfering with the identification of true merit (5)
Discussion of debate at MLA about plan to introduce blind submission to PMLA and comparison with sciences and social sciences.
-
-
Local file Local file
-
ouble-blind) peer review became an established component of thepost-war scientific bureaucracy (Chubin & Hackett, 1990,pp. 19 –24)
history of peer review
-
- Apr 2016
-
scholarlykitchen.sspnet.org scholarlykitchen.sspnet.org
-
Does peer review work? Is peer review broken? The vast majority of authors believe it improves their final work, and since it’s evolving from this solid base, it’s clearly not broken. But before we can have a useful discussion about its purpose and effectiveness, we need to agree on which approach to peer review we’re talking about, then whether our expectations of it are reasonable and accurate.
-
Here are some variables around peer-review we have to understand before we know what kind of peer review we’re actually talking about: Is it blinded? If it is blinded, is it single-blinded or double-blinded? Is there statistical or methodological review in addition to external peer-review? Are the peer reviewers truly experts in the field or a more general assemblage of individuals? What are the promises and goals of the peer review process? What type of disclosure of financial or other potential competing interests is made? Are reviewers aware of these? Is there a senior editor of some sort involved along with outside peer reviewers? Is the peer-review “inherited” from another body, such as a committee or a preceding journal process (e.g., in “cascading” title situations or when expert panels have been involved)? Are there two tiers of peer review within the same journal’s practices? Is the peer-review done at the article level or at the corpus level (as happens with some supplements)? Is plagiarism-detection software used as part of the process? Are figures checked for manipulation? Is the peer reviewer graded by a senior editor as part of an internal evaluation and improvement process?
-
-
-
White (1984, cited by Vaughan, 1991) reported on a study conducted at California State University in which two essays were tucked into a huge sample of essays and read a year apart by the same readers using a 6-point scale. The reading a year later produced scores that were identical to the first in only 20 per cent of the cases. The scores differed by one point or less in 58 per cent of cases and 2 points or less in 83 per cent of the cases. As White points out, a 1-point difference is generally considered unproblematic, but on a 6-point scale the difference between a 3 and a 4 is the difference between a pass and a fail. Obviously, then, changes in examiner severity/leniency over-time have implications for maintaining standards, and must be monitored. Research has been conducted into variations in examiner severity/leniency during the marking of a particular allocation of scripts, a marking period, and over more extended periods of time.
intrarater reliability is only 20%
-
According to Stemler, consistency estimates of interrater reliability assume that it is not necessary for judges to share a common meaning of the rating scale, so long as each judge is consistent in their classifications.
Wittgenstein's beetle in a box
-
(2004) notes that most research papers describe interrater reliability as though it is a single, universal concept. He argues this practice is imprecise and potentially misleading. The specific type of interrater reliability being discussed should be indicated. He categorises the most common statistical methods for reporting interrater reliability into one of three classes: consensus estimates; consistency estimates; and measurement estimates.
Stemler 2004
-
-
Local file Local filetitle3
-
in the latter both the wide differential in manuscript rejection rates and the high correlation between refereerecommendations and editorial decisions suggests that reviewers and editors agree more on acceptance than on rejection.
In "specific and focussed" fields, the agreement tends to be more on acceptance than rejection.
-
In the former there is also much more agreement on rejectionthan acceptance
In "general and diffuse" fields, there is more agreement on paper rejection than in "specific and focussed."
-
. Referees ofgrant proposals agree much more about what is unworthy of support than about what does have scientific value. In
Grant referees are better at agreeing on inadequate work than adequate
-
- Mar 2016
-
download.springer.com download.springer.com
-
Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H. D., & O’Mara, A. (2009). Gender effects in the peerreviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevelapproaches.Review of Educational Research, 79, 1290–1326
-
-
download.springer.com download.springer.com
-
One mid-career scientist told a story of how he and others in his lab counteractedan abuse of power by his mentor, a senior scientist, while he was in training. Hismentor received a manuscript to review that was authored by a ‘‘quasi-competitor.’’It presented results of experiments similar to those that were going on in thementor’s lab. The scientist continued, ‘‘That paper ... basically would have beat usto the punch. They would have published these results before us, and they wouldhave gotten credit, and not us. And my mentor, God bless him, sat on the paper.’’The mentor not only delayed writing the review but asked someone working in thelab to write it (a move of questionable ethicality in itself). That lab person and ourrespondent decided, in response, to stall their own work, so that their lab would nothave an unfair advantage over the group who submitted the paper for review. In theend, the original group got credit for the findings, while the respondent’s lab wasalso able to publish their slightly different findings. He ended his story with,‘‘Sometimes you’re in an awkward position, and you try to do the best thing you canunder the circumstances, within your own internal ethical clock or whatever. Andsometimes it’s ugly and it’s imperfect, but it’s the only thing you can do. If we hadgone to the mentor and voiced this objection, our careers would have been over. Ifwe had approached the journal—God forbid, forget it.’’ The speaker qualified thisstory by saying that it made him sound much more ethical than he actually is.
peer review deliberately delayed in order to slow competitor
-
The focus-group discussions showed, however, that scientists see peer review asaffording a unique, even protected opportunity for competitors to take advantage ofthem. In this sense, competition infects the peer review process, not only throughscientists’ competition with other applicants, but also through scientists’ distrust ofthe reviewers themselves, as competitors. The following exchange among mid-career discussants shows their sense of vulnerability
Evidence that peer-reviewers are competitors.
-
I really hate to admit this, but you do the same thing with your competitors asyou do with grant agencies. You sucker-punch them. You might have—when Isubmit a paper, I already have the next two or three papers’ worth of data. Imean, I know what I’m going to be publishing a year from now, mostly. But thepaper that comes out of my lab is Part A. Parts B and C are mostly on my desk.And I’ve put things in part A to basically entice my competitors into making anass out of themselves, or to second guess, or say, ‘‘Oh that must be wrongbecause of that, or something.’
Gaming referees.
-
- Feb 2016
-
onlinelibrary.wiley.com.ezproxy.alu.talonline.ca onlinelibrary.wiley.com.ezproxy.alu.talonline.ca
-
On the cost rejected papers add to the peer review system (p. 119): "The cost to the academic community of refereeing was estimated by Tenopir and King in 1997 to be $480/article (based on an average time 3–6 hours per article by each of 2–3 referees). At 2004 levels this is approximately $540 per submitted article. Clearly,the percentage ofpapers which are rejected makes a difference to the over- all cost to the journal; in a reasonable quality journal at least 50% of papers will be rejected, while some top journals (e.g. Nature) may reject as many as 90%. Most articles get published somewhere, and as they work their way through the system, being refereed for different journals, they accumulate additional cost; indeed, it couldbe said that a poor (or, at least, inappropriately submitted) article costs the system much more overall than does a good one."
-
-
www.readability.com www.readability.com
-
As I have mentioned in previous posts, several platforms have appeared recently that could take on this role of third-party reviewer. I could imagine at least: libreapp.org, peerevaluation.org, pubpeer.com, and publons.com. Pandelis Perakakis mentioned several others as well: http://thomas.arildsen.org/2013/08/01/open-review-of-scientific-literature/comment-page-1/#comment-9.
-
- Jan 2016
-
www.readability.com www.readability.com
-
Below I list a few advantages and drawbacks of anonymity where I assume that a drawback of anonymous review is an advantage of identified review and vice versa. Drawbacks Reviewers do not get credit for their work. They cannot, for example, reference particular reviews in their CVs as they can with publications. It is relatively “easy” for a reviewer to provide unnecessarily blunt or harsh critique. It is difficult to guess if the reviewer has any conflict of interest with the authors by being, for example, a competing researcher interested in stalling the paper’s publication. Advantages Reviewers do not have to fear “payback” for an unfavourable review that is perceived as unfair by the authors of the work. Some (perhaps especially “high-profile” senior faculty members) reviewers might find it difficult to find the time to provide as thorough a review as they would ideally like to, yet would still like to contribute and can perhaps provide valuable experienced insight. They can do so without putting their reputation on the line.
-
-
www.readability.com www.readability.com
-
With most journals, if I submit a paper that is rejected, that information is private and I can re-submit elsewhere. In open review, with a negative review one can publicly lose face as well as lose the possibility of re-submitting the paper. Won’t this be a significant disincentive to submit? This is precisely what we are trying to change. Currently, scientists can submit a paper numerous times, receive numerous negative reviews and ultimately publish their paper somewhere else after having “passed” peer review. If scientists prefer this system then science is in a dangerous place. By choosing this model, we as scientists are basically saying we prefer nice neat stories that no one will criticize. This is silly though because science, more often than not, is not neat and perfect. The Winnower believes that transparency in publishing is of the utmost importance. Going from a closed anonymous system to an open system will be hard for many scientists but I believe that it is the right thing to do if we care about the truth.
-
PLOS Labs is working on establishing structured reviews and we have talked with them about this.
-
It should be noted that papers will always be open for review so that a paper can accumulate reviews throughout its lifetime.
-
-
dlsanthology.commons.mla.org dlsanthology.commons.mla.org
-
‘review of the reviews’ (with reviewers weighing in on the issues raised by others),
Breaking down hierarchies...
-
- Dec 2015
-
opennessinitiative.org opennessinitiative.org
-
We believe that openness and transparency are core values of science. For a long time, technological obstacles existed preventing transparency from being the norm. With the advent of the internet, however, these obstacles have largely disappeared. The promise of open research can finally be realized, but this will require a cultural change in science. The power to create that change lies in the peer-review process.
We suggest that beginning January 1, 2017, reviewers make open practices a pre-condition for more comprehensive review. This is already in reviewers’ power; to drive the change, all that is needed is for reviewers to collectively agree that the time for change has come.
-
- Sep 2015
-
m.plantphysiol.org m.plantphysiol.org
-
It would avoid any possible awkwardness on either side, and if the discussion were constructive, then both author and commenter would profit from the outcome.
What's not to like?
Oh.
Tags
Annotators
URL
-
- Jul 2015
-
www.digital-science.com www.digital-science.com
-
Put simply, various stakeholders seem to have different perspectives on how research assessment works currently and how it should work in the future. In order to move forward, we must first identify and then address a number of misunderstandings.
This might be of interest, a project that we've put together for the Scholarly Communication Institute this year titled "The Qualities of Quality – Validating and justifying digital scholarship beyond traditional values frameworks": http://trianglesci.org/2015/05/15/the-qualities-of-quality/
-
- May 2015
-
nymag.com nymag.com
-
Broockman has ideas about how to reform things. He thinks so-called “post-publication peer review” — a system which, to oversimplify a bit, makes it easier to evaluate the strength of previously published findings — has promise.
-
-
researchity.net researchity.net
-
The key here is to not just democratize content sharing (the web has done that already) but to democratize and diversify reputation and trustworthiness. And to disrupt the monopoly academia has had on knowledge.
-
-
blogs.plos.org blogs.plos.org
-
Author and peer reviewer anonymity haven’t been shown to have an overall benefit, and they may cause harm. Part of the potential for harm is if journals act as though it’s a sufficiently effective mechanism to prevent bias.
-
Peer reviewers were more likely to substantiate the points they made (9, 14, 16, 17) when they knew they would be named. They were especially likely to provide extra substantiation if they were recommending an article be rejected, and they knew their report would be published if the article was accepted anyway (9, 15).
-
- Feb 2014
-
www.rpajournal.com www.rpajournal.com
-
Alternatively, Daphne Koller and Andrew Ng who are the founders of Coursera, a Stanford MOOC startup, have decided to use peer evaluation to assess writing. Koller and Ng (2012) specifically used the term “calibrated peer review” to refer to a method of peer review distinct from an application developed by UCLA with National Science Foundation funding called Calibrated Peer Review™ (CPR). For Koller and Ng, “calibrated peer review” is a specific form of peer review in which students are trained on a particular scoring rubric for an assignment using practice essays before they begin the peer review process.
-