847 Matching Annotations
  1. Last 7 days
    1. Do ‘broad’ approaches to improving effective governance, and ultimately serving the farfuture, tend to be more or less effective in expectation than ‘narrow’ approaches (such asworking on reducing the risk of bioengineered pandemics)?

      A very big question -- would be helpful to pose some possible building blocks to answering this question that gives people a hint at how to take a stab at it.

    2. How can evidence be disseminated mosteffectively?

      Disseminate: By whom, to whom, with what theory of change/path to impact?

    3. Under what conditions should a social planner preserve ‘option value’ bydelaying an important, irreversible decision to acquire more information, thereby delegatingdecision-making authority to future agents with potentially different values and preferences(cf. Bishop 1982; Dixit and Pindyck 1994)?

      To me this seems distinct from the rest of the bullet point

    4. hen et al 2023; Toma andBell 2023

      biblio entries missing

    5. Intergenerational governance and policy-making

      It's unclear whether we are talking about A. "intergenerational governance and international policymaking" or

      B. "1. International governance and 2. Policymaking in general".

      The latter bullet points and cited papers (e.g., Vivalt and Coville) do not seem to always relate to intergenerational governance

    6. Vivalt, Coville, KC 2023

      This seems relevant for The Unjournal's consideration/evaluation (but this may fall into the 'ask authors' permission' category). It is empirical and apparently rigorously quantitative and seems highly-relevant to policymaking and impact evaluation research, and influencing policy, all crucial to 'the impact agenda'. Hopefully also follows open, robust science standards (prereg, etc.).

    7. Gonzalez-Ricoy and Gosseries 2017

      biblio entry missing

    8. Can we design mechanisms to ensure that AI systems exhibit desirable behavioursuch as truth-telling or a lack of deception?

      Perhaps this should be elaborated and defined somewhat more formally. Reference a particular issue in mechanism design here that is particularly relevant to AI systems, perhaps.

    9. govern powerful non-state entities

      I'm not sure what is meant by 'powerful non-state entities' here. This seems under-defined.

    10. inequalities,

      political inequalities?

    11. Read and Toma, 2023

      biblio entry missing. Would be useful to know what this one is.

    12. Song et al 2012)

      biblio missing

    13. Healy and Malhorta 2009
    14. economic models predict the impact of advancedAI systems on political institutions and inequalities

      A reference would be very helpful here. It's hard for me to see what sort of economic models are relevant here.

    15. (Acemoglu, 2023

      biblio missing. This seem potentially relevant for an Unjournal evaluation, although we tend not to focus on 'broad think piece' work, which this might be

  2. Nov 2023
    1. Bersiroglu andErdil, 2023

      biblio entry missing

    2. Shulman, C., & Thornley, E. (2023). How Much Should Governments Pay to Prevent Catastrophes? Longtermism'sLimited Role. In Barratt, Greaves, Thorstad (eds.) Essays on Longtermism.

      interesting but probably not quantitative/formally specific enough for The Unjournal

    3. Alexandrie and Eden, forthcoming

      biblio missing

    4. E.g. Jordà et al. 2022

      not sure this is getting at 'long run' in the sense that longtermists care about

    5. ased on the historical record of such events, what is the tail distributionof harmful impacts (e.g., fatalities) from pandemics, asteroids, wars, and other potentialdisasters? (E.g. Marani 2021;

      not really economics but that's not so important

    6. Aschenbrenner2019

      biblio entry missing

    7. 2023

      This is mainly about the welfare tradeoff between economic growth and x-risk in a theoretical sense; I don't think it's about the 'impact of growth on GCRs' per se

    8. Klenow et al. 2023

      biblio entry missing

    9. To what extent are forecasting methods informative for assessing the probability of globalcatastrophic risks and other future events of special importance for social welfare? (Karger

      empirical and seems very relevant and strong; adding it to the Unjournal database

    10. Karger, E., Rosenberg, J., Jacobs, Z., Hickman, M., Hadshar, R., Gamin, K., ... & Tetlock, P. E. (2023). ForecastingExistential Risks Evidence from a Long-Run Forecasting Tournament. FRI Working Paper No. 1.
    11. Kalai & Kalai 2001)

      biblio entry missing

    12. Andreoni 2018

      this is a practical applied policy paper that seems informative for donors considering their own charity decisionmaking

    13. 1.1 Strategic issues in altruistic decision

      they largely mention theory papers (micro theory, optimization, axiomatic/normative), not empirical work here

    14. (cf.Andreoni & Payne 2003)

      The Andreoni and Payne paper is about the government crowdout of private philanthropy (there are a bunch of papers about this), not about the reverse nor about crowding out among donors.

    15. Whatdetermines the optimal spending schedule for altruistic decision-makers?

      Practically speaking, this seems largely about the impact of interventions (funded by charity) over time; however it does connect with donors' to the extent it involves personal finance and issues like value drift.

      OK but the Trammell paper is addressing something different -- coordination in a public goods provision model.

    16. n altruistic decision-maker that funds a charitable intervention may crowd out fundingfrom other actors (e.g., governments or philanthropists)

      I might add a related issue -- decisions to give to one charity may crowd out other donations; the extent to which this is the case ('donations are substitutes') informs strategies for convincing people to give 'more effectively' vs 'give to effective causes.' See my notes: https://daaronr.github.io/ea_giving_barriers/chapters/substitution.html

    17. Research agenda draft for GPI Economics

      Does anyone know if this is the most updated statement of GPI’s economics agenda?

      “Economics ‘draft agenda’” Anyone know when it was updated?

  3. Oct 2023
    1. single chat requests: chatgpt_single()

      chatgpt_single(prompt_role = 'user', prompt_content = 'say something relevant' , temperature = 0.8 , n = 2 , max_tokens = 15)

      n = 2 -- 2 responses max_tokens = 15 ... maybe it means 'try 15 times' to get the best 2 from

      prompt_role = 'user' -- just ask a question for the gpt ('assistant') to respond. I'm not sure why it would make sense for single chat requests to choose the role 'system' or 'assistant' as this wouldn't persist (?)

    1. ecause military shocks generate plausibly exogenous variationin economic production, the findings improve upon the correlational evidence in Section 3.Second, the implied biodiversity-GDP elasticities from the quasi-experiment are largerthan their OLS counterparts. By leveraging shocks in a 2SLS setting, the quasi-experiment-based elasticity estimates alleviate classic measurement error and endogeneity problems.However, the fact that the two methods produce elasticities of similar order of magnitudeadds confidence to the overall credibility of the estimates.

      are they ignoring the heterogeneity and "LATE" issue?

    2. states’relative differences in response to aggregate military buildups (which are themselves largelydriven by geopolitical factors) – are unlikely to be correlated with unobservable determinantsof local biodiversity. That is, we assume the United States will not increase national mili-tary spending because states that receive larger military procurement contracts have worsebiodiversity

      but couldn't both of these be driven by a third factor ... the state is less environmentally friendly. or maybe I am missing something

    3. First, we producecausal estimates of the elasticities between biodiversity outcomes and air pollution. We use aresearch design that isolates variation in local pollution driven by transported pollution fromdistant, upwind cities (e.g., Deryugina et al., 2019; Anderson, 2020). We show that “upwindpollution” coming from areas over 300 km away generates substantial variation in local airquality, and these imported pollution shocks cause reductions in local biodiversity outcomes.Second, we estimate the impact of the military spending shocks on air pollution, and multiplythese estimates by the biodiversity-pollution elasticities we obtain from step one. Together,these exercises give us the expected impact of the military shocks on biodiversity throughair pollution. We find that pollution accounts for 20-60 percent of the reduced form effect ofmilitary shocks, suggesting air pollution is a first-order pathway underlying the production-biodiversity link

      have they successfully shown this 'mediation' channel ... with 2 separate sources of exogenous variation? (That's always very challenging to identify)

    4. Second, there is substantial distributional heterogeneity, where the negativeassociation at the lowest decile of biodiversity is almost twice as large as the average.

      But could this reflect something mechanical like a nonlinearity?

  4. Aug 2023
  5. Jul 2023
    1. We are now planning a further update in response to additional comments (e.g., from James Snowden and GiveWell). We expect this will include updating our analysis with recently completed studies  and refining some technical aspects of the analysis, including:Our systematic review, and the weight we place on different sources of evidenceEstimated spillover benefits for household membersCost estimatesTechnical details, such as:How long do the effects of psychotherapy last?How important is the expertise of the deliverer or number of sessions?Are the effects of psychotherapy affected by publication bias?

      This seems extremely high-value and potentially ideal for the Unjournal's non-academic stream. Ryan 'had this in mind too'

    2. but longtermists often claim priorities such as AI alignment and preventing pandemics are important, even if we solely consider present wellbeing, so we shouldn’t dismiss the possibility.

      I don't see how this argues against the 'suspicious convergence' claim... OK, I see Jack Malde's comment now, which basically gets at my doubts here.

    3. we have a full list of research ideas that we hope to explore

      this is the list you linked above under 'organizations'. fwiw it's an interesting list but it's very sparsely populated (most columns have a name only). Some fleshing out and ranking/prioritizing could be helpful here.

    4. If such views are true, that would count against longtermism

      I don't see this as a promising research agenda. My sense of it is that it is pretty intractable. (I'm not saying if it is true/false/wrong/right, just that I am not sure if there will be a lot of practical value in pursuing it? OK I see some approaches that might be helpful, if one has a tractible way to model welfare considerations with PAV it might win some people over.)

    5. credible cause for longtermists.

      'cause' or indirect instrumental goal?

    6. We’ve published two working papers on moral uncertainty: The property rights approach to moral uncertainty and Wheeling and dealing: An internal bargaining approach to moral uncertainty, which both explore a novel, bargaining-based approach to acting when you’re uncertain what’s morally right. (This is very roughly akin to the ‘moral parliament’ approach.) We’re currently working with two external  co-authors  on a new paper that combines these ideas, which we plan to publish in an academic journal.

      Potential relevant to #unjournalresearchprioritization, depending on the approach

    7. 5.1 Using WELLBYs to compare the value of extending lives against improving lives

      Somewhat relevant to #unjournalresearchprioritization

    8. Although unlikely, we may also do some work relating to animal welfare; a challenge is that we prefer to rely on self-reports, which animals can’t give.

      this could be relevant for Unjournal. How unlikely? If it's so unlikely, why mention it?

    9. Assess the social desirability bias and other self-reporting biases in SWB data (for example: Do people give answers surveyors want? Is it a problem? If so, can anything be done?)Explore whether the measure of SWB matters (for example, if the key outcome is happiness rather than life satisfaction, do we get different priorities?)


    10. Our working paper A Happy Possibility about Happiness (and other) Scales, a working paper attempts to provide the first overview of both the theory and evidence of the comparability of subjective wellbeing scales (e.g., is your 7/10 the same as my 7/10?). We plan to revise this for publication in an academic journal.
    11. Our article To WELLBY or not to WELLBY? sets out the WELLBY method, its strengths, weaknesses, and areas for future work. To expand on this, we are:

      This seems very relevant for Unjournal

    12. 3. The nature of wellbeing

      Probably not relevant for the Unjournal at this point, but there may be some overlap

    13. existing work (where a public document is available)

      academics (at least in my field) would distinguish a fourth stage 'having been accepted in a journal after peer review'. Not sure how important that distinction is for you.

      Note that The Unjournal is trying to make that last stage less burdensome and more informative by commissioning public evaluation and rating of work (rather than relying on tedious and imprecise the 'which journal was it published in' measure)

    14. Some work is both existing and current (where we have extant research we are updating).

      that's the best, I like to see all research as 'permanent alpha' mode

    15. which means we have a number of ongoing projects.

      'which means' --- the implication is not clear here

    16. The notable ones are:

      for me the more concrete measurement issues are at least as important ... you include these, but I don't see it in this paragraph.

    17. We also have a long list of organisations we would like to explore, including the Shamiri Institute, Action for Happiness, and Koko.
      1. The airtable view is linking interventions and cause areas, not organizations
      2. Why and how did you choose and prioritize these? It's a huge space to explore?
    18. e expect others will provide different types of mental health interventions, such as social-emotional learning. We expect to examine Friendship Bench, Sangath, and CorStone unless we find something more promising.

      Does that mean you will need to assess (and consider research evidence) on these other non-psychotherapy interventions? If so, that deserves its own section perhaps?

    19. Based on our cause area report on mental health and our cost-effectiveness analysis of psychotherapy, we think mental health is a promising area in which to find cost-effective interventions to improve wellbeing.

      this paragraphs seems unneeded and repetitive. Or am I missing something here?

    20. so we will also update our assessment of StrongMinds after we update our psychotherapy evaluation.

      Maybe restate this to clarify that you are not reevaluating SM as an organization again, but will update the evaluation of their impact in light of your updated evaluation of the intervention?

    21. 2.1 Updated evaluation of psychotherapy

      this part still seems like an intervention not an organization

    22. Are the effects of psychotherapy affected by publication bias?

      Pedantic: I'd say the 'estimated effects' here ... obviously the effects themselves are not affected by this bias

    23. e’ve found that psychotherapy for depression is several times more cost-effective than cash transfers for improving happiness, deworming has an unclear long-term effect,

      in the statement in the 1-pager, you stated

      we’ve found that psychotherapy is several times more cost-effective than cash transfers or deworming for improving happiness.

      That's not entirely consistent with this sentence

    24. From this work, we’ve found that psychotherapy is several times more cost-effective than cash transfers or deworming for improving happiness. We concluded that comparing psychotherapy to antimalarial bednets, a life-saving intervention, depends heavily on various philosophical assumptions: treating depression ranges from about as good as to several times better than antimalarial bednets, depending on the assumptions.

      these sentences repeat the sentences above in the one-pager

    25. evaluated the cost-effectiveness of organisations that provide  psychotherap

      The psychotherapy report doesn't seem to be about a particular organization. I'm a bit confused about the structure here. How does this differ from a 'cause area exploration' at this point?

    26. e, including psychedelics, opioids, poverty, loneliness, sleep, and air pollution.

      These again combine problems with potential remedies and interventions. And 'psychedelics' -- is that aimed at curing 'problems' or boosting the upper end joys of life?

    27. Longlist of future cause areas to explore

      this needs a header (it now is under 1.4). Not sure this long list is helpful here though? What's actionable about this? Is it linked to an appeal for more funding?

    28. 1.2 Child development effects (e.g., abuse, trauma, nutrition)

      I suspect some strong Unjournal/academic research links here. Also to the house improvement ones and possibly the fistula ones too.

    29. that may have large impacts on wellbeing as well. So far we have completed shallow reviews on pain, lead exposure, and immigration.

      For unjournal research prioritization, I guess I will have to dig into those reviews to identify the most pivotal research to have evaluated?

      These link articles but don't contain a 'list of works cited' at bottom. Could you provide that ... even better an 'annotated/categorized/prioritized list' explaining which ones you rely on most heavily, and which you have the most uncertainty over?

    30. Research agenda

      In the previous agenda you tried

      to articulate, within each research area, where additional research seems more (or less) useful, and therefore what our research agenda is for the next one to two years.

      This seemed particularly relevant to helping the Unjournal help you. Not sure this new agenda does this as much.

    31. A working paper exploring a bargaining-based approach to moral uncertainty

      How are you defining and considering a 'working paper' here?

      "Units of value" .. maybe add a few more words to clarify this?

      I assume this will be a theoretical paper (i.e., no surveys or data?)

    32. A revised paper on the theory and current evidence on scale cardinality (e.g., is your 7/10 the same as my 7/10?)

      I see a lot of benefit in engaging with academics on this paper, and getting and responding to feedback, possibly within The Unjournal's framework

    33. An academic paper setting out our method for measuring impact using wellbeing

      Not sure what is meant by 'an academic paper'. Typically it would be hard to publish a paper in an academic journal that simply 'describes' (or even justifies) the approach that a particular organization takes.

      You might have to frame it more as answering or providing evidence on a question of general interest, and/or formally arguing for the 'most appropriate approach' under certain defensible criteria.

    34. We will conduct new research on how to measure and interpret subjective wellbeing measures:

      A 2021 priority was "Examining how best to convert between different SWB, as well as other, measures (1.2.1)" This seems to have strong academic links relevant to the Unjournal. Is it still a priority?

    35. 2. Organisation evaluations

      I expect Unjournal-evaluated research to provide inputs relevant to these evaluations, but not to directly evaluate particular organisations. However, we might be able to cover some of this within our 'less academic stream'.

    36. Non-mood-related mental health issues (e.g., psychotic and trauma-related disorders)Child development effects (e.g., abuse, trauma, nutrition)Fistula repair surgeryBasic housing improvements (e.g., concrete floors)

      I suspect there are a range of academic papers (in development economics, health economics, psychology, policy, and the social-sciency side of biomedicine) that will inform this, that Unjournal might evaluate.

      This can include work that constitutes - impact evaluation of specific interventions, including RCTs non-experimental causal inference - work exploring the impact of specific paths to impact through these interventions (e.g., the career costs of childhood trauma) - work exploring costs and impacts on the market (e.g., impact of housing improvements on the local economy, price elasticities, etc.)

    37. 1. Cause area explorations

      In the 2021 agenda you stated "Our main current focus, and where the majority of our eΛort will go, is Area 2.3: using subjective well-being scores to compare the cost-effectiveness of highly-regarded health and development interventions used in low-income countries."

      Is this still your priority? Is this in line with the 'Cause area explorations" category here?

    38. Conduct further theoretical work:

      The boundary between theoretical and applied is not always clear here. Some research, maybe the methodological and measurement research in particular, has both theoretical aspects and very applicable and even empirical aspects. Calling this 'theoretical' might confuse people who would conflate theoretical with philosophical. E.g., research into which survey and other reporting instruments are more reliable, better reflect the actual measures of interest ... this seems very applied to me, and probably relevant to The Unjournal's scope as well.

    39. asurement of wellbeing. This has included evaluating philosophical views of wellbeing and life satisfaction, pioneering methods to conduct cost-effectiveness analyses using wellbeing, and conducting novel research on wellbeing measurement. We

      I think you may have moved a lot of the content outlined in the previous Research Agenda into those linked reports?

    40. Where relevant, we hone in and compare the top organisations implementing those interventions.

      Is this necessary in the wheelhouse of HLI? If your focus is on assessing impact from the wellbeing perspective, does that interact with things like 'organizational capabilities' at all?

      Maybe better to outsource the latter?

    41. picking new cause areas to investigate, then narrowing down to the specific organisations – which will enable us to look broadly and deeply at the same time.

      When will you do each? Do you anticipate returning to broad cause areas you've previously decided not to pursue?

      How much will you defer to other orgs and researchers in the broader prioritization?

    42. large, solvable, and unduly neglected.

      why not just namecheck the ITN framework here?

    43. broad analyses of different causes.

      how do you divide up the 'cause space' and define each 'cause'? Give some examples here? E.g., is "animal welfare' a cause area ... or 'farmed animal welfare' or 'chicken welfare' or 'promoting regulation of chicken farms' (the latter is more of an intervention IMHO)

    44. An ultimate goa

      one of several ultimate goals? To be pedantic, the 'improve global wellbeing' would seem to be the ultimate goal .... the 'identify the opportunities' is instrumental to that

    45. We will explore whether we should improve the wellbeing of people alive now or in future generations:

      Why not both? Maybe rephrase this?

      Also, will you consider the empirical tradeoffs here, or deeper philosophical issues, or?

    46. An academic journal book review of Will MacAskill’s What We Owe The Future

      what academic journal are you thinking?

    47. An experimental survey to test assumptions about subjective wellbeing measures, including comparability, linearity, and the neutral point

      I talked to someone recently who had done some survey work in this area -- maybe remind me to get back to you on it.

    48. An academic paper on life satisfaction theories of wellbeing

      This seems underexplained ... what and why? (add a bit or link)

    49. the cost-effectiveness of several organisations (partially informed by our cause exploration work)

      mention how you will do it better/different or add value to what other orgs are doing?

    50. Applied research to maximise global wellbeing

      Doing this yourselves? Synthesizing work? Sponsoring work?

    51. Non-mental health organisations (organisations TBD)

      Non mental-health orgs within the global health or global health and development space, or much more widely ranging, across vastly different causes etc?

    52. To find new promising solutions to the biggest problems,

      this phrase seems a bit vague? Also, are these 'cause areas' or possible interventions, or a mix?

    53. From this work, we’ve found that psychotherapy is several times more cost-effective than cash transfers or deworming for improving happiness.

      I think you would be more convincing if you linked or footnoted some of the critiques of this, as well as your responses. This reads well for a general audience but maybe not for a research and EA audience?

    54. engineering a paradigm shift towards a wellbeing approach among decision-makers

      This seems to move a bit towards advocacy, perhaps in contrast to the more neutral approach you mention elsewhere or previously. In other writings it's more like 'get people to consider a well-being based approach, and whether and when it makes sense to use it'.

    55. Ultimately, we measure impact in WELLBYs (wellbeing-adjusted life years), a method born in academia

      A citation/link here would be great

    56. For the first time in human history

      Very small comment ... 'for the first time in human history' tends to come across as overblown whenever people use it. At least it has that connotation

    57. The idea that the quality of a society should be judged by the happiness of its people is an old idea, stretching back at least to the Enlightenment, if not Aristotle.

      Your previous "Research Agenda and Context" sent a lot of time defining and arguing for this. I don't see that here (for better or for worse).

    58. [This post contains the Happier Lives Institute's research agenda for next 18 month. After a foreword, we give a brief summary of our plans, then go into more depth]

      Typo -- '18 month'

  6. May 2023
  7. evalresearch.weebly.com evalresearch.weebly.com
    1. Making referee payments or charity donations: Three-quarters of our respondents said that referees would do a better job if they were better rewarded for their effort. Among them, about 75% indicated that referees should be paid for timely completion of the report. This payment could take many forms e.g., a donation to a charity or research fund.


    1. Phase 2: EAMTT – Bringing together and engaging Academics, EA orgs, and marketers

      Jack - single biggest barrier (stated) is "you should give where you live" ...

      Move people who are somewhat aligned?

      Effektiv Spenden ... some people are drawn

      Which segment to appeal to?

    1. The output shows we need to set a prior on sigma, the Intercept, and on the male coefficient.

      I'm trying to interpret the output. So it's suggesting a 'flat' (uniform?) prior on b male (or also on another b? -- but what is b?) and a students' T distribution for the intercept and for sigma, maybe with the latter being truncated?

      Why does it make these particular choices of distributions?

      And it doesn't seem to be saying anything about the distribution of the outcome around it's mean, correct?

  8. Apr 2023
    1. Post-event (complete) response rates, ratings (1-6), by school

      something clearly went wrong here, but I think it's fixed now. Will push the results again

  9. Mar 2023
    1. we can change the question to “What is the probability that this intervention is better than 1x (i.e. cash transfers)?” We can set a critical value for that threshold (e.g. we accept programs that we are 90% sure are better than cash transfers). As above, that value comes straight out of the distribution from our PSA: it’s simply the proportion of outcome results from our PSA-generated distribution which are ≥1.

      But this would require some assumptions over the underlying distribution of effectiveness

    2. as a preview of what might happen with a full accounting of uncertainty. Code, data, and modified workbook are available.

      This seems to be done in the file sensitivity analysis.R, pulling parameters from the linked Gsheet

    3. Looked for a handful of key parameters pertaining to the overall effectiveness of the program and prevalence of the issue being addressed Traced those parameter values back to the original data, and located the statistical sampling uncertainty provided

      Focused on the statistical uncertainty only

    4. we see that there is a substantial bias in the programs that we select. Programs that we select have a large positive bias on average.

      the standard 'winners' curse'

    5. n the first tab (“True vs false rejections”), we see the distribution of programs we accept and reject compared with whether or not they were truly better or worse than cash transfers. As expected, we generate many false rejections, due largely to the decision threshold of 3 being a “hedge” of sorts. More importantly, we observe false positives (i.e. programs that got “lucky”).
      1. "Selected" contains only where estimated CE > 3
      2. "Rejected" are all other programs (estimated CE<3)
    1. nce this set of matched communities has been generated generalised linear mixed models (e.g., multilevel models) will be used to assess changes in outcomes before and after the intervention, at different time periods, while controlling for other variables including whether the area is a control or intervention area.

      somewhat non-specific

    1. All we need to do is change the units of the calculation and see if the result changes because of it. If it does, the calculation violates scale invariance, and for some reason the result depends on the units of measure that are used to calculate it.

      that is awesome!

    1. Major Future Additions​

      What about multivariate/correlated distributions? Or is there an easy compositional way to do this that I'm overlooking? Like maybe a 'shared random variable' that feeds into two distributions? But I'm not sure if that can be done in the current system, because ... can the 'draws from one distribution' be carried over as inputs into the 'draws from another distribution'?

    2. Static / sensitivity analysis Guesstimate has Sensitivity analysis that's pretty useful. This could be quite feasible to add, though it will likely require some thinking.


    3. Right now Squiggle mostly works with probability distributions only, but it should also work smoothly with probabilities.

      not sure what thi means

    1. Gallery

      do any of these allow correlations between the elements we are uncertain about? (I guess in principle, correlated variables those could be combined into a distribution of some function of these variables, but that seems like part of the work Squiggle is meant to do)

    1. Some distribution operations (like horizontal shift) return an unnormalized distriibution.

      explain what this means

    2. distriibution


    3. Second argument to SampleSet.fromDist must be a number.


    4. Recall the three formats of distributions. We can force any distribution into SampleSet format

      Don't say 'recall' because this only comes up later!

    5. For every point on the x-axis, operate the corresponding points in the y axis of the pdf.

      Explain better how this differs from adding the distributions.

      A comparison like uniform(3,4) - uniform(0,1)


      uniform(3,4) .- uniform(0,1)

      Could be helpful.

      Also note the false intuition 'the distribution of the difference between draws uniform distributions should be uniformly distributed' can be checked by thinking about and plotting

      uniform(0,1) - uniform(0,1)

      However, that distribution should be triangular, and the simulated distribution in your plot looks somewhat far from this. Why not make that an analytical computation?

    6. Pointwise operations are done with PointSetDist internals rather than SampleSetDist internals.

      I have no idea what this means

    7. TODO: this isn't in the new interpreter/parser yet.

      It seems to work in the playground though

    8. A projection over a stretched x-axis.

      for consistency with the above, you should characterize this mathematically

    1. Samples are converted into PDF shapes automatically using kernel density estimation and an approximated bandwidth. Eventually Squiggle will allow for more specificity.

      I thought Kernels can smooth things. Above it seems like a linear interpolation

    2. mixture(1,2,normal(5,2)), the first two arguments will get converted into point mass distributions with values at 1 and 2.

      and it gives 1/3 mass to each of 1, 2, and the distribution

    3. mixture(pointMass(1),pointMass(2),pointMass(5,2)).

      this throws an error in the Playground

    4. Array of Distributions Input

      Not sure what this is doing

    1. Most functions are namespaced under their respective types to keep functionality distinct. Certain popular functions are usable without their namespaces.

      not sure what point you are trying to make here. Note the first one crashes the playground

    2. For example,

      In the playground, the first entry a = List.upTo(0, 5000) |> SampleSet.fromList

      Throws "This page crashed. Minimum discrete weight must be an integer

      Try again"

    3. Squiggle dictionaries work similarly to Python dictionaries. API.

      OK these just store collections of things?

    1. Example

      I don't get what these are supposed to do. This snippet throws an error when I try it in the playground:

      Error merge is not defined Stack trace: <top> at line 19, column 12

    1. mixture​

      this needs more documentation perhaps?

    2. If both values are above zero, a lognormal distribution is used. If not, a normal distribution is used.

      This should be highlighted elsewhere!

    1. [5].

      Not sure these footnotes line up

    2. Say that $2B to $20B, or 10x to 100x the amount that Open Philanthropy has already spent, would have a 1 to 10% chance of succeeding at that goal [5].

      What is this benchmarked against? If I had said a 1-3% chance or a 10-50% chance, would that have seemed equally plausible?

    3. [0]. This number is $138.8 different than the $138.8M given in Open Philanthropy's website, which is probably not up to date with their grants database.

      What does this mean? the two 138.8's here suggest a typo

    4. For completeness, I do estimate the impacts of a standout intervention as well.

      This means for some 'great intervention' ... best in class or something

    5. cost = 2B to 20B

      Another huge wild guess? But should the cost really vary? Shouldn't this just be done for a particular level of cost?

      Also, I guess the prob. of success is likely to be related to the amount spent

    6. probabilityOfSuccess = 0.01 to 0.1 # 1% to 10%.

      Huge wild guess, and probably should be correlated to the acceleration and reduction in prison pop terms?

    7. counterfactualAccelerationInYears = 5 to 50

      huge wild guess

    1. We guess that implementation challenges would limit effectiveness and funding opportunities. As a result, we do not anticipate doing further research on this program in the near future.

      What is the model (a VOI model?) for what to focus GW attention on?

    1. we estimate that this net distribution will reduce the number of deaths each year within this population from 12 to about 11.4

      how does this number 'depend' on the 12.0 used above?

    2. Step Four: 12 of those people are expected to die every year of any cause In order to estimate how many lives these nets might save, we first need to know how many people in this population would have died without the protection of the nets. The mortality rates and population demographics in Guinea suggest that about twelve out of 1,431 people would have died per year of any cause (including malaria).8

      It's not clear how this would be included in the equation. Show the equation

    1. skeptical that a 4- to 8-week program like StrongMinds would have benefits that persist far beyond a year.

      is this a reasonable justification for a skeptical prior?

    2. t conclusions.

      the table below should be better formatted

    3. they seem unintuitive to us and further influence our belief that StrongMinds is less cost-effective than HLI’s estimates.

      But this seems a bit overly driven by priors/double-counting

    1. This post provides an overview and analysis of the Doing Good Better  book giveaway through Effective Altruism New Zealand (EANZ). The analysis covers data collected from survey responses between 05-Jan-17 and 17-Dec-19, for which there were a total of 298 responses, with appreciable variance in the amount of the survey which was completed. This analysis was initially completed around Jan 2020 so any reference to "to date" refers to then.

      Hypothes.is comments are different -- that's the functionality I was looking for, more or less

    1. We found that Fundraising Pages which received the £5 donation raised £118 more (on average) than the pages in the control group.

      We should try to replicate for effective pages

  10. Feb 2023
    1. ngle claim published in a paper to evaluating the credibility of published papers more holistically. In phase 2, which began in 2021, 200 "bushel" papers were evaluated holistically. Participants working in IDEA groups evaluated the seven credibility signals:

      I'm a little unclear on what this is. Is there a concise explanation of a 'bushel' or of how this is 'holistic'?

  11. Jan 2023

      test comment

    1. somemustuseIDEAprotocol,butmostcanuseasingleroundofelicitation.Whatthey allhaveincommonisthatthey mathematically aggregatejudgments abouttheprobability ofsomeeventorsubjectivedegrees ofbelief,intoasingle,value.

      Will this work for continuous outcomes like the Unjournal is currently asking for?

    1. Interestingly, this also means that the prior for σ is now dependent on the prior for the slope, because

      come back to this, we might be able to put this exlpicitly into the model

    2. This means that the estimate for sigma is the square root of 1 minus the variance of the slope estimate (0.75²). I

      Could/should we make this explicitly part of the model, i.e., constrain this?

    3. prior(normal(0, 0.5), class = "b", lb = -1, ub = 1)

      seems, with brms, you can set lb and ub on classes but not on individual parameters

    4. add_predicted_draws(model_height_weight) %>%

      here we draw 'predicted entries'

    5. add_epred_draws(model_height_weight) %>%

      draws from the slope parameter

  12. Dec 2022
    1. CEARCH discovering a Cause X every three years and significantly increasing support for it.

      This seems like 'assuming the result' ... why every 3 years?

    1. sample_prior = TRUE,

      Does the 'prior predictive simulation' stuff here too

    2. The output shows us that we need to set two priors, one for the Intercept and one for sigma. brms also already determined a default prior for each, but we’ll ignore that for now.

      It's not clear to me what get_prior is doing here, or what its logic is. It would seem to be using the data to suggest priors, which McElreath seems to be against (but the 'empirical bayes' people seem to like)

      Of course, it does at least remind you what objects you need to set priors over

    3. The prior for the slope is a lot easier now. We can simply specify a normal distribution with a mean of 0 and a standard deviation equal to the size of the effect we deem likely, together with a lower bound of 0 and upper bound of 1.

      Update: I was wrong on the below, the SD is not 1 here, because it's the SD for the residual term in the linear model, not the SD for the raw outcome variable.

      Previous comment:...

      I’m ‘worried’ that if you give it data you know has sigma=1, but you allow it to choose any combination of beta and sigma, you may be getting it to do give a weird posterior to both of the parameters, in a way you know can’t make sense, in order to find the most likely parameters for the weird geocentric model you imposed.

      on the other hand I would have thought that it would tend to converge to a sigma=1 anyways as the most likely, as that is ‘allowed’ by your model

      my take is that the cauchy prior you impose in that part is heliocentric; well let me expand on this. I think you know that the true std deviation of the ‘standardized heights from this population’ is 1 what you don’t know is whether it is indeed normal (i.e., whether family = gaussian is right here) thus it might be finding ‘a sigma far from 1 is likely’ under this model, because that makes your ‘skewed’ or ‘fat tailed’ data seem more likely under the normal prior A better approach might be to allow a different distribution with some sort of ‘skew’ parameter, but imposing the sd must be 1

    4. Apparently our prior was still very uninformed because the posterior shows we can be a confident in a much narrower range of slopes!

      so here the priors mattered!

    5. I increased the adapt_delta, as suggested in the documentation, from .8 to .9.

      what does this mean?

    6. he Rhat values did not show this was problematic but

      what are rHat values and where do we see them?

    7. egression model into a simple correlation analysis. That way we can specify a prior on what we think the correlation should be

      to me, in this case, with physl interpretable data, it sounds more difficult to consider correlations. The 'small medium large' thing is from psychometrics I believe

    1. Please note, not all rigor criteria are appropriate for all manuscripts.

      Sciscore seems to have failed to be meaningful here

    2. ScreenIT Sep 27, 2021 SciScore for 10.1101/2021.09.22.461342: (What is this?)Please note, not all rigor criteria are appropriate for all manuscripts.

      Can we use any tools like this? E.g., Statcheck.io (for APA/Psych papers)

      somewhat important

    3. Is the study design appropriate and are the methods used valid? Yes

      as noted before, this yes/no tickboxing is generally not optimal for our case. These things are on a spectrum.

    4. Some details of the methods are lacking. For example, the MUpro provides two methods, it is necessary to specify which method was used in the analysis. The confidence score of each prediction should also be provided. Besides, some results from I-Mutant and MUpro were conflicting, the authors may want to discuss the discrepancy.

      again, the markdown numbering is failing here

    5. Discussion, revision and decision Discussion and Revision Author response We would like to thank the reviewers for their valuable comments. Below we provide pointwise response and the changes made in the revised manuscript. To Dr. Jyotsnamayee Sabat

      Nice, but

      1. I'd like to be able to see this full screen
      2. A heading/table of contents would be very helpful here

      fairly important

    6. PeerRef Dec 15, 2021 Discussion, revision and decision

      I would hope we could replace 'decision' with 'ratings and predictions' or something ... and make those ratings prominent


    7. Author response

      The 'order by recency' is good but sometimes limiting. I think readers would probably prefer to see the 'major comments and discussion' first, before the specific detailed small comments and clarification questions.


    8. Nov 26, 2021 Peer review report Reviewer: Hurng-Yi Wang Institution: Institute of Ecology and Evolutionary Biology, National Taiwan University email: hurngyi@gmail.com Section 1 – Serious concerns Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable Section 2 – Language quality How would you rate the English language quality? Medium quality Section 3 – validity and reproducibility Does the work cite relevant and sufficient literature? No Is the study design appropriate and are the methods used valid? No Are the methods documented and analysis provided so that the study can be replicated? Yes Is the source data that underlies the result available so that the study can be … More Peer review report Reviewer: Hurng-Yi Wang Institution: Institute of Ecology and Evolutionary Biology, National Taiwan University email: hurngyi@gmail.com

      Nice. Is there a way we could put this at the top, or make a quick link to it?

      Ideally, this would have the ratings/rankings/predictions show up first on the page, as some sort of table (and also metadata if we dare to dream),


    9. Read the original source

      This is a bit misleading here. The 'original source' is basically the same stream of text

    10. I agree to change to Verified manuscript.

      what does this mean?

    11. and are shown below.

      these are not shown below. Are graphics possible here? Obviously a direct hyperlink to the revised section of the paper would be convenient here

    12. We would like to thank the reviewers for their valuable comments. Below we provide pointwise response and the changes made in the revised manuscript.

      @gavin @ annabell -- this might read better if each comment quickly linked to the section of the hosted paper and/or the comments were inserted in that part of the hosted paper with hypothes.is

    13. Pt-12:

      what do the prefixes like PT-12 mean here? I guess it's the reviewer number?

    14. The “Analysis of the Mutational Profile of Indian Isolates” should be moved to Materials and Methods.

      The markdown numbering failed here!

    15. Read the full article

      I clicked this link, and it is not coming up, or it's very slow

    16. Article activity feed Version 2 published on bioRxiv

      having trouble interpreting this. The linked version was published on Bioarxiv after the PeerRef? So which version was evaluated?

      OK, I guess the post-PeerRef version is published above ... so this is going from 'newest to oldest'. Maybe there's a way to make that clearer to someone visiting the page for the first time

    17. AgarwalNita Parekh

      why a 'full stop' (period) here after authors' names?

    18. Abstract

      abstract of which version?

    19. In this study we carried out the early distribution of clades and subclades state-wise based on shared mutations in Indian SARS-CoV-2 isolates collected (27 th Jan – 27 th May 2020). Phylogenetic analysis of these isolates indicates multiple independent sources of introduction of the virus in the country, while principal component analysis revealed some state-specific clusters. It is observed that clade 20A defining mutations C241T (ORF1ab: 5’ UTR), C3037T (ORF1ab: F924F), C14408T (ORF1ab: P4715L), and A23403G (S: D614G) are predominant in Indian isolates during this period. Higher number of coronavirus cases were observed in certain states, viz ., Delhi, Tamil Nadu, and Telangana. Genetic analysis of isolates from these states revealed a cluster with shared mutations, C6312A (ORF1ab: T2016K), C13730T (ORF1ab: A4489V), C23929T, and C28311T (N: P13L). Analysis of region-specific shared mutations carried out to understand the large number of deaths in Gujarat and Maharashtra identified shared mutations defining subclade, I/GJ-20A (C18877T, C22444T, G25563T (ORF3a: H57Q), C26735T, C28854T (N: S194L), C2836T) in Gujarat and two sets of co-occurring mutations C313T, C5700A (ORF1ab: A1812D) and A29827T, G29830T in Maharashtra. From the genetic analysis of mutation spectra of Indian isolates, the insights gained in its transmission, geographic distribution, containment, and impact are discussed.

      I really don't like this font, finding it very hard to read, but that's probably a taste thing. Still, I'd like if we could use a font that 'looks more like a journal'.

    20. Pt-13: I want to know how the representative sequences were selected for different states. Is it based on no. of sequences submitted or positivity rate of a particular region? All the Indian isolates available in GISAID for the period 27th Jan – 27th May 2020 were download and considered for analysis. NO state-wise selection was done.

      these authors seem to have use quotation the opposite way I would have done. I would have done

      reviewer's comment here

      My response here (unquoted)

    21. Demographic Analysis of Mutations in Indian SARS-CoV-2 Isolates

      would be nice to have keywords up top

    22. Demographic Analysis of Mutations in Indian SARS-CoV-2 Isolates

      Commenting on the format here

    1. What cause area(s) is/are you interested in working in if there was a role or project that was a good fit? (select all that apply)

      In the view I'm seeing here, the list is very vertically long. Maybe a way to have fewer spaces or 2 columns for less scrolling?

      If you are trying in general to learn from this rather than about specific people, you might have the survey tool randomise the list order

    2. What type of role(s) would you be interested in working in? (select all that apply)

      where does this list come from? 'Research' is rather vague

    3. What obstacles are holding you back from changing roles or co-founding a new project? (select all that apply)

      What is the purpose of this question? It seems like you are suggesting things they might not have thought of here.

    1. Add the SurveyMonkey account’s OAuth token to your .Rprofile file. To open and edit that file, run usethis::edit_r_profile(),

      For me this opened up some other profile. Maybe because I'm working in Rstudio with a Quarto?

      When I just opened the .Rprofile file listed at the root of my repo and where the .Rproj is stored, it worked

    1. 15desire to “pay it forward” for other donors by supporting the matching fund after receiving matching funds. This possibility may be explored infuture research. About a third ofdonors werewilling to support the matching fund with some or all of their donation. This provided enough matching funds to cover the matching funds received by donors, making the micro-matching system self-sustaining.Despite a long history of altruism,including centuries of organized philanthropy, humans have only recently attempted to systematically measure the cost-effectiveness of altruistic endeavors with the goal of doing as much good as possible10,11. The effective altruism movement is growing and has been notably successful in securing large commitments from relatively few people32,33. Effective altruism’s potential for more widespread adoptionis unknown. The sevenstudiesandproof-of-conceptdemonstration presented here are cause for optimism, grounded in a more detailed understanding of altruistic motivation. Today, relatively few donors prioritize effectiveness. But our results suggest that effective giving can be a satisfying complement to giving based on personal feelings, adding a “competence glow”27to the proverbial “warm glow” of giving. Some donors are willing to incentivize bundle donations in others, promoting a chain of giving that is both personally meaningful and effective. The stakes are high, as ordinary people have the power to do enormous good. The limited proof-of-concept demonstrationreported onhere raised funds sufficient to provide 100,700deworming treatments and 17,500 malaria nets, among other benefits. (See Supplementary Materials.) A better understanding of moral motivation, and how to channel it, could dramatically increase the impact of human altruism.MethodsAll reported studies, including the final proofofconcept,were pre-registered, except forStudy 7 (which was a pre-test for the proof of concept). Formore detailed descriptions of the methodsand results, please refer to our Supplementary Materialsavailable at https://osf.io/zu6j8/?view_only=28050663bd6b4b5cae0a73ad8068bc34. Across Studies 1-7w

      I see the code and data here, but I can't find the study materials

  13. Nov 2022
    1. add_predicted_draws

      not sure I get the syntax here. Why is this called add_predicted_draws?

    2. howed us the posterior distributions of the two parameters

      I think you plotted the 'marginal posteriors' for each (for each case, averaging over the posterior for the other). Technically, there is a joint posterior distribution, which you could plot as in those heatmap plots in Kurz.

    3. Apparently our posterior estimate for the Intercept is 154.63

      They call it an 'estimate' in the code but that seems like terminology McElreath would disagree with. That's the maximum a-posteriori value ... but the estimate is the distribution (actually, the joint distribution of the parameters)

    4. Here we see that the posterior distribution

      would be interesting to plot this for 2 different posteriors

    5. Notice that we sample from the prior so we can not only visualize our posterior later, but also the priors we have just defined.

      not sure what this means. Also what does 'run the model mean? Calculate a posterior? With which approach?