REF impact ... frameworks for The Unjournal?

What's 'empirical evaluations' here?

The Unjournal supports this. I think we have done this in more depth and rigor than other initiatives in economics and social science

We should discuss and analyze this further. What's a reasonable interpretation? How much of the variation do each of these explain?

I'd like to dig into this to find some specific instances. Are these all the 'real CIs' and not the ones we imputed from the earlier likert CI data?

Some comparison numbers would also be helpful. I asked chatgpt

give some comparison alpha's from other contexts to help understand this measure

Response:

Krippendorff's alpha is used across a variety of disciplines to measure inter-rater reliability. Below are some examples from different contexts to provide a point of reference for interpreting the values in your data:

1. Content Analysis (Media Studies):

2. A typical Krippendorff's alpha for coding reliability in content analysis (e.g., categorizing themes in media) might range from 0.6 to 0.8. Values below 0.67 are often considered insufficient for drawing strong conclusions, though thresholds can vary depending on the field. For example, a study evaluating news articles might report an alpha around 0.7.

3. Healthcare (Medical Diagnostics):

4. In medical diagnostics, particularly in assessments involving subjective judgment (e.g., radiology, pathology), alpha values are expected to be relatively high—often above 0.75. An alpha of 0.8 or higher is often considered acceptable for diagnostic consistency. For instance, a study on agreement among radiologists in interpreting mammograms might report an alpha of around 0.81.

5. Education (Grading & Evaluation):

6. In educational assessment, inter-rater reliability for grading essays or open-ended responses typically yields alphas in the range of 0.6 to 0.9, depending on the specificity of the rubric and training provided to raters. A study on grading consistency among multiple teachers might report an alpha of 0.65 to 0.75.

7. Psychology (Behavioral Coding):

8. In psychology, for tasks like coding behavior or assigning labels based on psychological interviews, Krippendorff’s alpha values of 0.4 to 0.6 are relatively common, but higher values (above 0.7) are preferred for more reliable and replicable findings. For example, coding nonverbal behaviors in therapy sessions might produce an alpha of 0.5.

9. Social Science (Survey Coding):

10. When coding open-ended survey responses, social science researchers often see alpha values ranging from 0.6 to 0.7, reflecting moderate agreement. An alpha of 0.65 might be reported in a study on the reliability of coding responses to survey questions about political preferences.

11. Linguistics (Speech Annotation):

12. In linguistics, particularly in tasks such as speech transcription or annotation, inter-rater reliability can vary widely depending on the complexity of the task. Alphas around 0.7 are often considered acceptable. For example, a study transcribing dialectical variations in speech might report an alpha of 0.72.

General Guidelines for Interpreting Krippendorff's Alpha:

• >0.80: Strong agreement, generally considered acceptable for high-stakes decision-making.
• 0.67 - 0.80: Moderate to strong agreement, often considered sufficient for exploratory studies or content analysis.
• 0.50 - 0.67: Moderate agreement, acceptable in some fields, but may require more consistency for high-stakes or formal evaluations.
• <0.50: Low agreement, indicates significant variability in ratings; further training or clarification of criteria may be needed.

Comparison to Your Data:

• Your highest alpha values (e.g., 0.804 for "gp_relevance" and 0.778 for "merits_journal") indicate strong agreement, similar to the reliability found in high-stakes fields like medical diagnostics or well-defined content analysis tasks.
• The lower values (e.g., 0.271 for "adv_knowledge" and 0.134 for "methods") suggest substantial disagreement among evaluators. These values are below what would be considered acceptable in most research contexts, even for exploratory analysis, and might require addressing the clarity or consistency of evaluation criteria.

These comparisons provide a framework for assessing where your data stands in relation to typical expectations in various research areas.

I think you are missing the basic characterization here ...

1 indicates perfect agreement, 0 indicates no agreement beyond chance, and negative values indicate systematic disagreement.

Do we have an interpretation of this? Are these high, low, reasonable?

(By the way if you wanted to integrate this into a permanent dashboard you might not want to add a narrative about the actual values, but you could still add a general discussion of 'what is considered a high alpha')

We should find some way to integrate this in,. There's so little data it's a shame to drop these.

Very rarely 2. I'm guessing it's only about 10% repeat evaluators atm.

This seems like a wild oversimplification. This would be if they both gave uniformly distributed random ratings.

A more reasonable baseline for 'meaningless ratings' would be something like "they both draw from give the underlying distribution of all papers, without using any information about the paper being rated itself." Perhaps some of the other statistics you mention get at this?

Is there any way to make these boxes foldable in the blog post format?

We should look at this more closely.

1. Which were misspecified? The newer interface actually doesn't permit this.

2. As I noted above, some people simply refused to give CIs at all ... which is essentially giving a 'degenerate' interval

3. For journal ranking tiers I'd still like to see the results in some way. If evaluators understood how we intended this measure to work, they should basically never give degenerate intervals.

However, IIRC some people didn't give bounds on the CIs at all, or they gave meaningless bounds; maybe we dropped those bounds?

You left out one thing I had suggested: Do evaluators with different attributes (country, field, anonymity choice) rate the papers differently? Again, this need not imply biases, but it might be suggestive.

Should we reference 'construal validity' here?

Can we give some links here to data and code? Even a setting to 'turn on the code' or some such? I like to do this for transparency and other reasons we've discussed.

OK we do link the github repo -- maybe add a note about that, and about how to find the code that produces this blog for people who want to see/check?

Do we want this to show here? Maybe some note to the reader is in order?

Formatting here: text is a bit bunched up, a bit harder to read

I think the 'blog post' might be a bit buried here? Perhaps it should be made more prominent?

I thing the spacing and fonts might be improved? Too much vertical space perhaps

Probably worth making this text punchier

Unjournal 'pivotal questions'

By our above standards, this is far too broadly defined; however, there is a wealth of recent (empirical, theoretical, and methodological) work on this; some of which (e.g., ‘WELLBYs’) seems to be influencing the funding, policies and agendas at high-impact orgs. Elicit.org (free version) summarizes the findings from the ‘top-4’ recent papers:

Recent research highlights the complexity of measuring wellbeing, emphasizing its multidimensional nature. Studies have identified inconsistencies in defining and measuring wellbeing among university students in the UK, with a focus on subjective experiences and mental health (Dodd et al., 2021). Global trends analysis reveals distinct patterns in flourishing across geography, time, and age, suggesting the need for comprehensive measures beyond single-item assessments (Shiba et al., 2022). Scholars argue for well-being indicators that capture physical, social, and mental conditions, as well as access to opportunities and resources (Lijadi, 2023). The concept of "Well-Being Capital" proposes integrating multidimensional wellbeing indicators with economic measures to better reflect a country's performance and inform public policy (Bayraktar, 2022). These studies collectively emphasize the importance of considering subjective experiences, cultural factors, and ecological embeddedness when measuring individual and aggregate wellbeing, moving beyond traditional economic indicators like GDP.

This suggests a large set of more targeted questions, including conceptual questions, psychometric issues, normative economic theory, and empirical questions. But I also suspect that 80k and their funders would want to reframe this question in a more targetted way. They may be particularly interested in comparing a specific set of measures that they could actually source and use for making their decisions. They may be more focused on well-being measures that have been calibrated for individuals in extreme poverty, or suffering from painful diseases. They may only be interested in a small subset of theoretical concerns; perhaps only those that could be adapted to a cost-benefit framework.

^[Asking it to “...please focus on recent work in economics and decision theory, and measurements that have been used for low and middle-income countries.” yields [excerpted]

Recent research … aim[s] to move beyond traditional economic indicators like GDP. The Global Index of Wellbeing (GLOWING) proposes a simple, meaningful measure using secondary ecological data (Elliott et al., 2017). The Multidimensional Wellbeing Index for Peru (MWI-P) incorporates 12 dimensions based on what people value, revealing disparities among subgroups (Clausen & Barrantes, 2022). Another approach suggests using the Condorcet median to aggregate non-comparable wellbeing facets into a robust ranking (Boccard, 2017). New methods for measuring welfare include stated preferences over aspects of wellbeing, life-satisfaction scales, and the WELLBY approach, as well as comprehensive frameworks like Bhutan's Gross National Happiness Index (Cooper et al., 2023).]

This seems promising as the basis for a ‘research prioritization stream’. We would want to build a specific set of representative questions and applications as well as some counterexamples (‘questions we are less interested in’), and then we could make a specific drive to source and evaluate work in this area.

Unjournal 'pivotal questions' -- not quite operationalized but seems close to workable 5/10

Unjournal 'pivotal questions' -- operationalizable but out of our current scope.

Unjournal 'pivotal questions' -- operationalizable but out of our current scope.

Unjournal 'pivotal questions' -- operationalizable but out of our current scope.

Unjournal 'pivotal questions'

Reframe as ‘what has been the long term benefit {defined in terms of some specific measure} of investment in scientific research’ – 6/10 as reframed

[The “e.g.”, is 9/10 although it’s on the borderline of our scope]

Unjournal 'pivotal questions'

The “e.g.”, is 9/10 although it’s on the borderline of our scope

Unjournal 'pivotal questions'

Could be operationalized ... what measure of ability, which domains (or how to define this?)

4/10

Unjournal 'pivotal questions'

Could this be operationalized? 3/10

Unjournal 'pivotal questions' ‘Could’ is vague. ‘Very bad outcomes’ needs a precise measure. Reframe as ~‘what is the increase in the risk of {specific VBO} as a function of the level of AI progress {specifically defined}.

3/10

Unjournal 'pivotal questions'

‘Global issues’ is vague; we need more specific categorizations. Cost-effectiveness needs a metric –

5/10

Unjournal 'pivotal questions'

This is fairly definite, although it’s not clear what the precise motivation is here.

8/10

Unjournal 'pivotal questions' -- Needs some refinement – what measure of growth, what units of x-risk, etc.

6/10

could they make these reports dynamic ?

These ad-hoc adjustments seem particular sketchy and this ripe for critique and a more systematic approach

The hover content is helpful

The linked Google sheet is hard to parse and hard to read. This makes it less than fully transparent. E.g., the columns are frozen in a way that you can barely navigate the by-region columns.

Linking something people can't use doesn't add transparency, it just wastes people's attention. If you feel the need put these links at the bottom, in a 'data section' or something. Anyone who wants to dig into it will need to do so as part of a separate and intensive exercise -- not just a glance while reading this. At least that's my impression.

But also note that a code notebook based platform can be far more manageable for the reader.

I think all reports should have an 'update date' prominently listed. OK it says 'April' below

But can they actively rule out a substantial effect ... equivalence test. I see some positive effects. Maybe it is just underpowered.

Is this the usual pre-trend diagnostic for DiD?

I want to see the voting levels by group

robustness tests -- what are some other 'fake' control groups?

But I suspect they might have reported this result as supporting their hypothesis. Thus a multiple comparison test seems appropriate.

they did some reasonable robustness checks. What about restricting it to all Diesel cars?

I'm surprised it's so large ... hmmm

a key first table of results. Looks like a linear probability model, here.

That would be other plausible outcomes?

Any good way to correct for multiple comparisons here?

so it should understate the effecgt

Hmm... Note that the treated group is also more male. But I don't think any of this matters after you control for these demographics, so their comment is misguided.

OK this is probably something we can ignore.

to do: I would want to see a table including vote shares here

Leading robustness check suggestion: The key outcome variable is a percentage of the vote total, I guess. But the treated and groups may have started from a different baseline level of voting. If the true model is nonlinear, the results here may be misleading. E.g., suppose the true effect on voting was the same for both groups as a percentage of the initial vote. Or suppose the impacts have diminishing returns, a sort of ceiling effect.

Other possible robustness checks ... what are some other plausible forms? Car type random effects? Limit analysis to diesel only? Euro-5 only?

Use demographic controls differently (?interact with a random effect or something?) Note, this DiD does not involve any time difference.

the 'placebo test' -- Possible robustness test: what if we made them part of the control group?

compensation scheme

Possible robustness test: compare other plausible vehicle treatment and control groups

the policy

"seem to" is vague

I guess they mean 'after the policy these people became relatively more environmentally friendly'

Check: significantly larger?

main claim, quantified

But how big a violation is 'substantial enough to matter' here?

In the paper, they are basically making the stronger claim that "this policy CAUSED these people to vote for Lega". That's the argument behind their discontinuity DiD approach .

The DiD approach

the thing they want to measure

the data

A claim of a 'null finding' (bounded or just underpowered?) And is the difference significant?

A secondary claim

this seems like a sort of ad-hoc way of breaking up the data. What exactly is the question here, and why is this the best way to answer it?

This seems visually the case, but I don't see metrics or statistical inference here.

But couldn't this just be because people who think there is high Xrisk think others are likely to think like themselves? Is it more finely grained 'better reciprocal accuracy' than that?

The differences in the total x-risk are not quite so striking-- about 2:1 vs 6:1 What accounts for this? Hmm, this look different from the 'Total Extinction risk' in table 4. Here a notebook would be helpful. Ahh, it's because this is for catastrophic risk, not extinction risk.

This paragraph seems a bit loosely argued.

They seem to have displayed the questions along with particular “Prior Forecasts” — is that appropriate? Could that be driving the persistent difference between the superforecasters and experts?

What are 'general x-risk experts'? Give some examples.

fairly substantial incentives ... but it may have been time consuming; how many hours did it take?... and how much variation was there in the incentive pay/how sensitive was it to the predictions?

Would this attrition matter?

with incentives for accuracy (or 'intersubjective' accuracy) at each stage, or only at the very end? Aldo incentives for making strong comments and (?) convincing others/

Recruitment of experts.

Is this approach valid? Would it give powerful incentives to be persuasive? What is are these rationales used for? Note that 'intersubjective accuracy' is not a ground truth for the latter questions.

How does this 'uncover the reasoning behind predictions'?

Is the 'reciprocal scoring' rule likely to motivate honest (incentive-compatible) predictions? Is it likely to generate useful information in this context?

This is a bit confusing. The language needs clarification. What exactly is 'intersubjective accuracy'?

I appreciate these links to the full question content.

I don't see the connection to household bargaining here

This seems logically incoherent. Why would conflict challenge gender roles? And if the opposite, why would greater inequality (favoring men, I guess) make them want to resort to violence to 'reassert' their power?

relevant to prioritization

whole population or only 90%?

This sentence is confusing. And why would I expect that 'violence ... [for[ intrahousehold bargaining' would be particularly driven by having been assigned to a conflict zone?

I think you need to explain a possible workflow here. E.g.,

1. Open the tex, markdown, etc. file in a text editor
2. Apply the suggested suggestions using the above syntax and save it with a different name
3. Use the 'suggs' tools below to manage this

Or 1. Open the old.txt (tex, markdown, etc. file) in a text editor 2. Just make your suggested changes (and comments?) and save it as new.txt, don't use the syntax 3. Use the utility 'suggs diff' to make a third file that highlights these suggestions (what is the name of the new file? do I need to pipe it into something?)

Also, would you want to have this syntax and style of suggs mapped for key text editors for syntax highlighting and maybe shortcut keys?

I'm curious how this will work -- who does it attribute the changes to?

Can you signpost or tease whether this will be done manually or whether there will be suggested shortcut keys etc?

why two "+" signs and not just one?

See above suggestion about how people grok the "@". I suggest --David or something like this instead... maybe even three dashes to avoid confusion with actual double-dash content.

"@" always makes me think you are flagging the OTHER guy ... and you expect it alerts them somehow. Maybe a double dash instead?

%%[ This clarifies the argument, right, @stephen? --Reinstein ]%%

Typo -- close bracket

Formatting of this documentation file -- I can barely see the text in those boxes ... make the boxes taller

A web site/wiki thing with dynamic explanations seems better for this section

tables should be forematted better

This seems very much ad-hoc and not meant for a specialist audience. There is a whole literature on this, and much more theoretically grounded approaches, as you know. The justification given here is rather incomplete.

This stuff could be better presented as a dashboard/hosted Quarto type thing

this seems somewhat contradictory

if this were a part of the project being evaluated we would ask for a reference here ('who are these people?'). But maybe OK for exec. summary.

I"m not sure a pdf is the best format for this. I suspect more interactive web presentation would be better

A very big question -- would be helpful to pose some possible building blocks to answering this question that gives people a hint at how to take a stab at it.

Disseminate: By whom, to whom, with what theory of change/path to impact?

To me this seems distinct from the rest of the bullet point

biblio entries missing

It's unclear whether we are talking about A. "intergenerational governance and international policymaking" or

B. "1. International governance and 2. Policymaking in general".

The latter bullet points and cited papers (e.g., Vivalt and Coville) do not seem to always relate to intergenerational governance

This seems relevant for The Unjournal's consideration/evaluation (but this may fall into the 'ask authors' permission' category). It is empirical and apparently rigorously quantitative and seems highly-relevant to policymaking and impact evaluation research, and influencing policy, all crucial to 'the impact agenda'. Hopefully also follows open, robust science standards (prereg, etc.).

biblio entry missing

Perhaps this should be elaborated and defined somewhat more formally. Reference a particular issue in mechanism design here that is particularly relevant to AI systems, perhaps.

I'm not sure what is meant by 'powerful non-state entities' here. This seems under-defined.

political inequalities?

biblio entry missing. Would be useful to know what this one is.

biblio missing

sp: Malhotra https://www.cambridge.org/core/journals/american-political-science-review/article/abs/myopic-voters-and-natural-disaster-policy/039708A3223EC114365ADF56F1D26423

A reference would be very helpful here. It's hard for me to see what sort of economic models are relevant here.

biblio missing. This seem potentially relevant for an Unjournal evaluation, although we tend not to focus on 'broad think piece' work, which this might be

biblio entry missing

interesting but probably not quantitative/formally specific enough for The Unjournal

biblio missing

not sure this is getting at 'long run' in the sense that longtermists care about

not really economics but that's not so important

biblio entry missing

This is mainly about the welfare tradeoff between economic growth and x-risk in a theoretical sense; I don't think it's about the 'impact of growth on GCRs' per se

biblio entry missing

empirical and seems very relevant and strong; adding it to the Unjournal database

https://forecastingresearch.org/s/XPT.pdf

empirical

biblio entry missing

this is a practical applied policy paper that seems informative for donors considering their own charity decisionmaking

they largely mention theory papers (micro theory, optimization, axiomatic/normative), not empirical work here

The Andreoni and Payne paper is about the government crowdout of private philanthropy (there are a bunch of papers about this), not about the reverse nor about crowding out among donors.

Practically speaking, this seems largely about the impact of interventions (funded by charity) over time; however it does connect with donors' to the extent it involves personal finance and issues like value drift.

OK but the Trammell paper is addressing something different -- coordination in a public goods provision model.

I might add a related issue -- decisions to give to one charity may crowd out other donations; the extent to which this is the case ('donations are substitutes') informs strategies for convincing people to give 'more effectively' vs 'give to effective causes.' See my notes: https://daaronr.github.io/ea_giving_barriers/chapters/substitution.html

Does anyone know if this is the most updated statement of GPI’s economics agenda?

“Economics ‘draft agenda’” Anyone know when it was updated?

chatgpt_single(prompt_role = 'user', prompt_content = 'say something relevant' , temperature = 0.8 , n = 2 , max_tokens = 15)

n = 2 -- 2 responses max_tokens = 15 ... maybe it means 'try 15 times' to get the best 2 from

prompt_role = 'user' -- just ask a question for the gpt ('assistant') to respond. I'm not sure why it would make sense for single chat requests to choose the role 'system' or 'assistant' as this wouldn't persist (?)

are they ignoring the heterogeneity and "LATE" issue?

but couldn't both of these be driven by a third factor ... the state is less environmentally friendly. or maybe I am missing something

have they successfully shown this 'mediation' channel ... with 2 separate sources of exogenous variation? (That's always very challenging to identify)

But could this reflect something mechanical like a nonlinearity?

For fortran

Compiler for C++

Tells R which C compiler to use

It's a work in progress

This seems extremely high-value and potentially ideal for the Unjournal's non-academic stream. Ryan 'had this in mind too'

I don't see how this argues against the 'suspicious convergence' claim... OK, I see Jack Malde's comment now, which basically gets at my doubts here.

this is the list you linked above under 'organizations'. fwiw it's an interesting list but it's very sparsely populated (most columns have a name only). Some fleshing out and ranking/prioritizing could be helpful here.

I don't see this as a promising research agenda. My sense of it is that it is pretty intractable. (I'm not saying if it is true/false/wrong/right, just that I am not sure if there will be a lot of practical value in pursuing it? OK I see some approaches that might be helpful, if one has a tractible way to model welfare considerations with PAV it might win some people over.)

'cause' or indirect instrumental goal?

Potential relevant to #unjournalresearchprioritization, depending on the approach

Somewhat relevant to #unjournalresearchprioritization

this could be relevant for Unjournal. How unlikely? If it's so unlikely, why mention it?

unjournalresearchprioritization

This seems very relevant for Unjournal

Probably not relevant for the Unjournal at this point, but there may be some overlap

academics (at least in my field) would distinguish a fourth stage 'having been accepted in a journal after peer review'. Not sure how important that distinction is for you.

Note that The Unjournal is trying to make that last stage less burdensome and more informative by commissioning public evaluation and rating of work (rather than relying on tedious and imprecise the 'which journal was it published in' measure)

that's the best, I like to see all research as 'permanent alpha' mode

'which means' --- the implication is not clear here

for me the more concrete measurement issues are at least as important ... you include these, but I don't see it in this paragraph.

1. The airtable view is linking interventions and cause areas, not organizations
2. Why and how did you choose and prioritize these? It's a huge space to explore?

Does that mean you will need to assess (and consider research evidence) on these other non-psychotherapy interventions? If so, that deserves its own section perhaps?

this paragraphs seems unneeded and repetitive. Or am I missing something here?

Maybe restate this to clarify that you are not reevaluating SM as an organization again, but will update the evaluation of their impact in light of your updated evaluation of the intervention?

this part still seems like an intervention not an organization

Pedantic: I'd say the 'estimated effects' here ... obviously the effects themselves are not affected by this bias

in the statement in the 1-pager, you stated

we’ve found that psychotherapy is several times more cost-effective than cash transfers or deworming for improving happiness.

That's not entirely consistent with this sentence

these sentences repeat the sentences above in the one-pager

The psychotherapy report doesn't seem to be about a particular organization. I'm a bit confused about the structure here. How does this differ from a 'cause area exploration' at this point?

These again combine problems with potential remedies and interventions. And 'psychedelics' -- is that aimed at curing 'problems' or boosting the upper end joys of life?

this needs a header (it now is under 1.4). Not sure this long list is helpful here though? What's actionable about this? Is it linked to an appeal for more funding?

I suspect some strong Unjournal/academic research links here. Also to the house improvement ones and possibly the fistula ones too.

For unjournal research prioritization, I guess I will have to dig into those reviews to identify the most pivotal research to have evaluated?

These link articles but don't contain a 'list of works cited' at bottom. Could you provide that ... even better an 'annotated/categorized/prioritized list' explaining which ones you rely on most heavily, and which you have the most uncertainty over?

In the previous agenda you tried

to articulate, within each research area, where additional research seems more (or less) useful, and therefore what our research agenda is for the next one to two years.

This seemed particularly relevant to helping the Unjournal help you. Not sure this new agenda does this as much.

How are you defining and considering a 'working paper' here?

"Units of value" .. maybe add a few more words to clarify this?

I assume this will be a theoretical paper (i.e., no surveys or data?)

I see a lot of benefit in engaging with academics on this paper, and getting and responding to feedback, possibly within The Unjournal's framework

Not sure what is meant by 'an academic paper'. Typically it would be hard to publish a paper in an academic journal that simply 'describes' (or even justifies) the approach that a particular organization takes.

You might have to frame it more as answering or providing evidence on a question of general interest, and/or formally arguing for the 'most appropriate approach' under certain defensible criteria.

A 2021 priority was "Examining how best to convert between different SWB, as well as other, measures (1.2.1)" This seems to have strong academic links relevant to the Unjournal. Is it still a priority?

I expect Unjournal-evaluated research to provide inputs relevant to these evaluations, but not to directly evaluate particular organisations. However, we might be able to cover some of this within our 'less academic stream'.

I suspect there are a range of academic papers (in development economics, health economics, psychology, policy, and the social-sciency side of biomedicine) that will inform this, that Unjournal might evaluate.

This can include work that constitutes - impact evaluation of specific interventions, including RCTs non-experimental causal inference - work exploring the impact of specific paths to impact through these interventions (e.g., the career costs of childhood trauma) - work exploring costs and impacts on the market (e.g., impact of housing improvements on the local economy, price elasticities, etc.)

In the 2021 agenda you stated "Our main current focus, and where the majority of our eΛort will go, is Area 2.3: using subjective well-being scores to compare the cost-effectiveness of highly-regarded health and development interventions used in low-income countries."

Is this still your priority? Is this in line with the 'Cause area explorations" category here?

The boundary between theoretical and applied is not always clear here. Some research, maybe the methodological and measurement research in particular, has both theoretical aspects and very applicable and even empirical aspects. Calling this 'theoretical' might confuse people who would conflate theoretical with philosophical. E.g., research into which survey and other reporting instruments are more reliable, better reflect the actual measures of interest ... this seems very applied to me, and probably relevant to The Unjournal's scope as well.

I think you may have moved a lot of the content outlined in the previous Research Agenda into those linked reports?

Is this necessary in the wheelhouse of HLI? If your focus is on assessing impact from the wellbeing perspective, does that interact with things like 'organizational capabilities' at all?

Maybe better to outsource the latter?

When will you do each? Do you anticipate returning to broad cause areas you've previously decided not to pursue?

How much will you defer to other orgs and researchers in the broader prioritization?

why not just namecheck the ITN framework here?

how do you divide up the 'cause space' and define each 'cause'? Give some examples here? E.g., is "animal welfare' a cause area ... or 'farmed animal welfare' or 'chicken welfare' or 'promoting regulation of chicken farms' (the latter is more of an intervention IMHO)

one of several ultimate goals? To be pedantic, the 'improve global wellbeing' would seem to be the ultimate goal .... the 'identify the opportunities' is instrumental to that

Why not both? Maybe rephrase this?

Also, will you consider the empirical tradeoffs here, or deeper philosophical issues, or?

what academic journal are you thinking?

I talked to someone recently who had done some survey work in this area -- maybe remind me to get back to you on it.

This seems underexplained ... what and why? (add a bit or link)

mention how you will do it better/different or add value to what other orgs are doing?

Doing this yourselves? Synthesizing work? Sponsoring work?

Non mental-health orgs within the global health or global health and development space, or much more widely ranging, across vastly different causes etc?

this phrase seems a bit vague? Also, are these 'cause areas' or possible interventions, or a mix?

I think you would be more convincing if you linked or footnoted some of the critiques of this, as well as your responses. This reads well for a general audience but maybe not for a research and EA audience?

This seems to move a bit towards advocacy, perhaps in contrast to the more neutral approach you mention elsewhere or previously. In other writings it's more like 'get people to consider a well-being based approach, and whether and when it makes sense to use it'.