CCRs
Oscar add domain generalisabiliyt from computer science.
CCRs
Oscar add domain generalisabiliyt from computer science.
Transfer performance of LBA based on RR (a) and PLS (b) measured
should we not here also include CCR and aCCR?
five words
why 5. also, is this five words all together or to one of the seven questions? Also, were the quetions part of the text so that the LLM know what the patient genreated text was in response to?
also consider adding a Suppl. table including the mean number of words sd etc for each response text for the different dataset (e.g., see the textDescriptives() function ).
META-FBZ
super nice dataset :) love the questions included.
Questionnaires
Rating scales?
im thinking that open-questions with open-response formats can be called and be part of a questionniare...
However, current supervised LBA models risk conflating the target construct with correlated traits (e.g., socioeconomic status, gender), raising concerns about construct validity. Here we propose that the critical test is domain-generality: the ability of a model trained in one domain/setting to transfer successfully to another.
Im not sure i fully understand the link here. Is this what is meant:
If a model has learned to exploit confounds rather than the construct itself, those confounds are unlikely to travel across domains -- a model trained on social media may leverage markers of socioeconomic status or gender that simply aren't present in clinical interview text. Domain-generality therefore serves as a practical test of construct validity: a model that transfers successfully is more likely to have converged on the construct itself.
but im not sure this assumption holds? that is: The construct itself might be domain-specific. depressive language might manifest differently enough across contexts that even a valid model struggles to transfer -- not because it learned confounds, but because the construct's linguistic expression varies. A model could fail the domain-generality test despite being valid within its training domain.
(the more simplified argument could simply be that generalizable models are more useful. but also that undersanding what generalises and what doesnt can provide insights for theory and modelling)
Psychometric Vectors
is this a new term? missing something about the central question being domain-generality? something like: Theory-Driven vs. Supervised Language-Based Assessment of Depression: A Cross-Domain Validity Evaluation
motiona
this is cool!
Author Note
Move this section to the end of the paper.
Build info
I think the build info should include more information regarding what is already prepared to be "updated" such as analyses parameters. And, open science aspects such as open data, open code, etc; link to github, osf account etc.