again, since formula involves "r" this is maybe better called an approximation to the standard error.

Although it does depend on r, so in that sense it does have a sampling distribution.

Get rid of the "i"s on the expectations and make sure expectations involve Greek letters (i.e. alpha) rather than english ones.

Fix notation to be consistent with Chapter 3 (there you use an equals sign with an infinity over it to indicate asymptotic equivalence).

closer to perfect.

missing the bars

Say "Sometimes \alpha is unknown..." Clearly it is not true as written. For instance, later in this chapter you give an example of an attenuated correlation due to a median split on X and the correction factor is 2/sqrt(2*pi). This correction factor is a a known quantity and does not need to be estimated.

Maybe add something like "even if a_i is an unbiased estimate of alpha"

It is the estimator that is unbiased, not the effect size itself

I think you mean "some errors are systematic and not random"

This seems like a non-standard usage of "range restriction" to me. It is usually used to mean that the instrument is unable to distinguish between differences in the underlying latent construct of interest NOT that the observed range of the measurements is restricted due to a selection effect when choosing the sample.

I believe you mean "high school"

"all" is a strong word here. Maybe back it down a little.

This section (including the example) doesn't make sense to me. I don't see how you can plausibly derive an appropriate correction her unless you have additional (strong) assumptions such as that misclassifications are random.

Do you have a citation for this? This seems like a roundabout way to get a correction.

Is the picture supposed to represent folks randomly misclassified? How is it generated?

only if the misclassification process is uncorrelated with the true scores. For instance, in the extreme, if all the As are classified as Bs and all the Bs are classified as As we get the same difference in observed means but just of opposite signs.

Typo in second equation, subscript should be B on LHS of equation.

I assume you meant to say the "observed mean difference".

Well, you wouldn't call this a control group in this case. I guess you'd call it the non-depressed group.

I assume you mean the true proportion of people in each group in the population are the same?

I assume this is the usual chi-square statistic for independence in contingency tables? In any event, you need to provide a formal definition of the statistic.

If you are going to talk about the phi coefficient you need to provide a formal definition

delete the "most"

Please provide reference for the formula

Is this using the "Total sample reliability" approach that you describe above? One of the other approaches? Be clear.

Is it usual to correct SMD effect sizes for unreliability. To me it seems more natural to think about the mean difference relative to observed score variance than relative to true score variance.

If so then the correlation between the reliability and effect size estimate should be accounted for, which the formulas you provide do not do.

In general point-biserial correlations do not behave in the same way as Pearson's r ((https://stats.stackexchange.com/questions/589128/converting-cohens-d-to-pearsons-r-ne-calculated-pearsons-r)), so it is not clear to me that this conversation process will work. Do you have a reference?

It seems like a lot of work to do that likely depends on lots of assumptions and it is perhaps cleaned to just assume the total reliability applies separately within each group.

As noted above, your previous formula for when SMD and reliability are computed on the same sample is wrong, so I don't trust this one either.

correct

Why? if the groups differ only in the mean then this will not be true. If they differ in terms of true score or error variance then it will be true.

The appendix appears to deal only with correlations and doesn't provide any particular justification for this formula.

It seems odd to me to compute these separately, since reliability is a function of observed score variance. There is clearly a strong covariance in the estimates that isn't being dealt with here. See also comment on 1 below.

Again, this is the correct equation if the reliability is assumed to be known ahead of time (in which case you would not need to pool estimates of it, which points out an inconsistency in your presentation). It is NOT the correct equation if both the reliability and the SMD are estimated on the same sample.

I guess we are assuming that the sampling error is uncorrelated with the measurement error?

For this table to be useful you are likely going to have to define what some of these things are.

The easier thing would be to just cut the whole table since the sources of measurement error are not relevant for how the corrections are used. Thus, the section/table only serves as a distraction from the main point of the chapter.

which of the two corrections that you describe above are you implementing (i.e. reliability estimates obtained from independent sample or from same sample as observed score correlation

Also, the derivations in Bobko and Rieck all depend on multivariate normality.

Equation (5.8) does not appear in Bobko and Rieck. They do provide a similar derivation from which (5,8) can be inferred. However, the Boboko and Rieck version uses n rather than n-1 in the denominators and only considers correcting for unreliability in one variable.

Looking at the Bobko and Rieck reference, this is the formula when the reliability coefficients are considered to be fixed, known, quantities. It is NOT the SE formula when they are estimated on the same sample as the correlation.

Is this a script that you found on the internet, that you wrote yourself?

similar to comments in previous chapters, change the notation. Don't use "se" but rather SE(est), where "est" represents that estimator for which you are providing the standard error.

But that is not so, because, as you noted last chapter r_xy is biased as estimate of rho_TU and as I noted in my comments on the last chapter the proposed corrections depend on normality.

Additionally, even if we can ignore the above objections the statement would only be true if the reliability estimates are independent of the observed score correlation estimate.

Not sure what you are saying here. Just say "Than the true score variance".

Since correlations are symmetric (i.e. they do not care which variable is dependent and which is independent) I would avoid the "dependent/independent" language when describing correlations. Instead just talk about the two constructs being correlated

make this "only two"

proposed by

Adding an "i" subscript here (and ff) would be useful to underscore that every person in the population has a true score and so when you later talk about sigma^2_T it refers to variability across the "i" subscript.

Not clear to me why test-retest gets a check in column 2 but others like coefficient alpha do not.

Use "may" instead of "likely"

Not totally sure how this is distinct from #1 (is #1 just meant to be a catch all for error that doesn't fit elsewhere?)

This alludes to statistical models that are much more complex than what you have presented so far. If you want the reader to contemplate more complicated variance component models (besides just true score and error variance components) you need to explicitly introduce what those models look like.

I think this can be fixed by just switching the order of 5.4.2 and 5.4.3

as a measure of reliability within the classical test framework

It is odd that you provide an example involving raters here but then you don't discuss what you call rater/observer specific error until much later in the chapter.

If your example is going to address the distinction between standardized and raw versions of coefficient alpha then your chapter needs to explain the distinction in more detail than what you do here.

I am not sure what you mean by this phrase

OK, what are the assumptions

Important to note that coefficient alpha is making the "parallel tests" assumption described above.

Well, unless the ratings were perfectly negatively correlated

I am confused by the usage of the word "formative" here.

I don't think that the sigma^2_XT notation has been defined.

LEss familiar with this one than the Hedges correction, but I suspect it also depends on normality.

Presumably using equation 4.1

This is too vague. A specific estimator of the pooled variance is unbiased. You need to say what that estimator is.

Again, only if we maintain a normality assumption

I don't think this statement can be made generally. Whether standard deviation estimates will be biased down or up would seem to me to depend on the distributional properties of the individual random variables. See here for some insight: https://stats.stackexchange.com/questions/11707/why-is-sample-standard-deviation-a-biased-estimator-of-sigma

Again, are you sure this is true generally. For instance, Hedges' correction to the standardized mean difference results in an unbiased estimator only under normality.

Are you sure this is true generally for all other possible artifact correction? Is so, how can you be sure?

Do you have a citation for this claim? It seems to me that it is likely false. I see no reason to believe that the bias of a ratio estimator will be small of the denominator and the numerator are independent.

Also, per my earlier comment, I'm not even sure I know what it means for e_i and \hat(a_i) to eb independent, since one is an unobserved structural parameter and the other is a sample estimate.

You should be clear that equation 3.9 is based on a normal approximation to the non-central t distribution

OK, in that case I would wait until next chapter to introduce the R code. There isn't much value in showing someone how to get an answer in R that differs from your hand worked example

Where does this formula for the variance of the correlation coefficient come from? I believe this is an expression of the asymptotic variance and/or an expression that is valid under normality and it may not be accurate generally. E.g., see here[(https://stats.stackexchange.com/questions/196689/expected-value-and-variance-of-sample-correlation#:~:text=Most%20of%20the%20sources%20I,follow%20a%20bivariate%20normal%20distribution.)

This cannot be true in general. It may be true under certain conditions, which should be carefully described before you make the claim.