On 2016 Jul 21, Tom Kindlon commented:

Another study raises serious questions about the use of the role emotional subscale of the SF-36 in the diagnosis of CFS

A CDC group (Reeves et al (2005)) proposed a way to operationalise chronic fatigue syndrome (CFS) criteria including subscales of the SF-36[1,2]: "We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [] on the physical function (≤ 70), or role physical (≤ 50), or social function (≤ 75), or role emotional (≤ 66.7) subscales of the SF-36." This method has been used in numerous subsequent CDC CFS studies but not by external groups, apart from Leonard Jason and colleagues who have examined the effect of this operationalisation of the criteria.

A newly published study looked at using the SF 36 subscales to define reduced activity in CFS (and ME)[3]. Six of the 8 subscales (Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality and Social Functioning) correlated with the sum of current hours of activity. Two subscales did not correlate (Role-Emotional and Mental Health).

The same 6 subscales (negatively) correlated with percentage reduction in the total hours of activity. Again, two subscales did not correlate (Role-Emotional and Mental Health).

Also, a "ROC analysis was used to assess which of the SF-36 subscales best discriminated between patients with ME and CFS and healthy controls. The ROC analysis used a plot of sensitivity versus 1-specificity for scores on the SF-36. The area under the curve (AUC) assessed the degree to which each subscale was able to discriminate between patients and controls. The closer the AUC is to 1, the better discriminatory power that SF-36 subscale has." The same 6 subscales (Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality and Social Functioning) had AUCs of 0.89 or better. By comparison, the Mental Health Functioning subscale had a AUC of 0.59. The AUC for the Role Emotional subscale was even worse at 0.47.

Each of these results demonstrates that the Role Emotional subscale is not suitable to be used to operationalise criteria for CFS.

References:

[1] Reeves WC, Wagner D, Risenbaum R, et al. Chronic fatigue syndrome – a clinically empirical approach to its definition and study. BMC Med. 2005;3(19):1–9.

[2] Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): conceptual framework and item selection. Med Care. 1992;30:473–483

[3] Thorpe T, McManimen S, Gleason K, Stoothoff J, Newton J, Strand EB, Jason LA (2016). Assessing current functioning as a measure of significant reduction in activity level. Fatigue: Biomedicine, Health & Behavior. Published online: 19 Jul 2016 doi: 10.1080/21641846.2016.1206176

On 2014 Feb 28, Tom Kindlon commented:

Using two MFI scales ("General Fatigue" or "Reduced Activity") to ensure patients satisfying the definition have "severe fatigue"

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#304567 but the formatting has gone from that page)

Initially when I read this paper, where it says "we defined severe fatigue as >= medians of the MFI general fatigue (>=13) or reduced activity (>=10) scales", I thought this referred to medians of the general population.Hearing other people commenting on it, that's how some other people have been interpreting it also. It is probably somewhat natural to do this as the sentence before reads: "We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [11] on the physical function (<=70), or role physical (<=50), or social function (<=75), or role emotional (<=66.7) subscales of the SF-36."

However from looking at the scores for controls in other papers, these MFI scores do not look like medians for the whole US population but in fact are medians for this particular group of patients. This seems a strange way to set cut-off points for a CFS definition that is used for numerous studies into the illness, given the cohort that is being used as a basis: "This population-based case control study enrolled 227 adults identified from the population of Wichita with: (1) CFS (n = 58); (2) non-fatigued controls matched to CFS on sex, race, age and body mass index (n = 55); (3) persons with medically unexplained fatigue not CFS, which we term ISF (n = 59); (4) CFS accompanied by melancholic depression (n = 27); and (5) ISF plus melancholic depression (n= 28)." i.e. this is not a random sample of the US population but a group of people selected for a specific purpose (or purposes) (not necessarily to design a definition, but as a follow-up study of people previously diagnosed with CFS or given some other label).

Some of the groups are of different sizes - if the relative size of these groups had been changed, with relatively more people taken from some classification groups and less people taken from other groups, the median scores would likely have been different.

It should also be remembered that in this context the categories listed in the last paragraph refer to their classification when they evaluated years before (from 1997 to 2000), and not necessarily at the time when they were evaluated in this study (December 2002 to July 2003) (as is clear from the tables in this paper).

I thought it would be interesting to look at MFI scores in some other papers on CFS that did not use the "empirical definition".

I don't claim this is a definitive list but, at the same time, mean MFI scores with standard deviations only seem to be listed in a small percentage of papers. The papers use cohorts from a variety of locations: England [3], The Netherlands [4], Germany [5] and the USA (New Jersey) [6]. I did not see any ranges given which would be useful given the task at hand (selecting cut-off points for a definition).Unfortunately not all of the papers I found used the Fukuda [1] definition for CFS; some also used the Sharpe [2] definition for CFS. I indicate which definition is used in each case.

MFI: General Fatigue (i) Sample/(ii) Sample Size/(iii) Mean/(iv) SD/ (v) (Mean - 13)/SD/ (vi) Definition

Weatherley-Jones [3] 53 18.4 1.7 3.176470588 Sharpe (1991)

Vermeulen (Group 1) [4] 30 18.6 1.9 2.947368421 Fukuda (1994)

Vermeulen (Group 2) [4] 30 18.4 1.8 3 Fukuda (1994)

Vermeulen (Group 3) [4] 30 19.1 1.4 4.357142857 Fukuda (1994)

Gaab [5] 21 17.7 0.5 9.4 Sharpe (1991) and Fukuda (1994)

Brimacombe [6] 65 18.41 2.02 2.678217822 Fukuda (1994)Combining these give a sample of 229 patients with a mean "General Fatigue" score of 18.45655022.

This data suggests that a threshold of >=13 will have a very very high sensitivity. This would suggest that another measure would not be necessary (unless it was being used as an extra criterion to increase the specificity, which isn't done with this definition).

However for completeness, I'm including the "Reduced Activity" data from the same papers:Reduced activity (MFI) (i) Sample / (ii) Sample Size / (iii) Mean Score / (iv) SE (v) (Mean-10)/SD (vi) Definition

Weatherley-Jones [3] 53 16.1 3.1 1.967741935 Sharpe(1991)

Gaab [5] 21 15 0.7 8.714285714 Sharpe (1991) and Fukuda(1994)

Brimacombe [6] 65 15.93 4.55 1.340659341 Fukuda 1994

Combining these give a sample of 139 patients with a mean Reduced Activity score of 15.85431655.

Note: the Vermeulen paper[4] did not collect the MFI scores for Reduced Activity, just "the fatigue axes of the Multidimensional Fatigue Inventory" (which they defined as the MFI scores for General fatigue, Physical fatigue, Mental fatigue).

It seems strange in the definition of Chronic Fatigue Syndrome defined in this paper (i.e. Reeves et al) that the "severe fatigue" criterion can be satisfied by a patient having a low score on a subscale of the MFI testing activity levels (as opposed to one of the 3 subscales measuring fatigue), especially when the function of the SF-36 is to "measure functional impairment". Just because someone is inactive doesn't mean they have severe fatigue. Allowing patients to be included if they simply have a "Reduced Activity" score of 10 or more (without necessarily having a low score on one of the fatigue axes of the MFI) risks reducing the specificity of the definition.

References:

[1] Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A. The chronic fatigue syndrome; a comprehensive approach to its definition and study. Ann Int Med 1994, 121:953-959.

[2] Sharpe MC, Archard LC, Banatvala JE, Borysiewicz LK, Clare AW, David A, Edwards RH, Hawton KE, Lambert HP, Lane RJ, et al. A report--chronic fatigue syndrome: guidelines for research. J R Soc Med. 1991 Feb;84(2):118-21.

[3] Weatherley-Jones, E., Nicholl, JP., Thomas, KJ., Parry, GJ., McKendrick, MW., Green, ST., Stanley, PJ and Lynch, SPJ. A randomised, controlled, triple-blind trial of the efficacy of homeopathic treatment for chronic fatigue syndrome. Journal of Psychosomatic Research, 2004, 56, 2, 189-197.

[4] Vermeulen, RCW and Scholte, HR. Exploratory open label, randomized study of acetyl- and propionylcarnitine in chronic fatigue syndrome. Psychosomatic Medicine, 2004, 66, 276-282.

[5] Gaab J, Hüster D, Peisen R, Engert V, Heitz V, Schad T, Schürmeyer TH, Ehlert U. Hypothalamic-pituitary-adrenal axis reactivity in chronic fatigue syndrome and health under psychological, physiological, and pharmacological stimulation.Psychosom Med. 2002 Nov-Dec;64(6):951-62.

[6] Brimacombe, Michael; Lange, Gudrun; Bisuchio, Kim; Ciccone, Donald S.; Natelson, Benjamin. Cognitive Function Index for Patients with Chronic Fatigue Syndrome Journal of Chronic Fatigue Syndrome, 2004, vol 12; number 4, pages 3-24

On 2014 Feb 28, Tom Kindlon commented:

Why is this definition being referred to as an "empirical definition"?

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#301566 but the formatting has gone from that page)

I believe most people's understanding of "empirical criteria" or an "empirical definition" would be that the data would speak for itself; it "would decide" the cut-off points through methods such as cluster analysis (for example).

Indeed this would seem to have been William Reeves' understanding of an empirical definition. For example, in a presentation on the CDC's CFS research program (to a Task Force Meeting on the Epidemiology of Interstitial Cystitis)[1], he said: "The problem with the CFS criteria was that they were not specific enough and not empiric-based. For example, one of the criteria stated that the research subject must have at least four of eight symptoms, among them, impaired concentration or memory and postexertional worsening of physical or mental fatigue. "The accompanying symptoms need to be defined in and of themselves," Dr. Reeves said.

The 1994 International Study Group also hypothesized that fatigue led to patients' symptoms rather than the reverse. The CDC is currently conducting population studies to develop an empiric definition of CFS that is based on statistical modeling."At the inaugural meeting of the US Department of Health and Human Services' Chronic Fatigue Syndrome Advisory Committee (CFSAC), Dr Reeves said the CDC team of research would "derive an empirical case definition based on data".[2]

The definition presented here does not seem to have been based either on "statistical modeling" or "data". It seems to involve relatively arbitrary cut-off points; for example, of the 8 subscales of the SF-36, four are chosen and, for each of these, the 25th percentile of the published US population is chosen as a cut-off point. A patient is required to be in the bottom quartile for just one of these subscales to satisfy the criteria. Where did this cut-off point come from? There is no mention of it in the paper that suggested the use of the SF-36[3]; nor is there any mention that these particular subscales should be chosen or that one would be sufficient. One of the authors of the paper[3] has confirmed that cut-off points were never chosen nor was it decided which sub-scales would be used.

Given that the CDC's definition of CFS tends to go on to be used in numerous studies, would it not be better to investigate which thresholds give a "better" definition e.g. with a higher specificity and sensitivity - for example, for some of the SF-36 subscales, perhaps (say) the 13th, 15th, 20th or even 30th percentiles may be more appropriate.

The cut-off points suggested in this paper may or may not be useful. But is it really accurate to suggest that they are "empirically" derived?

References:

[1] Epidemiology of Interstitial Cystitis - Executive Committee Summary and Task Force Meeting Report October 29th, 2003. http://www.niddk.nih.gov/fund/reports/ic/task_force_summary.pdf

[2] US Department of Health and Human Services - Chronic Fatigue Syndrome Advisory Committee (CFSAC). Inaugural Meeting. September 29th, 2003Meeting Summary. http://www.hhs.gov/advcomcfs/CSFAC_mins_2003.09.29R.pdf

[3] Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason LA, Bleijenberg G, Evengard B, White PD, Nisenbaum R, Unger ER, International Chronic Fatigue Syndrome Study Group: Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution. BMC Health Services Research 2003, 3:25

On 2014 Feb 28, Tom Kindlon commented:

This may not be a representative group of those who would be diagnosed in a random sample using the "standardized clinically empirical criteria"

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#290587 but the formatting has gone from that page)

This "empirical" method of operationalizing the CDC 1994 CFS criteria[1] has subsequently been used in a population study[2]. It found a prevalence rate for CFS of 2540 per 100,000 persons 18 to 59 years of age[2].

This is considerably higher than the prevalence rates found in earlier studies.

For example, a previous study using this cohort using a "previous" method of operationalizing the CDC 1994 CFS criteria[1] found a prevalence rate of 235 per 100,000[3]. Given the way the cohort in this current study was drawn up, using 58 people who had previously been diagnosed using a "previous" method of operationalizing the CDC 1994 CFS criteria, the group satisfying the new method of operationalizing the CDC 1994 CFS criteria, the "empirical" criteria, in this study may well not be the same sort of people that would show up if the method was used on a random sample of the population. So for example the results in Table 6 may not be similar to the results one can get in a random sample.

Unfortunately the paper giving the prevalence rate for Georgia[2] does not give the same pieces of information as is in Table 6 in this study. However we do have a paper which uses a group from the Georgia cohort[4]. Table 1 of this study[4] includes similar data. Some of the numbers are somewhat similar. However one that particularly stands out is the Role Emotional score. It was 35.6 (95% CI: 26.3-44.8). That compares to the value in this paper of 55.8+/-42.2.

Perhaps other data will be published in time. The main point of this comment is to point out or remind people that the data presented in this paper may not be representative of those that would be diagnosed using the empirical criteria.

References:

[1] Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, & Komaroff A. (1994). The chronic fatigue syndrome: A comprehensive approach to its definition and study. Annals of Internal Medicine, 121 (12):953-959. http://www.annals.org/cgi/content/full/121/12/953

[2] Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M, Devlin R. Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural Georgia. Population Health Metrics 2007, 5:5 doi:10.1186/1478-7954-5-5http://www.pophealthmetrics.com/content/5/1/5

[3] Reyes M, Nisenbaum R, Hoaglin DC, Unger ER, Emmons C, Randall B, Stewart JA, Abbey S, Jones JF, Gantz N, Minden S, Reeves WC: Prevalence and incidence of chronic fatigue syndrome in Wichita, Kansas. Arch Int Med 2003, 163:1530-1536.

[4] Nater UM, Maloney E, Boneva RS, Gurbaxani BM, Lin JM, Jones JF, Reeves WC, Heim C. Attenuated Morning Salivary Cortisol Concentrations in a Population-based Study of Persons with Chronic Fatigue Syndrome and Well Controls. J Clin Endocrinol Metab. 2007 Dec 26

On 2014 Feb 21, Tom Kindlon commented:

Data from another population study found scores on the RE subscale are similar in CFS patients to those found in healthy controls

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#285575 but the formatting has gone from that page)

I previously said that I questioned the value of using the Role Emotional (RE) subscale to satisfy functional impairment criteria (http://www.biomedcentral.com/1741-7015/3/19/comments#284561). Researchers deciding whether to follow the method of operationalizing the Fukuda [1] used in this study, might be interested at looking at Table 2 in Jason et al [2].

The subjects were also obtained from a random-digit population study. Here is what the authors said in the text on this part of the results: "A MANCOVA for the Medical Outcomes Study SF-36 Health Survey (controlling for the effects of work status) revealed significant differences in gradations of disability across the diagnostic categories of CFS only, MCS only, FM only, more than one diagnosis, and no diagnosis on seven of the eight subscales (F(4,208) = 1.82, p < .05). The role-emotional scale was the only scale that did not reveal significant differences between the groups (see Table 2). Significant post hoc tests revealed that individuals with CFS demonstrated greater disability than those with no diagnosis on the role-physical; bodily pain; vitality; and social functioning scales. Individuals with MCS demonstrated greater disability than the no diagnosis group on the physical functioning; role-physical; bodily pain; general health; vitality; social functioning; and mental health scales. Individuals with FM demonstrated greater disability than the no diagnosis group on the physical functioning; role-physical; bodily pain; and social functioning scales. In addition, individuals with more than one diagnosis demonstrated greater disability than those in the no diagnosis group on the physical functioning; role-physical; bodily pain; vitality; and social functioning scales. Means for each of the Medical Outcomes Study subscales are reported in Table 2."

This issue of how the Fukuda criteria [1] are operationalized is not a trivial matter. Using the previous method of operationalizing the criteria, a CDC team found a prevalence for CFS of 235 per 100,000 [3]. Using the method of operationalizing the criteria outlined in this study, the prevalence rate for CFS was found to be 2.54% or 2540 per 100,000 [4] or 10.81 times the previous prevalence rate!

References

[1] Fukuda, K., Straus, S.E., Hickie, I., Sharpe, M.C., Dobbins, J.G., & Komaroff, A. (1994). The chronic fatigue syndrome: A comprehensive approach to its definition and study. Annals of Internal Medicine, 121 (12):953-959. http://www.annals.org/cgi/content/full/121/12/953

[2] Jason, L.E., Taylor, R.R., & Kennedy, C.L. "Chronic Fatigue Syndrome, Fibromyalgia, and Multiple Chemical Sensitivities in a Community-Based Sample of Persons With Chronic Fatigue Syndrome-Like Symptoms." Psychosomatic Medicine 62:655-663 (2000). http://www.psychosomaticmedicine.org/cgi/reprint/62/5/655

[3] Reyes M, Nisenbaum R, Hoaglin DC, Unger ER, Emmons C, Randall B, Stewart JA, Abbey S, Jones JF, Gantz N. Prevalence and incidence of chronic fatigue syndrome in Wichita, Kansas. Arch Intern Med. 2003;163:1530–1536. doi: 10.1001/archinte.163.13.1530.

[4] Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M, Devlin R. Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural Georgia. Population Health Metrics 2007, 5:5 doi:10.1186/1478-7954-5-5

On 2014 Feb 21, Tom Kindlon commented:

MDDm should be resolved for more than 5 years before a CFS diagnosis can be given

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#285572 but the formatting has gone from that page)

In this paper, it says: "Following recommendations of the International CFS Study Group, only current MDDm was considered exclusionary for CFS." However, part of the specific recommendations of the International CFS Study Group [1] was that MDDm had to have been resolved for more than 5 years: "The 1994 case definition stated that any past or current diagnosis of major depressive disorder with psychotic or melancholic features, anorexia nervosa, or bulimia permanently excluded a subject from the classification of CFS ... we now recommend that if these conditions have been resolved for more than 5 years before the onset of the current chronically fatiguing illness, they should not be considered exclusionary." It might not be important to point this out for definitions for some illnesses: however if one looks at table 2, 6 of the 16 who are said to have CFS using the "current classification" of CFS, had been diagnosed with MDDm at a previous assessment which suggests it is important in this context.

Reference:

[1] Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason LA, Bleijenberg G, Evengard B, White PD, Nisenbaum R, Unger ER, International Chronic Fatigue Syndrome Study Group: Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution.BMC Health Services Research 2003, 3:25. http://dx.doi.org/10.1186/1472-6963-3-25

On 2014 Feb 21, Tom Kindlon commented:

MDDm should be resolved for more than 5 years before a CFS diagnosis can be given

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#285572 but the formatting has gone from that page)

In this paper, it says: "Following recommendations of the International CFS Study Group, only current MDDm was considered exclusionary for CFS." However, part of the specific recommendations of the International CFS Study Group [1] was that MDDm had to have been resolved for more than 5 years: "The 1994 case definition stated that any past or current diagnosis of major depressive disorder with psychotic or melancholic features, anorexia nervosa, or bulimia permanently excluded a subject from the classification of CFS ... we now recommend that if these conditions have been resolved for more than 5 years before the onset of the current chronically fatiguing illness, they should not be considered exclusionary." It might not be important to point this out for definitions for some illnesses: however if one looks at table 2, 6 of the 16 who are said to have CFS using the "current classification" of CFS, had been diagnosed with MDDm at a previous assessment which suggests it is important in this context.

Reference:

[1] Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason LA, Bleijenberg G, Evengard B, White PD, Nisenbaum R, Unger ER, International Chronic Fatigue Syndrome Study Group: Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution.BMC Health Services Research 2003, 3:25. http://dx.doi.org/10.1186/1472-6963-3-25

On 2014 Feb 21, Tom Kindlon commented:

Data from another population study found scores on the RE subscale are similar in CFS patients to those found in healthy controls

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#285575 but the formatting has gone from that page)

I previously said that I questioned the value of using the Role Emotional (RE) subscale to satisfy functional impairment criteria (http://www.biomedcentral.com/1741-7015/3/19/comments#284561). Researchers deciding whether to follow the method of operationalizing the Fukuda [1] used in this study, might be interested at looking at Table 2 in Jason et al [2].

The subjects were also obtained from a random-digit population study. Here is what the authors said in the text on this part of the results: "A MANCOVA for the Medical Outcomes Study SF-36 Health Survey (controlling for the effects of work status) revealed significant differences in gradations of disability across the diagnostic categories of CFS only, MCS only, FM only, more than one diagnosis, and no diagnosis on seven of the eight subscales (F(4,208) = 1.82, p < .05). The role-emotional scale was the only scale that did not reveal significant differences between the groups (see Table 2). Significant post hoc tests revealed that individuals with CFS demonstrated greater disability than those with no diagnosis on the role-physical; bodily pain; vitality; and social functioning scales. Individuals with MCS demonstrated greater disability than the no diagnosis group on the physical functioning; role-physical; bodily pain; general health; vitality; social functioning; and mental health scales. Individuals with FM demonstrated greater disability than the no diagnosis group on the physical functioning; role-physical; bodily pain; and social functioning scales. In addition, individuals with more than one diagnosis demonstrated greater disability than those in the no diagnosis group on the physical functioning; role-physical; bodily pain; vitality; and social functioning scales. Means for each of the Medical Outcomes Study subscales are reported in Table 2."

This issue of how the Fukuda criteria [1] are operationalized is not a trivial matter. Using the previous method of operationalizing the criteria, a CDC team found a prevalence for CFS of 235 per 100,000 [3]. Using the method of operationalizing the criteria outlined in this study, the prevalence rate for CFS was found to be 2.54% or 2540 per 100,000 [4] or 10.81 times the previous prevalence rate!

References

[1] Fukuda, K., Straus, S.E., Hickie, I., Sharpe, M.C., Dobbins, J.G., & Komaroff, A. (1994). The chronic fatigue syndrome: A comprehensive approach to its definition and study. Annals of Internal Medicine, 121 (12):953-959. http://www.annals.org/cgi/content/full/121/12/953

[2] Jason, L.E., Taylor, R.R., & Kennedy, C.L. "Chronic Fatigue Syndrome, Fibromyalgia, and Multiple Chemical Sensitivities in a Community-Based Sample of Persons With Chronic Fatigue Syndrome-Like Symptoms." Psychosomatic Medicine 62:655-663 (2000). http://www.psychosomaticmedicine.org/cgi/reprint/62/5/655

[3] Reyes M, Nisenbaum R, Hoaglin DC, Unger ER, Emmons C, Randall B, Stewart JA, Abbey S, Jones JF, Gantz N. Prevalence and incidence of chronic fatigue syndrome in Wichita, Kansas. Arch Intern Med. 2003;163:1530–1536. doi: 10.1001/archinte.163.13.1530.

[4] Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M, Devlin R. Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural Georgia. Population Health Metrics 2007, 5:5 doi:10.1186/1478-7954-5-5

On 2014 Feb 28, Tom Kindlon commented:

This may not be a representative group of those who would be diagnosed in a random sample using the "standardized clinically empirical criteria"

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#290587 but the formatting has gone from that page)

This "empirical" method of operationalizing the CDC 1994 CFS criteria[1] has subsequently been used in a population study[2]. It found a prevalence rate for CFS of 2540 per 100,000 persons 18 to 59 years of age[2].

This is considerably higher than the prevalence rates found in earlier studies.

For example, a previous study using this cohort using a "previous" method of operationalizing the CDC 1994 CFS criteria[1] found a prevalence rate of 235 per 100,000[3]. Given the way the cohort in this current study was drawn up, using 58 people who had previously been diagnosed using a "previous" method of operationalizing the CDC 1994 CFS criteria, the group satisfying the new method of operationalizing the CDC 1994 CFS criteria, the "empirical" criteria, in this study may well not be the same sort of people that would show up if the method was used on a random sample of the population. So for example the results in Table 6 may not be similar to the results one can get in a random sample.

Unfortunately the paper giving the prevalence rate for Georgia[2] does not give the same pieces of information as is in Table 6 in this study. However we do have a paper which uses a group from the Georgia cohort[4]. Table 1 of this study[4] includes similar data. Some of the numbers are somewhat similar. However one that particularly stands out is the Role Emotional score. It was 35.6 (95% CI: 26.3-44.8). That compares to the value in this paper of 55.8+/-42.2.

Perhaps other data will be published in time. The main point of this comment is to point out or remind people that the data presented in this paper may not be representative of those that would be diagnosed using the empirical criteria.

References:

[1] Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, & Komaroff A. (1994). The chronic fatigue syndrome: A comprehensive approach to its definition and study. Annals of Internal Medicine, 121 (12):953-959. http://www.annals.org/cgi/content/full/121/12/953

[2] Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M, Devlin R. Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural Georgia. Population Health Metrics 2007, 5:5 doi:10.1186/1478-7954-5-5http://www.pophealthmetrics.com/content/5/1/5

[3] Reyes M, Nisenbaum R, Hoaglin DC, Unger ER, Emmons C, Randall B, Stewart JA, Abbey S, Jones JF, Gantz N, Minden S, Reeves WC: Prevalence and incidence of chronic fatigue syndrome in Wichita, Kansas. Arch Int Med 2003, 163:1530-1536.

[4] Nater UM, Maloney E, Boneva RS, Gurbaxani BM, Lin JM, Jones JF, Reeves WC, Heim C. Attenuated Morning Salivary Cortisol Concentrations in a Population-based Study of Persons with Chronic Fatigue Syndrome and Well Controls. J Clin Endocrinol Metab. 2007 Dec 26

On 2014 Feb 28, Tom Kindlon commented:

Why is this definition being referred to as an "empirical definition"?

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#301566 but the formatting has gone from that page)

I believe most people's understanding of "empirical criteria" or an "empirical definition" would be that the data would speak for itself; it "would decide" the cut-off points through methods such as cluster analysis (for example).

Indeed this would seem to have been William Reeves' understanding of an empirical definition. For example, in a presentation on the CDC's CFS research program (to a Task Force Meeting on the Epidemiology of Interstitial Cystitis)[1], he said: "The problem with the CFS criteria was that they were not specific enough and not empiric-based. For example, one of the criteria stated that the research subject must have at least four of eight symptoms, among them, impaired concentration or memory and postexertional worsening of physical or mental fatigue. "The accompanying symptoms need to be defined in and of themselves," Dr. Reeves said.

The 1994 International Study Group also hypothesized that fatigue led to patients' symptoms rather than the reverse. The CDC is currently conducting population studies to develop an empiric definition of CFS that is based on statistical modeling."At the inaugural meeting of the US Department of Health and Human Services' Chronic Fatigue Syndrome Advisory Committee (CFSAC), Dr Reeves said the CDC team of research would "derive an empirical case definition based on data".[2]

The definition presented here does not seem to have been based either on "statistical modeling" or "data". It seems to involve relatively arbitrary cut-off points; for example, of the 8 subscales of the SF-36, four are chosen and, for each of these, the 25th percentile of the published US population is chosen as a cut-off point. A patient is required to be in the bottom quartile for just one of these subscales to satisfy the criteria. Where did this cut-off point come from? There is no mention of it in the paper that suggested the use of the SF-36[3]; nor is there any mention that these particular subscales should be chosen or that one would be sufficient. One of the authors of the paper[3] has confirmed that cut-off points were never chosen nor was it decided which sub-scales would be used.

Given that the CDC's definition of CFS tends to go on to be used in numerous studies, would it not be better to investigate which thresholds give a "better" definition e.g. with a higher specificity and sensitivity - for example, for some of the SF-36 subscales, perhaps (say) the 13th, 15th, 20th or even 30th percentiles may be more appropriate.

The cut-off points suggested in this paper may or may not be useful. But is it really accurate to suggest that they are "empirically" derived?

References:

[1] Epidemiology of Interstitial Cystitis - Executive Committee Summary and Task Force Meeting Report October 29th, 2003. http://www.niddk.nih.gov/fund/reports/ic/task_force_summary.pdf

[2] US Department of Health and Human Services - Chronic Fatigue Syndrome Advisory Committee (CFSAC). Inaugural Meeting. September 29th, 2003Meeting Summary. http://www.hhs.gov/advcomcfs/CSFAC_mins_2003.09.29R.pdf

[3] Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason LA, Bleijenberg G, Evengard B, White PD, Nisenbaum R, Unger ER, International Chronic Fatigue Syndrome Study Group: Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution. BMC Health Services Research 2003, 3:25

On 2014 Feb 28, Tom Kindlon commented:

Using two MFI scales ("General Fatigue" or "Reduced Activity") to ensure patients satisfying the definition have "severe fatigue"

(This was originally posted here http://www.biomedcentral.com/1741-7015/3/19/comments#304567 but the formatting has gone from that page)

Initially when I read this paper, where it says "we defined severe fatigue as >= medians of the MFI general fatigue (>=13) or reduced activity (>=10) scales", I thought this referred to medians of the general population.Hearing other people commenting on it, that's how some other people have been interpreting it also. It is probably somewhat natural to do this as the sentence before reads: "We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [11] on the physical function (<=70), or role physical (<=50), or social function (<=75), or role emotional (<=66.7) subscales of the SF-36."

However from looking at the scores for controls in other papers, these MFI scores do not look like medians for the whole US population but in fact are medians for this particular group of patients. This seems a strange way to set cut-off points for a CFS definition that is used for numerous studies into the illness, given the cohort that is being used as a basis: "This population-based case control study enrolled 227 adults identified from the population of Wichita with: (1) CFS (n = 58); (2) non-fatigued controls matched to CFS on sex, race, age and body mass index (n = 55); (3) persons with medically unexplained fatigue not CFS, which we term ISF (n = 59); (4) CFS accompanied by melancholic depression (n = 27); and (5) ISF plus melancholic depression (n= 28)." i.e. this is not a random sample of the US population but a group of people selected for a specific purpose (or purposes) (not necessarily to design a definition, but as a follow-up study of people previously diagnosed with CFS or given some other label).

Some of the groups are of different sizes - if the relative size of these groups had been changed, with relatively more people taken from some classification groups and less people taken from other groups, the median scores would likely have been different.

It should also be remembered that in this context the categories listed in the last paragraph refer to their classification when they evaluated years before (from 1997 to 2000), and not necessarily at the time when they were evaluated in this study (December 2002 to July 2003) (as is clear from the tables in this paper).

I thought it would be interesting to look at MFI scores in some other papers on CFS that did not use the "empirical definition".

I don't claim this is a definitive list but, at the same time, mean MFI scores with standard deviations only seem to be listed in a small percentage of papers. The papers use cohorts from a variety of locations: England [3], The Netherlands [4], Germany [5] and the USA (New Jersey) [6]. I did not see any ranges given which would be useful given the task at hand (selecting cut-off points for a definition).Unfortunately not all of the papers I found used the Fukuda [1] definition for CFS; some also used the Sharpe [2] definition for CFS. I indicate which definition is used in each case.

MFI: General Fatigue (i) Sample/(ii) Sample Size/(iii) Mean/(iv) SD/ (v) (Mean - 13)/SD/ (vi) Definition

Weatherley-Jones [3] 53 18.4 1.7 3.176470588 Sharpe (1991)

Vermeulen (Group 1) [4] 30 18.6 1.9 2.947368421 Fukuda (1994)

Vermeulen (Group 2) [4] 30 18.4 1.8 3 Fukuda (1994)

Vermeulen (Group 3) [4] 30 19.1 1.4 4.357142857 Fukuda (1994)

Gaab [5] 21 17.7 0.5 9.4 Sharpe (1991) and Fukuda (1994)

Brimacombe [6] 65 18.41 2.02 2.678217822 Fukuda (1994)Combining these give a sample of 229 patients with a mean "General Fatigue" score of 18.45655022.

This data suggests that a threshold of >=13 will have a very very high sensitivity. This would suggest that another measure would not be necessary (unless it was being used as an extra criterion to increase the specificity, which isn't done with this definition).

However for completeness, I'm including the "Reduced Activity" data from the same papers:Reduced activity (MFI) (i) Sample / (ii) Sample Size / (iii) Mean Score / (iv) SE (v) (Mean-10)/SD (vi) Definition

Weatherley-Jones [3] 53 16.1 3.1 1.967741935 Sharpe(1991)

Gaab [5] 21 15 0.7 8.714285714 Sharpe (1991) and Fukuda(1994)

Brimacombe [6] 65 15.93 4.55 1.340659341 Fukuda 1994

Combining these give a sample of 139 patients with a mean Reduced Activity score of 15.85431655.

Note: the Vermeulen paper[4] did not collect the MFI scores for Reduced Activity, just "the fatigue axes of the Multidimensional Fatigue Inventory" (which they defined as the MFI scores for General fatigue, Physical fatigue, Mental fatigue).

It seems strange in the definition of Chronic Fatigue Syndrome defined in this paper (i.e. Reeves et al) that the "severe fatigue" criterion can be satisfied by a patient having a low score on a subscale of the MFI testing activity levels (as opposed to one of the 3 subscales measuring fatigue), especially when the function of the SF-36 is to "measure functional impairment". Just because someone is inactive doesn't mean they have severe fatigue. Allowing patients to be included if they simply have a "Reduced Activity" score of 10 or more (without necessarily having a low score on one of the fatigue axes of the MFI) risks reducing the specificity of the definition.

References:

[1] Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A. The chronic fatigue syndrome; a comprehensive approach to its definition and study. Ann Int Med 1994, 121:953-959.

[2] Sharpe MC, Archard LC, Banatvala JE, Borysiewicz LK, Clare AW, David A, Edwards RH, Hawton KE, Lambert HP, Lane RJ, et al. A report--chronic fatigue syndrome: guidelines for research. J R Soc Med. 1991 Feb;84(2):118-21.

[3] Weatherley-Jones, E., Nicholl, JP., Thomas, KJ., Parry, GJ., McKendrick, MW., Green, ST., Stanley, PJ and Lynch, SPJ. A randomised, controlled, triple-blind trial of the efficacy of homeopathic treatment for chronic fatigue syndrome. Journal of Psychosomatic Research, 2004, 56, 2, 189-197.

[4] Vermeulen, RCW and Scholte, HR. Exploratory open label, randomized study of acetyl- and propionylcarnitine in chronic fatigue syndrome. Psychosomatic Medicine, 2004, 66, 276-282.

[5] Gaab J, Hüster D, Peisen R, Engert V, Heitz V, Schad T, Schürmeyer TH, Ehlert U. Hypothalamic-pituitary-adrenal axis reactivity in chronic fatigue syndrome and health under psychological, physiological, and pharmacological stimulation.Psychosom Med. 2002 Nov-Dec;64(6):951-62.

[6] Brimacombe, Michael; Lange, Gudrun; Bisuchio, Kim; Ciccone, Donald S.; Natelson, Benjamin. Cognitive Function Index for Patients with Chronic Fatigue Syndrome Journal of Chronic Fatigue Syndrome, 2004, vol 12; number 4, pages 3-24

On 2016 Jul 21, Tom Kindlon commented:

Another study raises serious questions about the use of the role emotional subscale of the SF-36 in the diagnosis of CFS

A CDC group (Reeves et al (2005)) proposed a way to operationalise chronic fatigue syndrome (CFS) criteria including subscales of the SF-36[1,2]: "We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [] on the physical function (≤ 70), or role physical (≤ 50), or social function (≤ 75), or role emotional (≤ 66.7) subscales of the SF-36." This method has been used in numerous subsequent CDC CFS studies but not by external groups, apart from Leonard Jason and colleagues who have examined the effect of this operationalisation of the criteria.

A newly published study looked at using the SF 36 subscales to define reduced activity in CFS (and ME)[3]. Six of the 8 subscales (Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality and Social Functioning) correlated with the sum of current hours of activity. Two subscales did not correlate (Role-Emotional and Mental Health).

The same 6 subscales (negatively) correlated with percentage reduction in the total hours of activity. Again, two subscales did not correlate (Role-Emotional and Mental Health).

Also, a "ROC analysis was used to assess which of the SF-36 subscales best discriminated between patients with ME and CFS and healthy controls. The ROC analysis used a plot of sensitivity versus 1-specificity for scores on the SF-36. The area under the curve (AUC) assessed the degree to which each subscale was able to discriminate between patients and controls. The closer the AUC is to 1, the better discriminatory power that SF-36 subscale has." The same 6 subscales (Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality and Social Functioning) had AUCs of 0.89 or better. By comparison, the Mental Health Functioning subscale had a AUC of 0.59. The AUC for the Role Emotional subscale was even worse at 0.47.

Each of these results demonstrates that the Role Emotional subscale is not suitable to be used to operationalise criteria for CFS.

References:

[1] Reeves WC, Wagner D, Risenbaum R, et al. Chronic fatigue syndrome – a clinically empirical approach to its definition and study. BMC Med. 2005;3(19):1–9.

[2] Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): conceptual framework and item selection. Med Care. 1992;30:473–483

[3] Thorpe T, McManimen S, Gleason K, Stoothoff J, Newton J, Strand EB, Jason LA (2016). Assessing current functioning as a measure of significant reduction in activity level. Fatigue: Biomedicine, Health & Behavior. Published online: 19 Jul 2016 doi: 10.1080/21641846.2016.1206176