On 2014 Jan 08, Tom Kindlon commented:

Thresholds for recovery were set very very "low" (perhaps the bottom percentile and 5th lowest percentile of the healthy adult population on the two scales used)

(I originally posted this as a comment here: http://www.biomedcentral.com/1472-6963/8/175/comments. However all the paragraph breaks have been deleted so I doubt many would read it there)

The thresholds for recovery seem very very low: "Patients were defined as being CSI at post treatment if they had a reliable change index > 1.96 on the CIS fatigue severity subscale [22], a fatigue severity score <= 35 and a Rand-36 physical functioning score > = 65".

Many of the patients already likely had a "physical functioning score >=65" given the mean (SD) values before treatment were: "Physical impairment (Rand 36) 54.0(23.4)" And the threshold for recovery was only 0.47 SDs above the initial mean score. I am aware of the questions on the SF-36 PF subscale (scores can range from 0 to 100 with the higher the score, the better their "physical functionaling") and I don't believe most healthy adults would believe scoring 65 on the SF-36 PF scale would mean they were recovered. As a study[1], that was co-written by one of the authors of this study (Gijs Bleijenberg), pointed out, a community study found that "healthy adults without a chronic condition" had "a mean score of 93.1 (SD 11.7)." The authors of that study[1] pointed out they did not know the exact distribution of the SF-36 subscales - they just made the assumption that the mean - 1SD would represent a threshold for the 85th percentile and rounded this figure to 80.The threshold in the current study is 65. That is 2.4 SDs below the healthy population's mean score. If the same assumptions were made (i.e. that the curve was normally distributed), this would represent the bottom percentile!

For the CIS fatigue severity subscale (where the possible scores are 8-56 with the higher the score, the greater the fatigue), that same study that Gijs Bleijenberg co-wrote[2] used (to calculate thresholds i.e. from another study) a "normal group of 53 healthy adults with a mean age of 37.1 (SD 11.5)" who had "a mean score on the CIS-fatigue of 17.3 (SD 10.1)."[3] The ages of those healthy adults are similar to the ages of the CFS patients in this study: Mean (SD) 38.1 (10.2). In that study[1], they estimated that the 85th percentile (mean+1SD) would be 27 (due to rounding). This study uses 35 or the mean + 1.7525SD or the 95th percentile. Put another way, patients in this study could be considered recovered if they scored in the bottom percentile on the physical functioning subscale (of the SF-36) and in the 5th lowest percentile on the CIS-fatigue scale!

References:

[1] Knoop H, Bleijenberg G, Gielissen MF, van der Meer JW, White PD. Is a full recovery possible after cognitive behavioural therapy for chronic fatigue syndrome? Psychother Psychosom. 2007;76(3):171-6.

[2] Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, te Velde A, Verrips E: Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease population. J Clin Epidemiol 1998; 51: 1055-1068.

[3] Vercoulen JHMM, Alberts M, Bleijenberg G: De Checklist Individual Strength (CIS) (The Checklist Individual Strength). Gedragstherapie (Behavioural Therapy) 1999; 32: 642-649.

On 2013 Oct 24, Tom Kindlon commented:

Another paper by the authors provides information on the effects of self-efficacy and fatigue severity on health use

I find it strange that in this paper, there is no mention of: "Determinants of health care use in chronic fatigue syndrome patients: a cross-sectional study." That study was written the four authors of the current study plus one other person.¹

I'm far from an expert on health economics but it seems to have information and data relevant to the current study or at least worthy of mention in the discussion section.

They found for example, that: "self-efficacy showed a positive instead of a negative relation with health care use. We checked the direct correlation between self-efficacy and health care use, which also appeared to be positive (Pearson's R=0.12, p=.04). It thus seems that using more health care services might form an aspect of, or is stimulated by, a high self-efficacy instead of being the result of a low self-efficacy."

One of the aims of CBT for CFS is to increase self-efficacy. As they say in the paper: "Subsequently, dysfunctional fatigue related cognitions are being challenged to diminish somatic attributions of fatigue, to improve a sense of control over symptoms and to facilitate behavior change. Finally a plan for work rehabilitation is outlined and worked out. Patients without a paid job focus on rehabilitation in other personal activities. The last session deals with relapse prevention and further improvement of self-control."

Self-efficacy is a commonly used term in the literature on CBT for CFS.

That study¹ also found that: "Fatigue severity itself showed no relation with health care use." So improving fatigue scores will not necessarily change health care use.

References:

[1] Scheeres K, Wensing M, Severens H, Adang E, Bleijenberg G. Determinants of health care use in chronic fatigue syndrome patients: a cross-sectional study. J Psychosom Res. 2008 Jul;65(1):39-46.

On 2013 Oct 24, Tom Kindlon commented:

(This follows on from an earlier comment that I thought was already long enough)

Some of the calculations in this study use in the study use the recovery rate of 37%.

For example: "Given the recovery rate of 37% the COR of implementing CBT for CFS was 5.320 per recovered CFS patient. The COR acceptability curve (figure 3) shows that the probability that implementing CBT for CFS has a favorable COR is 100% when the decision maker values a recovered CFS patient at least 6.500."

They become very different if the threshold for recovery was much different.

Alternatively, with the lax definition for recovery they used, the amount of people who would have "recovered" without treatment may be a lot higher than the 5% assumed in this study.

"Finally, to get an impression of this study's results when compensating for spontaneous recovery, an additional analysis was performed. This was done from the health care perspective, presuming a spontaneous recovering rate of 5% [2], implying a recovery rate due to treatment of 32%. It revealed that the COR would rise from 5.320 to about 5.969 per recovered patient."

I also don't understand why a "recovered group" should have a much worse average score in a domain e.g. around the lower end of any scale. Surely the null hypothesis would be that there is no difference between the means and if this is not satisfied, it's not a "recovered group". What could be done would be that some people would have to be taken out of the "recovered group" until one got to a situation where the mean of the recovered group was within a confidence interval for the mean of the normal population (the variance of a mean is of course much smaller than the variance of an individual entry i.e. the average value would have to be "pretty close" to the mean of the healthy population). Of course, it could be the case that there would only be a recovered group of 1 (say) as none of the values were above the mean.

On 2013 Oct 24, Tom Kindlon commented:

(contd.)

In a review of CFS treatments, Whiting et al⁶ recommended objective outcome measures be used: "Outcomes such as "improvement," in which participants were asked to rate themselves as better or worse than they were before the intervention began, were frequently reported. However, the person may feel better able to cope with daily activities because they have reduced their expectations of what they should achieve, rather than because they have made any recovery as a result of the intervention. A more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

Actometers or pedometers were not used in the study. So by the measures the review⁶ suggested, this paper doesn't prove that CBT is an "effective treatment" let alone that it leads to a recovery rate of 37%.

This review⁶ also recommended long-term follow-up:

"The relapsing nature of CFS suggests that follow-up should continue for at least an additional 6 to 12 months after the intervention period has ended, to confirm that any improvement observed was due to the intervention itself and not just to a naturally occurring fluctuation in the course of the illness."

This also brings up the question of recovery being assessed by questionnaires at one point at time. This seems a strange way to define recovery especially in a condition with a condition like CFS where the symptoms can intensify and reduce over a period.

So based on the information in my two comments, I think it's difficult to sustain that 37% of the patients recovered.

References:

1 Kindlon T. Thresholds for recovery were set very very "low" (perhaps the bottom percentile and 5th lowest percentile of the healthy adult population on the two scales used). Comment at: http://www.biomedcentral.com/1472-6963/8/175/comments

2 Wessely, S: Chronic Fatigue Syndrome-Trials and Tribulations. JAMA.2001; 286: 1378-1379.

3 Malouff, J. M., et al., Efficacy of cognitive behavioral therapy for chronic fatigue syndrome: A meta-analysis. Clinical Psychology Review (2007), doi:10.1016/j.cpr.2007.10.004

4 Cohen J: Statistical power analysis for the behavioural sciences. Edited by: 2. New Jersey: Lawrence Erlbaum; 1988.

5 Deale A, Husain K, Chalder T, Wessely S. Long-term outcome of cognitive behavior therapy versus relaxation therapy for chronic fatigue syndrome: a 5-year follow-up study. Am J Psychiatry. 2001 Dec;158(12):2038-42.

6 Whiting P, Bagnall AM, Sowden AJ, Cornell JE, Mulrow CD, Rammrez G.Interventions for the treatment and management of chronic fatigue syndrome: a systematic review. JAMA. 2001;286:1360-1368 http://jama.ama-assn.org/cgi/content/full/286/11/1360

On 2013 Oct 24, Tom Kindlon commented:

Information on occupational performance does not suggest 37% of the patients were recovered

In a comment,¹ I pointed out that the thresholds for recovery using were set very low. However it is useful to investigate the other data to see what it might say about recovery rates. Also other data from other studies.

This paper claims that CBT led to a recovery rate of 37% in CFS patients. That is one of the highest if not the highest recovery rate I can recall from a CBT study.

It is my impression that, outside the Netherlands, claims of CBT leading to recovery in CFS are not that common. For example, Prof. Simon Wessely, who could be said to be one of the chief proponents of CBT for CFS worldwide over the last couple of decades has said that CBT is not "remotely curative".²

Recently, a meta-analysis of the efficacy of CBT for CFS was published.³ The studies involved a total of 1371 patients. This involved calculating the size of an effect measure, the Cohen's d value.

They calculated d using the following method: "Separate mean effect sizes were calculated for each category of outcome variable (e.g., fatigue self- rating) and for each type of outcome variable (mental, physical, and mixed mental and physical). Studies generally included multiple outcome measures. For all analyses except those that compared different categories or types of outcome variables, we used the mean effect size of all the relevant outcome variables of the study."

d was calculated to be 0.48.

For anyone unfamiliar with Cohen's d values, they are not bounded by 1; also, the higher the score, the bigger the "effect size" i.e. the more "effective" a treatment was found to be. Cohen's d values are considered to be a small effect size at 0.2, a moderate effect size at 0.5, and a large effect size at 0.8.⁴

So this suggests CBT leading to a 37% recovering would be unusual.

I noticed in the discussion section, we are told: "Concerning work productivity, fewer patients had a paid job after treatment than before, but the mean hours of paid work per week had increased after treatment." We are not given the percentage of patients who work.

What we are told is that the median number of hours actually worked before the treatment was 0 hours. That means that at least 50% of the patients were working 0 hours per week. For the whole group, they worked a mean number of 9.4 hours. Some of these people who were not actually working were actually in paid employment at the start as the median number of hours they were contracted to work was 7 hours. Like most studies of adults with CFS, these patients were of working age. The mean age was 38.1 with a standard deviation of 10.2. 34% were male (larger than most studies) and 66% were female.

Given the sample group, one would think that an "effective" treatment for CFS which is supposed to improve functioning and which the authors claim led to a recovery rate of 37% would dramatically effect the professional functioning of the patients.

Indeed at least one study of CBT has used hours worked in its definition of recovery: "Predetermined criteria for "complete recovery" required that patients no longer met chronic fatigue syndrome criteria, were employed full-time, and scored less than 4 on the Fatigue Questionnaire and more than 83 on the Medical Outcomes Study Short-Form General Health Survey physical functioning scale."

However in the current study we find that the mean number of hours work they are contracted to work actually decreased from a mean (SD) of 16.2 (16.3) to 14.9 (16.2) and given that median number of hours actually worked after the intervention is 0 hours, more than 50% of the patients are still not working at the end.

[The number of hours actually worked did increase from a mean (SD) of 9.4 (13.5) to 11.4 (14.7) but a quick t-test suggests this isn't statistically significant].

(contd.)

On 2013 Oct 24, Tom Kindlon commented:

Information on occupational performance does not suggest 37% of the patients were recovered

In a comment,¹ I pointed out that the thresholds for recovery using were set very low. However it is useful to investigate the other data to see what it might say about recovery rates. Also other data from other studies.

This paper claims that CBT led to a recovery rate of 37% in CFS patients. That is one of the highest if not the highest recovery rate I can recall from a CBT study.

It is my impression that, outside the Netherlands, claims of CBT leading to recovery in CFS are not that common. For example, Prof. Simon Wessely, who could be said to be one of the chief proponents of CBT for CFS worldwide over the last couple of decades has said that CBT is not "remotely curative".²

Recently, a meta-analysis of the efficacy of CBT for CFS was published.³ The studies involved a total of 1371 patients. This involved calculating the size of an effect measure, the Cohen's d value.

They calculated d using the following method: "Separate mean effect sizes were calculated for each category of outcome variable (e.g., fatigue self- rating) and for each type of outcome variable (mental, physical, and mixed mental and physical). Studies generally included multiple outcome measures. For all analyses except those that compared different categories or types of outcome variables, we used the mean effect size of all the relevant outcome variables of the study."

d was calculated to be 0.48.

For anyone unfamiliar with Cohen's d values, they are not bounded by 1; also, the higher the score, the bigger the "effect size" i.e. the more "effective" a treatment was found to be. Cohen's d values are considered to be a small effect size at 0.2, a moderate effect size at 0.5, and a large effect size at 0.8.⁴

So this suggests CBT leading to a 37% recovering would be unusual.

I noticed in the discussion section, we are told: "Concerning work productivity, fewer patients had a paid job after treatment than before, but the mean hours of paid work per week had increased after treatment." We are not given the percentage of patients who work.

What we are told is that the median number of hours actually worked before the treatment was 0 hours. That means that at least 50% of the patients were working 0 hours per week. For the whole group, they worked a mean number of 9.4 hours. Some of these people who were not actually working were actually in paid employment at the start as the median number of hours they were contracted to work was 7 hours. Like most studies of adults with CFS, these patients were of working age. The mean age was 38.1 with a standard deviation of 10.2. 34% were male (larger than most studies) and 66% were female.

Given the sample group, one would think that an "effective" treatment for CFS which is supposed to improve functioning and which the authors claim led to a recovery rate of 37% would dramatically effect the professional functioning of the patients.

Indeed at least one study of CBT has used hours worked in its definition of recovery: "Predetermined criteria for "complete recovery" required that patients no longer met chronic fatigue syndrome criteria, were employed full-time, and scored less than 4 on the Fatigue Questionnaire and more than 83 on the Medical Outcomes Study Short-Form General Health Survey physical functioning scale."

However in the current study we find that the mean number of hours work they are contracted to work actually decreased from a mean (SD) of 16.2 (16.3) to 14.9 (16.2) and given that median number of hours actually worked after the intervention is 0 hours, more than 50% of the patients are still not working at the end.

[The number of hours actually worked did increase from a mean (SD) of 9.4 (13.5) to 11.4 (14.7) but a quick t-test suggests this isn't statistically significant].

(contd.)

On 2013 Oct 24, Tom Kindlon commented:

(This follows on from an earlier comment that I thought was already long enough)

Some of the calculations in this study use in the study use the recovery rate of 37%.

For example: "Given the recovery rate of 37% the COR of implementing CBT for CFS was 5.320 per recovered CFS patient. The COR acceptability curve (figure 3) shows that the probability that implementing CBT for CFS has a favorable COR is 100% when the decision maker values a recovered CFS patient at least 6.500."

They become very different if the threshold for recovery was much different.

Alternatively, with the lax definition for recovery they used, the amount of people who would have "recovered" without treatment may be a lot higher than the 5% assumed in this study.

"Finally, to get an impression of this study's results when compensating for spontaneous recovery, an additional analysis was performed. This was done from the health care perspective, presuming a spontaneous recovering rate of 5% [2], implying a recovery rate due to treatment of 32%. It revealed that the COR would rise from 5.320 to about 5.969 per recovered patient."

I also don't understand why a "recovered group" should have a much worse average score in a domain e.g. around the lower end of any scale. Surely the null hypothesis would be that there is no difference between the means and if this is not satisfied, it's not a "recovered group". What could be done would be that some people would have to be taken out of the "recovered group" until one got to a situation where the mean of the recovered group was within a confidence interval for the mean of the normal population (the variance of a mean is of course much smaller than the variance of an individual entry i.e. the average value would have to be "pretty close" to the mean of the healthy population). Of course, it could be the case that there would only be a recovered group of 1 (say) as none of the values were above the mean.

On 2013 Oct 24, Tom Kindlon commented:

Another paper by the authors provides information on the effects of self-efficacy and fatigue severity on health use

I find it strange that in this paper, there is no mention of: "Determinants of health care use in chronic fatigue syndrome patients: a cross-sectional study." That study was written the four authors of the current study plus one other person.¹

I'm far from an expert on health economics but it seems to have information and data relevant to the current study or at least worthy of mention in the discussion section.

They found for example, that: "self-efficacy showed a positive instead of a negative relation with health care use. We checked the direct correlation between self-efficacy and health care use, which also appeared to be positive (Pearson's R=0.12, p=.04). It thus seems that using more health care services might form an aspect of, or is stimulated by, a high self-efficacy instead of being the result of a low self-efficacy."

One of the aims of CBT for CFS is to increase self-efficacy. As they say in the paper: "Subsequently, dysfunctional fatigue related cognitions are being challenged to diminish somatic attributions of fatigue, to improve a sense of control over symptoms and to facilitate behavior change. Finally a plan for work rehabilitation is outlined and worked out. Patients without a paid job focus on rehabilitation in other personal activities. The last session deals with relapse prevention and further improvement of self-control."

Self-efficacy is a commonly used term in the literature on CBT for CFS.

That study¹ also found that: "Fatigue severity itself showed no relation with health care use." So improving fatigue scores will not necessarily change health care use.

References:

[1] Scheeres K, Wensing M, Severens H, Adang E, Bleijenberg G. Determinants of health care use in chronic fatigue syndrome patients: a cross-sectional study. J Psychosom Res. 2008 Jul;65(1):39-46.

On 2014 Jan 08, Tom Kindlon commented:

Thresholds for recovery were set very very "low" (perhaps the bottom percentile and 5th lowest percentile of the healthy adult population on the two scales used)

(I originally posted this as a comment here: http://www.biomedcentral.com/1472-6963/8/175/comments. However all the paragraph breaks have been deleted so I doubt many would read it there)

The thresholds for recovery seem very very low: "Patients were defined as being CSI at post treatment if they had a reliable change index > 1.96 on the CIS fatigue severity subscale [22], a fatigue severity score <= 35 and a Rand-36 physical functioning score > = 65".

Many of the patients already likely had a "physical functioning score >=65" given the mean (SD) values before treatment were: "Physical impairment (Rand 36) 54.0(23.4)" And the threshold for recovery was only 0.47 SDs above the initial mean score. I am aware of the questions on the SF-36 PF subscale (scores can range from 0 to 100 with the higher the score, the better their "physical functionaling") and I don't believe most healthy adults would believe scoring 65 on the SF-36 PF scale would mean they were recovered. As a study[1], that was co-written by one of the authors of this study (Gijs Bleijenberg), pointed out, a community study found that "healthy adults without a chronic condition" had "a mean score of 93.1 (SD 11.7)." The authors of that study[1] pointed out they did not know the exact distribution of the SF-36 subscales - they just made the assumption that the mean - 1SD would represent a threshold for the 85th percentile and rounded this figure to 80.The threshold in the current study is 65. That is 2.4 SDs below the healthy population's mean score. If the same assumptions were made (i.e. that the curve was normally distributed), this would represent the bottom percentile!

For the CIS fatigue severity subscale (where the possible scores are 8-56 with the higher the score, the greater the fatigue), that same study that Gijs Bleijenberg co-wrote[2] used (to calculate thresholds i.e. from another study) a "normal group of 53 healthy adults with a mean age of 37.1 (SD 11.5)" who had "a mean score on the CIS-fatigue of 17.3 (SD 10.1)."[3] The ages of those healthy adults are similar to the ages of the CFS patients in this study: Mean (SD) 38.1 (10.2). In that study[1], they estimated that the 85th percentile (mean+1SD) would be 27 (due to rounding). This study uses 35 or the mean + 1.7525SD or the 95th percentile. Put another way, patients in this study could be considered recovered if they scored in the bottom percentile on the physical functioning subscale (of the SF-36) and in the 5th lowest percentile on the CIS-fatigue scale!

References:

[1] Knoop H, Bleijenberg G, Gielissen MF, van der Meer JW, White PD. Is a full recovery possible after cognitive behavioural therapy for chronic fatigue syndrome? Psychother Psychosom. 2007;76(3):171-6.

[2] Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, te Velde A, Verrips E: Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease population. J Clin Epidemiol 1998; 51: 1055-1068.

[3] Vercoulen JHMM, Alberts M, Bleijenberg G: De Checklist Individual Strength (CIS) (The Checklist Individual Strength). Gedragstherapie (Behavioural Therapy) 1999; 32: 642-649.