On 2016 Feb 16, Robert Courtney commented:

A study that uses questionable assumptions rather than empirical evidence leads to conclusions that stretch credibility.

Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.

Chalder et al. were working to this model and acknowledged that they failed to establish a clear temporal separation between variables, and therefore did not empirically establish a causal mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.

However, despite the lack of robust empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”

The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by unhelpful or maladaptive illness beliefs, fear, and an avoidance of activity.

However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.

Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.

Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. (e.g. This would mean that a decrease in activity would cause an increase in symptoms.) However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that increased symptoms would be the direct cause of activity avoidance in any illness, rather than vice versa. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.

In the case of fatigue, flu-like malaise and other symptoms of ME/CFS, it seems reasonable to consider the possibility that, as the symptoms fluctuate, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.

Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.

References:

1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.

2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.

3. IOM (Institute of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press. http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx

4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.

On 2015 Sep 15, Tom Kindlon commented:

(Contd.)

References:

1 Torjesen I. Tackling fears about exercise is important for ME treatment, analysis indicates. BMJ 2015;350:h227 http://www.bmj.com/content/350/bmj.h227

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 14 Jan 2015, doi:10.1016/S2215-0366(14)00069-8.

3 Burgess M, Chalder T. Manual for Participants. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-participant-manual.pdf (accessed: January 17, 2015)

4 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME. Information for Participants http://www.pacetrial.org/docs/get-participant-manual.pdf (accessed: January 17, 2015)

5 Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Kirsch I, Israel E, Kaptchuk TJ. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. N Engl J Med. 2011;365(2):119-26.

6 Whiting P, Bagnall AM, Sowden AJ, Cornell JE, Mulrow CD, Ramírez G. Interventions for the treatment and management of chronic fatigue syndrome: a systematic review. JAMA. 2001 Sep 19;286(11):1360-8.

7 Burgess M, Chalder T. PACE manual for therapists. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-therapist-manual.pdf (accessed: January 17, 2015)

8 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al, for the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.

9 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808. doi: 10.1371/journal.pone.0040808

10 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212. Epub 2010 Jan 5.

11 Heins MJ, Knoop H, Burk WJ, Bleijenberg G. The process of cognitive behaviour therapy for chronic fatigue syndrome: which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue? J Psychosom Res. 2013 Sep;75(3):235-41. doi: 10.1016/j.jpsychores.2013.06.034. Epub 2013 Jul 19.

12 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Apr;65(4):423-42. doi: 10.1002/jclp.20551.

13 Knoop H, Prins JB, Stulemeijer M, van der Meer JW, Bleijenberg G. The effect of cognitive behaviour therapy for chronic fatigue syndrome on self-reported cognitive impairments and neuropsychological test performance. Journal of Neurology and Neurosurgery Psychiatry. 2007 Apr;78(4):434-6.

14 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist manual): http://www.pacetrial.org/docs/get-therapist-manual.pdf (accessed: January 17, 2015)

15 O'Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme. Health Technology Assessment, 2006, 10, 37, 1-140.

16 Knoop H, Wiborg JF. What makes a difference in chronic fatigue syndrome? Lancet Psychiatry 13 Jan 2015 DOI: http://dx.doi.org/10.1016/S2215-0366(14)00145-X

17 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bulletin of the IACFS/ME. 2011;19(2):59-111http://iacfsme.org/BULLETINFALL2011/Fall2011KindlonHarmsPaperABSTRACT/ta...

18 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. Six minute walking test for assessing exercise capacity in chronic heart failure. Br Med J (Clin Res Ed) 1986. 292:653–655. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1339640/pdf/bmjcred00224-001...

19 Marin JM, Carrizo SJ, Gascon M, Sanchez A, Gallego B, Celli BR. Inspiratory Capacity, Dynamic Hyperinflation, Breathlessness, and Exercise Performance during the 6-Minute-Walk Test in Chronic Obstructive Pulmonary Disease. Am. J. Respir. Crit. Care Med. 2001 63(6):1395-1399.http://171.66.122.149/content/163/6/1395.full

20 Goldman MD, Marrie RA, Cohen JA. Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Multiple Sclerosis 2008. 14(3):383-390. http://pocketknowledge.tc.columbia.edu/home.php/viewfile/download/65399/The six-minute walk test.pdf

21 Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake. BMC Pulm Med. 2010 May 26;10:31. PMID 20504351.http://www.biomedcentral.com/1471-2466/10/31

22 Camarri B, Eastwood PR, Cecins NM, Thompson PJ, Jenkins S. Six minute walk distance in healthy subjects aged 55–75 years. Respir Med. 2006. 100:658-65 http://www.resmedjournal.com/article/S0954-6111(05)00326-4/abstract

23 Troosters T, Gosselink R, Decramer M. Six minute walking distance in healthy elderly subjects. Eur Respir J. 1999. 14:270-4. http://www.ersj.org.uk/content/14/2/270.full.pdf

24 Rapport d'évaluation (2002-2004) portant sur l'exécution des conventions de rééducation entre le Comité de l'assurance soins de santé (institué auprès de l'Institut national d'assurance maladie invalidité) et les Centres de référence pour le Syndrome de fatigue chronique (SFC), Bruxelles, juillet 2006. (French language edition)

25 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. Available online:https://drive.google.com/file/d/0BxnVj9ZqRgk0QTVsU2NNLWJSblU/edit (accessed: January 17, 2015) (Dutch language version)

26 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58)https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf (accessed: January 17, 2015)

On 2015 Sep 15, Tom Kindlon commented:

Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial

[Originally posted here: http://www.bmj.com/content/350/bmj.h227/rr-10]

This BMJ article and a flurry of articles in the lay media this week followed the publication in Lancet Psychiatry of an analysis of the mediators of change in the important PACE Trial, a chronic fatigue syndrome (CFS) trial which cost UK taxpayers £5 million[1,2]. What seems to have been lost in the coverage is that, although there were some modest improvements in the self-report measures, there was an almost complete absence of improvements in objectively measured outcomes for cognitive behavioural therapy (CBT) and graded exercise therapy (GET) compared to the control group (specialist medical care only (SMC)).

This is a non-blinded trial, where participants were told CBT and GET had previously been found to be effective in CFS and other conditions[3,4]: one way to look at the mediation results for subjective measures when there was a lack of objective improvements is that they may merely tell us how response biases and/or placebo effects are mediated[5].

The focus on subjective measures in some CFS studies was previously criticised in a systematic review published back in 2001 (long before the PACE Trial started)[6]. They suggested instead "a more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

The model presented for cognitive behaviour therapy (CBT) in the PACE Trial manuals posits that the impairments and symptoms are reversible with the therapy[3,7]. However, the latest paper shows that fitness, as measured by a step test, didn't improve following CBT[2]. An earlier PACE Trial publication reported that the addition of CBT to SMC did not result in an improvement in 6-minute walking test scores compared to SMC alone[8].

The PACE Trial was part funded by the UK Department of Work and Pensions, a rare move for them, presumably done due to an expectation that the therapies would improve measures of employment and levels of benefit receipt. However, again CBT brought about no improvement using objective measures, such as days of employment lost, levels of disability benefits received and levels of receipt of insurance payments[9].

These results are in line with earlier studies of CBT. For example, an analysis of three randomized controlled trials of CBT interventions for CFS found no improvement in objectively measured activity, despite participants reporting a reduction in (self-reported) fatigue and (sometimes) functional impairments[10]. Similar results were found in another uncontrolled trial where changes in objectively measured activity did not predict fatigue levels, and objectively measured activity on completion remained low compared to population norms[11]. An uncontrolled study found improvements in self-reported physical functioning and fatigue were reported despite a numerical decrease in (objectively measured) activity[12]. In another study, the level of self-reported cognitive impairment in CFS patients decreased significantly after CBT, however, cognition had not improved when it was measured objectively using neuropsychological test performance[13].

It is unsurprising that 15 sessions of CBT (and the associated homework exercises and management program) might alter how participants respond to self-report questionnaires. A PACE Trial manual itself says "the essence of CBT is helping the participant to change their interpretation of symptoms": this could lead to altered or biased fatigue scores, one of the two primary outcome measures[14]. Also, one of the aims of CBT (for CFS) has been said to be "increased confidence in exercise and physical activity"[15]. The possible responses for the other primary outcome measure, the SF-36 physical functioning subscale, are "yes, limited a lot", "yes, limited a little" and "no, not limited at all" to questions on a range of physical activities. Such responses could be easily be artificially altered following a therapy like CBT for CFS.

The results were not that different with the GET cohort in the PACE Trial. Again the manuals predicted that the impairments and symptoms are reversible using the intervention[4,15]. The model said there was no reason participants should not be able to get back to full functioning. Deconditioning was posited to be an important maintaining factor. However, GET did not result in an improvement in fitness, as measured by the step test. GET did result in a small improvement on the six minute walking test to a final distance of 379 metres, or 35 metres more than the SMC-only group[7]. However, as Knoop and Wiborg commented in an accompanying editorial in Lancet Psychiatry: "an increase in distance walked during a test situation without an increased fitness suggests that patients walk more because of a change in cognitive processes (eg, daring to do more or an increased self-efficacy with respect to activity), not because of a change in physiological capacity”[16]. The result remained very poor given that normative data would suggest a group of similar age and gender should walk an average of 644 or so metres[17]. The distance walked remained comparable to people with many serious conditions[18-21], and considerably worse than the distance walked by healthy elderly adults[22,23], despite the PACE trial cohort having a mean age of only 40[8]. Also, to be allowed entry into CFS research studies such as the PACE Trial one can not have a range of chronic illnesses so with genuine recovery one would expect results comparable to healthy people[8].

As with CBT, measures relating to employment showed no improvement following GET in days of work missed, which remained very high, nor a reduction in levels of benefits (financial support from the state) or payments from insurance companies[9].

These results are in line with an audit of Belgian rehabilitation centres for CFS offering CBT and GET[24-26]. Some improvements in subjective measures were found, but there was no improvement in the results of the exercise test and hours in employment actually decreased.

Probably the main contribution of the PACE Trial has been to add to a growing body of evidence that while CBT and GET for CFS have resulted in some changes on subjective measures, they haven't lead to improvements on objective measures.

Competing interests: I am a committee member of the Irish ME/CFS Association and perform various types of voluntary work for the Association.

(continues)

On 2015 Sep 15, Tom Kindlon commented:

Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial

[Originally posted here: http://www.bmj.com/content/350/bmj.h227/rr-10]

This BMJ article and a flurry of articles in the lay media this week followed the publication in Lancet Psychiatry of an analysis of the mediators of change in the important PACE Trial, a chronic fatigue syndrome (CFS) trial which cost UK taxpayers £5 million[1,2]. What seems to have been lost in the coverage is that, although there were some modest improvements in the self-report measures, there was an almost complete absence of improvements in objectively measured outcomes for cognitive behavioural therapy (CBT) and graded exercise therapy (GET) compared to the control group (specialist medical care only (SMC)).

This is a non-blinded trial, where participants were told CBT and GET had previously been found to be effective in CFS and other conditions[3,4]: one way to look at the mediation results for subjective measures when there was a lack of objective improvements is that they may merely tell us how response biases and/or placebo effects are mediated[5].

The focus on subjective measures in some CFS studies was previously criticised in a systematic review published back in 2001 (long before the PACE Trial started)[6]. They suggested instead "a more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

The model presented for cognitive behaviour therapy (CBT) in the PACE Trial manuals posits that the impairments and symptoms are reversible with the therapy[3,7]. However, the latest paper shows that fitness, as measured by a step test, didn't improve following CBT[2]. An earlier PACE Trial publication reported that the addition of CBT to SMC did not result in an improvement in 6-minute walking test scores compared to SMC alone[8].

The PACE Trial was part funded by the UK Department of Work and Pensions, a rare move for them, presumably done due to an expectation that the therapies would improve measures of employment and levels of benefit receipt. However, again CBT brought about no improvement using objective measures, such as days of employment lost, levels of disability benefits received and levels of receipt of insurance payments[9].

These results are in line with earlier studies of CBT. For example, an analysis of three randomized controlled trials of CBT interventions for CFS found no improvement in objectively measured activity, despite participants reporting a reduction in (self-reported) fatigue and (sometimes) functional impairments[10]. Similar results were found in another uncontrolled trial where changes in objectively measured activity did not predict fatigue levels, and objectively measured activity on completion remained low compared to population norms[11]. An uncontrolled study found improvements in self-reported physical functioning and fatigue were reported despite a numerical decrease in (objectively measured) activity[12]. In another study, the level of self-reported cognitive impairment in CFS patients decreased significantly after CBT, however, cognition had not improved when it was measured objectively using neuropsychological test performance[13].

It is unsurprising that 15 sessions of CBT (and the associated homework exercises and management program) might alter how participants respond to self-report questionnaires. A PACE Trial manual itself says "the essence of CBT is helping the participant to change their interpretation of symptoms": this could lead to altered or biased fatigue scores, one of the two primary outcome measures[14]. Also, one of the aims of CBT (for CFS) has been said to be "increased confidence in exercise and physical activity"[15]. The possible responses for the other primary outcome measure, the SF-36 physical functioning subscale, are "yes, limited a lot", "yes, limited a little" and "no, not limited at all" to questions on a range of physical activities. Such responses could be easily be artificially altered following a therapy like CBT for CFS.

The results were not that different with the GET cohort in the PACE Trial. Again the manuals predicted that the impairments and symptoms are reversible using the intervention[4,15]. The model said there was no reason participants should not be able to get back to full functioning. Deconditioning was posited to be an important maintaining factor. However, GET did not result in an improvement in fitness, as measured by the step test. GET did result in a small improvement on the six minute walking test to a final distance of 379 metres, or 35 metres more than the SMC-only group[7]. However, as Knoop and Wiborg commented in an accompanying editorial in Lancet Psychiatry: "an increase in distance walked during a test situation without an increased fitness suggests that patients walk more because of a change in cognitive processes (eg, daring to do more or an increased self-efficacy with respect to activity), not because of a change in physiological capacity”[16]. The result remained very poor given that normative data would suggest a group of similar age and gender should walk an average of 644 or so metres[17]. The distance walked remained comparable to people with many serious conditions[18-21], and considerably worse than the distance walked by healthy elderly adults[22,23], despite the PACE trial cohort having a mean age of only 40[8]. Also, to be allowed entry into CFS research studies such as the PACE Trial one can not have a range of chronic illnesses so with genuine recovery one would expect results comparable to healthy people[8].

As with CBT, measures relating to employment showed no improvement following GET in days of work missed, which remained very high, nor a reduction in levels of benefits (financial support from the state) or payments from insurance companies[9].

These results are in line with an audit of Belgian rehabilitation centres for CFS offering CBT and GET[24-26]. Some improvements in subjective measures were found, but there was no improvement in the results of the exercise test and hours in employment actually decreased.

Probably the main contribution of the PACE Trial has been to add to a growing body of evidence that while CBT and GET for CFS have resulted in some changes on subjective measures, they haven't lead to improvements on objective measures.

Competing interests: I am a committee member of the Irish ME/CFS Association and perform various types of voluntary work for the Association.

(continues)

On 2016 Feb 16, Robert Courtney commented:

A study that uses questionable assumptions rather than empirical evidence leads to conclusions that stretch credibility.

Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.

Chalder et al. were working to this model and acknowledged that they failed to establish a clear temporal separation between variables, and therefore did not empirically establish a causal mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.

However, despite the lack of robust empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”

The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by unhelpful or maladaptive illness beliefs, fear, and an avoidance of activity.

However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.

Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.

Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. (e.g. This would mean that a decrease in activity would cause an increase in symptoms.) However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that increased symptoms would be the direct cause of activity avoidance in any illness, rather than vice versa. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.

In the case of fatigue, flu-like malaise and other symptoms of ME/CFS, it seems reasonable to consider the possibility that, as the symptoms fluctuate, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.

Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.

References:

1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.

2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.

3. IOM (Institute of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press. http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx

4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.