Garssen stated in his letter (Garssen, Reference Garssen2014) that the methods we applied in our study (Van der Zanden et al. Reference Van der Zanden, Galindo-Garre, Curie and Cuijpers2014) were not suited to demonstrate a circular process of change. We appreciate his critical appraisal of our article and we are glad that he gives us the possibility to explain some of the issues in more detail. But we would also reply to his critical comments. He built his opinion on a few arguments and we would like to answer to each of these.
First, according to Garssen there was a lack of definition of ‘circular process’ in our article. On pages 2 and 3 of our article we referred to circularity in processes of change as ‘bidirectional relationships between changes in mediators and outcomes’. Indeed, as Garssen suggested and based on previous research outlined in the Introduction, we hypothesized that a decrease in depression may trigger an increase in mastery, which in turn may lead to a further decrease in depression etc.
Second, Garssen stated that no conclusions can be drawn on our findings about processes of change, due to the limited number of measurements in our mediation models. As described in the Method section, we used three measurements in our study: baseline, post-treatment and follow-up. During the intervention no measurements took place. In the mediation analysis, we used the change scores. That is, we used in the mediation models the difference scores of the outcome variables, that is the difference between pre-post measurement (t0–t1 scores). And because, according to Kazdin (Reference Kazdin2007), ‘a more rigorous test of mediation would require that changes in specific mediatory variables temporally precede changes in the outcome variables’ (p. 17), we analysed also with regression analysis whether the t0–t1 scores of the significant mediating variables could predict later change in these variables at follow-up (t1–t2 scores).
In our opinion we discussed the consequences of the limited number of measurements appropriately in the Limitations section (p. 9): ‘First, the absence of assessments during the course of the intervention prevented us from analysing the precise sequence of changes. Our study contained three assessments: baseline, post-intervention and follow-up. Additional interim measurements could have given more detailed insights into the process of recovery from the beginning (when most changes appear to occur; Garratt et al. Reference Garratt, Ingram, Rand and Sawalani2007; Warmerdam et al. Reference Warmerdam, van Straten, Jongsma, Twisk and Cuijpers2010) to the end of the intervention and follow-up …, etc.’ Unfortunately, more measurements were lacking in our study, like in many other mediation analyses studies in our field (e.g. Meulenbeek et al. Reference Meulenbeek, Spinhoven, Smit, van Balkom and Cuijpers2010; Warmerdam et al. Reference Warmerdam, van Straten, Jongsma, Twisk and Cuijpers2010).
Third, Garssen suggested that there may be conceptual overlap between anxiety, depression, and mastery which could have biased the results. Indeed, our results could have been affected by this phenomenon. In this context, we cited in our Method section (page 5) Preacher & Hayes (Reference Preacher and Hayes2008), who refer to the ‘collinearity effect’, ‘meaning that the indirect effect could have been attenuated, to the extent that the mediators are correlated in the model (in our study: depression and anxiety in model 2; depression and mastery in model 3)’. This phenomenon can compromise the significance of particular specific indirect effects (Preacher et al. Reference Preacher and Hayes2008), as could be the case in our study. To exclude the possibility of a collinearity effect, we calculated the variance inflation factor (VIF) (see Method section). The results of the VIF analysis indicated that there was no collinearity effect in our mediation models (see Results section).
Fourth, Garssen stated that we overlooked a plausible explanation for the fact that we detected a greater early decrease in depression scores corresponded with a smaller late decrease in anxiety scores and vice versa. He suggests that because of the high correlation between depression and anxiety, and most changes occur at the beginning of psychological treatment, one could expect that a greater early reduction in depression is associated with a smaller later change in anxiety (and a greater early reduction in anxiety is associated with a smaller later change in depression).
We agree this might be the case, along with the other hypotheses we put forward in our Discussion section. We suggested our findings were in line with the circularity hypotheses (Discussion section). ‘When relationships are bi-directional and the change process is circular in nature, it is less likely to find in a time schedule one variable is predictive for the other. The results could also be due to the fact that most of the change in our sample took place between baseline and post-treatment, leaving little change to detect from post-treatment to follow-up’ (p. 9). We disagree that we used our finding as ‘a proof of circularity’ as Garssen suggested. We evaluated this result as an indication of circularity (and put forward alternative hypotheses) which is less substantial.
Finally, as in the field of psychological treatment knowledge of mediators and mechanisms of change is still very limited, we would bring into the discussion with Garssen the ideas of Kazdin (Reference Kazdin2007). We cited his opinion in the Discussion section of our article on p. 10, as Kazdin (Reference Kazdin2007, p. 6) has emphasized, ‘the case of a mediator is built by a sequence of studies that may vary in the set of criteria they address and the clarity of the demonstration’. He underlined that several studies are needed, meeting a range of criteria (strong associations, specificity, consistency, experimental manipulations, gradient, demonstrated timelines, plausibility), before one can conclude that an intervening process explains change. In fact, the same has to be applied to the circularity hypothesis. We would consider our study as one in a series of studies that are needed to prove circularity in change processes during psychological treatment. For this reason, we would call on for ‘next step research’, as Kazdin (Reference Kazdin2007) did.
