Open Access
Subscription Access
Prediction of treatment outcome in daily generalized mental health care practice: first steps towards personalized treatment by clinical decision support
Abstract
Rationale, aims and objectives: Results from prediction studies are often of limited value because potential predictors are measured with instruments that are not routinely used, the results are presented in terms that are difficult to translate to individual patients and the relations between predictors and outcome are complex. These problems were faced by deriving decision trees for classification using information routinely accessed in generalized mental healthcare for intake purposes and treatment monitoring.
Method: Positive treatment outcome was defined as symptom improvement, measured with the symptom distress scale of the Outcome Questionnaire (OQ-45.2). The analysis process consisted of 3 phases, derivation of a possible decision tree, selection of the best 10 trees and assessment of classification performance after integration of these 10 trees. This analysis was performed 3 times, for all patients without any missing data, for the full set of patients and for the patients in which intermediate outcome data were available.
Results: The prediction performance of the 3 integrated classifiers varied from poor (AUC 0.68) in the complete sample including patients with missing variables, to good (AUC 0.83) including early response as predictor. Complex interactions between variables were found.
Conclusion: The present study shows the need for registration of clinical and sociodemographic variables and outcome monitoring in a systematic way to prevent missing variables and automated decision support systems to use complex interactions between variables for outcome prediction.
Keywords
References
Clarkin, J.F. & Levy, K.N. (2003). The influence of client variables on psychotherapy. In: Handbook of Psychotherapy and Behavior Change (ed. M. Lambert), pp. 194-226, 5th edn. New York: Wiley & Sons.
Barkham, M., Stiles, W.B., Connell, J., Twigg, E., Leach, C., Lucock, M., Mellor-Clark, J., Bower, P., King, M., Shapiro, D.A., Hardy, G.E., Greenberg, L. & Angus, L. (2008) Effects of psychological therapies in randomized trials and practice-based studies. British Journal of Clinical Psychology 47, 397-415.
Van der Lem, R., van der Wee, N.J., van Veen, T. & Zitman, F.G. (2012). Efficacy versus effectiveness: a direct comparison of the outcome of treatment for mild to moderate depression in randomized controlled trials and daily practice. Psychotherapy and Psychosomatics 81, 226-234.
Hannan, C., Lambert, M.J., Harmon, C., Nielsen, S.L., Smart, D.W., Shimokawa, K. & Sutton, S.W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology 61, 155-163.
Slade, M., Gask, L., Leese, M., McCrone, P., Montana, C., Powell, R., Stewarta, M. & Chew-Graham, C. (2008). Failure to improve appropriateness of referrals to adult community mental health services - lessons from a multi-site cluster randomized controlled trial. Family Practice 253, 181-190.
Avorn, J. & Choudry, N.K. (2008). Using computer-based decision support to close the “Know-Do” gap in lipid-lowering therapy. Circulation 117, 336-337.
Moons, K.G., Altman, D.G., Vergouwe, Y. & Royston, P. (2009). Prognosis and prognostic research: application and impact of prognostic models in clinical practice. British Medical Journal 338, b606.
Bright, T.J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R.R., Samsa, G., Hasselblad, V., Williams, J.W., Musty, M.D., Wing, L., Kendrick, A.S., Sanders, G.D. & Lobach, D. (2012). Effect of clinical decision-support systems: a systematic review. Annals of Internal Medicine 157, 29-43.
Lutz, W., Saunders, S.M., Leon, S.C., Martinovich, Z., Kosfelder, J., Schulte, D., Grawe, K. & Tholen, S. (2006). Empirically and clinically useful decision making in psychotherapy: differential predictions with treatment response models. Psychological Assessment 18, 133-141.
De Jong, K. (2012). A chance for change: building an outcome monitoring feedback system for outpatient mental health care. Leiden University, Clinical, Health and Neuropsychology, Faculty of Social and Behavioural Sciences.
Osei-Bryson, K-M. (2004) Evaluation of decision trees: a multi-criteria approach. Computers & Operations Research 31, 1933-1945.
Lambert, M.J., Morton, J.J., Hatfield, D.R., Harmon, C., Hamilton, S., Shimokawa, K., et al. (2004). Administration and scoring manual for the OQ-45.2 (Outcome Questionniare) (3 edn). Wilmington DE: American Professional credentialing Services LLC.
De Jong, K., Nugter, M.A., Polak, M.G., Wagenbor, J.E.A., Spinhoven, P. & Heiser, W.J. (2007). The Outcome Questionnaire (OQ-45) in a Dutch population: A cross-cultural validation. Clinical Psychology & Psychotherapy 14, 288-301.
Hakkaart-Van Roijen, L. (2007). Handleiding Trimbos/iMTA questionnaire for Costs associated with Psychiatric Illness (TiC-P). Rotterdam: Institute for Medical Technology Assessment Erasmus Universiteit.
Jacobson, N.S. & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59, 12-19.
Quinlan, J.R. (1986). Induction of Decision Trees. Machine Learning 1, 81-106.
Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. Burlington: Morgan Kaufmann Publishers.
Höhle, D., Pieterman, C.R.C., Valk, G.D, Hermus, A.R., Koppeschaar, H.P.F, Meyer, J. & de Lange, R.P.J. (2011). Classifying the decision to perform surgery in MEN1 cancer patients using decision trees. 24th International Symposium on Computer-Based Medical Systems (CBMS) 2011, 1–6. doi:10.1109/CBMS.2011.5999108.
Bower, P., Kontopantelis, E., Sutton, A., Kendrick, T., Richards, D.A., Gilbody, S., Knowles, S., Cuijpers, P., Andersson, G., Christensen, H., Mayer, B., Huibers, M., Smit, F., van Straten, A., Warmerdam, L., Barkham, M., Bilich, L., Lovell, K. & Liu, E.T. (2013). Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. British Medical Journal 26, 346:f540.
Van der Lem, R., Stamsnieder, P.M., van der Wee, N.J., van Veen, T. & Zitman, F.G. (2013). Influence of sociodemographic and socioeconomic features on treatment outcome in RCTs versus daily psychiatric practice. Social Psychiatry and Psychiatric Epidemiology 48, 975-984.
Schat, A., van Noorden, M.S., Noom, M.J., Giltay, E.J., van der Wee, N.J., Vermeiren, R.R. & Zitman, F.G. (2013). Predictors of outcome in outpatients with anxiety disorders: the Leiden routine outcome monitoring study. Journal of Psychiatric Research 4712, 1876-1885.
NICE, 2009. Depression in adults. The treatment and management of depression in adults. www.guidance.nice.org.uk/cg90.
Fournier, J.C., DeRubeis, R.J., Shelton, R.C., Hollon, S.D., Amsterdam, J.D. & Gallop, R. (2009). Prediction of response to medication and cognitive therapy in the treatment of moderate to severe depression. Journal of Consulting and Clinical Psychology 77, 775-787.
Lutz, W., Hofmann, S.G., Rubel, J., Boswell, J.F., Shear, M.K., Gorman, J.M., Woods, S.W. & Barlow, D.W. (2014). Patterns of early change and their relationship to outcome and early treatment termination in patients with panic disorder. Journal of Consulting and Clinical Psychology 82, 287-297.
Knaup, C., Koesters, M., Schoefer, D., Becker, T. & Puschner, B. (2009). Effect of feedback of treatment outcome in specialist mental healthcare: Meta-analysis. British Journal of Psychiatry 195, 15-22.
Shimokawa, K., Lambert, M.J. & Smart, D,W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology 78, 298-311.
Carlier, I.V., Meuldijk, D., van Vliet, I.M., van Fenema, E., van der Wee, N.J. & Zitman, F.G. (2012). Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. Journal of Evaluation in Clinical Practice 18, 104-110.
Rubel, J., Lutz, W. & Schulte, D. (2013). Patterns of Change in Different Phases of Outpatient Psychotherapy: A Stage-Sequential Pattern Analysis of Change in Session Reports. Clinical Psychology and Psychotherapy 22 (1) 1-14.
DeRubeis, R.J., Cohen, Z.D., Forand, N.R., Fournier, J.C., Gelfand, L.A. & Lorenzo-Luaces, L. (2014). The Personalized Advantage Index: translating research on prediction into individualized treatment recommendations. A demonstration. PLoS One 9 (1) e83875.
Simon, G.E. & Perlis, R.H. (2010). Personalized medicine for depression: can we match patients with treatments? American Journal of Psychiatry 167, 1445-1455.
DOI: http://dx.doi.org/10.5750/ejpch.v4i1.1044
Refbacks
- There are currently no refbacks.