Open Access
Subscription Access
Point-of-service questionnaires can reliably assess patients’ experiences
Abstract
Rationale, aims and objectives
Point-of-service patient experience measures are increasingly used to evaluate health services but their psychometric properties have not been reported. The current study determined the test-retest reliability of a point-of-service patient experience questionnaire used in an inpatient rehabilitation centre.
Method
Patient experience data were collected from inpatients by two community volunteers using an electronic hand-held five-item Likert-style questionnaire. Eighty-one participants completed the questionnaire at Test 1 and Test 2 (two to four hours apart). Twenty three participants completed the questionnaire at Test 1 and Test 3 (one week apart).
Results
There were small and non-significant differences between test and retest scores for individual questions and the five-item total score. Kappa and intra-class correlations coefficients indicated moderate to good agreement. Subgroup analysis indicated that participants who had previously completed the questionnaire during their inpatient stay gave more consistent answers between Test 1 and Test 2 than first time responders.
Conclusion
Patients participating in sub-acute inpatient rehabilitation can provide reliable answers to a point-of-service patient experience questionnaire.
References
Centers for Medicare and Medicaid Services. CAHPS hospital survey (HCAHPS) quality assurance guidelines, version 7.
Coulter, A., Fitzpatrick, R. & Cornwell, J. Point of care. Measures of patients’ experience in hospital: Purpose, methods and uses. London: The Kings Fund; 2009. p. The Kings Fund.
Dirocco, D. N. & Day, S. C. (2011) Obtaining patient feedback at point of service using electronic kiosks. American Journal of Managed Care, 17 (7), e270-276.
Jensen, H. I., Ammentorp, J. & Kofoed, P-E. (2010) User satisfaction is influenced by the interval between a health care service and the assessment of the service. Social Science & Medicine, 70, 1882-1887.
Sitzia, J. (1999) How valid and reliable are patien satisfaction data? An analysis of 195 studies. International Journal for Quality in Health Care, 11 (4), 319-328.
Lavrakas, P. J. (2008) Test-retest reliability. Thousand Oaks, CA: SAGE Publications.
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hrobjartsson, A., Roberts, C., Shoukri, M. & Streiner, D. L. (2011) Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Journal of Clinical Epidemiology, 64, 96-106.
Hopkins, W. G. (2000) Measures of reliability in sports medicine and science. Sports Medicine, 30 (1), 1-15.
Victorian Department of Health. Victorian patient satisfaction monitor. Wave 18 report for mckellar centre. Melbourne, Victoria: Department of Health, 2010.
Customer Feedback Systems. Customer Feedback Systems Australasia. Pymble, NSW: Customer Feedback Systems Australasia; 2006. Available from: http://www.cfs-australasia.com/.
Boone, H. N. & Boone, D. A. (2012) Analyzing likert data. Journal of Extension, 50 (2), 2TOT2.
Sim, J. & Wright, C. C. (2005) The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy, 85, 257-268.
Atkinson, G. & Nevill, A. M. (1998) Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine, 26 (4), 217-238.
Shrout, P. E. & Fleiss, J. L. (1979) Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86 (2), 420-428.
Landis, J. R. & Koch, G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
IBM SPSS statistics 21. IBM; 2012.
The r project for statistical computing; 2013. Available from: http://www.r-project.org/
Walter, S. D., Eliasziw, M. & Donner, A. (1998) Sample size and optimal designs for reliability studies. Statistics in Medicine, 17 (1), 101-110.
Brink, Y. & Louw, Q. A. (2012) Clinical instruments: Reliability and validity critical appraisal. Journal of Evaluation in Clinical Practice, 18, 1126–1132.
Jakobsson, U. (2004) Statistical presentation and analysis of ordinal data in nursing research. Scandinavian Journal of Caring Sciences, 18, 437–440.
Staniszewska, S. H. & Henderson, L. (2005) Patients' evaluations of the quality of care: Influencing factors and the importance of engagement. Journal of Advanced Nursing, 49 (5), 530-537.
Jenkinson, C., Coulter, A. & Bruster, S. (2002) The picker patient experience questionnaire: Development and validation using data from in-patient surveys in five countries. International Journal for Quality in Health Care, 14 (5), 353-358.
Frost, M. H., Reeve, B. B., Liepa, A. M., Stauffer, J. W. & Hays, R. D. (2007) What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10 (2), S94-S105.
DOI: http://dx.doi.org/10.5750/ejpch.v2i4.834
Refbacks
- There are currently no refbacks.