| Patient Satisfaction |
Patient satisfaction and priority setting - an economic approachAuthors: Ingemar Eckerlund, PhD, National Board of Health and Welfare and Centre for Health Economics, Stockholm School of Economics Jan A Eklöf, Associate Professor, Department of Economic Statistics, Stockholm School of Economics Jörgen Nathorst-Böös, MD, PhD, Department of Obstetrics and Gynaecology, Karolin-ska Hospital, Stockholm
Ingemar Eckerlund National Board of Health and Welfare SUMMARYIn recent years, various methods for measuring patient satisfaction have been applied as part of quality improvement programmes. However, the validity and reliability as well as the applicability and change-orientation of adopted methods have been ques-tioned. Furthermore, most methods pay no specific attention to economic aspects. The purpose of this study is to analyse if and how priorities are influenced when an eco-nomic perspective is explicitly included. Data were compiled by a patient survey at a gynaecology clinic, inquiring about the patients´ views on various quality dimensions, and their willingness to pay for proposed improvements. The parameters of the model are estimated with simultaneous equation methods, based on partial least squares tech-nique. We compare the ranking of proposed improvements derived from a patient sat-isfaction index, a cost-effectiveness analysis and a cost-benefit analysis, respectively. Our results show that even a methodologically appropriate measurement of patient sat-isfaction may lead to a cost-ineffective¬ priority setting unless economic consequences are explicitly taken into consideration. Further, it is demonstrated how an analysis in-cluding patient preferences as well as economic aspects may be carried out. Keywords: patient satisfaction, cost-effectiveness, cost-benefit, willingness-to-pay 1. INTRODUCTIONPatient preferences measured in terms of satisfaction indicators,
are being increasingly recognised as an important basis for priority
setting in the health services and for im-proving the quality of care
[1, 2, 3]. The most common method for systematic assess-ment of patient
satisfaction in health care is patient surveys (questionnaires). In
Swe-den, the first generation of patient questionnaires was introduced
in the mid-1980´s [4]. They helped to focus on the patient perspective,
but had certain deficiencies both in terms of validity and reliability,
and with regard to their potential as a basis for quality improvements.
That is, even if they tell you that the patients are dissatisfied with
some aspects of the service, they are generally not specific enough to
point out what kind of changes/improvements to make, and which of these
are most important. Furthermore, since the costs are not taken into
account, we can´t say which changes should be im-plemented with highest
priority (according to their cost-effectiveness). However, sev-eral new
methods were launched in the 1990´s, claiming to meet demands for
statisti-cal quality as well as change-orientation and relevance better
than earlier methods [5, 6, 7, 8, 9]. Still, economic aspects are
seldom included, i.e., measures identified as ef-fective for improving
patient satisfaction are usually not evaluated with regard to their
costs and benefits. The following presentation starts with a description of the theoretical model for patient satisfaction measurements and an outline of the empirical approach in Section 2. The results of the patient satisfaction study, the cost-effectiveness calculations and the cost-benefit analysis are presented in Section 3. The paper ends with a discussion and some concluding remarks in Section 4. 2. METHODS AND MATERIALThe method applied for quality assessment based on a patient satisfaction approach, QSP (Quality, Satisfaction, Performance), is illustrated in Figure 1. FIGURE 1 The QSP model consists of three integrated components. One component measures the degree of patient satisfaction (PSI), usually through three questions. Patient-perceived quality levels of various quality dimensions (quality factors) are also measured, which are, a priori, assumed to explain the variation in patient satisfaction. Each quality di-mension is represented by three to six questions in the patient questionnaire, intended to represent a specific quality factor as thoroughly and reliably as possible. The ques-tions should be specific enough to provide an operative decision-making basis for quality improvement. Finally, the model also contains a component on goals, with questions directed at what patient satisfaction should ultimately lead to, e.g., increased trust, increased likelihood of positive recommendation and related loyalty indicators. This aspect of the measurement model links patient satisfaction to the goals of the health care provider. The QSP method is based on a microeconomic model of consumer behaviour devel-oped by professor C Fornell. The statistical methodology used in the measurement model is based on a state-of-the-art multivariate analysis based on principles applied in the partial least squares methodology, PLS [10, 11]. The analytical model consists of two components: 1) a structural model describing how quality factors, patient satisfaction, and goal fac-tors are related (see Figure 1);
More formally, the structural model may be described as follows:
where The formal description of the measurement models is presented as follows:
where
Matrices A (m x p) and B (n x q) show how the manifests
correlate with their respective latent factors. In the statistical
analysis, the impact parameters (a and b) are determined, and the rating associated with the latent variables In earlier studies in the health services, the QSP method has been
shown to produce rele-vant results [5, 12, 6]. Especially, it
identifies the most effective measures (drivers) for improving PSI.
Thus, incontrast to most other methods, it is clearly change-oriented.
Among other things, this means that the method itself tells – through
the operative ques-tion technique, and the impact coefficients derived
– what should be given the highest priority. The approach also offers a
way of conducting quality assessment of the results as indicators of
explanatory power are derived through the model estimation stage.
Further-more, PSI has shown to be an important indicator for
determining trust and loyalty in many service industries and public
sector functions (12). A few recent studies have also addressed the
issue of model validity for various patient groups in the health sector
(15). Thus, instrument validity has been demonstrated in several
studies. The current study is based on an earlier study of patient satisfaction at the gynaecology clinic at the Karolinska Hospital in Stockholm, Sweden, carried out in 1997 with the QSP method (7). In that study, three main areas calling for improvement were identi-fied, namely accessibility, environment and participation. Now, these areas are ana-lysed in further detail in terms of current patient satisfaction. This follow-up study is combined with a study of patient valuation – regarding satisfaction and willingness to pay, respectively – of possible improvements and relative to the calculated costs of these improvements. Thus, we use a reduced model including three of the seven quality dimensions in the above- mentioned QSP-study. We single out the three dimensions (latent variables) which were found to be important in determining the patient satisfaction level and, at the same time, being among the least appreciated by the patients (high impact and low scores). Each of the latent variables – accessibility, environment and participation – is measured with four questions, i.e., manifest variables. In the WTP-study, two of the manifest questions – relating to waiting-room functionality – were pooled together.
Thus, the study consists of three parts. This first part constitutes
a replication of the above-mentioned study conducted in 1997. It should
be noted that the replication was done in a most condensed form as the
original study included seven different latent variables. In addition
to the variables studied here, the four latent variables medical care,
doctor's attention, nurse's attention and information were included in
the initial study. All of these received very high scores and/or low
impact levels, and were thus considered not to be so important for
focused improvement work. However, the rela-tionship between costs and
patient valuation was not explicitly considered for them. Accessibility• Extended telephone hours by two hours per working day (telephone hours); Environment• Improved sign-posts guiding the visitor to the clinic (sign-posts); Participation• The patient is always offered to choose the appointment time for her next visit (next visit); In the third part, the respondents were asked to indicate whether they would be willing to pay a certain amount (on top of the current patient charge) to obtain the indicated improved service/treatment. Eight different payment alternatives ranging from SEK 10-150 were defined and randomly given to an equal number of the patients. Implementation of the studyThe study was conducted during the period August 1998 – February 1999 at the gy-naecology clinic at Karolinska hospital, Stockholm. All patients visiting the clinic in this period (the first time during the period they visited the clinic) received a survey questionnaire to be filled in either directly after the visit, or later and if so, to be re-turned by mail to the clinic. The survey questionnaire included a total of 43 closed re-sponse questions and two open questions (where the respondents could indicate any positive and/or negative aspects relating to the clinic). By and large, the same manifest questions as in 1997 were asked but a number of these were re-phrased. The question-naire design was pre-tested on both employees of the hospital, and actual patients. The respondents were explicitely prompted to the fact that their perceived valuation is asked for. The structure of the questionnaire, which took on average some 15 - 20 minutes to fill in, is shown in the Appendix. In total, 800 original questionnaires were distributed to patients in the period, coincid-ing with the number of first time visits in the study period. About 100 patients received a reminder including a new questionnaire and this procedure gave a gross response rate of 88 percent. After discarding forms with less than 70 percent filled-in answers (or in-consistencies in answers according to the test program), 657 forms (82 percent) were accepted and thus included in the analysis. 3. RESULTSPatient satisfaction FIGURE 2 Table 1 shows current scores, relative weights for the various
questions and estimated impact on PSI, and also the scores if the
proposed improvements are implemented. It is seen that the highest actual scores (where the patients are most satisfied) are ob-tained for opening hours, visit length and the presence of medical students, while the lowest scores are noted for wardrobe and (doctor's) call back. The relative weights (standardised to 1 for each latent variable) are fairly equal for the majority of the mani-fests, while a few are significantly different (wardrobe and medical students are lower than the average; and telephone hours, sign-posts, waiting-room comfort as well as visit length are higher than the average). The column “impact on PSI” indicates how much a one-unit increase in the score for the respective manifest variable would increase the PSI-score (based on using the ab-solute – not relative – weight of each manifest variable). From the above table, it is seen that the factor relating to waiting-room comfort has the strongest single effect (with telephone hours and sign-posts in shared second place). The effects of all the proposed improvements are shown in the last column of Table 1. Using the same impact coefficients and relative weights, the figures tell us what would happen if all the proposed improvements were implemented at the same time. It is seen that the PSI would increase by 14 units (from 78 to 92) and loyalty by 8 units (from 79 to 87). It should be noted that the questions in the survey were spelled out in such a way that the respondents considered one proposed improvement at a time. As it would be realistic to assume that the proposals are not independent, the presented results should be seen as a simplification of the real effects. Possible interactions between packages of improvements have not been considered so far. As shown in Table 1, all latent and manifest variables are increased
as a consequence of the proposed improvements. This means that the
patients find them all to be positive (representing real improvements).
This is true also for almost each individual respon-dent answer (hardly
any “inconsistencies” were found in the database is this respect). Cost-Effectiveness AnalysisWhat distinguishes the QSP-model from most other methods used for quality meas-urement in health care is that it not only measures the degree of patient satisfaction but also the impact of various quality dimensions (factors) on satisfaction. Consequently, one of the model’s strengths is that the patients’ quality opinions as well as their qual-ity dimension preferences are identified. Thus, the identified opportunities for change are based on actual patients’ opinions. However, the model does not account for the cost-effectiveness of the potential changes. In order to remedy this inherent weakness, we estimated the costs as well as the effects (impact on Patient Satisfaction Index, PSI) of various specified potential changes in the three main areas (domains) considered – accessibility, environment and participation. The costs – mainly in terms of costs for additional personnel needed for extended telephone hours, etc, but also for waiting-room equipment, and so on – were estimated in collaboration with the clinic’s financial controller and only costs at the clinic level are included. The cost-effectiveness ratios in Table 2 were calculated as the monetary cost to achieve a one-unit increase in PSI from the respective manifest vari-able. TABLE 2 It is noted that the cost-effectiveness ratios are varying within
very wide margins, which is mainly due to large differences in cost
between the various improvement measures. The lowest cost-effectiveness
ratio (the cheapest way of increasing PSI) has been calculated for the
improvement consisting of the doctor calling back the same day the
patient has called. The highest cost-effectiveness ratio, SEK 126.73,
stands for ex-tended length of visits. Cost-Benefit AnalysisThis part of the study was carried out using the standard methodology for measuring willingness-to-pay that has been developed and applied in various societal contexts in recent years [13]. Patients were asked whether they were willing to pay an additional X SEK per visit (on top of the current patient charge, which amounts to 120 SEK), pro-vided that the respective improvements were implemented. The patients were ran-domly distributed into eight groups of equal size that received different bids (X), from an additional 10-150 SEK (1 US$=10.50 SEK) per visit and improvement. Thus, 100 patients were allocated to each pay alternative. The result of the WTP-study differs between components. The mean willingness to pay was calculated for each of the considered alternatives. A logistic as well as a probit model have been estimated. The results of the willingness-to-pay study are summa-rised in Table 3, which shows the average willingness-to-pay in SEK for the various improvements. It is based on the Logit model estimation. The results are rather similar also in the Probit model case. However, due to the fact that the distribution between al-ternatives is not normal, we decided to use the Logit alternative in the further analysis. TABLE 3 Figure 3 shows the estimated percentage of the patients (based on the Logit model) willing to pay an additional X SEK per visit for the respective alternatives associated with the accessibility latent variable. It is shown that a relatively smooth downward function is estimated for each of the four considered improvement situations. The es-timated models predict the correct classification in 70 - 75 percent of the cases. In ad-dition to the simple Logit model, also models with additional explanatory variables (age of patient, visit number and number of births) have been estimated. It was found that the explanatory power of all these additional variables were low (not statistically significant). FIGURE 3 A comparison between the costs of realising the proposed improvements and the pa-tients’ valuation (willingness-to-pay) demonstrates that willingness-to-pay is higher for some improvements while for some, they are lower than the cost. Table 4 shows the ra-tio between the average (mean) willingness-to-pay (Table 3) and the cost per visit (Ta-ble 2) for the various improvements. The effect on patient satisfaction (PSI) of running the improvements with a WTP-cost ratio equal to or above 1 will be an increase by four units (from 78 to 82). At the same time, the loyalty index would increase by three units (from 79 to 82). The cost increase per visit for this improvement package is estimated at 9 SEK. (Based on an estimated 4 200 visits per year, the total costs would be 37,800 SEK). TABLE 4 Using the alternative approaches in priority settingTo summarise our findings on whether and how the priorities based merely on patient preferences are influenced by the inclusion of an economic perspective, we finally compare the rank order of the various improvements according to the alternative prior-ity rules, i.e., PSI-change, willingness-to pay, cost-effectiveness and benefit-cost ratio. TABLE 5 These results may be interpreted in the following way: Thus, carrying out a traditional patient satisfaction analysis and using those results di-rectly for priority setting is not enough. To appreciate the patients’ valuation of possi-ble improvements, either a PSI-change study or a WTP study is needed, both of which enable us to rank the alternative improvements. However, to reach cost-effective deci-sions, we must estimate the “PSI production function”, which is done in the cost-effectiveness analysis. To find out whether various improvements are worth their costs, we also need a WTP-approach. 4. DISCUSSION AND CONCLUDING REMARKSPatients’ perceptions of quality are essential for determining effectiveness and effi-ciency in health care delivery, and should thus be utilised as a basis for decisions on quality improvements. Even if this is done to an increasing extent, economic aspects are seldom included in a proper way. One of the advantages of the QSP-model is its focus on change. Several studies have shown that the QSP approach can produce very useful and policy relevant results. However, from a health economics perspective, the lack of economic content in the basic model is a serious deficiency. For instance, this means that in rating the various quality factors, patients do not take cost-effectiveness into account. This, in turn, means that the priorities derived by the model may be cost-ineffective [14]. This study is an attempt to remedy that deficiency by estimating the costs, as well as the effects, of various improvements. In that way, we are able to rank the potential im-provements according to their cost-effectiveness ratios. In addition we also asked the patients to evaluate the benefits in terms of their willingness to pay for the changes, in order to rank the changes according to their benefit-cost ratios. In order to achieve in-ternal consistency in the analysis (and for minimising the risks of misleading interpre-tation by various stakeholders) we propose that both issues should be considered to-gether. As shown, the ranking between various improvements is strongly influenced by the in-clusion of an economic dimension. This points at the importance of looking at both the benefit and the cost side, when considering alternatives as concerns changes. We also found disagreement between cost-effectiveness and cost-benefit ranking in a few in-stances. This indicates a need for further analysis and model development. The improvement call back gets the highest rank except for the WTP-ranking. Other-wise, the ranking differ much between the ranking order lacking an economic dimen-sion and those based on cost-effectiveness and benefit (WTP)-cost ratio, respectively. The second and third priorities according to “PSI-change” (same doctor and telephone answer) are relatively cost-ineffective, i.e., they have relatively high cost-effectiveness ratios. Moreover, they have WTP-cost ratios of less than 1, which means that the pa-tients are not willing to pay the costs of the changes. Assuming a decision rule that defines a cut-off value for the WTP-cost ratio of at least 1 means that those improvement measures, which cost less than or equal to what the patients are willing to pay, shall be implemented. In our study this would mean that the following improvements – and only these – should be undertaken. • The doctor calls back the same day as the initial contact (call back) We conclude that even if patient views are very important as a basis for improvement decisions, it is necessary to supplement it with economic analysis in order to avoid “wrong” decisions, i.e., sub-optimisation. Naturally, it is important to try to meet pa-tient demand for continuity, i.e., seeing the same doctor on a follow-up visit. But it is not enough in terms of a basis for priority setting. There are other improvements that – according to patient views – are more important when economic aspects are consid-ered, and thus should be given higher priority. To sum up, our findings suggest that it is possible to expand the QSP model with an economic dimension. Thereby, one can get a clear picture of the costs and effects of various improvement measures – and a relevant basis for continuous, cost-effective quality improvement. REFERENCES
3. Guadagnoli, E, Ward, P. Patient participation in decision-making. Soc. Sci. Med. 1998;47:329-339. 4. Patienterna svarar : patienternas syn på vårdkvaliteten. Stockholm : Spri, 1989. (Spri rapport 271). (In Swedish) 5. Eckerlund, I, Jönsson, B, Tambour, M, Westlund, A. Change-oriented patient ques-tionnaires : testing a new method at three departments of ophthalmology. International Journal of Health Care Quality Assurance 1997;10:254-259. 6. Nathorst-Böös J, Munck I, Eckerlund I, Ekfeldt-Sandberg C. An evaluation of the QSP and the QPP: two methods for measuring patient satisfaction. International Journal for Quality in Health Care 2001;13257-264. 7. Höglund, E. Patienternas erfarenheter : en utgångspunkt för förbättringsarbete. Rapport från studier med Picker-metoden. Stockholm : Spri, 1999. (Spri rapport 488). (In Swedish) 8. Larsson, G, et al. Refinement of the questionnaire ´Quality of care from the Patients´ Perspective´ using structural equation modeling. Scandinavian journal of caring sci-ences 1998;12(2):111-118. 9. Wilde, B., et al. Patienten värderar vården : vägledning till frågeformuläret KUPP, Kvalitet ur patientens perspektiv. Stockholm : Vårdförbundet SHSTF, 1995. (Rapport 45). (In Swedish). 10. Dijkstra, TK. ”Some comments on maximum likelihood and partial least squares methods”. Journal of Econometrics 1983;22:67-90. 11. Fornell, C, Cha, J. Partial least squares. In : Bagozzi, P. (Ed). Advanced Methods of Marketing Research. Blackwell, 1995, pp. 52-78. 12. Eklöf, J, Westlund, A. User satisfaction in the public sector : level and trends based on the Swedish Customer Satisfaction Index. Stockholm School of Economics. Research report (in press). 13. Johansson, P-O. Evaluating health risks : an economic approach. Cambridge Univer-sity Press, 1995. 14. Eckerlund, I, Eklöf, J, Nathorst-Böös, J. Patient satisfaction and priority setting in am-bulatory health care. Journal of Total Quality Management 2000;7:S967-S978. 15. SKI 2002 Öppen sjukvård, mätning av patientupplevd kvalitet mm. Swedish Quality Index Health sector Report. Stocholm 2002. (in Swedish). Appendix QuestionnaireExamples of questions on ”Environment” (page 3 in the 5-page questionnaire)
Today: The physical standard of the clinic has improved
substantially since 1997. However, there is no TV, coffee machine or
lockable wardrobe in the clinic waiting room. - the possibility to find your way to the clinic
How would you then grade Lowest grade Highest grade - the possibility to find your way to the clinic
- there were better sign-posts to the clinic yes no - the waiting-room was equipped with TV, coffee machine and new magazines yes no Would you be willing to pay an additional SEK X (10-150) per visit (above the ordinary charge) if - lockable wardrobes were installed close to the waiting-room yes no
Figure 1. The QSP-model
TABLE 1. Patient satisfaction - Base case and change results
Table 2. Cost-effectiveness ratios for proposed changes
TABLE 3. Willingness-to-pay (WTP) for various changes (based on the Logit model)
Table 4. Willingness-to-pay (WTP)-cost ratios for various changes
Table 5. Summary table (rank order in parentheses)
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||



















