1. Introduction
Patient satisfaction surveys produce data about patients’ perspectives of care and create objective meaningful comparisons of hospitals in terms of quality of care. The Hospital Consumer Assessment of Healthcare Providers and Systems Survey (HCAHPS), developed in 2005, was the first nationally standardized survey that measures how patients perceive the care that they receive in hospitals. The survey questionnaire has three broad goals: (a) to produce data about patients’ perspectives of care that allow objective and meaningful comparisons of hospitals on topics that are important to consumers; (b) to create new incentives for hospitals to improve quality of care; and (c) to enhance accountability in health care by increasing transparency of the quality of hospital care provided in return for the public investment [1] . Data from the HCAHPS survey also empower patients to select the appropriate hospital.
Effective communication was identified as the most important and valued feature and in particular, patients cited “understanding” and “explanation” of information and physicians’ “listening” to patients, as participants would consider changing hospitals for these experiences [2].
The focus of the HCAHPS is quality of care and results in six summary measures: (a) how well physicians and nurses communicate with patients, (b) responsiveness of the hospital staff to patients’ needs, (c) pain management, (d) how well the staff communicates with the patient about medications (e) cleanliness and composure of the facility environment, and (f) pertinent information provided at discharge [1]. The HCAHPS survey consists of 27 items, including specific questions on the six summary measures, items for global rating, overall rating of the hospital, and whether the patient would recommend the hospital to others. Several questions that pertain to summary measures are answered on a 4-pointLikert-scale (1 = never, 2 = sometimes, 3 = usually, and 4 = always). Additionally, patients’ perspectives on their hospital experiences are measured using an ordinal scale from 0 to 10, with 0=worst and 10=best. According to the CMS (2007) [3] guidelines, the HCAHPS survey must be administered 48 hours to 6 weeks after discharge to eligible acute-care hospital patients, excluding patients who received psychiatric care, via one of four methods: (a) mail only (b) telephone only, (c) mixed (mail with telephone follow-up) or (d) active interactive voice response.
2. Background of the Problem
The term “vulnerable population” refers to social groups with higher risk or susceptibility to health-related problems, and this vulnerability is evident in higher mortality rates, lower life expectancy, reduced access to care, and diminished quality of life compared to non-vulnerable populations [4].The population who is unable to read or comprehend written information is “vulnerable,” and low literacy levels have been found to affect health and well-being negatively, leading to poor health outcomes. Further, clients with documented low literacy are found to have a 52% higher risk of hospital admissions when compared with those with functional literacy, even after controlling for age, social and economic factors, and self-reported health [5]. Although illiteracy was considered in the development of the HCAHPS, reducing the reading level below that of the sixth grade was determined to compromise essential elements and goals of the survey [6].
The National Center for Education Statistics [7] reported that approximately 23.0% of individuals 16 years of age and older lacked prose literacy skills for the state of California for the year 2003, compared to the national average of 14.5%. In Los Angeles County, the percentage is even higher, with approximately 33.0% of the population 16 and order’s lacking literacy skills. The National Center for Education Statistics [7] found that adults who spoke only Spanish before starting school had the lowest average health literacy, equivalent to below basic health literacy. Below basic literacy was defined as ranging from being non-literate in English to being able to locate easily identifiable short information, following written information in short documents, and/or locating concrete and simple, familiar mathematical information.
Individuals with low literacy have poorer health outcomes and increased healthcare costs, which are as much as four times greater for those who read at or below the second-grade level than for the general population [8]. The inability to comprehend written information, understand verbally communicated medical instructions, and articulate health concerns to healthcare professionals makes it difficult, and sometimes impossible, for low-literate adults to obtain the care that they need through the use of available societal or environmental resources [9].
Paasche-Orlow, et al. [10] found that limited health literacy was prevalent in the United States and that the prevalence was associated with education, ethnicity and age, indicating a significant health disparity for the country. They concluded that simplification of health services and improving health education is essential for individuals with limited health literacy.
Because patient satisfaction has become an increasingly important parameter in measuring the quality of healthcare [11], it is important that a survey to measure this information obtains accurate information, particularly when individuals have low levels of literacy. Further, effective strategies to enhance patients’ understanding of healthcare information are warranted for individuals with low literacy. Limited studies have assessed the adequacy and accuracy of survey data in patients with low levels of literacy. Al-Tayyib, et al. [12] found evidence for the potential benefits of an “interview,” e.g., audio computer-assisted self-interviewing technologies, when obtaining survey data, as this method does not require respondent literacy. Bickmore, et al. [13] concluded that, regardless of health literacy, participants were more likely to sign a consent form when it was clarified by the computer agent. Participants with “sufficient” health literacy showed the highest level of understanding of the computer agent-based clarification. Nevertheless, overall, participants with limited health literacy displayed poor comprehension levels in all areas of the study conditions [13].
Hospitals that serve individuals with low literacy frequently encounter a low return rate of patient satisfaction survey questionnaires. A potential reason for the low return rate is the inability to read or understand information, particularly for individuals for who English is a second language. Typical of such hospitals is one in Southern California, where over one-third of the population have less than a high school education, and where there have frequently been low rates of return as well as incomplete survey questionnaires. Although there have been more than 7,300 inpatients cared for at this facility monthly, the average survey questionnaire return rate in some departments is reported to be less than 1%. Without an understandable survey questionnaire for individuals with low literacy, these vulnerable patients’ view of quality of hospital care received as well as their concerns specific to their overall health, post-discharge, may not be obtained, or, if obtained, the information may lack validity. The inability of this population to adequately comprehend and use resources, including the HCAHPS, may affect their choices, including access to care and utilization of healthcare services, and may increase their risk for disease and worsen their health outcome.
3. Theoretical Framework
The vulnerable population model [14] is a “population based model that focuses on the collective health status of the individual and its community” and comprises three interrelated concepts of availability of resources, risk factors, and health status. In this study, the concept of resource availability was adapted from the model to denote the importance of the HCAHPS as a resource in regard to risk and health status among a vulnerable population with high rates of illiteracy who receive care at this Southern California hospital.
4. The Study
4.1 Purpose
The purpose of this study was to determine the best possible means to achieve high rates of return and item completion on satisfaction surveys that seek perspectives on health and quality of care from patients with a reading level of less than sixth grade or who speak English as a second language, post-hospital discharge. The need for this feasibility study was warranted, based on the high percentage of individuals with less than a high school education serviced by this hospital and the current low rate of HCAHPS return from patients at this facility, post-discharge.
4.2 Objectives
The specific objectives were to determine the rate of return and number of completed HCAHPS items from adults with low literacy who received the English or Spanish version by mail versus by phone at post-hospital discharge as well as differences in the rate of the HCAHPS returns and number of completed survey items between the two groups.
4.3 Research questions
- What was the rate of return by adults with low literacy who received the HCAHPS by mail?
- What was the number/percentage of items completed by adults with low literacy who received the HCAHPS by mail?
- What was the survey rate of return by adults with low literacy who were read the HCAHPS by phone?
- What was the number/percentage of items completed by adults with low literacy who were read the HCAHPS by phone?
- What were the differences in the overall HCAHPS return rate in adults with low literacy who were disseminated the survey via mail compared to those who were disseminated (read) the survey by phone?
- What were the differences in the number/percentage of completed HCAHPS survey items in adults with low literacy who were disseminated the survey via mail compared to those who were disseminated (read) the survey by phone?
5. Methodology
5.1 Research design
Approval for the study was obtained from the Institutional Review Board of the selected hospital. An experimental design was used to conduct this descriptive/comparative study. Convenience sampling was used, and participants were randomly assigned to one of two groups. The informed consent used for this study was developed for a sixth-grade reading level so that the participants had the option to read and review the consent form.
5.2 Participants
A total of 386 adult patients, aged 18 years and older, who were undergoing hospital discharge were approached to inquire about voluntary participation in the study and to complete a demographic assessment tool. Inclusion criteria included an age of at least 18 years, completion of the demographic instrument, self-reported education level less than 12th grade, and ability to speak, write, and read English or Spanish. The participant was excluded if he or she had mental illness. Of the 386 patients, 333 (86.3%) patients agreed to answer the preliminary survey questions and to participate in the study, but 47 (14.1%) did not meet the eligibility criteria. As a result, 286 participants were included in the study.
5.3 Setting
The study was conducted at a large, 343-bed, non-profit hospital located in Southern California, in a 85-bed (three units) medical surgical departments. According to demographic data obtained in 2014, the racial/ethnic makeup of the city serviced by the hospital is as follows: Hispanic (64.8%), non-Hispanic White (17.3%), African- American (11.2%), Asian/Pacific Islander (4.6%), American Indian/ Alaskan (0.3%), and “other” (1.8%). In addition, of the population aged 25 years and over, 31.5% did not complete high school (less than a 12th-grade education) compared to the state rate of 19%, while 26.7% had graduated high school [15].
5.4 Instrumentation
All data were examined through statistical analysis using the 2007 NCSS Statistical Software, and Statistical Package for Social Sciences, Version 12.0 [16, 17]. To analyze the comparative data, the researcher conducted chi-square statistics, using a two-sided test and an alpha level set at 0.05.
The demographic instrument used was a survey questionnaire that consisted of six self-reported demographic items: (a) current age, (b) highest grade completed in school, (c) race/ethnicity, (d) languages(s) that the patient feels that he or she speaks and reads/writes well, (e) US- or foreign-born, and(f) prior schooling if attended school in the United States. The English and Spanish version of HCAHPS was used, as these are the predominant languages spoken within the community serviced by the hospital.
6. Procedures
6.1 Modes of dissemination
The HCAHPS was disseminated either by mail or by telephone. Group 1 (control group) was mailed the HCAHPS with no follow-up, and Group 2 (intervention group) received the HCAHPS via telephone (read to each participant), with item clarification as needed. The one researcher who participated in the telephone dissemination received training to ensure consistency. The demographic tool included the hospital discharge date so that the study would adhere to the 48-hour to six-week dissemination guidelines of CMS. The HCAHPS was disseminated within one week following discharge.
Mail dissemination: HCAHPS surveys were mailed to Group 1 participants, using the current mailing address supplied by the hospital, one week after discharge, based upon the date of discharge on the demographic instrument. The envelope included the HCAHPS, instructions, and a stamped envelope for return. The participant’s name and address were obtained from the numeric coded data on the demographic instrument, and the code was aligned with the database. Once mailed, the participant was given 30 days from the mailing date to complete and return the HCAHPS before it was considered “not returned.” Those who completed and returned the survey within the 30-day time frame were considered for the purpose of this study as having “returned surveys,” and the survey questionnaire was analyzed for item completion.
Phone dissemination: Participants in Group 2 were phoned and asked whether they would like to continue in the study by completing the HCAHPS survey. The survey was disseminated one week after hospital discharge based upon the recorded date of discharge on the demographic instrument. The participant’s name and address were obtained from the numeric coded data on the demographic instrument, and the code was aligned with the database. Specifically, the HCAHPS was read verbatim to all participants in Group 2. Individuals who required more information to aid in understanding (based upon request for clarification) of questionnaire items had the items reread or were provided with additional information to assist with understanding and clarification.
Participants who agreed to have the survey read were considered for this study’s purpose as having “returned surveys,” and the survey questionnaire was analyzed for item completion. Participants who phoned and who requested not to continue in the study were considered as having their “survey not returned.” Those individuals who were phoned and were unavailable (e.g., not at home, line busy, no answer) were called again within one day after the initial phone call or phoned at a time convenient or requested by the participant if he or she was “busy” at the time of the initial phone call. Participants who failed to respond after five phone attempts or had incorrect or disconnected phone numbers were considered “attrition.”
7. Results
7.1 Demographic characteristics
Table 1 provides a demographic summary of the two groups. Educational level served as a proxy for literacy level, with low literacy defined as a self-reported highest educational level as “not completing high school” (grade completion less than Grade 12). Educational status, like that of age, was not normally distributed for this sample (Shapiro-Wilks test= 0.8972629). Self-reported grade completion for the 286 participants ranged from no schooling to grade 11 (Mdn = Grade 8). A total of 97 (34.0%) did not attend high school in the United States, all of whom self-reported their race/ethnicity as Hispanic/Latino.
The demographic characteristics of the two groups (mail versus phone) are presented in Table 2. With the exception of age, all demographic variables were analyzed at the nominal level ofmeasurement by modes of dissemination (nominal level), and the chi-square statistic was conducted to determine comparability of the groups.
The mean age of the participants who received the HCAHPS by mail was 51.4 years (SD = 17.7) compared to 47.4 years (SD= 18.2) for those who received the HCAHPS by phone. There was no statistically significant difference between the two groups by age (t= 1.87, p = .06). With the exception of the demographic characteristic of race/ethnicity, the two groups were comparable for the demographic characteristics of education; place of birth; language spoken, read, or written well; and attendance of high school in the United States. The members of both groups comprised similar numbers/percentages of African- Americans; however, participants who reported their race/ethnicity as White were fewer in number in the phone group compared to that of the mail group. A higher percentage of participants who self-reported their ethnicity as Hispanic/Latino also were found in the phone group as compared to that of the mail group. There were 12 and 8 people in the mail and phone dissemination groups, respectively, who could not be contacted. Excluding these people, the total number of participants in the mail and phone group was131 and 135, respectively.
7.2 Research questions
The results for each research question are presented below.
- What was the rate of return by adults with low literacy who received the HCAHPS by mail? Data on the HCAHPS return rates were stratified by modes of dissemination. Of the 143 survey questionnaires sent to members of Group 1 (mail dissemination), 12 were returned to the hospital due to incorrect addresses, leaving a total of 131 surveys received. Of the 131 surveys, 77 (58.7%) were returned within the designated onemonth time period, and 54(37.8%) were not returned within the designated one-month period.
- What was the number/percentage of items completed by adults with low literacy who received the HCAHPS by mail? A total of 9 (6.3%) of the 77 participants who returned the HCAHPS surveys by mail did not complete all 27 items. The items not completed varied by the participant. The number of patients and number of completed items were as follows: 68 (88.3%) of the participants completed all items, two completed 25 items, three completed 18 items, two completed 15 items, one completed 11 items, and one completed 5 items. The most frequently missed items among these participants were the later questions of the survey.
- What was the survey rate of return by adults with low literacy who were read the HCAHPS by phone? A total of 143 of the participants were phoned, using the demographic information (name and phone number) obtained. Of these, 15 (10.5%) refused to proceed with the study, including one who was “too busy” to participate, four (2.8%) were not home (repeated unsuccessful attempts, removed from the study), three (2.1%) had incorrect phone numbers, and one (0.7%) had a phone number that was continuously busy on each call attempt. Of the 135 participants, 120 (83.9%) responded to the caller and answered the HCAHPS items.
- What was the number/percentage of items completed by adults with low literacy who were read the HCAHPS by phone? Of the 120 participants contacted by phone and read the HCAHPS, all completed the 27 survey items, and 45 (31.46%) requested clarification on one or more items.
- What were the differences in the overall HCAHPS return rate in adults with low literacy that were disseminated the survey via mail compared to those who were disseminated (read) the survey by phone? A chi-square analysis was utilized to determine whether there were differences in the overall HCAHPS completion rate for the two groups. The results revealed a statistically significant difference in the completion rate. Participants who were phoned and read the survey had higher proportions of returned surveys compared to those who were mailed the survey (X²= 43.87, p<.001). Participants in the phone group were 7.4 times more likely (95% confidence intervals [CI] 3.92, 14.01) to complete the HCAHPS as compared to those who received the HCAHPS by mail.
- What were the differences in the number/percentage of completed HCAHPS survey items in adults with low literacy who were disseminated the survey via mail compared to those who were disseminated (read) the survey by phone? All of the participants who were contacted by phone (n =120) completed all 27 items of the HCAHPS. Of the 77 participants who returned their survey by mail, nine failed to complete all items. There were statistically significant differences by dissemination mode in the proportion of individuals who completed all items (X² = 14.7; p = .0001).Participants in the phone group were more likely to complete the entire survey items compared to those who were mailed the survey (OR, 33.5; 95% CI 3.3, 128.9).
8. Discussion
The HCAHPS rates of return have been lower among patients with low literacy at this Southern California hospital, which warranted the need for this study. In this study, higher rates of return as well as 100% item completion were obtained among those with low literacy when HCAHPS dissemination was conducted by phone, each survey item read, and item clarity provided to respondents, as compared to those who received the surveys by mail. These findings support the major concepts of the VPCM used in this study as well as prior research on survey use among patients with low literacy.
In this current study, the use of traditional dissemination of HCAHPS to patients post-discharge was found to result in low rate of returns in a hospital where the typical patient is of low socioeconomic status, uneducated, and with low literacy. Prior studies have shown that effective strategies to enhance patient’s understanding of healthcare information are warranted for individuals with low level literacy [13]. Like the findings of Bickmore et al., [13] this study’s findings support the need to clarify information for individuals with low literacy. In keeping with this, Al-Tayyib et al. [12] reported a connection between low literacy and the participant’s inability to accurately complete a self-administered questionnaire. These findings have important implications for the survey measurement of health-related and other behaviors.
The differences between the two groups might be attributed to the mail-dissemination group’s lack of understanding of HCAHPS items. This is consistent with the finding that the clarification of information for patients based upon their literacy level results in improvement of the HCAHPS returns rate as well as the number of items completed. Thus, effective communication is integral to successful return rates for this vulnerable population. In fact, Sofaer et al. [2] reported that effective communication was study participants’ most important and valued feature, particularly in regard to “understanding” and “explanation’” of information.
9. Limitations and Recommendations
The study was limited to one hospital located in Southern California. Thus, the findings cannot be generalized to other settings or populations of individuals with low literacy. However, despite this limitation, the findings are supported by prior studies that reported the significance of providing clarification to patients with low literacy. Moreover, the findings support the major concepts of the VPCM, indicating the importance of effective available resources to improve outcomes.
Another limitation is that the findings are preliminary; nevertheless, they are useful in supporting the need for a change in survey dissemination for patients with low literacy. What is still needed, however, is the means to identify those who are at risk for low literacy while hospitalized so that appropriate resources can be implemented to ensure comprehension of health information and improve survey response rates. Institutional policy change in regard to how the HCAHPS is distributed at this hospital may be warranted, based upon these findings. Further studies are recommended to determine whether the current HCAHPS is a suitable tool for patients with low literacy, despite its sixth-grade reading level. This is important given the culturally diverse populations in the United States, particularly in Southern California, and the lower educational levels among the poor and underserved.
A final limitation was the inability to know the patient’s true literacy level (ability to read and understand and obtain meaning from the HCAHPS survey). The demographic instrument was utilized to assess information about the individual’s highest level of education. While level of education serves as a proxy for literacy, the true literacy level of the individual, as determined by reading, writing, and understanding the HCAHPS, was not known.
This study had components of an experimental design, including an intervention (phone dissemination of HCAHPS), control group (traditional mail dissemination), and controls to reduce extraneous variables that could confound findings and randomization. The strengths of this study included the random assignment of the participants to the two groups, enabling comparability of the groups and reducing threats to internal consistency. Utilizing one individual to disseminate the survey via telephone reduced threats to validity by providing uniformity and constancy of conditions, thus reducing error. Strict protocols for data collection also enabled constancy of conditions, which further limited threats to validity. In addition, the use of an experimental design is a measure of causality regarding the effect of the intervention (phone dissemination) in improving outcomes. A recommendation for future study includes replicating the study at other facilities to confirm the findings.
The HCAHPS return rate may be the result of numerous factors. Previous studies have shown that racial/ethnic minorities and individuals of lower socioeconomic status often respond less frequently to surveys compared to Whites or individuals of higher socioeconomic status [18]. In this study, additional factors such as socioeconomic status were unknown and not considered, as literacy level was defined by educational level of less than 12th grade. The findings, therefore, are limited to the literacy as defined by educational level. The need for future research is warranted to ensure patient understanding of vital health information and feedback. Simpler surveys should be further explored, as the use of the current surveys may be disadvantageous for those low-literacy patients with a lack of understanding of health information.
10. Implications for Nursing Practice
Nurses need to take into consideration the literacy level of their patients. Nurses have an obligation to ensure that patient satisfaction surveys are aligned with the literacy skills and delivery preferences of the patients. Responses from hospital surveys can be of vital importance to nursing practice, as the feedback obtained from the surveys assists in meeting the specific needs of the population while considering their cultural and personal desires as well as their literacy skills and delivery preferences. Developing a simple and effective survey may improve health outcomes.
Competing Interests
The authors have no competing interests with the work presented in this manuscript.