Improving the quality of healthcare data through information design

Improving the quality of patient care, generally referred to as Quality Improvement (QI), is a constant mission of healthcare. Although QI initiatives take many forms, these typically involve collecting data to measure whether changes to procedures have been made as planned, and whether those changes have achieved the expected outcomes. In principle, such data are used to measure the success of a QI initiative and make further changes if needed. In practice, however, many QI data reports provide only limited insight into changes that could improve patient care. Redesigning standard approaches to QI data can help close the gap between current norms and the potential of QI data to improve patient care. This paper describes our study of QI data needs among healthcare providers and managers at Vancouver Coastal Health, a regional health system in Canada. We present an overview of challenges faced by healthcare providers around QI data collection and visualization, and illustrate the advantages and disadvantages of different visualizations. At present, user– centred and evidence–based design is practically unknown in healthcare QI, and thus offers an important new contribution.


Introduction
Quality improvement (QI) in healthcare refers to the varied initiatives across healthcare organizations that aim to improve health outcomes for patients.The QI movement emerged in force after 1999 when the Institute of Medicine (IOM) reported that over 90,000 patients in the U. S. die each year as a result of medical errors (2000).Similar findings were subsequently reported in other countries.In the 18 years since the IOM report, QI has become deeply embedded in health care practices, regulation, and culture.Although practitioners have sought to improve healthcare throughout history, QI as a distinctive field is relatively new.Approaches to QI continue to evolve.
Examples of QI initiatives include: reducing hospital-acquired infections and patient falls, implementing evidence-based care protocols, involving patients and families in care planning, and reducing wait times for surgical procedures.QI leaders often quote the motto, "If you can't measure it, you can't fix it" which speaks to the central role of QI data.This paper presents the introduction of user-centred data design as a new opportunity in the evolution of QI.
Healthcare organizations routinely collect vast amounts of patient care data, primarily for administrative purposes such as program planning, accounting, and regulatory oversight.Additional data are collected to measure performance of the health care organization as a whole.This "big data" is collected across the organization every day for every patient.
Different audiences use these data-from those who manage and oversee healthcare organizations (accreditation agencies, healthcare boards, senior executives, and managers) to healthcare providers at the bedside (physicians, nurses, and other healthcare professionals) (Aitken, Joy & Dyck 2015).
Each of these audiences is engaged in QI, but from different perspectives.In the boardroom or among hospital leaders, the focus is on comparing long-term trends across many different topics.Their interaction with the data is typically in a committee meeting room with a focused discussion.They reflect on data aggregated across multiple care settings and hospitals, then summarized in scorecards (Figure 1).QI in this case might take the form of changing policies, reallocating resources, or setting priorities.
In contrast, the focus of healthcare providers is on specific QI initiatives at the unit level and on individual patients.They need "small data." Their interaction with the data will typically be in brief meetings during which they review a multitude of patient needs, and later by viewing the data displays posted on walls that are cluttered with information (Figure 2).The goal of QI data in this context is to identify unit-level changes that will improve patient care, and then motivate and enable staff to make those changes.
Scorecards designed to meet the needs of senior healthcare leaders will not necessarily meet the needs of health care providers.Although scorecards can be subdivided to provide unit-level data, these data may still not reflect the sphere of influence for healthcare providers.For example, most scorecard data are reported as rates (or percentages), which is useful to compare large sample sizes, but not useful to study how to improve care of individual patients.
Another difference in the type of data needed pertains to the type of metric reported.For example, rates of post-surgery infections are commonly used in QI scorecards but these are influenced by many factors beyond care provided at the beside.Healthcare providers would generally find different metrics more actionable, such as whether wounds are properly assessed during a hospital stay, since proper wound care helps prevent post-surgery infections.
Every day, healthcare providers make hundreds of decisions with major impacts on patients' lives.The cognitive burden is high and the work can be stressful.Indeed, burnout of healthcare providers has been reported as an unintended consequence of QI efforts to improve patient care (Bodenheimer & Sinsky 2014).
Decades of research have demonstrated the impact of design on how data is perceived.Yet, the role of information design in meeting the needs of healthcare providers for QI has received very little attention.In 2015, we commissioned an experienced research librarian in our organization to search for publications on the topic.She found fewer than five articles that addressed this topic, none of which focused on healthcare providers working in hospitals-despite the critical need for data to improve patient care.Erwin and Krishnan (2016) note that "design methods, which many industries outside healthcare already recognise as critical to meeting the needs of the people they serve, have the potential to make delivery science more effective." Design can help present information in a form that is easy to use and understand.Data comprehension and interpretation are affected by factors such as whether the data are presented as percentages or frequencies, how the information is framed (e.g., reporting the number of people who die vs. those who survive), the graphic format (e.g., bar charts or pie charts), and the visual presentation (e.g., colours, layout, and typography) (Feldman-Stewart, Brundage & Ztovo 2007).
Understanding numerical information can be difficult for many people (Public Health England & UCL Institute of Health Equity 2015).Even experts misinterpret data.Elting, Martin, Cantor and Rubenstein (1999) presented identical data showing the results of a hypothetical clinical trial to a group of physician researchers in four different formats.When asked to decide whether the data would justify stopping the clinical trial, the accuracy of their decisions differed significantly, depending on the data format.Interestingly, user preference for data visualization did not match the formats that were interpreted most accurately.Similarly, Agostinelli et al. (2012) presented identical data to 105 physicians and hospital medical directors, in four formats.More than three-quarters failed to recognize that the data were the same in each of the different formats.Again, their accuracy in interpreting the data depended on the type of display.
Given the issues outlined above and in response to the frustrations expressed by those involved in QI initiatives, we undertook a study to investigate the QI data needs, wants, and information perceptions of healthcare managers, directors, and staff.Our setting was Vancouver Coastal Health (VCH), a regional health authority in Vancouver, Canada.VCH includes 13 hospitals, which together serve a population of over 1 million.Based on our collective experience working with over a dozen healthcare organizations, the approaches to QI data at VCH are common across other organizations.

Methodology
The research plan included three phases: defining, scoping, and understanding constraints.One-on-one interviews and discussion groups formed the main body of the research.These focused on defining the current state of QI data visualization at VCH.Details of each phase and methods are summarized in Table 1.
Participants were selected because each played a key role in providing care, planning and implementing QI initiatives, reporting data about patient safety, performance measurement, and/or clinical optimization.
Defining: Six one-on-one interviews and twelve discussion groups were conducted to understand what QI data VCH managers and healthcare providers were receiving, what QI data they would like to see (content), and how they would like to see it (format).
Scoping: A large discussion group (a regional, inter-hospital forum) was held with managers and directors of areas related to data management, performance, quality and optimization.Seven leaders spoke, and thirty participants were organized in tables of about 5 people each.Each table had a leader that took notes.
Understanding constraints: Five one-on-one interviews were conducted, three months after the forum, with staff whose roles ranged from software development to data management.The goal was to identify technological and IT system constraints and possibilities that would inform the design of new QI data solutions at VCH. Verbatim notes were taken during the interviews and discussion groups, and a thematic analysis was performed.The first step was to highlight key words to discover themes.Then, themes were used to index the data.The data was organized and sorted in a matrix.Two researchers corroborated that each theme accurately described the words under it.All the data was reviewed using the same analytic process.
In addition, site visits were conducted to observe and document how QI data are displayed in hospital hallways, wards, and public spaces.

Findings: main themes
The four main themes arising from the research were: (1) Unmet QI data needs: content; (2) Unmet QI data needs: format; (3) Feasibility: Difficulties and frustrations of generating QI data; and (4) Opportunities: Designing QI data displays to drive action.The order of the themes does not indicate a hierarchy of importance.Findings 2 and 4 are analysed in the discussion.Another participant noted that rates are misleading when they are based on a few cases such as when working at the unit level, where a 20% rate might just represent 4 patients.
Challenges in presenting the data at a unit level: Very small numbers [create] issues of interpretation and display.We've seen people get really concerned about very minor fluctuations.Interviewee 4 Although rates are useful for managers and senior leaders who monitor trends across multiple programs, they are often not useful to healthcare providers who want to know how they should change the care they provide.
Other challenges were related to the need for meaningful information that is easy to understand.
One of the major data struggles is understandability of the data, making the data translatable for the frontline staff to work with.Large Discussion group Interviewee 2 2.2.3 Feasibility: Difficulties and frustrations of generating QI data.Staff described several challenges and frustrations.Some of these concerned technology.Centralized data management systems are not always agile enough to fit the ever changing QI data needs at the unit level.
Data managers described frustrations arising from the many ad-hoc requests for data made by leaders and unit managers.Ad-hoc requests require creating tailored visualizations for the unit level.Responding in a timely way to the unique QI data needs of different users is a major challenge, since different teams use different tools to both collect and display QI data on various topics.
Staff who create data reports want to better understand healthcare providers' information needs and to automate as much as possible to keep their workload manageable.They mentioned: It' d be good to give the frontline exposure to these different tools [for automating data reports] and give them mock-ups.That's the best way to get feedback.Interviewee 3 2.2.4 Opportunities: Designing QI data displays to drive action.Participants mentioned that data should be meaningful, staff-centred, self-explanatory and appealing to help them drive action and improve quality of care.Nurses stated: We want to see trends: whether positive (help us know when we're doing well) or negative (that helps us know how to get better).…Information needs to come to us in many different ways.…We need a multi-pronged approach that quickly tells us what we need to know.

Discussion group 1 (5 nurses)
Nurses want data to help them understand how their efforts are helping.
We want to know that we're making a difference.
[We] want to know that we're actually improving care.Discussion group 1 (5 nurses) Both nurses and managers want data to inform action and drive change, and to see how their performance changes over time.
Data managers mentioned the need to develop data skills among healthcare providers and present appealing and understandable visualizations.
The graphs we create are very plain because we want them to be functional… [Healthcare providers] like getting the information in a 'per patient' format [as opposed to aggregated info] because this is how they think.Interviewee 3 Visualizations must be engaging and must stand out from the rest of the information posted.A nurse mentioned: If I have a 5 minute coffee break, I' d rather read a story than try to figure out a graph….Numbers are important but they don't have the same impact as a real patient story.Discussion group 2 Only certain things catch our eye, the data needs to be visually appealing especially in a busy climate like the ED [Emergency Department].Large Discussion group Interviewee 6

Discussion
This section discusses ways to address Findings 2.2.2 and 2.2.4,which speak to the unmet needs related to how QI data are presented, and the opportunity to design QI data displays to drive action.Finding 2.2.1, that is, the unmet needs related to the content of QI data, should be discussed in a healthcare setting and are not discussed further in this article.The same is true of Finding 2.2.3, the difficulties and frustrations of generating QI data.
This discussion is organized in three sections.The first elaborates on unmet QI data formatting needs; the second section provides visuals that illustrate the challenges of designing displays to drive action, and the third proposes competencies that designers working in health QI data visualization should have.

Why isn't user-centered design more common in healthcare QI visualizations?
Given the enormity of data used in healthcare, it may be surprising to designers that user-centered design is not firmly entrenched in this field.However, this study demonstrates that the basic principles of user-centered design are not being used to create QI data visualizations-if the data is not meaningful to people, if it cannot be used to drive action, then there is a problem.
The problem is not related to healthcare providers' motivation or intelligence.Healthcare providers consistently express a desire to receive QI data to know how they are doing, and to make decisions to improve their care practices if necessary.This motivation is indispensable for QI.
Data managers who work with QI data express a desire to better understand healthcare providers' data needs.It is important that "frontline teams who are capturing the data and implementing the improvements understand and are motivated by what you are showing them" (NHS Institute for Innovation and Improvement 2011: 15).Human-centred design can help to better understand the users' needs.However, user-centred design has received little attention in healthcare QI.
Possible reasons for this include the lack of awareness or limited expertise in the principles and techniques of user-centered design, and the traditional focus on the methodological rigor of QI data, as opposed to users' need to act on the data (Searl, Borgi & Chemali 2010).
Managers mentioned the need to improve healthcare providers' data skills.Agostinelli, et al. (2012) stated that "there is a strong need of improving the comprehension of tables and graphs within the health-care system, … to improve informed medical decision-making" (p.85).Allwood, Hildon, and Black (2013) found that despite the fact that healthcare providers have higher education, including training in statistics, "explanatory cues to aid understanding are just as necessary as with other less highly educated groups" (p.92).

The challenge of responding to users' QI data needs: Five examples
QI problems come in many forms and need different data solutions.They may relate to very rare events, to those that occur hundreds of times each day, to time-sensitive care processes, or to processes that rely on interdisciplinary team consultations.Hence, the optimal method to communicate each problem will most likely require a specially designed solution.For example, visualizing how many falls occurred in a unit per month can be done using an icon display.However, visualizing what preventive fall tools the units are using requires a different approach, perhaps a narrative one.Allwood, Hildon, and Black (2013), found that clinicians were reluctant "to select only one format, instead recognizing the need for data to be presented in more than one way" (p.91).The type of question addressed determines the type of visualization required."Different graphs are better for different judgment tasks" (Hollands & Spence 1998: 173).It seems that, even within the same group of users and topic under investigation (i.e., inpatient falls), one solution will not fit all the needs.Figure 3 illustrates this challenge.

Different strokes for different folks:
The challenge to respond to the users' needs.This section uses five figures to illustrate the problems of using rates, showing trends, proportion and number of patients, and possible ways to overcome these problems.
The problem of rates.Currently QI data are mostly presented as rates, such as number of falls per 1,000 occupied bed days.Figure 4 uses hypothetical data on patient falls to illustrate this.Managers and healthcare providers mentioned the need to present the information in a "per patient" format.This format allows the reader to easily observe that 10 patients fell on Monday (Figure 5), instead, in a rate format the reader interprets that 2.4 patients fell (Figure 4).Rates are more difficult to translate into the daily situation of healthcare providers.
Showing trends.The line graphs in Figure 5 could help nurses identify trends.Fischer, Dewulf and Hill (2005) stated that "when [participants] see line graphs, they tend to describe trends (e.g., rising, falling, increasing, decreasing)" (p.953).However, in Figure 5 the reader is required to search and locate the relevant number and relevant day, and to integrate these following a reasoning process (Körner, Höfler, Tröbinger & Gilchrist 2014).
Showing proportions.Figure 6, an icon display, helps providers see how many falls occurred during a certain period of time.The reader can easily see the proportion of patients that fell each day in relation to the total number of occupied beds.Elting et al. (1999) found that physicians made more accurate decisions when using icon displays than when using other formats.Similarly, Brase (2014) found that undergraduate students did better at inferring when they were provided with an icon display.Unfortunately, in Figure 6, it is not easy to interpret the exact number of patients, forcing the reader to count.It is also difficult to see trends.Additionally, clinicians are not used to this format.Clinicians "preferred formats with which they were familiar, although… felt that once they had familiarized themselves, they would welcome the benefits" (Allwood, Hildon & Black 2013: 92).
Showing proportions, numbers and trends.Figure 7 helps to identify the number of inpatient falls, to understand the proportion compared with the total number of occupied beds, and to see trends.It also helps to compare across the days of the week.Hollands and Spence (1998)  reported that "subjects shown bar graphs… required more time and made larger errors than they did with divided bar graphs" (p.174).The visual attributes and the task to perform influence the effectiveness of bar graphs.As Fischer, Dewulf and Hill (2005) explained, "bar graphs are an efficient but task-dependent means of visualizing numbers" (p.954).
Visualizing in a task-appropriate way.Visualizations enhance comprehension when they are presented in a task-appropriate way (Schnotz 2005).Figure 8 shows which patients fell, how many times, and where, helping healthcare providers identify possible causes.
Nurses can see that patient 2 fell three times: this affects their understanding of the number of total falls.This information is masked in the previous figures.
Nurses could link the repeated falls with a cause, for example with medication side effects.They could observe that four falls took place in the bathroom and begin to analyse why falls are happening and what can be done to prevent them.To facilitate problem solving, the visualization should help to identify patterns or recurrent events, to check assumptions, to discover unexpected aspects of the data, to make comparisons, to foster inquiry, and to obtain insights (Gelman & Unwin 2013).
The purpose of visualization is insight, not pictures.
The main goals of this insight are discovery, decisionmaking, and explanation.Information visualization is useful to the extent that it increases our ability to perform these and other cognitive abilities (Card, Mackinlay & Shneiderman 1999: 6).
Facilitating comparison.Nurses emphasized the need to understand How are we doing?To know how a unit or hospital is doing requires also answering the question Compared with what?(Tufte 2006).Providing clues for comparison is fundamental for understanding."Comparison is a key strategy in applying clarity to information" (Frascara 2015: 46).Knowing that 3 patients fell in July is not enough to answer How are we doing?It is necessary to compare with other units, other hospitals, or any other framing information."Data without context is not information" (Frascara 2015: 47).

How human-centered design can help tackle the QI data conundrum
To create QI data visualizations that engage the hearts and minds of healthcare providers, a new approach is needed.Though human-centered design is an established and prolific field, the authors submit that QI teams in healthcare are generally unaware of these principles when designing QI data visualizations for healthcare providers.
Expertise in information design is key to arrive at effective visualizations.This involves considering the nature of the content, the purpose of the communication, the context of implementation, the goals of the users, and the need to stand out from other communications (Delin, Searle-Jones & Waller 2006).If the content of the visualization does not correspond to the healthcare providers' QI data needs, wants, preferences, and motivations, then the visualization will not be effective.
Once the appropriate content is determined, the design of the visualizations should be tailored to the needs of the healthcare providers.This implies a humancentred design approach, involving the users at every stage of the design process.Once the users have provided feedback on the visualization, it should be empirically tested using both quantitative and qualitative methods to measure response time, accuracy and preferences (Stahl-Timmins, Pitt & Peters 2010).Unfortunately, visualizations "of performance information for clinicians are often being adopted in the absence of much research evidence on the best format and content to use" (Allwood, Hildon & Black 2013: 86).
Data visualization is a complex human communication process that requires diverse skills.The design of visualizations, like other tools to facilitate comprehension, needs to be guided by a deep understanding of human perception and cognition (Meirelles 2013;Schnotz 2005).QI data visualization requires adequate expertise and resources to obtain effective results.
The visualizations commonly used with healthcare providers often provide little insight on what they should do to improve the quality of care for their patients.
Providers are left to fill in information gaps based on their subjective knowledge and personal memory, which is vulnerable to cognitive biases (Kahneman 2011).
As shown, the interpretation of information is influenced by the users' data needs and expectations, knowledge about the content, data skills, task to accomplish, and quality of the visualization, as well as by the users' level of familiarity with the type of visualization (Ancker, Senathirajah, Kukafka & Starren 2006;Gelman & Unwin 2013;Schriver 1997).Data should be designed to help healthcare providers understand what is going on in their unit and hospital, to inform action, and to drive change aimed at improving care.
Based on the discussion, it can be argued that healthcare providers' decisions related to improving quality of care are influenced by the quality of the visualizations they work with."Data visualization of health information is an important component of care, impacting both the accuracy and speed of decision making" (Le, Reeder, Thompson & Demiris 2013: 250).Poor visualizations result in poor decision-making.
This investigation shows that different groups of users have different QI data needs, and that each QI problem is unique; the type of data to display can vary Knowledge about organizing and signalling information to reduce cognitive load.QI requires solutions that go beyond aesthetics trends.
2. The purpose of QI data is to drive change, reflect change, and compel action.
Ability to design persuasive visualizations to engage hearts and minds, so that healthcare providers will change their care practices.Knowledge of typography and layout is not enough.
Ability to learn about the users' problems, propose solutions, test effectiveness, provide evidence that the solution proposed is appropriate for the healthcare context and users, and monitor implementation.
4. Healthcare systems are complex.Ability to understand healthcare environments and the variety of intersecting factors that affect performance.
5. Healthcare QI systems tend to see design as a luxury and not as a necessity.
Ability to make the case for the value of human-centred design to tackle the unmet QI data visualization needs.Show that meaningful visualizations can improve patient care.
from unit to unit, and might need to be changed every few months.The VCH QI data visualization challenge is a common problem in healthcare.In collaborating with 5 other health organizations in the Canadian province of British Columbia, the authors have found that all their counterparts share similar QI data struggles.From these collaborations, and work with numerous designers, the authors propose that designers creating healthcare QI data visualizations need to have the following competences to address the five issues listed in Table 2 (previous page).

Conclusion
The study provides an understanding of the current situation of QI data visualization at regional healthcare organization.Addressing the current QI data needs of healthcare providers is essential to improve the quality of healthcare.This paper presents specific and concrete examples demonstrating how different types of visualizations may support different understanding and reasoning.It also demonstrates the value of human-centred design to create effective QI data visualizations in healthcare.
The lack of comparable studies on QI data visualization and interpretation in hospitals presents a barrier when interpreting and supporting our findings, and indicates the need for further research.To our knowledge, this is the first study to address these data needs.

Figure 1 .
Figure 1.This scorecard is typical of health care organizations.Overall organizational performance is shown under the column "VCH, " followed by performance of individual hospitals (e.g., Richmond) or groups of hospitals.

Figure 2 .
Figure 2. Currently QI data is displayed on boards placed in different rooms and hospitals.The amount of information presented, the way it is presented and the lack of hierarchy makes it hard to know where to start reading.Where and how the data is displayed is also part of the visualization problem.

Figure 3 .
Figure 3.Although working on the same topic (i.e., falls), different QI teams might need to investigate different research questions.This presents barriers for automation and standardized visualization.Standardization might work only for certain types of questions.

Figure 4 .Figure 6 .Figure 7 .
Figure 4.A graph showing falls in a unit as a rate x/100 occupied bed days.Notice the difficulty that it presents for interpreting the number of patients that fell per day.

Figure 8 .
Figure 8.This visualization shows that patient #2 fell three times.A repeat faller can distort the number of total falls per day.This visualization shows the need for special attention to be paid to this patient.The location of falls is also visible, alerting staff to possible environmental hazards.

Table 1 .
To better understand the VCH QI data visualization challenge (and the resulting opportunities), extensive consultations were held with healthcare providers and leaders Unmet QI data needs: content.This theme relates to data that healthcare providers said they need, but do not receive.This pertains to what is measured, as opposed to how it is presented.These needs were organized by topics such as hand hygiene, stroke, and sepsis (a life-threatening condition caused by infection).For example, in relation to sepsis nurses wanted to know: Methods 1. Six one-on-one interviews Participants: Healthcare Managers Questions -How do you get QI data back to healthcare providers?-How (in what form & how frequently) do you want to see QI data?-What challenges do you face in getting timely and meaningful QI data back to healthcare providers?2. Twelve Discussion Groups Participants: Healthcare and data managers; and healthcare providers.The number of participants per group varied from 3 to 10. -How (in what form & how frequently) do you want to see QI data?To provide a breakdown per unit of [x] cases [per] week …would produce a graph that is almost entirely 0's with only two lines.Interviewee 3

Table 2 .
List of competencies designers require to visualize health data Healthcare providers work in a busy, cluttered environment.QI data are typically presented in the midst of many other types of information.