Using a "think-aloud" process to aid co-design in development of a medicine information needs assessment tool

Abstract

Medicine administration is a key component of nursing practice that encompasses consumer education. However, the information a consumer considers important in their health journey is difficult to ascertain in a timely manner. This paper describes one part of a multi-stage project aiming to develop a new tool to assist nurses, pharmacists and other healthcare professionals to better meet the medicine information needs of consumers. The development of this questionnaire involved a multi-step co-design process. Here we report on our findings to the research question of “How can co-design be used in the development of a patient-reported outcomes tool?”

Specifically, we report on the process of using a "think-aloud" process to aid co-design in the early testing and refining of the data collection tool (Medicines Information Needs for Individuals – Questionnaire: MINI-Q). Several rounds of "think-aloud" interviews with consumers were used to determine the tool’s acceptability for use, initially as a paper version and then as a digital version completed using a tablet device.

Consumers (n=40), with an average of three medical conditions and taking six regular medicines, were interviewed. They identified 315 issues with the MINI-Q. Their suggestions included clearer wording, reduced length, simplified sentence structure, changed formatting, type of response options, and additional information to collect. The "think-aloud" technique was helpful to improve the usability and acceptability of the tool. The next stage is to test the final version with a larger sample to determine acceptability and face validity.

Introduction

Enhancing consumer engagement with their health care is expected to lead to improved health outcomes (Hopkinson, 2018). Engaged consumers should be viewed as active stakeholders in any health information development and therefore their perspectives need to be considered when developing information resources (Featherall et al., 2018; Horne, 2017). However, often health information initiatives are provider-driven, with consumer perspectives ignored. This approach risks consumer needs being inadequately addressed (Featherall et al., 2018; Lee et al., 2019; Young et al., 2018). The role of consumers in managing their own health has changed with increased access to on-line information (Honey & Westbrooke, 2016). They use the internet to explore aspects of their health, including to seek information about their medications (Honey et al., 2018; Kusch et al., 2018). However, accurate and inaccurate sources of medicine information can leave them with questions (Honey et al., 2018; Sheridan et al., 2015).

It is imperative to better understand consumers’ medicine information needs so that nurses, pharmacists and other health professionals can meet these needs and potentially provide information aligned to consumer priorities (Lee et al., 2019; Sze et al., 2018; Young et al., 2018). Not considering consumer needs may lead to poorer health outcomes, reduced medicine adherence, and resource wastage. This is particularly important when considering the creation of any health information assessment tool, as any design that does not involve consumers may lead to a tool that does not achieve its aim (Ioannidis, 2016). Using a co-design approach is a method that can ensure consumer engagement and acceptable tools (Nakarada-Kordic et al., 2017).

This paper reports our findings from addressing the research question “How can co-design be used in research and practice to improve the development of a patient-reported outcomes tool?” The paper first defines co-design and "think-aloud" processes before describing the application of these to gain an understanding of consumer perception of the Medicines Information Needs for Individuals – Questionnaire: MINI-Q. These co-design and "think-aloud" processes occur as part of a larger project, and so the context of the work presented in this paper is provided. The iterative "think-aloud" process used is described before presenting the results from the "think-aloud" process with consumers using a paper version (n=30) and a digital version (n=10). The discussion considers the results in light of wider literature before identifying limitations and the next steps in this project.

 Co-design

Co-design is described as involving collaboration between researchers and end users (Goodyear-Smith et al., 2015). Slattery et al. (2020, p. 2) define co-design as “meaningful end-user engagement in research design and includes instances of engagement that occur across all stages of the research process and range in intensity from relatively passive to highly active and involved.” Terms used to describe co-design approaches include participatory research methods; participatory action research; consumer, stakeholder or community engagement; and inclusive health research (Slattery et al., 2020). Therefore, a defining feature of co-design is that it is emergent and flexible as feedback from consumers is included (Goodyear-Smith et al., 2015). Using a co-design approach ensuring consumer understanding is central to the development of the questionnaire or research tool rather than the researcher or health professional making assumptions about a consumer’s interpretation or understanding. Consumer-centered co-design approaches can be incorporated into research in a variety of ways including consumer involvement in research planning, product design, stakeholder engagement, collaborative data collection, shared analysis, and distribution of results (Slattery et al., 2020).

A 2020 overview of 23 literature reviews (Slattery et al., 2020) identified there were no comparative studies indicating co-design led to better health outcomes or cost-effectiveness compared to traditional research approaches. However, the benefits of co-design included ensuring research topics, questions and design of materials were more applicable and acceptable; increased consumer knowledge and confidence about managing their health; improved ease of access to participants by researchers, particularly recruiting from hard-to-reach groups; and increased acceptability by consumers of research instruments that were developed using co-design (Slattery et al., 2020).

 "Think-aloud" process

One way to collaborate with consumers and gain insight into what they are thinking is to use the "think-aloud" process that was first ascribed to Ericsson and Simon (1980). Think-aloud techniques are derived from cognitive psychology with the basic procedure being to ask the participant to verbalize what they are doing as they complete a task (Mes et al., 2019). Verbalizations are considered to reflect the cognitive and metacognitive processes and strategies used as participants engaged with the task (Charters, 2003), therefore giving insight into the participant’s understanding. While there is some thought that the "think-aloud" can be carried out either alongside the task or soon after completing a task (Peute et al., 2015) evidence supports concurrent task and "think-aloud" processes (Charters, 2003; Peute et al., 2015). The "think-aloud" process has been used successfully to understand the thinking process of patients as they complete questionnaires (Mes et al., 2019), including the Interpersonal Skills Questionnaire (ISQ) (Al-Jabr et al., 2018), Illness Perception Questionnaire (Aujla et al., 2020) and Quality-of-Life Questionnaire (Zhang et al., 2017). Involving consumers in the co-design of questionnaires reduces common issues in questionnaire design such as response error and incorporates people’s real-world experiences into the questionnaire design process (Mes et al., 2019).

 The project to date

This paper reports part of a larger project that aims to develop a tool to assist nurses, pharmacists and other health professionals to meet the medicine information needs of consumers. There are four phases to this project. Phase One was a needs assessment to explore what information consumers wanted to know about their medicines, when they want this information, and in what format they prefer it. Data collection for this phase involved 60 face-to-face interviews (30 mental health service users and 30 medical in-patients). The findings highlighted that consumer medicine information requirements are diverse and need to be individualised (Ashmore-Price, et al., 2017; Chan et al., 2020; Honey et al., 2018; Sheed et al., 2017).
The second phase consisted of a literature search to identify suitable tools to address these diverse needs. Four tools were found that potentially could be suitable: The Satisfaction with Information about Medicines Scale (SIMS) (Horne et al., 2001); The Beliefs about Medicines Questionnaire (BMQ) (Horne et al., 1999); The Intrinsic Desire for Information Scale (IDI) (Duggan et al., 2002); and The Extent of Information Desired Scale (EID) (Duggan & Bates, 2000). These tools were then reviewed by the research team to decide whether these could be used as they are, would need adaptation, or if a bespoke tool was needed. Of the relevant tools identified, the SIMS was considered to have the best coverage of the needs identified in Phase One; however, the SIMS was developed in the United Kingdom (UK) nearly 20 years ago as a measurement tool for medicine information audit and research purposes (Horne et al., 2001). The 17 items in the scale are written in the second person and assess UK consumer satisfaction with medicine information retrospectively. This means consumer perception is gained after medicine information has been provided rather than assisting in the identification of preferences and priorities prior to receiving the medicine information. After reviewing the tool, the research team concluded that adaptation and customisation for our purpose and to the New Zealand (NZ) context was needed, which was Phase Three of this project (Amende et al., 2019; Brackley et al., 2019).

Phase Three began with the development of the first iteration of the Medicines Information Needs of Individuals Questionnaire (MINI-Q), which was informed by the aim of the project, the data collected from consumers in Phase One, and the literature identified in Phase Two, the SIMS in particular. This paper focuses on the consumer engagement aspect of Phase Three, during which the usability and acceptability of the MINI-Q by general medical and surgical in-patients at one large urban hospital were explored using a qualitative approach collecting data via a "think-aloud" interview process.

 The Medicines Information Needs of Individuals Questionnaire (MINI-Q)

The first version of the MINI-Q was a 20-item tool designed to be completed by the consumer initially on paper, with a digital version developed once the content of the tool was finalized. The long-term goal is that the information gathered will be incorporated into consumers’ health records and be available to nurses, pharmacists and other health professionals to guide their medicine information provision, so it is individualised to the consumer’s needs, preferences and immediate priorities. The decision to have a digital version was informed by the long-term aim of being able to integrate the findings with the electronic healthcare record. The MINI-Q comprises three main domains (Table 1): information about the consumer, information requested, and preference for how and when to receive the information. The MINI-Q is designed to be completed by the consumer, and responses represent their needs at that point in time. This paper reports on the use of consumer co-design to test and further develop the MINI-Q tool by informing question and questionnaire refinement and future implementation of the questionnaire in practice and research.

Table 1. MINI-Q Domains with examples of the information sought.

Methods

A staged iterative "think-aloud" interview process was used to explore consumers’ understanding of the MINI-Q questions and their use of the MINI-Q in the paper and digital versions. The focus of these interviews was to identify inappropriate items in the tool and those that lacked clarity or were hard for consumers to understand. In addition, we sought to identify topics considered important to them that were not included in the tool and whether the instructions for completion were easily understood. The core research team reviewed the data collected from consumers every five interviews and used the findings to refine the tool before the next five interviews were conducted, in an iterative cycle of tool development in line with cognitive ‘think aloud’ methodology (Mes et al., 2019). Ethical approval for this study was received from the NZ Health and Disability Ethics Committee (reference: 18/NTA/137).

 Participants

Consumers, aged 18 years and over, admitted to a general medical or surgical ward at a large urban hospital during the study period (November 2018 to August 2019) who had previously taken medication were purposively recruited to capture the views of people with a range of ages, ethnicities and long-term conditions representing the population demographics of the study hospital catchment area. The aim of this purposive sampling was to be able to assess the usability of the questionnaire among a diverse range of interviewees to ensure its acceptability and applicability to all potential users of the tool. Potentially eligible consumers were identified from the daily patient list by clinical pharmacists. The researcher then discussed the consumer’s suitability to participate with the clinical pharmacist and nurse on each ward. Those invited to participate could consult with their family or support person before deciding to participate. They could also have a family member or support person present during the interview. The inclusion criteria for participation were 1) the ability to speak English at a sufficient level to understand the purpose of the study and give consent, 2) experience taking any medication (prescription, pharmacist-only, pharmacy-only or general sale), and 3) willingness to have the interview audio-recorded. Interviews were conducted at a time convenient for the consumer during the hospital stay in a private area of the ward. Any specific medicine information needs or concerns identified during the interview were reported to the ward clinical pharmacist for follow-up.

 Data collection

The interviews were undertaken by two researchers, and after initial training, two preliminary interviews were conducted as "pilots" to ensure researcher capability and consistency. Participants were given paper versions of the MINI-Q. Using the "think-aloud" technique, researchers conducted one-on-one semi-structured interviews in the ward. Interviewees were asked to share their thoughts as they completed the tool (Figure 1). Consumers were asked not to use a pen so that they would verbalise all thoughts and responses. Responses were audio-recorded, and field notes taken. Usability factors such as time taken to complete the questionnaire, question items skipped, and the order in which questions were answered were documented. Demographic data were collected on age, gender, ethnicity, long-term conditions, and number of medications. The prescription medicines reported by the consumer were then confirmed with their clinical record, with their permission. As a thank you for participation, a NZ$20 shopping voucher was offered to all participants.

Figure 1. Figure illustrating the think-aloud process undertaken and consumer input (co-design) into MINI-Q development

This process was undertaken in eight iterative stages with five participants at each stage (total of 40 participants; 30 participants for the paper version and 10 participants for the digital tablet version). After each stage the research team convened and decided on the changes to be made to the tool before the "think-aloud" task was repeated with the new version of the tool using another sample of five consumers. After six iterations using a paper-based version, the MINI-Q was transferred to a digital format. The iterations continued until few new comments or suggestions were received.

Results

Consumer characteristics

In total, 46 consumers agreed to participate in this study. However, two withdrew due to fatigue, two were unable to fully participate, and two were discharged or transferred from the ward before the interview, leaving a total of 40 who participated. One interview was only partially completed due to the consumer not feeling well enough to continue, and these data were included. The interviewees had an average of three long-term conditions, with one having seven long-term conditions. The average number of prescribed medications taken by each were six, with the maximum being 12. Of these 40, a third were NZ European, just over half were female, and their ages ranged between 19 and 92 years old (Table 2).

Table 2. Frequency and percentage distribution of the characteristics of consumers participating in the co-design study (n=40)

"Think-aloud" interviews

The 40 interviews, which lasted from 20 to 75 minutes, identified a wide variety of issues and suggestions. Across all interviews, participants identified a total of 315 issues with the MINI-Q. The highest number of issues identified was from the first group of five consumers. The number of changes suggested to the tool reduced with each version of the MINI-Q, in conjunction with a somewhat reduced interview time (Table 3). Changes suggested concerned the wording, length and structure of sentences; order and type of response options; and formatting. Consumers also suggested including three additional medicine information questions. Additional statements added to the MINI-Q included “changes to my medicines, e.g. tablet colour, dose etc.,” “what impact taking my medicine will have on my lifestyle,” and an item on concurrent use of natural health products.

Table 3. Issues and changes made for each iteration

Six iterations of the MINI-Q were needed before only one new comment or suggestion was received. Iterations seven and eight were completed using a table. Overall, consumers found the MINI-Q easy to use, and there were few new comments suggesting possible changes (see Digital Issues section).

Issues about wording and the need to improve clarity led to refinement of the MINI-Q. For example, the introduction in the first iteration had 101 words and stated: “We would like to ask you what you would like to know about your medicines. People often have preferences about what information they would like to know about their medicines and how important this information is to them. This will help us understand what is important to you. There are no right or wrong answers. We are simply interested in what you think. Your answers will help our team to best understand and cater for your information needs. We may not always be able to meet all your requirements, but we will do our very best to meet them wherever possible.”

This was finally modified and reduced to 46 words to read: “People often have preferences with the information they would like to know about their medicines. Your answers will help us understand what your preferences for medicines information are at the moment. There are no right or wrong answers; we are simply interested in what you think.”

Table 4 provides a summary of the main sections of the MINI-Q and the key changes that were made as a result of the interviews and consumer feedback gathered during the "think-aloud" process. As seen from Table 4, the co-design process with consumers was a critical part of questionnaire development as over 30 edits were made across all questionnaire domains, ranging from the introduction to the wording of the items and the format of the questionnaire.

Table 4. Description of MINI-Q sections and a summary of the changes made

Digital issues

When the MINI-Q was transferred to the tablet device, a further 10 consumers were interviewed. Few problems with the MINI-Q itself were identified; however, some device-specific issues relating to the electronic format of the MINI-Q were found. These included a small sidebar that made it difficult for one consumer to scroll and navigate down the page. Another participant accidently opened a file on the tablet from the tool bar and then required help to close it, and another noted a preference for larger text. Another interviewee shared that he found it daunting to use the tablet at first, as he had not used one before and did not think he would be able to. He was surprised at how easy it was to read and understand. Further comments included “easy survey to use on the tablet” (consumer aged 71) and “easy to use and the tablet was simple to navigate” (consumer aged 21).
Generally, consumers read each question once and immediately provided an answer. A number of them experienced interpretation problems with questions assessing similar topics such as “what my medicine does” and “how my medicine works”.  As the “think-aloud” technique asked interviewees to verbalize all answers, problematic items, identified by a delayed response where a consumer appeared to read an item two or three times before providing a response, could be readily identified and followed up with sensitive probing by the researcher to ascertain the exact nature of the issue.

Discussion

Interviews incorporating a staged iterative "think-aloud" process proved a useful technique for testing the usability and acceptability of a consumer-facing data collection tool and making tool iterative refinements in a time-efficient manner. The interviews resulted in many subtle refinements being made to the MINI-Q via a series of tool iterations. Many of the suggestions made either had not been considered by the research team or differing opinions existed within the team. Rather than the research team reaching a consensus on what wording or format was clearer, consumers themselves indicated the improvements needed. This led to a succession of improvements in the understandability of the MINI-Q that were led from a consumer perspective. These changes eventually resulted in a version of the MINI-Q that was deemed sufficiently understandable to be transformed into a digital version. The next sample of participants then completed the digital version of MINI-Q during the "think-aloud" interviews using a tablet supplied by the research team. We believe that this was a more inclusive, responsive and time-efficient way of receiving feedback than the traditional methods used during questionnaire design and development whereby often the first step is piloting one version of the tool with a small group of the target population (Polit & Beck, 2017).

With this method, clear communication between the researcher and interviewee is important. Spending time establishing a relationship with the participant and building rapport before a participant completed the MINI-Q was considered crucial by the researcher conducting the interviews. Establishing rapport enables the collection of rich data, i.e. comprehensive and truthful information, and a shared understanding of issues during the "think-aloud" interview process. This method also reduces the need to interpret participant responses as ambiguous since responses can be clarified during the interview. For consumers, rapport enabled them to feel comfortable answering questions and have open and easy communication and sharing. Similarly, establishing rapport with a healthcare professional increases patient satisfaction (Grobosch et al., 2018).

Increased consumer access to online information, including medicine information, brings challenges for nurses (Honey & Westbrooke, 2016). Consumers may need support to understand information they have located online, as well as assistance to discern if the information is relevant for them (Honey, et al., 2018; Kusch, et al., 2018; Sheridan, et al., 2015). Using tools, such as the MINI-Q, may support nurses, pharmacists and other health professionals to provide targeted information aligned to consumer priorities. In addition, providing the MINI-Q in a digital format could improve accessibility, as text can be enlarged if vision is impaired and a "read aloud" function can also be used. Having options to view the MINI-Q in different languages could further improve usefulness and accessibility of the tool, although information provided would likely have to be in the same language to ensure responsiveness to the consumers’ needs. The digital version may also be preferred by other groups. For example, there is evidence that young people prefer their medicine information in a digital format (Monkman et al., 2020). However, there have been reports of those with low health literacy using digital technologies less than those with higher health literacy (Levy et al., 2015).

As the MINI-Q is primarily intended for consumers to complete independently, consideration was given to the understandability of the MINI-Q by the development team. However, poor English literacy levels could still limit access to the MINI-Q as adequate skills in English reading and comprehension are required to complete it independently. In these cases, completion of the MINI-Q could be assisted by a health professional with the instructions, items and response options being read out and further explained if necessary. However, even if medicine information needs and preferences are identified and met, poor health literacy can be a barrier to understanding the medicine information provided (Kusch, et al., 2018). A North American study suggested that low health literacy, in terms of medicine information needs, results in poor understanding of medication information, forgetting to take medications, and ignoring side effects and instructions relating to food or interactions (Wali & Grindrod, 2016).
Nurses, pharmacists and other health professionals must be skilled in accessing and critically appraising, evidence to be able to answer consumer questions related to their medicines. These are the skills of evidence-based practice, yet research indicates that healthcare professionals vary in their ability to use these skills (Weng et al., 2013). This has implications for ensuring that evidence-based practice skills are included in the educational preparation of healthcare professionals, as well as in their continuing education. Being able to access credible medicine information will support healthcare professionals to better meet consumers' medicine information needs (Young, et al., 2018).

Limitations of this work include the small sample size, the use of purposive sampling, and the nature of the in-patient population that limits generalization of the findings to outpatients in the primary care setting. Purposive sampling aimed to ensure consumers represented the population characteristics and demographics of the study hospital catchment area. Such an approach in co-design is supported by Slattery et al. (2020), who considered co-design to be more effective and inclusive in reaching those who are sometimes omitted or under-represented. However, whilst purposive sampling led to the inclusion of consumers with diversity of ethnicities and ages, the sample size was small and cannot be considered representative. This study was undertaken with in-patients at one large urban hospital, and the participants were drawn only from general medical and surgical wards. Whilst these wards were selected to capture a range of consumers with a variety of health conditions, this approach may have limited participation of those with rarer health conditions. An area for future research and further development of the tool’s coverage and use will be to explore the medicine information needs of those with a broader range of health conditions, cultural backgrounds, ages and ethnicities. The next stage that is planned in the development of the MINI-Q is for a much larger group of health consumers with varying demographics and health conditions to complete the latest version of the MINI-Q independently across multiple sites. This should somewhat address this limitation, and a large sample may also support exploring reliability and validity.

The study reported here is part of Phase Three of the larger project. It is anticipated that the outcomes of this research project will be translated into practice and inform the development of services and strategies to address each patient’s specific medicine information needs. As mentioned above, the next step is to test the refined MINI-Q in a digital format using tablets with a large sample of consumers. In addition, feedback from nurses, pharmacists and other health professionals is needed to consider how the information arising from the completed MINI-Q can best be presented to allow health professionals to make use of it to meet the needs of consumers. This may be in a summarized or infographic format. Furthermore, this information could also be integrated into electronic health records. Long-term, it is hoped that the responses to the MINI-Q will be available to all nurses, pharmacists and other health professionals working in hospitals, so that individualized and targeted medicine information and education can be provided in the format the consumer prefers before discharge. Also, each consumer should be provided with a copy of their responses to share with other health professionals after discharge to facilitate communication between different points of care. Further research will be needed to assess the fit of the MINI-Q for community dwelling consumers.

Conclusions

Understanding consumers’ medicine information needs has the potential to benefit them as nurses, pharmacists and other healthcare professionals will be able to better meet their needs. The MINI-Q is a prototype medicine information needs assessment tool designed to collect data about the medicine information needs of individuals and present it to health professionals in a manner facilitating its use. Adopting a co-design approach and the "think-aloud" interview process with a small sample of consumers supported further development and refinement of the consumer-facing side of the MINI-Q. This study confirmed the consumer acceptability of the MINI-Q as later iterations of testing identified very few new issues. With further testing using a tablet and a larger sample of consumers, followed by co-designing the visualization of the outputs with health professionals, it is hoped this would maximize the potential of this tool prior to implementation into clinical practice.