The Process of Making Sense of ODL Data

Matthew Lee, Embedded Assessment Lead Researcher, Ph.D. Student in Human-Computer Interaction, Carnegie Mellon University 

One of the many reasons observations of daily living (ODLs) are interesting is because people don’t normally pay much attention to them as they live their everyday lives. If ODLs easily lent themselves to simple mental accounting, we would not need special sensors or mobile applications to log them. Thus, when we present ODL data back to the user, they often encounter an odd feeling that mixes familiarity (because, after all, the data show their own behaviors) and unfamiliarity (because they have never seen this particular behavior logged and presented this way). This showcasing of the mundane allows people to explore and attempt to make sense of this new kind of data. Is there a process by which people explore their own ODLs?

We had a chance to understand this process in our pilot study with two elders and four months of sensor data about their medication adherence. We gave them each a special pillbox to use that could keep track of when they picked it up and opened the pillbox doors. When we showed them their data, our participants went through a process that involved three basic steps: 1) looking for anomalies, 2) explaining these anomalies, and 3) confirming their explanations with the details in the data. When first presented with their data, our participants wanted to find the anomalous behavior — in this case, instances of days when they did not take their pills. The visualization we used actually made it rather difficult to find these days where no pill-taking event occurred, but nonetheless our users were persistent in finding the days with missing pills. After finding these anomalies, they wanted to find an explanation for why they might have missed their pills. They first tried to find a benign explanation such as being away on travel, but they also looked for explanations that might show their own forgetfulness and lack of structure in their routines. And finally, after coming up with possible explanations for why they might have missed their pills on one occasion, they were able to use the low-level details in the data to confirm their explanations.

We observed one instance with a study participant that illustrates the process involved in making sense of ODL data: As she reviewed the data, the participant noticed that she had missed her evening pills on a Friday evening two weeks prior. She thought back and explained that she visited her nephew that day and had probably taken her evening pills with her and took the pills at her nephew’s home. She then looked at the detailed view of the data for the pillbox for that day and noticed that she had the door open for 20 seconds in the morning — much longer than she thought was normal. She reasoned that she must have been taking her evening pills from that box and filling her travel pillbox in those 20 seconds.

This same sense-making process, which involves identifying anomalies, explaining them and confirming them with other or more detailed streams of data, can also apply to other times when people are exploring their ODLs for the first time. Granted, our example focuses on only one data stream, pillbox activity, but this process may very well be expanded and developed to account for multiple streams of data and different types of ODLs.

For more details on this process of making sense of ODL data, read our CHI 2011 paper.

Learn more about the Embedded Assessment project.
Read more posts from the Embedded Assessment team.

Creating an ODL Prescription [Video]

Libby Dowdall, Communications Coordinator, Project HealthDesign National Program Office

 

Deryk Van Brunt, Chronology.MD principal investigator, describes the social contract between a patient and a clinician that would comprise an ODL Prescription. This prescription, or joint agreement, would allow patients and clinicians to choose which observations of daily living (ODLs) the patient will track between clinical visits, as well as the frequency, duration and method of ODL capture.

Learn more about the Chronology.MD project.
Read the most recent post from the Chronology.MD team.

ODL Prescriptions and Patient Data Capture - Is Provider Liability an Issue?

Deryk Van Brunt, Dr.P.H., Chronology.MD Principal Investigator, University of California, Berkeley, Healthy Communities Foundation
Kathleen Morrison, M.P.H., Chronology.MD Project Manager, Healthy Communities Foundation

We live in a country where the American Trial Lawyer’s Association is one of the biggest contributors to political campaigns — to whichever party is in office. Although several attempts at federal legislative overhaul of our medical malpractice system have been introduced in the last 25 years, none has succeeded. The threat of medical malpractice lawsuits can influence the way providers practice medicine. One consequence is the practice of “defensive medicine,” which often results in more expensive care — including unnecessary tests and procedures, as well as recommended protocols. Although attempts have been made to reduce the fear and threat of malpractice liability in the medical profession (e.g., clinical guidelines, best practices, etc.), the reality is that this threat remains quite salient for many providers.

It should come as no surprise, therefore, that liability was a significant concern in the early phases of the design of our technology. On Pioneering Ideas, Steve Downs recently mentioned the Crohnology.MD ODL Prescription (read his post). The concept of the ODL Prescription, put forward here by Dan Bernstein, Nikolai Kirienko, Deryk Van Brunt and Kathleen Morrison, was developed out of a collaboration between iMedic8 and the Healthy Communities Foundation and has been embraced by the Crohnology.MD team.

The ODL Prescription is an agreement between patients and providers that details which biometrics and observations of daily living (ODLs) will be captured between visits. An ODL Prescription is the outcome of a patient/provider negotiation and represents a balance between collecting what’s important to both parties while also considering what’s most relevant in managing the patient's health. The ODL Prescription identifies which ODLs the patient will track and designates how often and when they will be collected. This patient ODL data will be captured and viewable by the patient, and it will then be shared with the provider during the next clinical appointment.

Some members of our team were concerned about the liability risk to our participating providers. For example, is it possible that once a patient logs ODL data, then the provider could be liable if a negative outcome were to result? And what if clues in the patient’s ODL data had pointed toward the negative result?

Our clinical partner, Dr. Jonathan Terdiman, quickly put the liability fears to rest. From his perspective, the ODL information that would be shared at the time of the clinical visit does not represent new information beyond what was previously being discussed; the information is just being captured and shared more thoroughly and reliably. Because the ODL data will only be shared with the clinician at the time of the clinical visit, this data is, in effect, no different from any other information a patient discusses with his or her provider at the time of the clinical appointment. The key element is that initially the system will not even allow the physician to retrieve and see the data between visits. This data workflow will be reinforced once we make our clinicians and patients aware of this protocol. Because there is an understanding that the data will not be shared between appointments, nor reviewed by the clinicians outside of the appointments, we see the ODL data as a visually enhanced — and perhaps more comprehensive — patient narrative.

This approach does, however, raise questions for future versions of our system — and other systems — as we consider sharing the ODL data in real time. Certainly, if a patient’s ODLs indicate that a situation is urgent and requires immediate clinical attention, then that data could be used to inform the clinician of the situation and to instruct the patient to seek treatment immediately. In such a case, would the clinical team be held responsible and liable if they do not act upon this data? The answer will likely relate to the expectations established and agreements made between the patient and his or her care team.

In the meantime, the ODL Prescription will allow our Chronology.MD patients to collect data that may be both clinically and personally significant, allowing us to evaluate whether the information positively impacts the efficiency and value of the patient narrative.

Learn more about the Chronology.MD project.
Read more posts from the Chronology.MD team.

Project HealthDesign, Meet Quantified Self

 Libby Dowdall, Communications Coordinator, Project HealthDesign National Program Office

This weekend, we’re sending Project HealthDesign National Program Director Patricia Flatley Brennan, R.N., Ph.D., out to California to participate in the inaugural Quantified Self Conference. The Quantified Self describes itself as a community of “users and tool makers who share an interest in self-knowledge through self-tracking.” Health and wellness topics are some of the most frequent topics for self-experimentation. 

Outside the conference, Quantified Self community members hold regular “Show & Tell” meetups in 15 U.S. cities and around the world. The largest and longest-standing meetup occurs in the San Francisco Bay Area and is hosted by Quantified Self Co-Founders Gary Wolf and Kevin Kelly, along with Quantified Self Director Alexandra Carmichael.

What types of things do Quantifies Self community members track, analyze and optimize?

• Seth Roberts spent several months tracking how his work efficiency varied by time of day and found that his efficiency improved when he began reviewing the data visually in a progress display graph.
  
  
• Matt Bell used a Zeo to track his sleep data in order to optimize his sleep quality (which he anticipated would boost his energy during the day). So far, he’s learned that meeting new people in the evening and having emotionally difficult conversations late at night negatively impact his sleep. Watch a video of Matt to learn more about his experiment.

Self-quantifiers track lots of different things as they seek to optimize their lives. Even simple experiments have yielded useful results.

Quantified Self, Meet Project HealthDesign

 

Project HealthDesign’s first nine project teams completed a user-centered design process to create health tools that provide people with actionable information to help them manage their health. The patients who interacted with these teams helped us to recognize the concept of observations of daily living (ODLs). ODLs are sensations, feelings, thoughts, attitudes and behaviors that provide cues to a person about their health state.

Our five current teams are examining how personal health technology tools can collect and integrate observations of daily living (ODLs) into personal health decision-making and clinical care. They’re currently conducting studies to find out which ODLs patients value and how tracking ODLs impacts people’s health in everyday living.

If you’re part of the Quantified Self community, we want to learn from you! Because ODLs are personally meaningful health indicators, many self-quantifiers are already experimenting with data about ODLs like mood, stress level, sleep quality, socializing, etc. Engage with our work by reading our blog or sharing your recent experiments with us on Twitter (@prjhealthdesign). We look forward to more Project HealthDesign/Quantified Self introductions!

Project HealthDesign Workshop and Emerging ODL Themes

Libby Dowdall, Communications Coordinator, Project HealthDesign National Program Office

We routinely hold workshops that allow the five Project HealthDesign teams, the national program office and other project advisers to meet in person. At a recent workshop at the Vanderbilt Center for Better Health, we focused on the evaluation phase of the project.

 

During the workshop, we identified several themes that resonate across the five teams:

Patients and clinicians have different perspectives on which ODLs are important.
The observations of daily living (ODLs) that are crucial to a particular patient’s daily life and health may not seem as important to his or her clinician. Or, a patient may grow tired of collecting data about an ODL that the clinician finds important. In “Whose ODLs Are They Anyway?” on Pioneering Ideas, Steve Downs, chief technology and information officer at Robert Wood Johnson Foundation, further examines this difference in perspective, and also details the Crohnology.MD team’s innovative approach to negotiating common ground between the two perspectives.

Patients may want flexibility in how they use ODLs.
Because the projects are breaking new ground as they look to see whether and how ODLs enter the clinical decision-making process, several teams left a measure of flexibility for patient preference. They recognized that even though patient and clinician focus groups helped to choose and prioritize the ODLs tracked in each study, each individual patient might value another ODL that is not part of the study’s standard ODL set. The iN Touch team, for example, allows participants to choose an extra function in The Carrot platform that they’re interested in tracking. Additionally, each participant in the BreathEasy study will be able to use a journal feature to write narratives pertaining to the ODLs that matter to him or her.

Clinical workflow varies by team, but several use “intermediaries” to review ODL data.
The five project teams were chosen, in part, for the various types of clinical partners with which they work. Thus, it is no surprise that each project team has taken a different approach to incorporating ODLs into the clinical workflow. We were intrigued, however, to see that several teams are using an “intermediary,” such as a health coach or nurse, to routinely analyze the ODL data and act upon the findings.

 

Our teams are beginning to recruit (and even enroll) patients for the evaluation phase of their projects. We’re excited to think that data is already being collected from sensors in several elders’ apartments as they use the phone, and dozens of overweight teens are meeting regularly with their health coaches and collecting ODLs. Rest assured that the teams will soon be sharing new developments as their studies progress!

Patient Engagement: A Key Ingredient in Accountable Care Organizations

Bob Belfort, Manatt, Phelps & Phillips, LLP

On March 31, the Centers for Medicare and Medicaid Services released a proposed rule governing the approval and operation of Accountable Care Organizations (ACOs) under the new Medicare Shared Savings Program.

The program creates financial incentives for providers to work together to treat patients across multiple care settings, including physician offices, hospitals, and in the home. Under the proposed rule, an ACO would consist of a group of providers that have agreed to collaborate to coordinate care for patients enrolled in Medicare Parts A and B. ACOs are intended to be patient-centered organizations that deliver integrated, efficient and cost-effective care. ACOs that successfully provide lower-cost care and that achieve quality targets will be eligible to retain a portion of the savings shared with the Medicare program.

ACOs would have to define processes to promote evidence-based medicine, patient engagement, and care coordination. They would also have to report on quality and cost measures.

In some cases, the proposed rule is prescriptive about the health IT tools providers should use to achieve these objectives. One example is the requirement that at least 50 percent of an ACO’s primary care physicians must be meaningful users of electronic health records at the beginning of the second year of the ACO’s existence.

In other cases, the proposed rule does not establish specific criteria. In the area of patient engagement, for example, the proposed rule allows ACOs the flexibility to choose tools that are most appropriate for their practitioners and patients. Activities that promote patient engagement may include, but are not limited to, use of decision-support tools and shared decision-making methods that help patients assess the merits of various treatment options in the context of their specific values. Patient engagement may also include methods for fostering basic knowledge about maintaining good health, avoiding preventable medical conditions and managing existing conditions.

It is worth noting that the proposed rule encourages ACOs to use remote monitoring and telehealth to coordinate care; however, the rule does not specify exactly when or how such tools should be employed.

We think that knowing a patient’s observations of daily living (ODLs) could help ACOs meet quality improvement goals under the Medicare Shared Savings Program. We also believe that the program, with its focus on patient engagement and care coordination, could go a long way toward convincing health care providers to incorporate patient-defined, patient-generated health information into the clinical care setting.

Do you agree? What are your predictions about what health care providers will do to satisfy the proposed rule’s patient engagement and care coordination requirements?  What about your big picture reactions to the ACO concept? Will ACOs help improve health care quality and efficiency or are they just another passing fad?

To learn more about the proposed rule, read the summary prepared by Manatt, Phelps and Phillips, LLC.

Listening to Health in Everyday Living

Patricia Flatley Brennan, R.N., Ph.D., Project HealthDesign Director

About three years ago, our Project HealthDesign project teams began hearing something odd and unexpected from the patients involved in their user-centered design activities — the sound of health in everyday living! Health doesn’t come labeled in the professional, clinical-speak of signs and symptoms. It sounds like a family’s laughter, strained dinner conversations or the peaceful snoring of a parent who’s sick. It sounds like the birds one hears while taking an especially long walk or the raspy breathing that precedes an asthma attack. It sounds like the words an elderly woman uses when she explains to her physician that she “just doesn’t feel right” after taking her medication.

We’ve called the phenomena these sounds reflect “observations of daily living,” or ODLs. ODLs are sensations, behaviors, environmental exposures, thoughts and social experiences that serve as clues to health and health activities. We have been trying to better understand the stories they tell.

We’ve learned that with ODLs, there is a jargon or rhythm to the sound of health, and that it doesn’t sound at all like the sounds of health care. What it does sound like may be idiosyncratic, or unique for each patient. Like the more familiar signs and symptoms, ODLs provide snapshots into a person’s health situation. However, unlike signs and symptoms, which have a scientifically proven link to an underlying health process, ODLs represent what a patient pays attention to; this is why ODLs  are so important for patient-centered care.

Our five current Project HealthDesign project teams are hearing about a whole new set of ODLs. Listen to what they are hearing:

The Embedded Assessment team at Carnegie Mellon University is using sensors embedded in household objects such as coffee pots, telephones and pill boxes to monitor how well elders are doing everyday activities. Understanding how well — or how poorly — an elder performs routine tasks may provide the early warning signals needed to bring help in a timely fashion. The sensors, if fired in the anticipated sequence, will show that the person is handling things just fine. However, sensors firing out of sequence or not at all suggests that some type of cognitive disruption is occurring — perhaps the elder can’t remember the steps to making coffee, or has forgotten to take meds at a certain time. Of course, one misstep doesn’t equate to profound confusion. That’s why we think of ODLs as sentinels, not signals. The ODL here is the firing sensors; this ODL must be interpreted and validated in order to serve as an indicator for action.

The iN Touch team from San Francisco State University is reaching out to teens who are both obese and at risk for depression. This group is going to use camera-equipped cell phones to prompt the teens to take photos of their food, record their menus on the spot and to rate things like: feelings, interactions with others, and activity levels. The ODLs here might sound like nutrition, mood, social activity and using the phone to capture information in real time. This information will be stored and reviewed later with a health coach, and could serve as the key for real success in weight management.

Again, awareness of ODLs is only part of the story. The other part is looking for patterns in the clues. It’s a lot like the sound of a crying baby — crying is an ODL, but distinguishing between the sound of a distressed cry and a bored cry means the difference between responding with comfort or stimulation. And, speaking of babies crying, we’ve wondered whether ODLs exist if you can’t hear them or know what they mean. But of course they do!

The FitBaby team at the University of California, Irvine is working with parents and young infants. They’ll be looking at how things like the number of diaper changes or the sound of a mom singing to a baby give clues to the quality of the parent-child relationships and the health of the baby. They’ve created a Mind Garden to appraise parents’ stress levels, and are using photos of baby poop to evaluate the adequacy of baby’s nutrition.

Over the next few weeks we’ll be introducing you to more sounds of health in everyday living. Join us on the journey to decode these clues and figure out how to use them to improve health!

Read more posts about ODLs.

Do People Hide “Bad” Observations of Daily Living?

Tim Patton, Project HealthDesign Ph.D. in Engineering Student, University of Wisconsin-Madison
Uba Backonja, Project HealthDesign Ph.D. in Nursing Student, University of Wisconsin-Madison

People might not be willing to share all of their observations of daily living (ODLs). This may be old news to health care providers, but it was a bit of a surprise to the engineers among us as we looked for information to help us design systems that support patients’ real needs in regard to personal health management, decision-making and monitoring behaviors.

Because they are indicators to which an individual pays attention and finds important to his or her health, ODLs are very specific to individual patients. To begin to obtain a systematic understanding of ODLs, we recently conducted a pilot in which we performed one-on-one interviews with 10 college students. The ODLs that emerged focused on health behaviors (e.g., exercise) and monitoring symptoms (e.g., pain). Because we framed the questions in terms of health, this is largely what we had expected. Throughout the pilot, we found that participants seemed willing to talk about very personal ODLs and discuss their personal habits’ and relationships’ impacts on their health.

Because of this apparent comfort with disclosing highly personal information, we assumed that participants would be comfortable disclosing all ODLs. However, we have some reservations regarding whether that assumption is true. During the interviews, not one student mentioned drinking alcohol, smoking, drug use, sex (unsafe or otherwise) or partying. In fact, none of the ODLs the students reported to us carried a negative social stigma, or could be seen as a “bad” ODL.

Were we simply lucky with our sample? The CDC estimates that 21.4% of adults ages 18–24 smoke and that  young adults and adolescents under 21 drink 11% of all alcohol consumed in the United States. Furthermore, the CDC estimates that 43% of all students report drinking in a high-risk manner at some point during college. Statistically, it is unlikely that such behavior was truly absent in our sample population.

Thus, we are left to ponder why the “bad” ODLs were missing. We have considered the following:

1. Our questions may have biased participants toward ODLs that improve or maintain health.
2. Participants didn’t think to observe the effects of “bad” things as ODLs.
3. Participants may deliberately avoid mentioning ODLs that might cast them in negative lights.

First, given that participants did mention things that detracted from health (e.g., lack of sleep, poor diet), we don’t think this can be explained by a positive bias in our interview questions. Second, other negative behaviors were noted (e.g., poor diet choices and exercise habits). Thus, we are left with a sense that people deliberately avoided telling us about “bad” ODLs.

Due to the anonymous nature of our interviews and the encouraging early results, we were hopeful we could avoid this potential issue and strive to account for it in future work. To properly design systems in support of ODLs, we must understand the full range of personal observations, including those that people prefer to keep to themselves. Thus, as we continue to push for systems that support personal health, we will likely need to find ways to safely and securely elicit all ODLs, whether good or “bad.”

Read more posts about ODLs.

Dogs as Animal Models for Research

Gillian Hayes, University of California, Irvine, FitBaby Principal Investigator

As a researcher who focuses on computing systems and their designs and uses, the checkboxes on all my various research oversight forms for preliminary animal research and animal models never made a lot of sense for my work. Two things happened this month, though, that made me rethink the whole idea of animal models for my research.

First, I watched a fascinating PBS documentary about dogs. As a dog-lover, I was keen to learn about the co-evolution of our species, both biologically and sociologically. Dogs are key animal models for research in the standard sense of being good disease models for certain human diseases. Beyond that, however, their evolution and behavior also serve to inform us about human history, migration patterns, the development of societies and more. From this documentary, I also learned the important role that dogs have in acting as “juveniles” in the lives of humans. In other words, we like them in part because they act so much like human children.

That got me thinking about one interesting way I have used my dog in my research lately: as a proxy for a premature infant. I have been debugging our FitBaby application for the last few weeks by logging information about my 9-year-old retriever. When my dog Savannah and I go out for a walk, I log in our application “Took baby for a walk;” when Savannah poops, I log another dirty diaper in the application.

As is to be expected, I have found in our software a lot of little bugs, which we are fixing. What surprised me, however, is how I began to change my behavior. I never sing to my dog. First of all, I am not a great singer, but secondly, it has never occurred to me that she might like it. The little icon on the screen that I am not earning by not singing, however, has a profound effect — one that we planned it to have on parents, who we hope will sing to their babies. I find it taunting me, wanting to be earned, and so I sing to my dog. She actually seems to like it. So, maybe we are on to a new tradition.

Learn about FitBaby.
Read more posts from the FitBaby project team.

PCAST Report Comment: Looking Toward Truly Patient-Centered Health Data

Patricia Flatley Brennan, R.N., Ph.D., Project HealthDesign Director

As director of the RWJF-funded Project HealthDesign initiative, I commend the President’s Council of Advisors on Science and Technology (PCAST) for the effort and care evident throughout their December 2010 report, "Realizing the Full Potential of Health Information Technology to Improve Healthcare for Americans: The Path Forward," and for the attention given to patients and their health information needs. I am encouraged to see that the technical standards and structures the council recommends will allow for broad and safe use of health information in the future. I also appreciate the report’s openness about the privacy and identity management concerns involved in this significant undertaking.

Yet even as I affirm the overall spirit of the report, I find myself searching for a more explicit focus on the patient. I was hoping for a report addressing health information technology, but this report focuses more on health care information technology.

Effective management and use of health data must contribute to the improvement of the health of individuals, families and societies. I echo the comments of Chris Gibbons of Johns Hopkins Urban Health Institute, who emphasizes that health data includes the life experience of the individual, which encompasses—but is not limited to—clinical encounters.

Through my work with Project HealthDesign, I have learned that clinical data tell only part of the patient’s story. We must also value any data that the patient values. Patient-defined, patient-generated data such as observations of daily living (ODLs) contribute to a full picture of the patient’s health and complement clinical care data.

ODLs serve as personal health sentinels, which give individuals cues about their health or alert them to take personal health action. Because they arise from the patient’s experience, not the clinician’s perception, ODLs may be the most truly patient-centered type of personal health data. I believe the standards and structures PCAST recommends will prove sufficiently robust to handle even these types of data.

From its beginning, Project HealthDesign has advocated a vision of health information technology (HIT) in which applications draw data from many sources and synthesize and present it in ways that guide action. As bodies such as the ONC and FDA deliberate the PCAST recommendations and use them to guide the criteria for demonstrating meaningful use of HIT, I exhort them to hold fast to the fundamental purpose of effective health data management, which is to improve health.

Related resources
• Read the full PCAST report [pdf]
• Read Project HealthDesign National Advisory Committee member Chris Gibbons' [bio] comment on the report
• Read Project HealthDesign National Advisory Committee member John Halamka's [bio] primer on PCAST-related technological terms