Envisioning the solution

The goal

Make Pulse device installation and ownership a delightful experience by reducing the number of customers receiving incorrect charges and speeding up the time to replug for users who unplugged their device.

The how: the Unplug Model

When the Pulse device software was updated to provide new accelerometer and gyroscope data, our team had an opportunity to fix this broken user experience. Our data science team used these new insights to build a model that produced an unplug score, which represented the degree of certainty that the Pulse device was purposefully removed. 

The data science Unplug Model (shown below) takes different user scenarios and assigns to them probabilities that the signal loss was intentional. 

1. For example, if a user drives an electric vehicle the model returns a score of zero; we can’t determine whether they unplugged their device or not, so we’re assuming innocence. 

2. If the user parks or drives in an area without a signal and the device has an internal battery, there will be no accelerometer data returned and the model will return a score of 0.6. 

3. If a car is in the shop and the device is unplugged by a mechanic, the model would likely return anything from a 0.95 to a 0.7. 

4. If a user purposefully unplugged their device, regardless of motive, the same scores would return as the mechanic unplug. (Maybe if they were really angry when unplugging they’d return a higher score!) 

5. For pulse devices with no internal batteries, the model doesn’t return an unplug score until after the device is reconnected to the car, so we couldn’t determine an unplug probability until after their signal loss was resolved. 

In the end, we decided any score above 0.7 at the time of signal loss would be coded as a purposeful unplug. 

Translating the model into experiences

With the buy-in from the team and informed by the user research, I established guiding design principles for the project:

1. Assume best intention There are times when Metromile doesn’t have the data to know what a user did. In those instances, give users the benefit of the doubt.
   When interacting with the experience, users should never feel like we’re pointing the finger.

2. Be as quiet as possible Users don’t want to have to think about their car insurance, especially people who drive infrequently, but when necessary make sure our users get the info they need.
   Don’t over burden our users.

3. Proactively provide support Handling the pulse device can be confusing.
   Give people the relevant resources to feel empowered.

I knew there were going to be many possible solutions and wanted to make sure we didn’t fall into the same pitfalls as the last design. 

I worked with the PM, data science, and the engineering team to determine the different user scenarios and map the data outputs to user groups. This way, I understood how our system would or wouldn’t be able to differentiate between the various user scenarios. I also wanted to minimize the number of flows needed to reduce the engineering work and service complexity, but also make sure the flows were user-centric. My main questions were:

• What are all of the possible scenarios where a device could lose signal?

• What is the ideal service experience for each of these scenarios?

• Given the data we’re getting back from the unplug service, which scenarios look the same quantitatively even if they’re qualitatively different? 

• Where is there a quantitative difference but qualitative overlap in the ideal service experiences? 

The Unplug Model reduced but did not eliminate the probability of mistakenly charging penalty miles. Therefore we had to craft a communications strategy that accounted for the lingering uncertainty. The key was creating a service design and crafting communications that addressed the worst-case user actions while not alienating users who had done nothing wrong. The team was onboard.

I then broke the flows out further, creating a rough, but more detailed comms flow chart detailing various communication options. This went through two iterations. Between each iteration, we collaborated with our cross-functional team to hone in on the design that would meet our user and business needs as well as adhere to our technical limitations: 

 After agreeing on which user scenarios would share an experience flow and nailing down the general timeline, I broke the flows out even further, detailing the communication timelines, user and system decision points, and communication strategy. For each step, I wrote out the goals (i.e. what we needed the user to know), assumptions, variables to include (e.g. vehicle make, model, and year), and tone guidelines.

Throughout this process, I continued meeting cross-functionally. As new information came to light, the flows changed significantly:

• we decided not to allow users who got a high unplug score to set their own grace period, (i.e. a period of time where their device is not required to send a signal) through their online Metromile portal. We didn’t want to make it too easy for users who had purposefully unplugged. However, users could still contact our customer experience team directly if they had unplugged for a valid reason. At this point, our customer experience team could listen to the user’s story, look at their device history, and then use their discretion to decide whether the grace period was warranted.

• in collaboration with the underwriting team, we also added in additional reporting to identify users whose devices initially returned no unplug score but then when signal was regained, returned a high unplug score.

I talked this flow through with my cross-functional partners: product, engineering, customer experience, and underwriting. Getting input early and often from all stakeholders was critical to the process and prevented the need for eleventh-hour critical changes. As changes to the flow were made, I would update the asset so we wouldn’t lose any essential details as we moved into the next phase where we actually wrote the communications. This asset was also the roadmap our engineering team used to build out the flow, so it was essential it served as a source of truth. 

Collaborating with creative

When reviewing user NPS comments, I saw an opportunity to improve some of our terminology around our Pulse device. Specifically, users often didn’t know what the Pulse device was, felt daunted by the term “install,” and didn’t understand why they were getting “Penalty Miles.” In contrast, telling a user to “plug in” their device felt like an easier task, and the term “No Signal charges” is both more descriptive and feels less punitive. Through presenting this evidence, I persuaded the Creative Director to rethink our use of language. With the support of her team, we established new brand guidelines that cascaded through all of our communications.

Additionally, as I was writing the comms, I worked with our (incredibly talented) animator to concept animations for the emails. We faced two big challenges with emails: convincing people it was worth their time to read them and reducing the frustration a user feels when receiving an email telling them their car insurance needs their attention. By including playful animations in the comms, we hoped to make emails more interesting and more positive feeling. 

Writing the comms

With the help of the marketing team, the final draft emails were then moved into a recently revamped Iterable template. Shown below is an assortment of the communications we created: