Inside Even


V1 app data: A retrospective (Part 1 of 3)

Even 2.0 is here, and with it fresh new app data. To get us started off on the right foot using this data, I recently sat down with our good friend and CTO Evan Goldschmidt to reminisce about how we used data to evaluate Even v1. This is the 1st of 3 little posts about that.

Our first client app, Even 1.5, launched the first week of October, 2015. As a result of a ton of pre-launch work to set up client side data collection, we were able to start looking at data from the launch right away. One of the first things we looked at was our member acquisition funnel.

The app data

The funnel showed that a lot of people were falling out of the onboarding flow after the stage called “vetting.” This was the point when new members got to see their new Even Pay number, accept it, and actually start using the app.

We were confused, because by the time sign-ups got to vetting, they’d already filled in many screens of information and connected their bank accounts to the app. So they’d done all the work, but then as soon as they got to the pay-off, they left. Why?

The follow-up

Data from a couple other sources helped us troubleshoot. First, the questions sign-ups were asking us via support chat on the day they became active were revealing. The most common question, by a solid margin, was some variation of “What is Even?”:

Image for post

Interviewing a few people who’d gotten “stuck” at vetting added to the picture. One of the main reasons they mentioned for not going ahead and using the app was they felt they didn’t understand it well enough:

“I’m just like, ‘Let me not do this, because I don’t really know what it is, and I don’t want my money to just go anywhere.’”

The user experience behind the funnel leading up to vetting became clear: people had been completing all that information and connecting their banks not necessarily because they understood the app and were psyched to start using it, but because completing those steps was the only way the app allowed them to move forward and hopefully learn more information before starting to use Even. When we finally failed to give them that information, they jumped ship.

The decision

These insights led to a raft of onboarding improvements designed to help explain the app better and put people at ease. We cut down on the information we required sign-ups to give us (making onboarding less “form-y”) and bulked up the information we gave to them (by adding a bunch of FAQs and in-app testimonials from real Even members along with their faces).

Image for post

Once we shipped a new build with these improvements, we were once again able to immediately observe the results — which were that people were tapping furiously on the new FAQs, and that more of them were making their way past vetting to becoming active.

The takeaways

  • Having logging set up prior to launch is clutch.
  • When something we’re seeing in app data seems counterintuitive to us, we can use qualitative methods to see things from the member’s perspective.

That’s it for Part 1. Now read Part 2!