Naim Kabir is a Data Scientist and Software Engineer who went from researching human decision-making at the University of Pennsylvania to studying how best to improve the financial lives of employees.
Evan Goldschmidt is Even's CTO & VP of Product. He joined Even to optimize for impact. Previously, Evan worked at Google.
“A desire to find the truth, not win arguments.” This is one of Even’s core values, and to us, it means caring more about getting the facts right than about being right — even if those facts are inconvenient, uncomfortable, or disappointing. We’ve cared deeply about this since the company was founded; Even’s very first hire outside the founding team was for a head of research to talk to real people outside Silicon Valley and figure out what their day-to-day lives are about. This was one of the original ways we tested our assumptions, with the goal of building a product that not only truly helped people, but was engaging enough that they’d actually use it enough to see value.
Since then, the company has grown, and along with it our approach to research, testing, and experimentation has grown too. Recently, we participated in a panel during EMERGE Insights, the Financial Health Network’s annual event for innovators across industries to further financial health featuring speakers from JPMorgan Chase, PayPal, MetLife Foundation, and Plaid. During the conversation, we harkened back to Even’s early days, when we were part of the Financial Solutions Lab’s investment accelerator program in 2015. Maria Lajewski, Director at Financial Health Network, kicked things off by explaining why innovative testing is so difficult — yet important — in the financial services space.
The Financial Solutions Lab, part of the Financial Health Network, runs an “accelerator” for fintechs. Small startups become part of the accelerator to receive funding capital, professional assistance, and mentorship from industry leaders. The goal of the Lab is to help companies provide innovative solutions for consumers, in spite of the fact that “understanding an individual’s financial life is complex.” This is especially true at resource-constrained startups, and as a result, Lajewski notes that entrepreneurs sometimes end up chasing “vanity metrics” — stats about things like downloads or signups, which don’t necessarily indicate whether users are getting any value from the product.
We knew that anything we built and brought to market would need the assumptions behind it tested — early, and often.
Lajewski explains this is the entire reason they started the Financial Solutions lab: to help startups like Even that are “all in on solving real problems for millions of households across the country.” She says that when the Lab’s staff first met Even, they saw a founding team which was “also skilled at taking a gnarly complicated problem like cash flow volatility and breaking it down.” Lajewski says that for a fintech to be successful, the people behind it must be acutely aware that they aren’t their own target customer — therefore, it’s important to constantly be testing assumptions. This is one of the best ways to know that the impact you’re having is actually aligned with the problem you’re trying to solve.
Lajewski commented during the EMERGE session that her team found Even “unusually adamant about surrounding ourselves with people who would hold us accountable.” We focused on this because we knew that anything we built and brought to market would need the assumptions behind it tested — early, and often.
This started by testing assumptions about the world in general. In Even’s early days, this meant spending a lot of time talking to the people who would be using our app. We interviewed them about their daily lives, personal finances, and experiences using our app. We also conducted analyses and interviews — anything we could do to get opinions from people who weren’t us, and see if our assumptions held up to scrutiny.
Five years later, we still invest heavily in one-on-one conversations with our members. But we’ve also invested in testing assumptions about how we’re changing the world. In other words: Is our product doing what we want it to do? We like to think that it is, but at the same time, we’re perpetually trying to prove ourselves wrong, so we run rigorous experimentation with randomization and control groups and use the learnings to adjust our product roadmap.
To operationalize testing and experimentation, it needs to become part of your culture.
For example, last year we ran an experiment to test the influence of behavioral economics strategies on our product design, specifically around building savings. In this experiment, we presented members with an active choice, or a “nudge” to automatically start saving parts of their paychecks, instead of relying on them being aware of the feature and taking the initiative to engage with it. The members in the treatment group who saw the “nudge” were 25% more engaged with our Automatic Savings feature, and had 40% higher balances after eight months. We were also excited to see that the introduction of that one extra screen resulted in zero churn.
People who work at Even are here because we all want to put a dent in the massive financial challenges faced by millions. But for the most part, the folks running the company, building the marketing campaigns, and writing the code weren’t experiencing the depth and breadth of these challenges. In order to be successful, we knew we needed to go outside our own experiences to learn and understand the scope of the problem.
To operationalize testing and experimentation, it needs to become part of your culture; and that has to come from the top. If it’s not identified as one of the core ways you expect your business to be successful, or even as a competitive advantage, then it will just become a side project. One key indicator of whether an organization has prioritized experimentation is budget, or lack thereof. Do the people doing the testing have the tools they need, and budget to recruit subjects?
At Even, the curiosity and drive to find the objective truth is part of our values and our interviewing process. We talk to candidates about their “desire to to find the truth, not seek arguments.” At the organization level, we bring different parts of the business together to form a multi-disciplinary testing culture. The people in charge of testing work closely with business partners outside of the technical organization: Salespeople, member advocates, marketers, and operations managers are all part of experimentation at Even.
We surround ourselves with people who are just as invested in experimentation and strong consumer outcomes as we are.
It’s also critical to lay a massive engineering technical infrastructure down if you want to be successful at this. You need the ability to toggle experiences on a per-member basis. We built our own experiments platform but these days there are great turnkey solutions out there. As members take actions in the app we record each state so we can reconstitute what happened, when. A solid data warehouse and interactive analytics platform let researchers derive insights from our experiments.
Finally, we surround ourselves with people who are just as invested in experimentation and strong consumer outcomes as we are. Over the years we’ve strengthened our ties with the good people at the Financial Health Network, and established new relationships with organizations like Stanford University, Washington University, and UC Berkeley. By partnering with experts in economics and social policy, we can build on top of decades of literature and hold ourselves accountable to researchers who demand objective truths.
Our goal is to help millions of people become financial secure with responsible earned wage access that also provides a path to building savings. But building a product that truly helps people, and is designed in a way that people want to keep using, requires digging deep for the truth. To do this a company must hold a deep reverence for the facts, and a desire to uncover them — but it also needs to build the foundations for that becoming part of the business, and the culture. At Even we do that by hiring people who are obsessed with finding the truth and building it into our products, and then we give them everything they need to get the job done.
Get updates around new research and findings in your email.