Measuring Product Market Fit and “Core” Data

This is Week 03 of 09 of Astra’s Y Combinator Startup School Journal.

Continuing last week’s focus on product and growth, this week’s Startup School lectures focused more on the latter aspect of the topic. Gustaf Alstromer (YC Partner, formerly Growth at Airbnb) presented “How to Get Users and Grow” and Suhail Doshi (Mixpanel) shared “How to Measure Your Product.” As a technical founder, I really enjoyed both presentations for their level of detail in an analytical approach to understanding your users, your product, and your growth.

There are common qualitative descriptions of what product market fit, “PMF” feels like, but fewer resources that define this quantitatively. Gustaf’s talk was particularly valuable in this regard. I enjoyed it so much in person that I’ve watched it twice more since the video was published - I highly recommend it.

As was we can see in the image above (and was prefaced in the required step from Week 01 of defining your metric), PMF is based uniquely on the product you are offering and can be measured today even with a small dataset. Note that for all but Facebook, the “metric that represents value” involves more than just logging into the product. This necessarily means that measuring PMF and the impact any key product improvement or marketing effort has on growth isn’t going to be as easy as integrating an out of the box analysis tool, like Google Analytics, that tracks sessions by default. Luckily there’s Mixpanel if you want to track events in app.

For Astra, our key metric is Transfers with an ideal frequency of Monthly. Since some of us get paid monthly and others weekly or bi-weekly, we chose Monthly as the minimum requirement. Additionally, since a key feature for our product is “transfer automation,” we execute your scheduled transfers in our business logic and don’t require you to be in the app. We posed this scenario after the lecture and Gustaf shared a tip from his prior experience: Astra’s transfers are like Airbnb’s bookings. To measure effectively one could split their data into “Core” for the value-based metric and “UI” for user actions and track them separately. Luckily we already started building our own dashboard to access Astra’s “Core” data so we are going to adopt this approach asap. Thanks, Gustaf! ????

---

Program:

This week we attended the main weekly lectures on Tuesday and our group’s Office Hours on Thursday. Our advisor is holding them Monday - Friday for 30 minutes, so this time there were 6 companies in attendance with us. I thought this made communication more effective. Aside from those core activities, this week we also:

• Continued to engage other founders in SUS via the forum, Slack workspaces, and in person.
• Offered a “feedback exchange” with other founders.
• Signed up for a meetup after next week’s lecture for SUS founders located here in the Penninsula.
• Shared our product release and blog updates on the SUS forum.

 

Product:

After playing the App Store waiting game for 5 days, our new app version was finally approved and released.

Tip: For each app version you review, include any additional detail that Apple has ever asked for to avoid back and forth resubmissions. For us, this time that was a live demo recorded showing the app in use.

We shared last week what the key motivators were for this update and now that it’s live we hope we will see some improvement around the onboarding flow. In the meantime, we got to work on development for the next update:

• Defined how we would enable additional bank accounts to be connected to the app - initially, we chose the simplest solution for our MVP but it came with some limitations.
• Refactored our navigation from react-router-native to react-navigation.
• Added more detailed error messages for views with fields.
• Highlighted the limit in the UI for a maximum per transfer amount of $5,000.

 

Growth:

While we were waiting on the App Store approval, we took the opportunity to make a push on content marketing initiatives. We compiled quite a bit of feedback already on the items that we knew we needed to address, so we optimized for new learning by not pushing too hard on getting some of our current users through the verification process until the new version was available. Along the way, while digging into some of our “Core” data, we found that we had just crossed $10,000 of transfer value in the 4 weeks since launch! Beyond a quick celebration for that milestone, we:

• • Launched the Astra blog officially and shared with our fellow SUS founders.
    • Tip: Original content hosted on your site equals more "juicy SEO" but it's a long game. We don't anticipate seeing this uptick in the near term.
  • Sent a ton of emails after the new app version was available on Saturday to follow up with our existing users and those we’d queued up to try it out.

---

Per our refined dashboard, this week’s user stats:

• 177 Total Users (+8)
• 162 Signed Up Users (+7)
• 45 Onboarded Users (+6)
• 28 Verified Users (+7 or 33%)

Our target was 10 Verified Users to get back on track to our target growth rate for the program. Considering we had the delay from the App Store and most of these came over the weekend, we will take it! Onward to more users and new features for the app.