Building in Public Update: Lessons from Pivoting Kalen

Earlier this year, I started working on Kalen, an AI productivity coach. The initial hypothesis was that a lightweight email-based touch would be effective. Since my initial launch a couple months ago, it's become clear that the hypothesis has been proven false. While email was a novel and interesting approach, it doesn't seem to solve users' needs, so I've pivoted Kalen to interact via chat on a webapp. It can now handle things like habits and tasks and help you hit your goals. The following are learnings on how I got to this pivot along with how I could have reached this conclusion sooner.

The Allure of Email (and Why It Failed)

When I first was thinking about Kalen, I wanted to use email as the primary UX. I thought it would be more lightweight overall and would remove any install friction. Both turned out to be true, and there was definitely a decent amount of interest in this approach. But I also saw users trying to have a conversation with Kalen and use it for habit tracking. I waited a bit too long to explicitly ask these users for feedback, and once I did, it became clear that Kalen's email was too slow and was getting buried. Proper feedback gathering is important because it helps reduce the risk of going down the wrong path. It might seem like you're going down the right path, but it's a local maximum. The main lesson here was that I should have been better about my qualitative feedback approach and collecting more feedback, which brings me to my second mistake...

Flying Blind Without Analytics

My second mistake was starting off without quantitative feedback. I waited too long to instrument analytics. I had basic Vercel web traffic analytics, but I didn't instrument the funnel (I use PostHog, they have a generous free tier) until much later. I was flying blind. I didn't have visibility on all the drop-off points. And the biggest drop-off was at the top of the funnel. Users would come to the site and not convert. And when they converted, they wouldn't actually use the product. They wouldn't email back. It felt like the email was getting lost in the noise. That corroborated what others were saying when I explicitly asked them for feedback. There was a clear pattern. Email wasn't a good medium for this. Productivity and accountability required its own medium.

Key Lessons Learned

I should have instrumented and measured the marketing site funnel before I built features. This would have allowed me to see where users were dropping off and make data-driven decisions about what to build next. I should have validated my ideas and product more quickly. I also learned that when I had conflicting opinions with qualitative feedback, which was common, I should have let data break the tie. It's a hard pill to swallow when you see low conversion numbers, but it's important to accept and adapt. Finally, I learned that I am effectively shipping a funnel. Everything is an optimization. If users aren't signing up or making it through the core flow, no amount of feature polish or technical excellence will save me. I can't build my way out of something people don't want.

Moving Forward

None of these lessons are novel, but I think I needed to experience them first hand. If you're an engineer like me, you're wired to build. Building feels productive and fun. Distribution and validation are much harder, but without them, it's easy to go down the wrong path and build the wrong thing. I don't know yet if this pivot will work, but if it doesn't, I hope to fail faster this time.