Stop Planning and Start Learning

The book “The Lean Startup” by Eric Ries swept through the product and business world in 2011, and it had a simple message: Organizations that learn the fastest win. He also brought the concept of Minimum Viable Product to life.

Unfortunately, like almost every idea rapidly spreading, its message quickly became diluted, so the messages never really stuck, but the vocabulary did.

So we have groups everywhere talking about their MVP, which only means their first software release. We have folks talking about validation, meaning they think hard about what their boss wants.

I take a certain amount of pride in the fact that I did what this book said. Applying the ideas and specifics found in this book was something I started a long time ago, and I still find myself alone in my willingness to experiment, learn, and treat an MVP as intended.

This article is about coming back to how to do experiments so that you learn quickly. Learning is at the heart of removing risk from whatever you attempt. Without the learning, the risk exists. You can hope it won’t show up—you can roll the dice, but you’ll be betting on luck. You’ll be risking your product or initiative when you could ensure its success.

Build Measure Learn

Build, Measure, Learn is the three-step, business-friendly experiment framework most people learned when The Lean Startup became all the rage. So what did people do? They started building! It’s the first step, after all.

The other two steps became afterthoughts. Measurement became a thing you did after you built, so you measured what you could with what you had. Learning was stunted because there was never a goal, so the data isn’t conclusive, and now we’re already invested in the solution.

So let me say this instead: You need to do this in reverse order.

Learn

Start with learning. Learning is about asking a question you want to learn the answer to. “What’s the plan?” is not the question you need to ask, no matter how much your organization wants it to be.

Instead, you want to answer questions that would cause you to change course depending on the answer. Here are some examples:

• What problems do my customers have today?
• Do they have the problem I’m solving?
• Will they pay the price?
• Can we reach our customers the way we thought?
• Is the failure in our system code or infrastructure?
• Will this new technology solve our constraint?
• Why do our tests fail all the time?

Once you start listing the questions like this, you’ll realize you have many of them. Start with the most dangerous one, then move to the next. We answer one question at a time.

Measure

Step two is measure. Measure is where we decide what data we need to answer our question. This step is the hardest thing for groups to grasp.

When we pick what data we need, we are basically taking a stand to say that if the data looks one way, we believe that to be a strong indication.

We don’t say that it’s conclusive or fact or worry about statistical significance. For a fun break in this article, define statistical significance.

At some point, we will have to take our data, make a call with it, and move on. Yes, there is an inherent risk here, but this risk of making a judgment call with data related to a defined question is far less than the risk you’re learning about.

To help come up with measurement criteria this you can use a template. I know everyone hates templates, but it forces you to answer the parts of this that are challenging and to be honest.

We believe that if ____________, then that means, ________________.

An example here would be: We believe that if we see 50 sign-ups on our landing page in one month, then that means our customers are interested in our solution.

Without getting this specific in your measure criteria, you are limited to a gut feeling instead of giving clear criteria for your decision.

A few quick tips about this measurement criteria—come up with these measurement criteria with your team. If you go solo, people will point out every imperfection. While we never claimed perfection to begin with, it will be an obstacle for folks. Also, pay attention to putting time boundaries on things. You have to know when to evaluate, or you will again be left feeling things out. Finally, it helps to have some basic background in your measure of choice. My example of sign-ups may or may not be reasonable. I could ask marketing folks, do a quick Google search, or say that I may have a flawed experiment.

When you do come around to looking at the data, the clear and concise criteria will force you to learn. It will force you to ask new questions and dive deeper. That’s the whole point.

Build

The last thing is to do the small amount of planning you need to execute the experiment.

If you don’t know where to start, conducting customer interviews is one of the most valuable skills to learn and will always provide valuable information. Even with lots of quantitative data available, you’ll get deeper insight with interviews.

So, start with conducting interviews. Getting started with customer interviews is often hard, but it is well worth it for how much you’ll learn.

Reverse It

Ok, now I’ve walked backward through the Build, Measure, Learn loop because this is how you plan it. You can execute once you have a plan in place with the question you’re answering, the measurement criteria to answer, and the plan for your experiment.

Execute the steps in the order of build, measure, and learn.

Build and execute your experiment, collect measures and data, and learn by answering your question.

Many folks see this as a lot of overhead to do something “Obvious,” yet I see so few people do this. They assume there is no risk, build for months and months, and have poor results. Let me paint this a little differently here.

I spent 1 hour deciding what questions I needed answers to, prioritizing them, coming up with measures for the first, and a plan for my experiment. I spent the next week on my experiment and learned my assumptions were wrong.

That attention to detail, in one case in particular, saved my client over $1 million in building out a product that nobody wanted.

It’s worth it.