A Common Framework & Post Launch Metrics – Podcast Transcript

Jael

We often underestimate what we can achieve in a year. And we overestimate what we can do in a day. That’s the mother of stress.

Intro

Welcome to the innovation metrics podcast, where we bring you the latest on innovation management. We provide insights on how to measure innovation, innovation, accounting, and managing the uncertain process of developing new, sustainable, and profitable business models. You can find links to the main topics covered in this episode and information about the guests and hosts in the show notes, or go to our blog on innovation metrics.co. Your host is Aaliyah island.

Elijah

So rather than me explaining who your background, welcome to the show jail. And do you mind telling us about yourself?

Jael

Sure. Thanks for having me off on a show today. Jail. I have been a corporate innovator for the last 12 years of my life in a large FMCG or CPG company in the last four years specifically, I was actually very much anchored around, innovating on a, you know, innovation system within a large corporate company and also driving the actual innovation strategy and how it translates into activities and projects on a smaller scale. So you can see from a system and innovation to innovating the strategy, as well as the innovation projects that then land in the markets. So that was, there was what I have been doing professionally for 12 and a half years in a company. Recently, I started my own gig doing consultancy and coaching for leaders and teams around innovation systems innovating as well as their own lives and their own career. So that’s really keeping me a lot excited this day.

Elijah

What’s what’s your company called? The current ones shared with

Jael

The current company is called level up, up with three please.com. Yeah. So you can find more in the show notes.

Elijah

Yeah, we certainly put all the links in here. Wonderful. Yeah. Fantastic. Great. Yeah. Cool. So, so today we want to start off with exploring the, the problems that corporates have specific corporate staff with after a product is launched. An innovative product is launched specifically, how do we, why do we need a different way of measuring? What do we want to look out for and why do we measure in the first place? Right. So I guess the issue is, and correct me. One of the issues is that when, when we finally decide that we should launch a product at, at a certain scale and it doesn’t perform the way we we anticipated. Right? So that, that happens right.

Jael

80% of the time, I think,

Elijah

Wonderful. 80% of the time. I think this is really an interesting, an interesting issue. I’ve been a while ago, I’ve been looking into it and it’s, you know, for obvious reasons, it’s really hard to find really good data. I remember that one of the, I forgot what industry it was. I think the highest success rate was around 40%. Like that was sort of the best thing in an industry. And that was probably more adjacent products, you know, like changing, changing small things or so I’m not quite sure.

Jael

Yeah. And because this episode is about metrics, I would like to understand the, when you talk about the 40%, what, what is the success metric for those?

Elijah

Yeah, that’s fantastic. Right. That’s a really good question. So I think what we, I don’t know that that’s a clear answer. I don’t know. I mean, it was a while ago when I looked into, into that is success even defined in most from, for most projects clearly is really the question to ask your bag. Do you think that is often the case that success is clearly defined or is there a failed criteria in with each launch clearly?

Jael

So from a corporate setting, success is very typically defined by the absolute person over or the gross margin or profitability. One of these financial metrics, right? The supply chain team may think of success as how many units can we shift out of the factory. I would be able to forecast the demand and the supply accurately. That could be their success metric, you know, for the general manager of the country, the success might be okay, are we profitable? Can we have a good story to tell the global HQ? So successes

Elijah

Wonderful, wonderful, perfect. And when we stay with the financial success criteria, I think very quickly, very quickly, we can see that this is what we would call typically, what you just described is what we would typically call a lagging indicator. So lagging indicator such as return on investment is something that is lagging, softer. Everything already happened for a while. So it doesn’t really tell us whether or not we need to change course. So as somebody like you, who really wants to know early on whether or not you have to change, course that is not satisfying. Is that correct?

Jael

Exactly. There’s, that’s very correct. And to build on that and to expand a little bit in, you know, traditional brick and mortar kind of product and channel channel is how we reach consumers and customers. The idea of pivoting is not very often heard. Anyway, if you think about the industrial style of innovating and putting products into the hands of the consumers or the customers is a very long set process that is kind of set in stone. It’s not like a button where you can stop and then tweak and then do it again, which is what I think, you know, I’ve been in the industry for 17, 18 years, and people don’t talk about pivots the ones, one thing, which is the market. And simply because it’s hard to do, I think in the 1980s, when Toyota started the idea of agile, that was very interesting, right?

Jael

Because in an auto mobile company or industry, you don’t really stop the factory to do tweaks because then the cost time to restart and so on, you lose productivity, but Toyota realized, okay, I’ve given the Liberty to one factory to do whatever they want. They learned how to make pivots along the way, even though in the first year of their experiments, they reduce production dramatically. After that, it took off. Why? Because the little pivots actually improved productivity so much. So bringing the analogy back into corporate innovation is actually, I would say at least in the company I used to work with people started to become more aware of the need to pivot and hence this whole conversation came about.

Elijah

So we know that the term pivot and probably most, most listeners who listen to a show like this would be familiar with that, but that’s because we’re mostly concerned with trying to find product market fit. And that is properly not very well defined either, but you know, trying to get to the point basically of saying, Hey, let’s scale this thing. Right? And then, then we see if we can scale it. And then we hit other issues. But is, was that, is that in the context of your former work, was that always clear? Was that said, Hey, this product has got product market fit. Now let’s scale. It is, that was that, is that what happened? What was the, or what was the terminology? Was that at least used? Yeah,

Jael

Smiling widely because there’s no such concept as incubation or a small scale launch. Remember we are coming from the industrial age. Everything is done at scale from day one. So if there is a mistake, that mistake is a huge mistake. Yeah. At least from the company that I was working with, we push a button, then everything gets made. Everything gets shit. If it doesn’t sell, it sits in the store houses or the warehouses and then writes offs happen. So if there is really no product market fit, nobody to buy the retailers or the customers and return this stock to us, we buy them back, you know, and then this products appear as free gifts for employees costing a lot of business waste. And now we are all into sustainability. You can imagine the amount of raw material waste that has gone into the way. And of course from an ROI perspective or return on investment perspective as a complete disaster for the team and the whole

Elijah

Experiment you can imagine. Right. Exactly.

Jael

So you’ll feel on a big scale.

Elijah

And that was every single time. So we were like, where you used to work at that time, at least that was, you did not have a, a different system implemented where or where you made small, smaller bets beforehand.

Jael

Yeah, indeed. The whole innovation system within the company has pivots as changed to allow for that smaller experiments upfront before we pushed, you know, make 10 million units button. Right. So before producing say 10 million units, we’ve learned that we needed to work on smaller scale experiments or would it be loading fast, fail fast in order to make a better success.

Elijah

Yeah. Wonderful. So within that system, then that, that was then eventually established. There was a, so you, that had that moment where you said, okay, this product has pre-market fit and it has enough risk reduced that we can launch it at larger scale, basically. Right. Did that have a name like th was that, was there an overarching system to that? Was there an overarching terminology such as something similar to product market fit, maybe a different term assault.

Jael

Okay. So we have different gatekeeping processes. So every time we pass a particular date, so he’s like a farmer, right. You’ve passed a particular date, then you can progress to the next thing where more investment is made, more commitments are made. And when, you know, there’s a product launched position, and again, there’s no incubation, so that’s no small scale. Let’s make 1000 units see how the sales go is like, okay, we make is a, is a proper launch. We have moved from the pilot plant where we make 1000 units for in-house experiments. And then we go and do the mass production. So that, that is the moment at which we decide we’re going to push launch.

Elijah

Wonderful. Great. Okay. So, so this is really nice. So for us, because of today, we want to see it often. What, what I’m busy with is to say, Hey, what metrics do we need before we say launch? So, you know, how, how, how large does the sample size need to be? Right. What kind of evidence, how much uncertainty is acceptable and so on and so on. But what we want to get to today is, Hey, even then we, there’s still a lot of uncertainty and we need to continue measuring and monitoring behaviors or others are the metrics that can tell us early on whether or not we’re actually on track, whether this, whether this thing actually works. Right. And so, yeah, just trying to, trying to find that a bit better. And, and I guess as you said, the, the, the, the magnitude of failure is just the same. Even with the old or the new system, you’re still producing at the same scale and the same people. Okay. Wonderful. It’s significantly harder to make a change once it goes in that face.

Jael

Yes. Once it reaches the hands of the retailers is really, really hard to make a change or people are resistant to making a change. In fact,

Elijah

Yeah, it will be great to, to, to, to name a few of these really big issues. Right. They might be obvious to everybody, but, but maybe not

Jael

That there are structural challenges, right. From a big corporate perspective. So for example, because our specialization, the people developing the product can be different from the people actually launching the product more so in a global, regional local setting where it’s a matrix organization, that people who came up with the consumer need or the consumer problem to be solved, develop the idea and actually producing the thing in the local market could be completely different people. So that sense of ownership, as it passes on from conception stage to actual prototype to actual thing that gets produced in a pilot plant to the thing that actually reaches the retailers and consumers, they could be different people and having differently. Yeah. So the idea of success is actually different. As we move across the whole process, then the black, the challenge in this case happens to be what happens after the product goes into the market, because oftentimes the development team is in then go, oh, great. Now you go deploy. You make a success out of it. I’m onto my next shiny new toy. This is what happens.

Elijah

Do you think there’s also an issue with, before we go on inside, is this, this is something that is personally dear to me, that the development team then, because they have proven two degree, nobody can see