Innovation Accounting – Ranges are Key – Podcast Transcript

Teaser

What are the principles we should be bringing into early stage prediction? It’s that, you know, first of all, quantification is not bad. The way that we are doing quantitation is bad. And the reason …

Intro Speaker

Welcome to the innovation metrics podcast, where we bring you the latest on innovation management, we provide insights on how to measure innovation, innovation accounting, and managing the uncertain process of developing new, sustainable, and profitable business models. You can find links to the main topics covered in this episode and information about the guests and hosts in the show notes, or go to our blog on www.innovationmetrics.co 

Your host is Elijah Eilert.

Elijah

So today we’ve got Tritan Kromer back on the show – hi Tristan.

Tristan

Hi Elijah, how is it going?

Elijah

Going great, thank you. For this time, I thought you could tell the listeners a little bit about yourself, like your background, maybe your company, what you’re doing. Are you happy to share that with us?

Tristan

Sure. My background is pretty haphazard. I have a philosophy degree. I was born in New York city. I was in the music industry for 10 years as a music producer writer. Pretty much anything I could earning a dollar from and transitioned from marketing into the IT security industry, where I stayed for five years, living in five different countries around the world, Taiwan, Germany, Vietnam, Switzerland, eventually landing in San Francisco and moving into the wide and wonderful field of innovation via being a startup entrepreneur. So that’s kind of me a little bit all over the place. My company now has worked with over 25 different accelerators around the world. I believe everywhere from Ramallah to Mexico, we’ve worked with startups in Japan and more focused recently on corporate and large organizational innovation, including working with the civil service in the United Kingdom. Companies, such as Unilever and fast-moving consumer goods and high technology companies as well. So it really just a diverse range of experience, but all focused on how do we get new products into new markets as fast as possible. That’s kind of been the driving force of my career, I think for the past 15, 20 years.

Elijah

Thanks for sharing that. The last time we spoke on this show, we spoke about how to measure teams. Today, we wanna talk about product.

Tristan

Ah, okay.

Elijah

So, basically, we want to talk about innovation accounting. We want to talk a bit about, why we need that. Why do we need anything else? And we’re not happy with what we have, why do we need to change anything? What are the problems,

Tristan

The answer to that should be relatively obvious. Nobody’s happy with what we have. Everybody knows it’s a sham. All of our projections are typically off by a very significant factor.

Elijah

That’s great. Let’s, dive a bit into, into the issues. Maybe we start with the issue of the business case, to have a really good starting point here specifically in the context of a larger organization, if you want to progress your project significantly, any project, but also an innovation project or venture, you need to provide a business case, right?

Tristan

Yes, and business cases, just some language for pitch, right? The business case you provide to venture capitalists is just a different type of business case. The business cases, the thing you have to provide inside a corporation to your boss or your business sponsor that says here, I will make you money. And if you put resources or venture capital funds or whatever the case may be into my organization or my startup, I am going to give you some sort of return on that investment, whether it is dollars or lives saved or actions taken or whatever, the output measurement is that you want is that your organization wants like we will get you stuff.

Elijah

Right. And that can work. I certain, circumstances.

Tristan

It is not an unreasonable request. Let’s just start out by saying that. I think in the last 10 years there’s been so much criticism. I mean justified criticism of give me your business case. And then the entrepreneur, the intrapreneur says well, I have had this idea for five days. It’s too early to speak of that. Or the, the data that I’m getting right now is purely qualitative in nature and nature. Therefore, I can’t give you a coherent business case. And I think that is a reasonable kind of defense and for a time that might’ve been reasonable because yeah, the argument we were having before is kind of dumb, right? Because it would always follow this cycle of somebody in a position of authority to, to grant or deny resources has given me a business case. And then the entrepreneur intrepreneur says, okay, well I think this project, this product is going to make a hundred million dollars and then eventually capitalist or the business sponsor says, well, that’s too small. I only care about things that are a billion dollars and above. Therefore I’m going to deny you money. And then the entrepreneur will turn around and say, oh, well, in that case, I’ve found another aspect of this business model and it turns out it we’ll make a billion dollars. And so there’s this kind of stupid game where everybody knows that the projections are a little bit ridiculous, but nonetheless, there are resources that can only be allocated until you promise to deliver an absurd amount of money. And lo and behold, those business cases are generally inaccurate because they’ve been asked to be an accurate. We have specifically asked the poor entrepreneur to come up with a number larger than they are comfortable with coming up with based on measly qualitative data.

Elijah

Right. So that’s great. I might just take one more step back first. All these tools and the methods in there, they may be very valid if we’re in a non-entrepreneurial non-innovative sphere, let’s say. So when we lots of historical data, I just want to frame that a bit more. It’s just the amount of uncertainty that we need to, or the amount of guesses that we need to pluck into these tools become so vast that as you say, we’re basically forced to make something up.

Tristan

Yeah. If it’s something super predictable like you’re sitting in Australia. So, you know, if you’re, if you’re digging gold out of the earth and yeah, you can shovel, you’re shaking your head at me?

Elijah

Finding gold is actually the perfect analogy. Because the chance of finding a gold deposit is, no even better, not just finding gold. You may find a little bit of gold. You may find a little bit of value.

Tristan

No, no. So let me clarify, right. So if you are digging out gold and last year you produced one ton of gold.

Elijah

Okay. Thumbs up.

Tristan 

Right. This is non-innovative. This is not exploratory, right. If we’re just looking for gold somewhere on the planet. Yeah. There’s a certain amount of risk involved. Right.

Elijah

10000 in from your first idea.

Tristan

Well, if you’re like, huh, I can see that there’s a gold mine here. And it’s just a question of extract again. Right? Like I know the rate at which we dig. I know the rate in which we process oil, there only one variable here. And that is the price of gold or, you know, there’s some variation in the weather and things like that, which might impede my process or something like that. But the fewer variables there are the better, right? So whether it’s, you are carting water from a stream to a local community or something like that and reselling it, or you’re doing something that’s very predictable and that it tends to happen the same. Maybe it’s even commoditized, basically, there aren’t a huge number of variables that are impacting your production. And you’ve done it before. Like you’ve got the data, then business cases tend to not be terrible, but the moment, anything kind of new applies like, well, I’m, I’m not actually digging gold anymore. I’m taking gold and I’m turning it into jewellery and nobody’s ever seen this type of jewellery before. Like now it starts to become a little bit riskier because there’s not only the volatility in the price of gold but there is volatility in terms of, well, does the consumer actually like your jewellery? Is the jewellery crafted to a high degree of precision and beauty? The more variables there are, the more uncertainty there is, the harder it is to make a coherent prediction. And the one variable that always, always, always causes a lot of uncertainty as a person humans, right? So the moment humans are at the end of that supply chain saying, give me the thing. Then inherently becomes a lot more unpredictable because humans tend to be weird. Humans are kind of lower, irrational, tend to vary their preferences over time. And they tend to respond to a lot of really weird dynamics in society. It’s just no longer, so easy to make a simple business case work.

Elijah

Yeah. Fantastic. I thought the other interesting problem with having to come up with Return on Investment or Internal Rate of Return, for example, is that when you have to make a case for creating a new asset and new business model. Maybe you want to create new assets, tangible intangible. If you draw in your case, on existing assets and you come in one case and then the other one you don’t. So you’re including existing assets of the company, basically your number would always, it’s like a bias towards continue working with what you already got, right? Like that we’ll win that case. We’ll win against the other case, always. So looking at this number as a deciding factor, internal rate of return, you basically shielding yourself systematically or you’re making it even harder to come up to do what you actually want to do. Yeah.

Tristan

I think that’s the general innovator’s dilemma that, that Christensen wrote about. If I could spend a dollar in my consistently good money making machine that always returns $2, then why would I ever spend that dollar on something innovative, which could return $2, but it could also return $0, which again is a fair argument. And I think this is, this is a kind of complaint I have with, with Andy Cars from Sweden. Andy was saying like, you should always invest in innovation. And, and my pushback there, even though I’m kind of an innovation person, is that now actually there is, there is sometimes good reasons not to invest in innovation. Like if your organization has lived out its life and has done what it’s needed to do, there’s very good reason not to invest in innovation. I worked for cancer research UK for a little bit of time. And frankly, like I think everybody at cancer research, UK and myself, and probably you, even though I think it would be very sad if they’d lost their jobs, it would be thrilled to shut the company down successfully because guess what? There’s no cancer. We’ve raised all the money and we’ve cured all the cancer. Like that would be a really great outcome. I don’t think you need to invest in like horizon three innovation initiatives in that particular organization because you know, hopefully you’re done and that’s great. Now Andy would make the argument that no, no, no. You’re a successful money. Raising scheme now go raise money for, I dunno – Alzheimer’s that would be good. Maybe you could do that. But my point being is sometimes the organization is a temporary institution and it should be shut down successfully when its purpose has been served.

Elijah

I love that. It reminds me of my aunt. She always said I would love lose my job. She was a nurse.

Tristan

Yeah exactly! People will no longer break their ankles and stumble and all those bad things. That would be great.

Elijah

I’m happy with not having to see that anymore.

Tristan

But then we’re facing a few other issues. If we say that they provide no predictive value, these business cases. They are very ingrained in organizations for many reasons and that’s the only way to release a lot of money. I am just trying to get to the next point of what we’re doing about it. So what else we can do other than that. So sometimes we just don’t do anything. We don’t have any modeling, any prediction of the future anymore. Very often for a certain innovation labs or initiatives or accelerators. So we say, look, just don’t need, just don’t need anything that is quantifiable. Right. Maybe we say, is there a problem? What’s the evidence of a problem? Very little in terms of predicting the actual impact into the future.

So just to better define what we mean when we say a basic business case is that essentially somebody is, is going to predict how much revenue, how much the, the output value of your, your startup or your product is. You’re going to predict dollar figures for year one, year two, year three, year four. And then it’s going to be plugged into a net present value score, which uses the company’s internal cost of capital and all that stuff. And it’s going to output like this number that says, Hey, this is a good investment. Or this is a not investment, a bad investment. It’s going to say something about the ROI, but it’s just based on these really, really arbitrary guesses that don’t seem to have any clear connection to the qualitative data that early stage entrepreneurs or intrepreneurs are coming out with.

Right. So it’s not that the question itself is bad. And I think that’s where we made the mistake. It’s not like it’s a bad thing to say, Hey, how is this business going to eventually generate money? Or how much like that’s, that’s not necessarily a terrible question. It’s just the way that we’re framing it is a way that entrepreneurs can’t answer effectively because they don’t have a good way to project how much money they’re going to have. Just like Twitter when it got started and had no idea how it was going to make money that Facebook didn’t necessarily know it was going to wind up in advertising, but there was a way to look at the business and said, wow, these businesses are growing at a phenomenal rate. They have a huge number of users here. And if this company can figure out a way to make even half a penny on each user here, they’re going to make a lot of money, right?

If they could make a dollar per user, they’re going to make an incredible amount of money, probably more what happened there. At least if they make a penny they’re going to be fine. So, you know, they may not have known exactly how they were going to make money, but they knew at the scale that they were operating, that it was almost certain to make money somehow. Whereas you look at companies like Uber, which has a higher cost to serve each user because you have car involved, got to pay the driver. If gas got to pay for insurance, there’s a lot of things where, where the margin isn’t so clear the cost per to serve a user for Facebook and Twitter is almost negligible. That means that almost any profits, just go straight to the bottom line. So it’s this of thinking that is totally fair to ask, you know, like, what are your margins like?

You know, is there some conceivable path for you to actually return money? That is absolutely 100% a fair question. What is not a fair question is, is basically to expect a, an exact figure and expect that figure to be correct in four or 10 years time. Like that’s just something that’s not reasonable for any early stage project. And if you even asked, you know, most organizations, what percentage of your business cases are actually accurate? You know, even the ones that are in the highly predictable, like, oh, we’ve been doing this for 10 years. Even those business cases are terrible. I think the highest figure I’ve heard is like, oh, 70%. But generally they’re all 70% wrong rather than 70% of them are correct. And 30% of them are, none of them are really that great. And most of them are like 50 or 30% accurate.

Elijah

Yeah. It reminds me, I don’t think I can fully accurately recall it, but when Warren Buffet once spoke about how he doesn’t like to have forecasts in the room, when he, when they make capital allocations basically. Just once and then it never came back, you know, because yeah, it’s just, it’s just ridiculous, you know? And, and he’s famous for investing in something has a long track record. Right. He’s like, because he is a proper historian in a sense, right. So they can analyze companies that have traded for a long time. So the company is very good at doing that? And so they have a lot to work with, and even there, they don’t really want forecasters in the room.

Tristan

Honestly, I have no clue as to how it works or Buffet runs his business, but perhaps he’s the forecaster and he’s just a better forecast everybody else, or is making longer term bets than everybody else.

Elijah

So we don’t think it’s reasonable to ask. It’s an unreasonable question basically was where you landed at to say, give me a number, give me a number in five years where we don’t even know yet all the variables that need to go in, you might not even know exactly the material. You don’t know your channel. You don’t know exactly your customer. You don’t know how many there are. We don’t really know your pricing. It could go up a lot. You might find ways of delivering even more value and so on. So it’s just completely unreasonable to ask for something specific. It is not unreasonable to quantify it at all. Right. So that is not unreasonable. I think that’s what we landed.

Tristan

It’s not unreasonable to ask the question of how does your business work and how could you eventually make money and how are you going to grow? And these kind of sub-questions, but the precision is totally unreasonable because what we need, what is not represented in that typical basic business case is uncertainty, right? They ask for very specific numbers and they have the specific output, which has your ROI is predicted to be 4% or 104%, whatever it is. But, that does not represent the most important thing for innovation, which is what is the level of uncertainty that you have about that number. And, that’s really, the thing that is lacking in the base of business case. There’s, there’s just no way to say something other than it’s really high or it’s really low. And of course the, the improvement, I would air quote, if we were not simply talking. The improvement that was made was to say, okay, well, give us your best case scenario and give us your worst case scenario. We will just divide those two numbers by two. And that will be our precise estimate. And that also turns out to the terrible, a way to represent things like it’s actually mathematically bad. But the improvement I will say is that if you don’t boil your best case scenario and your worst case scenario down into one number by just adding them up and divided by two, what you can see is you can see the range, right? Like the width between those two numbers. And so the moment I ask an entrepreneur, well, how much money are you going to make over the next four years? And if they say, well, it’s somewhere between a hundred million and zero. Well, okay. Now I actually have a really valuable piece of information hidden away in there. It’s like, well, the width of that range, like the Delta between zero and a hundred million is in fact a hundred million.

Tristan

So that expresses to me that the entrepreneur is telling me, there is a vast degree of uncertainty here. Whereas if you made the same prediction kind of on a, on a core business, one that has some historical track record, you might say, look, in our worst years, we’ve made 50 million. And in our best years we’ve made a hundred million. So I would say, you know, even without any information about this, this business, it’s gotta be somewhere between 150 million, 150 million, right? So that expresses that the level of uncertainty in this project is actually much, much lower. And that’s really what we’re after. Right? We need an understanding of what is the level of uncertainty in this project, because if you’re a venture capitalist and you’re looking at horizon three or highly disruptive projects or transformational projects, what you, what you should be seeing is actually a really wide range of numbers.

You should see the, the, the best case scenario is, is astonishingly large, especially if it’s a tech company, right. If there are exponential, if there’s an exponential equation, exponential factor hidden in there, some somewhere like it’s relying on, on a computing power doubling or something like that, it’s relying on network effects. It’s relying on a strong user base that is adding value as you grow more users. Well then your best case scenario should be this crazy exponential hockey stick graph. And your worst case scenario is it just, it stays at zero, right? So there should just be this massive range of uncertainty. And that’s actually what you’re looking for. If you want to invest in billion dollar tech startups, right? You, you want it to be incredibly unlikely for you to hit that top number, but it’s at least possible.

Elijah

Very nice, and I guess managing that is exactly innovation management. Right? Managing the answer in part.

Tristan

I think the, the thing that we can really draw from, or that we sh we should be drawing from the basic business cases, that the basic business case doesn’t represent uncertainty. And one of the things that we can add immediately to make deductions better is not to try to boil things down into one number, but allow for ranges. And the range expresses the uncertainty, right. That’s already a huge improvement over kind of traditional prediction methods, at least as far as they’re applied for innovation projects. But I think to be honest, there, that should be applied in almost any project, you know, use ranges to express things

Elijah

In any project. I like that. So, so we’re looking for ranges and that is really the best and most honest thing we can do. And in terms of predicting the future.

Tristan

I think so it’s also just the easiest, right?

Elijah

It is also the easiest. Ah, that’s good. Because it’s a bit daunting for some people to do that in the first place and we need to make it easier, I think, to enable, to enable us to actually do it. Right.

Tristan

Yeah. That’s, that’s one of the strong benefits that I think they’re there is too to just allowing entrepreneurs to add ranges in is, as you say, is very psychologically difficult. Let’s say to say, to commit now to, I’m going to earn $100 million in four years and it’s, it’s unreasonable given the amount of information they have because they will eventually be held to account for that. So, so yeah, like if you allow them to say, we might earn a hundred million dollars or we might earn one or even zero, you’re allowing them to express the level of uncertainty. And you’re also kind of letting the entrepreneur off the hook a little bit. You’re no longer forcing the entrepreneur to lie to you because if you just say you must earn a hundred million dollars or we won’t fund this project and you’ll be fired. Of course, they’re going to say a hundred million dollars you were asking to be lied to. But if you allow them to express the level of uncertainty, you’re basically kind of reducing the level of fear. You’re increasing the level of psychological safety and you’re increasing the likelihood. You’re going to get a straight answer out of the entrepreneur and all of that is good. Right. There’s nothing bad about any of those things.

Elijah

Yeah. And I guess what I’m passionate about here is always the fact that when you, when you, force people to make things up, then they also need to be delivered in a certain way because it’s not like the other side is this silly, right. It’s not like a CFO has, you know, didn’t go to university and has hasn’t acted in life.

Tristan

Oh, the CFOs are typically very smart people. Right? I mean, that’s, that’s kind of the funny thing about all of this, this ridiculous, like shared lie because the entrepreneur is saying like, okay, I’m going to run a billion dollars. And the CFO is thinking to themselves, well, this person is obviously extremely enthusiastic. And I know that this is very risky project, so it’s probably only 10% less likely to succeed. So they kind of in their minds, if not on paper, give it this, I’m going to discount 90% of that. And this person is realistically maybe only going to come up with a hundred million. Right. So that’s what they’re doing in their brains anyway. And it, it’s just kind of like, why go through all that? Why not just

Elijah

But they’re not subject to, I mean, they also have emotions, right. So I think what we’re, I think what, comes really into play here is, is the ability to tell that story. So in order to believe in, you know, to come to that point of intersubjective belief and fiction, we, kind of, the better you tell a story, the more likely it is for you to get funded is not necessarily the quality of your idea. Right. So it is even less. So, so basically

Tristan

True, but I guess that’s a that’s that seems like that’s a separate issue, right? Like there’s,

Elijah

I feel like it’s in the tool, right? It’s in the process. By having to, by forcing these systems into the organization or not changing him, we’re basically allowing or make it more likely for people who are very good, very good at that to succeed. I see that over and over again. And then biases, become much more important as well. Right. Cause you’re not just putting your trust in the numbers and in the facts, but into a person. And that activates all our biases. So like who gets funded from VCs. Like we see, you know, the vast discrepancy between male and female or whatever right. And so on. And I think that all plays a massive role there and ultimately in, yeah, I’m, I’m a bit passionate about that.

Tristan

I think you’re absolutely right. I don’t think, you know, I don’t think the area of innovation accounting where you and I become particularly passionate is necessarily going to help move the human biases part of this. Or like, there are very things that we know that can help solve some of the typical human biases, like sexism and racism. Right. And we know those things right. There are things like blind auditions for musicians. Amazon typically does not allow. PowerPoints does not allow presentations. You must provide all of your business case information. Your pitch must be in a six page document. Right. And it must be written out in narrative format. And everybody’s going to sit around a table and read it because there’s less bias because I can’t be charming in my presentation. I can’t, express white maleness in my presentation and confidence like, no, no, your pitch has to make sentence.

Like you have to have logic on a piece of paper in a way that is coherent and make sense. Of course, there’s still probably some biases there in terms of writing sound and things like that. But, but the methods of innovation, accounting, to be honest, like that impacts the content. It doesn’t impact the delivery system. We can still be thrown off. Even if you have a sophisticated Monte Carlo analysis based on estimations with fringes that you’ve created that, that a coherent financial projection, if you were pitching that in a PowerPoint, you are still subject to all of the human biases about race and vocal performance and sex and gender and all of those things. Like you’re not going to get away from that by having a better accounting method, unfortunately.

Elijah

Okay. Well, that’s destroying my dream.

Tristan

Yeah. As much as I might, like it. Basically do blind audition. That’s the one thing that I like about kind of having a format. A lot of people know that I, I generate a lot of templates on my website. We give them away as creative commons tools, but I’m also kind of known as the like, oh, don’t use this template. Like just take the idea and hack your own. You know, that’s why we created commons. So people can like take it and copy it and cut it up and make it their own type of template. But I kinda liked the idea of templates and an idea of having like what, this is what I want in your pitch.

I want these six slides, or I want this six page document in this format because it kind of removes some of the bias out there. Like you, you say that I’m not going to be impressed by your fancy graphics in your PowerPoint side. We just want it on a piece of paper. Like that’s a nice way of removing some graphics. Of course, if you have the wrong template, you’re totally screwed because now nobody can add in additional information that might be valuable that doesn’t fit into the template. But the idea is, you know, standardized and stuff to some degree and help remove some of the biases from the system.

Elijah

Lovely, So how does a team that have, that has, you know, has an idea, did a few custom interviews, ran a first AB test or landing page tests or comprehension tasks or something, you know, what would be the best approach in your mind, but avoid a business case with one number? What is it like? What does the process look like?

Tristan

Well, I think again, just for the purposes of, since we’re in the realm of audio here and nobody can see anything, but let’s just start with something super basic, right? So we asked the entrepreneur for prediction for your prediction and they said a hundred million dollars. Okay. And then we decided, well, wait a minute. Let’s, let’s not quite be so certain there, give me a range. Give me like, what is your worst case scenario here is you’re not gonna make any money. So we have a range from zero to a hundred million dollars say, now that’s an improvement. And that’s what we asked on day zero before they got started with a project. Now in the simple scenario here, the entrepreneur goes out, does the customer discovery works in their business model? Canvas, they’ve done some work. Let’s ask them again. Hey, how much money do you think you’re going to make over four years?

What’s the best case scenario. What’s the worst case scenario. And now what we want to hear from the entrepreneur or entrepreneur in this case is we want to hear that that range has gotten narrower, right? So if they come back to us and say, Hey, I think it could be $150 million idea or a $0 idea. Then something strange has happened, right? Because the range has just on wider. So somehow there is more uncertainty in the entrepreneur’s bond, which is interesting. Right. Okay. We can maybe ask some more questions to try and figure out why that is, but what should be happening is that the entrepreneur, maybe if they just went on and talked to 10 people, but you know what, nine out of 10 of those people were super excited and have this really distinct problem that the entrepreneur thought they could solve. Like, you know what the banking system is terrible. And I really want to send money around the world, to my families and relatives. We got so many transaction fees and the entrepreneur thinks there’s a way to solve that with, I’m just going to go with doge coin right now, because that’s just funny.

Okay. So they’re going to solve that with some sort of doge coin transaction exchange. Great. Now the entrepreneur should say, I believe that actually my, my best case estimate has increased. It’s now 150 million over four years, but my worst case scenario has decreased. I now think that we are at a minimum that earn $60 million. Okay. So, so my range, the distance between my best case, worst case is now 90 million before it was a hundred million. Now it’s 90 million. So what I can kind of see what the best and worst case numbers that they’ve expressed is that they’re actually more certain about the outcome here. They’re not only more certain, but that range is a lot higher than zero. And so that’s always a good thing, right? But the most important thing is even if the entrepreneur said, well, now my best case is 19 million. And my worst case is zero.

I see that they’ve generated some sort of information that has allowed them to narrow the range in some degree, like they become more confident in their prediction. And that’s what we want to see. Like if the range is getting wider, it means they’re getting less certain and whatever they’re doing is not helping them. Right. They’re not generating information. That’s useful. Most likely that’s the case. But if the range is getting more certain, then it means that whatever they’d been doing, talking to 10 people and nine out of 10, really excited that made them more confident because they’re look, they’re like, look, 10 people is not a representative. It is not a representative sample of the a hundred million people that we wish to serve. Right. But I know out of those, a hundred million people, at least nine are excited and have this issue. Right?

So, I know the number is not the, the worst case scenario is no longer zero. And the best case scenario is no longer 100 million. It’s a 999 million, 999,999. Right. Because one person was like, no, I don’t care about that. Right. So even with a very, very small sample size, like you’ve learned something. And when you have such a vast amount of uncertainty that your range going from zero to 100 million, and then even a little bit of information tells you a lot. And that’s a point that Douglas Hubbard who wrote this book, how to measure everything always makes is that a little bit of information in the state of extreme uncertainty is just going to tell you a tremendous amount. Like that information is very, very valuable. So that’s what we want to hear from the early stage entrepreneurs that they have an estimate. And that estimate is narrowing because they have done some sort of research and gained some information or some insight from the market. That’s how we should proceed initially by that was a long answer to a short question,

Elijah

Take this app is it really becomes about Rangers itself, right? So that’s why, why our range is so, so good and so powerful. Like we advocate for estimating ranges for, for a particular set of variables, you know, like, and then, and then run an analysis on it. It’s called Monte-Carlo. And I thought we would be talking about it, but I think this is really, this is really good to just stick a stick with it here. It’s also, as you said, it’s audio, it’s a bit hard.

I don’t know how to efficiently explain Monte Carlo simulation, Monte Carlo simulation. For those of you are scratching your head and saying like, what’s that like basically without gambling, right? There’ll be a link somewhere in here, but, but basically it’s just saying fancy math, fancy, fancy statistics based math that we can apply to those ranges. That’ll give a more accurate estimate, but that’s not really the important thing. Like, like we don’t need to apply the fancy math to understand that, that if I have this very, very wide range and I got information and the range narrowed, that is a good thing. I increased my predictive power is my insight here. So if we’re, if we’re trying to give some, some basic guidance as to like, what are the principles we should be bringing into early stage prediction? It’s that, you know, first of all, quantification is not bad. Okay. The way that we are doing quantitation is bad. The reason quantification is bad is because we are not allowing uncertainty to be represented in our equations. Right? Once you represent uncertainty in your equations, then the, the art of entrepreneurship is the art of reducing uncertainty, right. And that means going and getting the information, right? So the valuable activities of these startups, entrepreneurs or entrepreneurs should be doing is getting information.

And then maybe the other thing that we can add to this is to improve that basic business case is. I’ll throw in the other fancy term for another hyperlink is, is a for me breakdown or Fermi, what do they call it Elijah?

Elijah

Fermi Decomposition

Tristan

Fermi Decomposition right, which is a super fancy way of saying break it down into a simpler problem. Like Fermi was just a mathematician who was really good at this. But the, the simple explanation is if I ask everybody in Australia to tell me how Javier for our state building is, I’m expecting those answers to be kind of all over the place. I don’t know, Elijah, do you know how all the empire state building is? I’m a new Yorker and I don’t even know, right. That that’s a really hard thing to guess. Just like it’s hard for an entrepreneur to guess how much revenue they’re going to happen for a million dollars. But with the empire state building, you can decompose the question into a smaller set of questions that are easier to answer.

It’s kind of the difference in doing a top down market estimation versus a bottom up analysis of the business model. I can say, well, I know that basically the height of the empire state building is the antenna on top or tall. That is plus the height of the floors. And I know that the, the height of the building is the number of floors it has plus times the average height of each floor. And, okay, so now I’ve got some things that are a little bit easier to estimate, right? Because I know that an antenna, I don’t know, it’s, it looks pretty big on top of the empire state building. So I’m going to say that it’s, no shorter than 20 feet and maybe no taller than a 100 feet. Now that seems reasonable. Maybe I’d say 150 at the, at the Atlas range. And I know that the average height of the floor, again, I’m not big on certain because I haven’t lived in New York, 20 years, but I know that floors are generally not more than 15 feet tall. That’s a really, that’s a, that’s a really tall floor. In fact, I would say the advertisable floor is probably more like 10 feet, right. But if I wanted to be really, really safe and pad my kind of best case and worst case scenarios here, I might say that the shortest of floor could be, would be eight feet tall. And the tallest could be, was let’s just say, 12 seems reasonable. Right? And then I just need, then I only have one more number, right? Like I be composed the question of how Paul is the empire state building into three separate questions. Like how tall is the antenna? What is the advertisement for, and how many floors there are.

And now I just need to guess and say that there’s, you know, there’s gotta be more than 50, right? Because I’ve been in buildings in New York that were not as tall as the empire state building and are, you know, now I’m probably going to be called a liar, but I believe the empire state building has more than 50 floors. Right. And I know it has less than a hundred floors. Find the link below to get your answer. But now I’ve, I’ve broken down the question to something that is easier to estimate and, and that you can do with your business model as well. You can say that, well, I believe that the cost per unit is going to be this. And I believe the revenue per unit, the price that the customer’s willing to pay is going to be this. And I believe that there are this many customers buying this many times per month. Therefore the amount of revenue we are going to have per month is going to be this, because I believe that we can get 5% market share and you break it down into these much, much smaller questions that are both easier to estimate, and also easier to God and run experiments on because I can much, much more easily go out and run an experiment. Say, I am going to go into the factory and produce one unit of this, or I need, I’m just going to go and talk to a manufacturing expert and say like, Hey, give me a range. How much at the worst case scenario would it cost to, to produce this paperclip or produce, you know, how many engineering hours, how many engineers and how long do you think it will take to, to create a doge coin exchange system? And I can get a worst case in the best assessment, right? So I can break this down into smaller questions that somebody, hopefully you can find me an answer to.

Elijah

Yeah. That’s fantastic. And it’s also really nice that referring to the other episode that we recorded about how do we measure teams, right. It’s really lovely. Now how we can, for those who have listened to it or who want to listen to ties back really well.

Tristan

Yeah right, cause one of the metrics was the innovation velocity, right. So how fast are they running experiments or not innovation velocity, experiment velocity or insight velocity is how much information they’re bringing back. And we can’t even quantify the value of that information by saying, look it reducing the range on these estimates that we’ve created.

Elijah

Right. And then, yeah, that’s fantastic. So now we know does a teammate progress or not, and in different ways. One of them is, is there progress in, the product means, do we narrow any of these variables and therefore the entire range down.

Tristan

Yeah. And I think it would be fair to say that we have to be a little bit careful here in that we’re not kind of playing the same game again and insisting that the team narrows the range each time, like they shouldn’t be punished. Like if they run a few experiments and you’re like, you know what, we still don’t know it. So highly uncertain, it’s highly volatile. That could be the case. Like we don’t want to force them to like narrow the range on necessarily, but that is what they should be doing. Right. Like if they are going week after week after week a week after into the field, and none of the variables are, are getting a narrow range, then that is probably a problem.

Elijah

Yeah. But it’s also nice. Right. Cause then, then we can see if somebody needs help. You know, not again, it’s not always about punishing or rewarding. It’s about to see where, where do, where do we need to improve as an organization rather than like, huh. That team might need a bit of assistance there. Right. And that, that could also be just an indicator for that.

Tristan

That’s true. Although we should probably also have an indicator at some point for our company, like percentage of teams that are willing to ask for help, but that might be a, another good indicator for our corporate culture. Maybe that’s one for your, your metrics book. But, but yes, I think you’re absolutely right.

Elijah

Very nice. Yeah. I love it. Yeah. Fantastic. So I think, yeah, this episode is really about the, ranges and talking, talking in ranges. And then what we’re passionate about is how to bridge that, communication gap often between the innovation team and the finance team or the sponsor and this might be one of the most fundamental.

Tristan

Yeah, I think so because it really, yeah, it really lets, I mean the finance department has to take one step towards innovation and allow people to express themselves in uncertainty, which means ranges. But the moment they do that, then the entrepreneurs should also be able to take a step towards finance department and say, okay, well, now that I know that it’s okay to express my level of uncertainty, like I am willing to give you predictions and I’m willing to revise those predictions on the basis of the information that I’m gathering. And if you ask me next week, you’re going to get a different answer. You ask me a month from now, you’re going to get a different answer because I’m going out and getting information improving yesterday. That’s like, that should be the story that, that allows us to communicate. Yeah. And, and