Pretotyping@Work
Invent Like A Startup, Invest Like A Grownup
Jeremy Clark
Co-Founder, PretotypeLabs
First Edition
!
Copyr...
AUTHOR’S NOTE
This is an economics book. Before you drop it like it’s on fire and
run screaming from the room, let me expl...
CONTENTS
The bottom line of this book!! ! ! ! ! ! 3
Rational Inve(n/s)tor Behavior! ! ! ! ! ! 5
Invent Like a Startup
1! T...
THE BOTTOM LINE OF THIS BOOK
In the 1980’s, IBM was in discussions with several important customers
about a radical produc...
to apply immediately to breakthrough innovations of all types: prod-
uct, service and internal change.
Note this book is N...
RATIONAL INVE(N/S)TOR BEHAVIOR
When corporations innovate, the key economic actors are Inventors2 and
Investors. Think of ...
Inventors and Investors conduct their dialog within a typical frame-
work of processes and standards:
• The Annual Budgeti...
I’d like to see the early-stage dialog between Inventors and Investors
change, dramatically. The default pattern I describ...
This negotiation between Inventors and Investors on what level of mar-
ket interest will bring continued support is the cr...
Invent Like a Startup
Inventors, has this happened to you? A senior executive Investor an-
nounces that your company is se...
1. THE LAWS OF FAILURE
The first problem to confront when dealing with Inventors is idealism.
Inventors live in a world in...
or Almost Always” a cause of failure, and 41% selected “Lack of data
to validate that product addresses a real market need...
As you can see from the table, the majority of the new products
launched were classified as “Me too”. Why is this? This is...
As such, they can be shared with others, but not in any real way.
Ideas can only attract opinions by way of a response, wh...
2. False Negatives: Every idea can be a loser!
Who can ignore Twitter? But when you first heard of the service,
what was y...
is rational: radical-sounding ideas conflict with Investors’ sense of
how the company currently makes a living, and raises...
2. PLAYING SMARTER
Aside from the obvious - and critical - new element of “actual usage”,
this textbook definition will se...
Desirability
Feasibility Viability
This is an appealingly complete model, and indeed this seems to be how
many Inventors a...
the right channel built? What is the monetization model, and how will
we underwrite profits? Will the new product cannibal...
• “Will they buy repeatedly?”
• “Will customers from both city and suburbs buy?”
• “What mix of groceries (e.g., fresh vs ...
phase of innovation, such as prototyping. He and I did the same for
our recent workshop at Stanford Graduate School of Bus...
a fairy. Thus a Pinocchio pretotype is one in which an inanimate (or
“dumb”) artifact acts as a proxy for the real thing.
...
dress book, to-do list, and note taker) were the ones released on the
finished Pilot.
Design firms regularly employ Pinocc...
stature) would operate a pantograph-style lever arrangement connected
to the Turk’s arm to grip, move, and release pieces ...
4. The One Night Stand Pretotype
The One Night Stand pretotype is a model in which an interactive serv-
ice experience is ...
only that, but the data also showed whether the credit was cashed in
immediately (in most cases, yes) and the average up-s...
to provide the basis for the Roadster chassis. Lotus supplied Tesla
with a ‘glider’ Elise - a car without a powertrain - w...
• The Teaser, in which the Inventor creates a fully-functional sub-
set of the full solution, e.g., the first 3 chapters o...
3. A WRENCH IN THE INNOVATION TOOLBOX
Inventors schooled in innovation tools often ask two questions:
• “How is pretotypin...
The following graphic sums up where I believe pretotyping belongs in
the “front-end of innovation” (or FEI) toolkit12, as ...
Invest Like a Grownup
Investors, has this happened to you? A breathless Inventor team pre-
sents a new idea: it’s going to...
4. DON’T BELIEVE IT, PROVE IT!
US Law presumes that a defendant accused of a crime is innocent until
proven guilty. In thi...
Let’s work an example to illustrate RPI, returning to our old friends,
Webvan. Let’s pretend we are the Inventors and Inve...
RPI = .75 x 100 = 75 = 7500% cheaper
RPI = ________ ________
X
.75 $100M
1.00 $1M
The estimates doubtless have a wide conf...
RPI = .62 x 60 = 37.5x “faster learning”
RPI = ________ _________
X
.50 60 months
.80 1 month
Of course, any Investor migh...
Small representative
sample (~100-1,000)
I (1000)
Time
Target
market
Let’s say your target group of customers to whom you’...
OLI(t) = _________________________
# who tried ‘it’
# still using ‘it’ after time t
Note that the numerator from the ILI c...
I (10,000)
Time
T (843)
ILI = 843/10,000 = .084 = 8.4%
As Investors in Webvan, we should be encouraged by this initial res...
Listening to this example people often respond with: “But online grocery
ordering and home delivery is a successful busine...
5. BUILDING CONFIDENCE INCREMENTALLY
Pretotyping, and the RPI, ILI and OLI metrics that support it, are a
practical illust...
pace (R=7 days, R=14 days, etc) and meet again after every R
milestone to review progress. Your decision will be clear bas...
Discussion of this topic would not be complete without introducing the
Dead Cat Bounce. This charming term is used by Wall...
6. PRETOTYPING FOR ALL REASONS
So far we have only discussed end consumer-facing products and services,
and our case studi...
second adaptation is to limit the pretotype modes used to the four most
partnership-friendly: Pinocchio, Mechanical Turk, ...
APPENDIX 1 - PRETOTYPING WORKSHEETS
PretoStorming experiment design worksheet
Pretotyping Metrics I - Calculating RPI and ...
Pretotyping@Work! 45!
Pretotyping@Work! 46!
Pretotyping@Work! 47!
APPENDIX 2 - ABOUT THE AUTHOR
Jeremy Clark is a growth strategy and innovation expert, helping com-
panies to unleash inno...
of 49

Pretotyping@Work by Jeremy Clark (6.73MB)

http://www.pretotyping.org/resources.html
Published on: Mar 4, 2016
Published in: Design      
Source: www.slideshare.net


Transcripts - Pretotyping@Work by Jeremy Clark (6.73MB)

  • 1. Pretotyping@Work Invent Like A Startup, Invest Like A Grownup Jeremy Clark Co-Founder, PretotypeLabs First Edition ! Copyright © Jeremy Clark 2012 PretotypeLabs.com
  • 2. AUTHOR’S NOTE This is an economics book. Before you drop it like it’s on fire and run screaming from the room, let me explain. Economics is the study of resource scarcity and choice; it helps clarify the trade-offs we face when we make decisions about where to put our time and money, when and how much we should spend or save. In the context of innova- tion, economics informs the type and number of innovations attempted in a given period - how bold, how aggressively pursued, and how funded. This book describes an approach to innovation decision making that can break enormously wasteful historical trade-offs in resources. The goal of this book is to enable the practical application of that approach - pretotyping - within mature companies looking to improve the effectiveness of their front-end innovation processes. My col- league and friend Alberto Savoia is the originator of the term preto- type and much of the theoretical foundation for pretotyping. For an entertaining and rapidly-digestible primer on the method, I commend his excellent book Pretotype It1. I owe Alberto - and his many col- laborators at Google, where pretotyping abounds - a profound debt, and I heartily acknowledge his prior art. This book is based upon the Pretotyping@Work workshop materials I de- veloped with Alberto that makes pretotyping a teachable, repeatable method. As the book is intended to be readable by those new to preto- typing, there will be some duplication for the initiated, for which I compensate with new tools and perspectives. For invaluable feedback, reinforcement, and tempering challenge I must also thank both my wife Petra and Alberto for patiently reviewing manuscript drafts. Thanks finally to the participants of Alberto’s and my previous speeches and workshops, including a beta version of the workshop delivered at Stanford University GSB in June 2012. The picture on the cover is a trompe l’oeil (“deceives the eye”) image of a violin painted onto the inner door of the State Music Room at Chatsworth House in Derbyshire, England. It was painted by Jan van der Vaardt in the 18th century, and for me charmingly captures the es- sence of pretotyping: a captivating impression of the real thing that succeeds by being not quite what it appears to be. I dedicate this book to my late father, Peter Clark, mechanical engi- neer, committed pilot, lifelong fabricator of solutions. He was not, in today’s sense, a customer-focused man, but on him Britain might have placed its gold medal hopes if tinkering were an Olympic event. Pretotyping@Work! 1! 1 “Pretotype It: Make sure you are building the right it before you build it right”, Alberto Savoia, 2011, available on Amazon.com as a Kindle download.
  • 3. CONTENTS The bottom line of this book!! ! ! ! ! ! 3 Rational Inve(n/s)tor Behavior! ! ! ! ! ! 5 Invent Like a Startup 1! The Laws of Failure! ! ! ! ! ! ! 10 2! Playing smarter! ! ! ! ! ! ! ! 16 3! A wrench in the innovation toolbox! ! ! ! 28 Invest Like a Grownup 4! Don’t believe it, prove it!!! ! ! ! ! 31 5! Building confidence incrementally! ! ! ! 39 6! Pretotyping for all reasons!! ! ! ! ! 42 APPENDIX 1: Pretotyping Worksheets!! ! ! ! ! 44 ! PretoStorming worksheet: for experiment design ! Pretotyping metrics worksheets: for calibration APPENDIX 2: About the Author!! ! ! ! ! ! 48 Pretotyping@Work! 2!
  • 4. THE BOTTOM LINE OF THIS BOOK In the 1980’s, IBM was in discussions with several important customers about a radical product idea: hardware and software that could turn spoken words into a text on a screen. The fundamentals of the tech- nology were still years away, yet customers seemed very enthusiastic: many declared they would pay generously for such a solution. Traditionally, IBM would have launched an R&D effort to develop the algorithms and electronics necessary to demonstrate a prototype. In the case of the Speech-To-Text idea, however, a team member had an in- triguing alternative suggestion: they should pretend to have the solu- tion, to see how customers actually reacted to the capability. What the team did was to create a movie-set like testing lab, in the form of a typical office space of the day. Customer subjects would be briefed on the Speech-to-Text solution, then seated in the space. The subject would speak into a microphone, dictating a variety of office correspondence, and would almost immediately see their words appear on the screen on the desk in front of them. What the subjects didn’t know was that the electronic output was being produced by a typist in a nearby room, listening to the dictation through headphones. What the IBM team learned was that, in practice, customers didn’t like the solution, not because of flaws in the product (the transcribed text) but because of a host of hitherto-unseen environmental chal- lenges: speaking taxed the subject’s throat, there was concern for privacy surrounding confidential material that the speaker would not wish to be overhead, and so on. Actual exposure to the essence of the proposed solution completely reversed the earlier customer enthusiasm. What the IBM team had done was a pretend-otype: they faked it before making it. It was more than a concept board or an idea on a piece of paper, which is entirely hypothetical. It was less than a prototype, which is typically a primitive but functioning ancestor to a finished solution. It was something in between, a new experimental protocol that drove to the fundamental question at the heart of every break- through innovation: “Do they want it?”. This book turns what the IBM team did into a complete method, called Pretotyping, because while “pretending” is involved, the method owes more to hard behavioral science than the dramatic arts. The method borrows from entrepreneurial theory, but pretotyping is most relevant for mature companies with developed innovation processes looking to boost their breakthrough success rate. The book develops a theory and method of pretotyping illustrated with varied examples and graphics, and provides practical tools for readers Pretotyping@Work! 3!
  • 5. to apply immediately to breakthrough innovations of all types: prod- uct, service and internal change. Note this book is NOT a primer on the full spectrum of breakthrough innovation processes, from generating distinctive insights into cus- tomer needs through idea generation techniques to steady-state product management. This is a deep dive into the critical time between idea conception and product development, a treatise on getting there more quickly and with the smallest possible quantum of wasted resource and uncertainty. A WORD ON “THE RIGHT IT” Readers of Pretotype It will recall Alberto’s definition of the right “it”. Throughout this book, any reference to “it” refers to a new idea for a product, service or initiative that might be ultimately in- tended for sale to an end consumer, delivered internally with a com- pany as a change initiative, or used between two companies in a business-to-business (B2B) context. A shorthand list of synonymous phrases might help: An “it” might be... ...an Idea to Try ...an Innovative Technology ...an Internal Transformation ...an Innovation Tool. We keep the definition broad to underscore the wide relevance of pre- totyping principles to many types of breakthrough innovation, whether ultimately sold into a consumer market or not. More on this in Chap- ter 6. Pretotyping@Work! 4!
  • 6. RATIONAL INVE(N/S)TOR BEHAVIOR When corporations innovate, the key economic actors are Inventors2 and Investors. Think of them as the sellers and buyers in a market for ideas. This is critical because, just as in the economy as a whole, corporate innovation markets rarely function efficiently; in fact, most corporations systematically favor the most incremental, certain opportunities and are biased against riskier, breakthrough ideas. To understand why, we need to take a closer look at the game, its play- ers, and how they play. Many corporate employees play a role as Inventors, whether as a major or minor job focus. Inventor methods vary widely from organization to organization, but common activities include exploring customer needs and new markets, generating new product and service concepts, invent- ing new process approaches and business models, and driving these in- ventions through development towards successful launch. It’s a little cliché, but Inventors are revealed by their entrepreneu- rial traits: attuned to future possibilities, optimistic, action- biased, dismissive of risk, and often with a healthy disregard for the status quo. Investors are fewer in number but on average carry more authority. Investors are typically middle managers to senior executives who allo- cate the organization’s resources in support of innovation initia- tives. Investors are also easy to spot as the grownups of their or- ganizations: attuned to the current business, cautious, analysis- biased, contingency-driven, and often with a healthy regard for the status quo. No surprise, then, that Inventors often seem to Investors like bomb- throwing loonies, wreaking havoc with their imaginative but unproven theses about markets and opportunities where the company has little or no proven experience. And that Investors often seem to Inventors like green eye shade-wearing troglodytes, living in the dark and inexplica- bly unmoved by the buffet of exciting potential spread before them. And that’s just at the level of personal traits. Let’s consider the stage on which these actors play their roles, and the script contain- ing their lines. Pretotyping@Work! 5! 2 I will use the term Inventor and Innovator synonymously, to take craven advan- tage of the felicity of the Inve(n/s)tor phrase. Some innovation theorists argue that there are important distinctions between the two roles, but for the purposes of describing how innovations attract investment these distinctions are not mate- rial. The term Inventor also includes teams of Inventors.
  • 7. Inventors and Investors conduct their dialog within a typical frame- work of processes and standards: • The Annual Budgeting process defines the discretionary funds avail- able to invest in growth projects of all kinds. Most such processes are what economists call “inelastic” - they can’t flex much if cir- cumstances change - which tends to set a limit on how many innova- tion projects of a certain kind receive investment in a given year. • The Business Case is a document that encapsulates the Inventor’s as- sertions - some of which are assumptions, others data - about the market needs, solution characteristics, and economic opportunity. Companies that use Business Cases typically have a “hurdle rate” of return that innovations are expected to clear before securing in- vestment. • The New Product Development (NPD) process (of which a widely-used proprietary variant is Stage-Gate®3) is a structured approach to in- novation management. The process specifies development activities and levels of definition required to pass each “gate” or investment review, and the ranges of funding available at each level. Most NPD processes require Inventors to complete a series of exercises to es- tablish a tight connection between the opportunity, the company’s strategy, and its competences that become more elaborate before each successive hurdle. These methods - a syntax for the Inventor-Investor Dialog - are all well intended, but exist fundamentally to manage risk for corporations by circumscribing the conditions under which growth projects are con- sidered investment-grade. The tools and rules of this model - the strategy screen in the early stages of an NPD process, the hurdle rate within a business case, the inelastic budget - combine to produce at least three dangerous effects on breakthrough innovation: 1. Inventors go “EMO”, or exaggerate, manipulate, and obfuscate in order to make their case to Investors. Risks and unknowns are airbrushed away, and revenue projections take the form of the classic optimistic “hockey stick” shape in which a small short- term development cost appears trivial before the consistently- growing, increasingly-profitable revenues following launch. 2. Investors become skeptical of any breakthrough innovation fund- ing petition, discounting revenue projections and multiplying cost and time budgets. After all, a seasoned Investor has seen fanciful works of fiction from Inventors before! 3. A pernicious bias towards investing in opportunities that resem- ble today’s products, services, and markets. Pretotyping@Work! 6! 3 Stage-Gate is a registered trademark of Product Development Institute Inc.
  • 8. I’d like to see the early-stage dialog between Inventors and Investors change, dramatically. The default pattern I describe above is charac- terized by a small number of infrequent, detail-oriented reviews, where reputations are on the line but facts are in short supply. Whatever the hurdle rate, every projection will inevitably meet it, but it takes months and large investments before real customers de- liver their verdict in the marketplace. O p i n i o n s O p i n i o n s Business Case projection Gate 1 Hurdle rate Gate 2 Gate n Launch F a c t s ? ? ? Months O p i n i o n s Both sides deserve better. The right profile for that dialog is many short conversations, occurring frequently and punctuated by short ex- periments that provide real data on whether the right it has been found. The Inventor and Investor should negotiate thresholds of mar- ket interest that will encourage them to continue with experiments, then map out the first few tests to build their confidence level in small increments. Experiments 1 2 n Right it Wrong it ??? F a c t s F a c t s F a c t s ? ? ? Days/Weeks Pretotyping@Work! 7!
  • 9. This negotiation between Inventors and Investors on what level of mar- ket interest will bring continued support is the critical innovation here. It should happen for each opportunity. Note that this dialog doesn’t change the odds of success at all: it just produces a fact- based result much sooner, allowing more opportunities to be explored, and more agility in the innovation portfolio. Much more on this later. Instead of defaulting to NO, Investors should reflexively say YES to initial investigation, but expect Inventors to quickly return with DATA. Inventors should learn how to thrive on a short runway by pri- oritizing demand as the key variable, balancing many early-stage ideas in a state of revealed-preference testing4, and never falling in love with an unproven concept. It’s a beautiful dream, and I have a method to propose. It won’t be easy, because the default process is so deeply embedded in our insti- tutions. To understand what we stand to gain by changing the Inventor-Investor dialog, we must first confront the reasons behind the failure of most early-stage ideas. Pretotyping@Work! 8! 4 Revealed-preference testing is a crucial concept for pretotypers. Economists use the term to describe a test in which a subject’s behavior can be taken as identical to their beliefs. A better definition for our purposes is: design an experiment in which you can ask for a commitment rather than an opinion. Revealed preference tests ask “Will you...?” not “Would you...?”.
  • 10. Invent Like a Startup Inventors, has this happened to you? A senior executive Investor an- nounces that your company is seeking the NEXT BILLION DOLLAR IDEA: you are to find it by CHALLENGING THE OLD WAYS, SEEKING NEW CUSTOMER NEEDS, and above all, THINKING OUTSIDE THE BOX! Some weeks pass, you produce the fruits of your work. What happens? Shocker: the Investors select and fund only the safest, most incremental ideas! Cynicism and antacid consumption skyrocket. Why does this happen? Because the Investor class at your company runs the current business, and the funds the company invests in new ideas are overwhelmingly in their budgets. The more novel your idea, the less it looks like what generates the revenue streams of their busi- nesses today. And they don’t want to waste funds that could be used on line extensions or “Me toos” on your loony-sounding stuff. Sadly, the abysmal track-record of breakthroughs attempted by the company supports this worldview. How to get Investors to think differently? Show them you understand why your new idea will probably fail, and why a cheap quick test will let you all move on based on data not opinions. Pretotyping@Work! 9!
  • 11. 1. THE LAWS OF FAILURE The first problem to confront when dealing with Inventors is idealism. Inventors live in a world in which everything is possible, where noth- ing has yet been disproved or has disappointed, where breakthroughs - and the riches that surely follow - are just over the next horizon. Wake up, Pollyanna: MOST NEW IDEAS FAIL. This is what Alberto and I call the First Law of Failure, and what it lacks in profundity it makes up for in veracity. No-one wants to ad- mit it, especially those whose livelihoods depend upon preserving budgets for R&D facilities, innovation centers, and brainstorming off- sites. But the evidence is overwhelming. A recent Nielsen study followed the marketplace success of some 24,543 new products over the first year following launch. Their conclusions are framed in terms of success relative to pre-launch expectations: Failed 27% Disappointed 16% Cancelled 37% Success 14% Star 6% Total 100% The distinctions between the categories are irrelevant for our pur- poses, but as a sobering exercises just total up the “Failed”, “Disap- pointed”, and “Cancelled” categories and you get 80%. That leaves a 20% chance of hitting either “Success” or “Star” status, either of which for our purposes counts as a win. There is a crucial corollary to the First Law Of Failure: MOST NEW IDEAS FAIL, EVEN IF THEY ARE WELL EXECUTED5. In a 2011 survey of Consumer Goods companies, 70% of respondents re- ported that “Low Product Quality” was “Almost Never a Cause” of new product failure, while 67% similarly exculpated “Technical or Regula- tory Problems”. On the other hand, 45% of respondents cited “Lack of data regarding future financial value of the product” as a “Frequently Pretotyping@Work! 10! 5 “New Product Development: product launches hindered by major challenges” Con- sumer Goods Technology magazine, August 2011
  • 12. or Almost Always” a cause of failure, and 41% selected “Lack of data to validate that product addresses a real market need”. This means that the failure rates in the Nielsen study overwhelmingly result not from poor execution of a good idea, but from robust execu- tion and launch of a poorly-conceived premise. Or, as we prefer to put it, most failures result from a well-executed, but wrong, “it”. In practical terms, this means that the odds of any one idea becoming successful are very low (of the order of 20%). For small companies pursuing new product ideas, the odds are that they will run out of budget and time before they find a success. For funded startups, In- vestors (usually angels or Venture Capital firms) cope with these odds by keeping a tight rein on cashflow, and by pushing the Inventors (company founders) to “pivot” to a new product and/or business model strategy as soon as the current one looks likely to disappoint. Com- panies without such attentive and flexible Investors are likely to simply run out of cash and time trying to test their original idea. Larger, better established companies can absorb these losses better thanks to deeper pockets, but that only makes the cumulative story of squandered resources worse. Alberto calls this effect The Wheel of Failure: on average, each spin of the wheel - or ‘bet’ on a new prod- uct idea - yields a positive response from the marketplace on average 1 time in 5. The other 4 draw snake-eyes, and as ever the house al- ways wins. The Wheel of Failure odds are even worse for breakthrough innovations, new ideas that offer dramatically improved price-performance, or that transcend user expectations, relative to current offers. Studies con- firm the jaded consumer’s impression that breakthroughs constitute a very small proportion of all new product introductions. The respon- dents to the CGT/Sopheon study of company-reported product innovations classified 18% of new product introductions as “Highly Innovative”, while 61% were either “Line Extensions” or “Product or Packaging Changes”. The Nielsen study broke down 24,543 new product introductions into a number of categories according to their degree of innovativeness: Category # of products % of products Breakthrough 334 1.4% Line or category extension 1,705 6.9% “Me too” 18,814 76.7% Others (seasonal, etc.) 3,690 15.0% Total 24,543 100% Pretotyping@Work! 11!
  • 13. As you can see from the table, the majority of the new products launched were classified as “Me too”. Why is this? This is where the lowest risk launches occur; either the launching company or a competi- tor has already validated the presence of a market, the willingness of buyers to pay for solutions in those categories. No wonder nearly 19,000 of those 24,543 new products were “better/faster/cheaper” re- spins of existing solutions. The problem with this “following” strategy is revealed by comparing the first and second tables. If most new products are justified with reference to the actual market performance of proxy offers, how come so many end up Failed, Disappointed, or Cancelled? Competition, of course: each new offer joins the ranks of comparable offers and, in most cases, cannibalizes sales that would otherwise have gone to a competitor product. Most extensions and copycat products don’t in- crease the size of the market pie, they merely add to the number of slices cut from it. The tiny fraction of breakthrough innovations, on the other hand, rep- resent a company’s best chance for abnormal returns. Breakthroughs contribute disproportionately to revenues and profits, and that impor- tance is increasing6. Yet for breakthroughs, the Wheel of Failure odds are even worse: long-run average data suggests that only 5% of at- tempts, or 1 spin out of 20, are successful. On the face of it, it seems Inventors face Sophie’s Choice: pursue a 20% chance of mediocre returns by developing “Me too” products, or chase even 5% odds of abnormal returns by developing utterly unproven breakthroughs. Inventors could escape the paradox if only they could discover which ideas, especially the breakthroughs, in their early-stage portfolio were winners, and weed out the others. Short of a crystal ball or time-machine, how might Inventors pull off this trick? By placing bets at the Wheel of Failure in a smarter way: by spending much smaller quantities of time and money per idea to validate market de- mand. You can’t change the odds, but you can change how you play. To see how this works, we must go on a trip to a wonderful place called Thoughtland. THOUGHTLAND Thoughtland is the natural habitat of most Inventors and the birth- place of all ideas. It’s a wonderful place of infinite possibility. Ideas are abstractions, made of nothing more than conceptual material. Pretotyping@Work! 12! 6 “Survival of the Fattest” by Raynor, Ahmed & Guszcza, Deloitte Review, shows how the most dynamic, value-creating firms generate increasingly asymmetric returns. Since 2001, the average Return on Assets (ROA) of the 100th percentile of firms has ranged from 4x-8x that of firms even in the 90th-99th percentiles!
  • 14. As such, they can be shared with others, but not in any real way. Ideas can only attract opinions by way of a response, which presents two critical problems: 1. False Positives: Every idea can be a winner! Remember Webvan, the originators of the idea of groceries ordered online, then delivered to your door? Conceived during the first internet boom of the late 1990’s, the idea behind Webvan was an in- stant success in Thoughtland. Everyone gave it a thumbs-up, and why not? It sounded simple, convenient, it had that why-didn’t-I- think-of-that, forehead-smacking ring of genius. Webvan’s Inventors, led by Louis Borders (of Borders Books fame), proceeded to raise over $122M in capital from legendary investors including Goldman Sachs and Sequoia Capital. An Initial Public Of- fering in 1999 raised a further $375M; in total, Webvan raised over $1B in pursuit of its idea. They used these funds to build a so- phisticated e-commerce website as well as a network of refrigerated distribution centers in 26 major markets, and to buy a fleet of de- livery trucks. Launched to great fanfare, an initial surge of curiosity-driven orders rapidly dried up, leaving Webvan investors not so much chilled as out in the cold. Webvan filed for bank- ruptcy in July 2001. What went wrong? The Thoughtland data on Webvan was misleading: people who had been asked a hypothetical question about an abstract service idea, a “Would you use it?” question7, turned out to be much less enthusiastic when faced with a fully-developed “Will you use it?” service. False Positives are a widely-distributed phenomenon: they occur in every sector of the economy, they are led by acknowledged experts in the fields of investment, marketing, and product development. Here are a few famous False Positives that proved to be spectacular “wrong its” in the marketplace: • Disney’s movie “John Carter” (Cost: $275M + $100M in marketing) • Motorola’s Iridium satellite phone system ($6B for 66 satellites) • Segway transporter (~$180M in funding) • Pontiac Aztek ($200M+) • Google Wave (~$20-30M) • New Coke or Crystal Pepsi (est. $50M each). Pretotyping@Work! 13! 7 This is of course how Focus Groups work, and much of the waste in Inventor re- sources can be attributed to False Positive data gathered from conclaves of well- disposed existing customers being asked low-stakes hypothetical questions.
  • 15. 2. False Negatives: Every idea can be a loser! Who can ignore Twitter? But when you first heard of the service, what was your reaction? Some may have thought it an intriguing ex- periment in real-time micro-broadcasting (though what evidence there was that this was a gap for people is unclear to me). But surely few intuited that it would ultimately power the democratic revolutions of the Arab Spring. The elevator pitch for Twitter has that terrier-twisting-its-head-to-comprehend, temple-scratching ring of insanity. Nevertheless, its Inventors pursued the idea, developing the plat- form and launching the service in July 2006. By Spring 2012, Twit- ter’s 500M subscribers were posting 340M “tweets” per day, and the Twitterverse’s role in breaking important news including the crash of a US Airways jet into the Hudson River in January 2009 estab- lished the relevance of the service. Inevitably, so many attentive users creates revenue opportunities, and by some estimates Twitter expects income of roughly $250M in 2012. Furthermore, Inventors Jack Dorsey, Biz Stone and Evan Williams are now viewed as influen- tial seers in the new media landscape. So the original Thoughtland data on Twitter was again misleading, this time because it was bearish. So it’s clear that Thoughtland produces two dangerous effects: False Positives, which the Nielsen study suggests might be 80% of all new ideas and up to 95% of breakthroughs, and False Negatives, whose num- bers we’ll never know because by definition they are strangled at birth. The Twitters of this world are few and far between. This brings us to the Second Law Of Failure: TOO FEW CRAZY-SOUNDING IDEAS GET TRIED Human beings often rush to judgement on new ideas. We take our own personality, professional expertise, career experience, and behavior as consumers as an instant proxy for the probability of success. This effect is intensified in corporations and government departments: in these hierarchies, death is often instantaneous for crazy-sounding (or even merely odd-sounding) ideas. I’ve seen this scenario pan out time and again; see if it rings any bells. Having originally chartered a team to “think outside the box”, at the end of brainstorming the highest-ranking person present compli- ments the group on their energy and creativity, then dismisses the boldest ideas. Often this is done with subtlety, with a skeptical ex- pression or a fatal hesitation in delivering the verdict, but typi- cally democracy collapses at some point and the crowd favorites get the thumbs-down from a few influential Investors. At one level, this Pretotyping@Work! 14!
  • 16. is rational: radical-sounding ideas conflict with Investors’ sense of how the company currently makes a living, and raises awkward questions about who will raise this odd-looking orphan, and what customers will say about it. But it’s equally irrational in the face of shorter strategy lifecy- cles, ferocious competition, and the lightning-fast pace of change. Investors have to find a way to take responsible risks with a certain proportion of potentially breakthrough ideas in order to boost the av- erage returns to their growth portfolio. Interestingly, the solution to the Second Law Of Failure has precisely the same characteristics as the solution to the First Law: Inventors and Investors both need a dramatically more efficient way of testing the true market appeal for new product ideas. To avoid both False Positive and False Negative outcomes, revealed-preference market test- ing of reasonable proxies for the final product have to be achievable at much lower investments of time and money. This solution has a name: Pretotyping. Pretotyping@Work! 15!
  • 17. 2. PLAYING SMARTER Aside from the obvious - and critical - new element of “actual usage”, this textbook definition will seem intuitive to Inventors: of course, that’s what we already do! In practice it isn’t often the case: the actual usage is assumed to be untestable before at least a working prototype is available, so early development proceeds on gut feel and extrapolation of past experience. Let’s try a more memorable and practical definition: This is a much more useful call-to-arms for Inventors. It demands a commitment to reaching the target customers for their ideas, not with Thoughtland artifacts (such as concept boards) but with revealed pref- erence experiments. Those experiments must simulate the core experi- ence, but that does not mean they need to be working prototypes. Pre- totypes inhabit the middle ground between abstract ideas and tangible prototypes: they must be just sophisticated enough to represent a valid test of market interest, and no more. Finding that minimum scale is the core mindset and discipline of pretotypers. SETTING PRIORITIES To unpack the mindset of pretotyping, we must discuss the questions that Inventors are trying to answer. One common framework comes from the design community, and uses a blend of three design attributes to frame the early stage development process: Pretotype: To validate the market appeal and actual usage of a potential new product by simulating its core experience with the smallest possible investment of time and money. Pretotype: To make sure you are building the right it before you build it right. Pretotyping@Work! 16!
  • 18. Desirability Feasibility Viability This is an appealingly complete model, and indeed this seems to be how many Inventors approach their work: by addressing all three facets si- multaneously using a multi-disciplinary team: • Marketing gets to work on testing Desirability by exposing cus- tomers to concepts and seeking feedback. • Engineers and Scientists head to the lab to begin prototyping to test Feasibility. • Analysts fire up a spreadsheet to model Viability under various scenarios. Everyone feels productive and the overall effort looks to Investors like a gratifyingly coordinated and efficient effort. I prefer to think of the critical questions as a sequence: “Can we make money?” “Can we build it?” “Do they want it?” To understand why the sequence is so important, let’s unpack them in reverse order. “Can we make money?” relates to the business model that the Inventors plan to wrap around the new product. Do we have Pretotyping@Work! 17!
  • 19. the right channel built? What is the monetization model, and how will we underwrite profits? Will the new product cannibalize existing business? All very important to ultimate success, but irrelevant without first establishing that the product can even be made. “Can we build it?” is all about technical capability, and embeds so many unknowns that it is usually the most costly and time-consuming to answer comprehensively. Can we make it reliable? Will the battery last long enough? Will the major functions work, and work together? Do we have exciting colors/flavors/features? Do we have the right raw materials/business partners/suppliers in place, at mutually beneficial terms? And so on. Again, critical to address, but irrelevant unless customers want the solution. “Do they want it?”, of course, is all about market demand. Inventors think they have a solution to a market need, whether or not their cus- tomer insight data has yet established that need. Most Inventors pro- ceed based on indicative insights about market demand. Pretotyping brings discipline to the exploration of this first question, by put- ting it first in the sequence and by being thoughtful about priori- ties. To begin with, “Do they want it?” has a number of possible variants, depending upon the nature of the it in question: • Will they use it where they are? (Environment, context). • Will they adapt in order to use it? (Behavior, switching). • Will they use it if it looks like this? (Appearance). • Will they use it if it does/does not do X? (Functionality). • Will they buy it this way? (Channel). • Will they buy it if it costs more than X? (Price). Recall IBM and their Speech-to-Text concept. The questions under test in their pretotype experiment included: “Will customers use it?”, “Will they use it for all kinds of office communications?”, and “Will they use it intensively enough to switch from their current solution (tasking a typist)?”. Or in the case of Webvan, some of the “Do they want it?” questions they should have pretotyped include: • “What % of people will buy groceries online?” Pretotyping@Work! 18!
  • 20. • “Will they buy repeatedly?” • “Will customers from both city and suburbs buy?” • “What mix of groceries (e.g., fresh vs packaged) will they buy from us?” Some compelling breakthroughs defy the odds and succeed despite the lack of a clear consumer need being identified prior to launch (Ap- ple’s iPad comes to mind). These are few and far between, however, and cannot be taken as an indicator of infallibility (Apple’s Lisa desktop, the Newton). The smarter way to play is to prioritize the testing of actual market demand before making the big investment. Let me lay out the new grammar of Inventor-Investor dialog, in the form of pretotyping approaches, starting with the simplest. I’ll in- clude some real-life examples of pretotyping success stories to illus- trate the different techniques. Bear in mind that “success” here means a successful experiment, not necessarily a right it: not all of the products concerned were launched, but in all cases the go/no-go decision was made cheaply and quickly. PRETOTYPING TECHNIQUES 1. The Fake Door Pretotype A Fake Door8 is a marketing entry point for an as-yet-undeveloped idea. Inventors can create a Fake Door by advertising a new product or feature, then tracking the response rate to see who would be inter- ested in the product or feature. The solution doesn’t even have to exist, yet an initial indication of interest can be captured at next to zero cost. Web technology enables a very robust method that includes: • Testing customers’ responses to different phrases or words (using online ads tied to specific search words). • Links placed on websites (the clicks on which can be counted). • Simple response forms (such as asking customers for an email ad- dress). Fake Doors do not have to be web-based, however. Emails, posters, and other media can be used to simulate the existence of the solution. Alberto created a Fake Door to test demand for his book, Pretotype It. Reasoning that his likely readers would be interested in innovation, he purchased AdWords for words related to the development and testing Pretotyping@Work! 19! 8 Credit goes to Jess Lee of Polyvore for the name Fake Door.
  • 21. phase of innovation, such as prototyping. He and I did the same for our recent workshop at Stanford Graduate School of Business by creat- ing a short brochure for the class. In both cases, the responses we received (clicks through and email requests for further information) encouraged further development). The Fake Door pretotype is usually the simplest and best first option for demand testing. Consider using a Fake Door pretotype when: • Your idea can be concisely described and presented to potential customers where you would expect to find them. A restaurant owner could put a proposed new item, described and priced like all current options, on the menu to see if customers request it. A nutritionist could buy an online ad presenting her idea for an app that provides meal selection guidance when people search the term “HEALTHY MEALS”. • You are confident you can manage the expectations of enthusiastic customers by following up within an appropriate timeframe. At least one car manufacturer has deployed a Fake Door test before designing a new model, proving that some customers can be patient indeed, but it pays to think ahead! 2. The Pinocchio Pretotype As everyone knows, Pinocchio is the inanimate wooden puppet whose dreams of becoming a real boy come true thanks to the intervention of Pretotyping@Work! 20!
  • 22. a fairy. Thus a Pinocchio pretotype is one in which an inanimate (or “dumb”) artifact acts as a proxy for the real thing. The Pinocchio pretotype was inspired by the story of the early devel- opment of what became the Palm Pilot, the iconic Personal Digital As- sistant (PDA) of the 1990’s. Jeff Hawkins, the founder of Palm, was a handwriting recognition software expert and evangelist, and hoped the technology could revolutionize personal organization. But his experi- ence launching an earlier handheld computer, the GRiDPad, had proved sobering: Time Magazine called the device “an engineering marvel but a market failure because [Hawkins] says, it was too big”. Hawkins was determined not to repeat the mistake, and became focused on the form factor for his new device. He had a size and shape in mind: it should fit in a shirt pocket. Hawkins’ solution was to cut a block of wood to fit his shirt pocket, then wrap it with paper bearing the image of a simple interface (see below, beside the finished de- vice). He then carried it with him for several weeks, pretending it was a functioning computer, miming his interactions with it when he encoun- tered a need for its imagined functions. For example, if he received an invitation for lunch, he pulled the block from his pocket and pre- tended to check his calendar for the proposed date, then “recorded” the event with “stylus taps” from a short stick he carried along. Not only did Hawkins’ theatrical experiment validate his theory about the form factor’s utility - being right where he wanted it when called upon - but it gave him insight into the most useful functions. The four functions Hawkins used most during the experiment (calendar, ad- Pretotyping@Work! 21!
  • 23. dress book, to-do list, and note taker) were the ones released on the finished Pilot. Design firms regularly employ Pinocchios to get a good feel for criti- cal attributes, and a good example is the Diego surgical dissector tool, designed by IDEO. To test a surgeon’s ability to balance, posi- tion, and finely control the tool, the team resorted to office sup- plies to understand their customers’ performance requirements for one- handed operation. Consider a Pinocchio pretotype when: • Your solution requires a significant switching or behavioral ad- aptation by customers to develop a new habit (e.g., using a new app), learn a new form of body control (e.g., smartphone finger gestures or riding a Segway), or simply abandon an existing sub- stitute solution. • You expect demand to be sensitive to the appearance or form fac- tor of your solution, and you need to test a range of sizes, shapes, weights, materials, etc. 3. The Mechanical Turk Pretotype The Mechanical Turk was a chess-playing “automaton”, designed in the late 18th Century by a Hungarian courtier attempting to impress the Empress of the day. The box could be opened to reveal complex clock- work components, which appeared to drive the left arm of a head and shoulders mannequin (the Turk) atop the device. Its maker would chal- lenge a member of the audience to play a game against the Turk. The illusion was made possible by a cleverly-concealed hiding place inside the box for a human chess player who could “see” the moves made by his opponent by means of magnets that repositioned in response to a piece being moved on the outer surface of the chessboard. To make the automaton’s moves, the player (presumably long on talent but short of Pretotyping@Work! 22!
  • 24. stature) would operate a pantograph-style lever arrangement connected to the Turk’s arm to grip, move, and release pieces on the table top. A Mechanical Turk9 pretotype then simulates sophisticated technology that would be costly or time-consuming to build from scratch, using human power to substitute for the technology. A well-designed Me- chanical Turk test delivers target customers the essential experience of a proposed technology with a tiny fraction of the required develop- ment investment. The IBM Speech-to-Text pretotype is a superb example; the human typist simulated the hardware and software under test exactly as the diminu- tive chess player in the Mechanical Turk played the role of the clock- work grandmaster. Consider a Mechanical Turk pretotype when: • When the final product requires the development of expensive and complex technology whose actions and outputs could be simulated by humans. • The value of the solution depends on multiple interacting tech- nology elements. Pretotyping@Work! 23! 9 Also known as the “Wizard of Oz” technique or “Oz Paradigm”, named by Dr. John Kelley to describe his methods of conducting psychological experiments at Johns Hopkins University in the 1970s and ‘80s.
  • 25. 4. The One Night Stand Pretotype The One Night Stand pretotype is a model in which an interactive serv- ice experience is presented in a fairly complete fashion, but minus the undergirding of infrastructure that a permanent solution requires. The physical facilities (space, equipment, fittings, decoration) may be rented and presented like a Hollywood set for the duration of the test, then dismantled and returned. Best Buy, a former client of mine, had an idea for a new service. The idea was to see if customers could be encouraged to purchase new elec- tronic gadgets such as camcorders and televisions sooner by offering them some residual value for their gently-used items. They called this concept NextPlay, and the full solution was expected to consist of an in-store department that would receive customers, test whether the items were functional, and offer the customer a credit towards new purchases using a stored value card. Could this be tested at low cost? The team’s pretotype consisted of a tent pitched in the parking lot of a Best Buy store in Boca Raton, FL. The tent covered a temporary workspace made up of folding tables, a power strip run from the store, and a Kelley Blue Book. Some advertising of the service was done in the week prior to the test within a local newspaper. The test oc- curred over a weekend: people brought used camcorders, TVs, and cell- phones, which the team tested. Customers who brought in an item the team perceived had some useful life left were paid in store credit with reference to the KBB. The store’s Point Of Sale (POS) system enabled the team to track which stored value cards were used as part of a subsequent purchase. Not Pretotyping@Work! 24!
  • 26. only that, but the data also showed whether the credit was cashed in immediately (in most cases, yes) and the average up-spend over the credit given (an appealing multiple). Today the solution is known as Technology Trade-In at Best Buy, and is deployed in many stores. The service has undergone considerable fur- ther evolution and development - for example, it is now operated on Best Buy’s behalf by a third-party partner - but the initial valida- tion of the concept was performed quickly and cheaply in a parking lot. An interesting variant of the One Night Stand is the Provincial preto- type. A Provincial simply implies exposing the pretotype offer to a limited subset of markets and customers. Consider a One Night Stand or Provincial pretotype when: • The solution is - or depends critically upon - an interactive service experience. • You expect demand for the offer to vary significantly from one market to another. • You expect demand for the offer will be sensitive to the choice of channel, and you need to test a number of possible customer interception points. 5. The Impersonator Pretotype An Impersonator pretotype is one where an existing product or service gets a new wrapper or “skin” in order to pose as the new offer under test. This has the advantage that the existing product has known per- formance characteristics, and can therefore be relied upon when put in the hands of test customers. Think of a new food product idea, such as a ready meal or a soft drink. An existing product in the same or similar category could be repackaged to pose as the new offer. Given that the Inventors are still answering the “Do they want it?” question, the ability to test actual selection and purchase in a retail environment is all that is required of the impersonator. True taste tests, including flavor preference, satiety, portion size and so on, must wait until later in the process. An excellent example of an Impersonator comes from Tesla Motors. In 2003, the founders of Tesla had an ambitious idea (a pure electric 2- door sportscar) and a marketing challenge (Tesla was an unknown quan- tity as a carmaker). In order to convince potential buyers to order its car, Tesla created a pretotype of what the car would look like. The base for the pretotype was a Lotus Elise, the car whose chassis technology was ultimately licensed - and heavily modified - by Tesla Pretotyping@Work! 25!
  • 27. to provide the basis for the Roadster chassis. Lotus supplied Tesla with a ‘glider’ Elise - a car without a powertrain - which was filled with models of key components like batteries and AC motors. This was not a prototype, because the vehicle didn’t function, yet with a (relatively) trivial investment, Tesla was able to show prospective buyers a very close proxy for the final design. Lotus Elise Tesla Roadster As if this were not canny enough, Tesla also deployed a Fake Door pre- totype to further validate demand. Instead of meeting their prospec- tive customers in Thoughtland by asking them whether they “Would buy a Roadster” if Tesla built it, they asked “Will you put down a $5,000 deposit to secure a build date?”. This is a true revealed-preference test, from which Tesla secured several hundred deposits, a non-trivial result to reassure Investors. Variants of the Impersonator pretotype include: • The Infiltrator, in which the Inventor co-opts an existing prod- uct by stealthily changing or adding one or more new features, e.g., A/B testing of different web page layouts. • The Pretend-To-Own, in which the Inventor rents or leases equip- ment or props vs. Purchasing, e.g., renting a few Toyota Priuses to pretotype an eco-friendly car rental service. Pretotyping@Work! 26!
  • 28. • The Teaser, in which the Inventor creates a fully-functional sub- set of the full solution, e.g., the first 3 chapters of a novel, or the first 10 minutes of a movie. Consider an Impersonator, Infiltrator, Pretend-To-Own, or Teaser pre- totype when: • A test of the value of the solution depends on the customers’ ability to interact with a full-scale design, and you need to create a plausible stand-in for the size, shape, color, features, etc. of the solution. 6. The Minimum Viable Product (MVP) Pretotype An MVP10 is the transition from pretotyping to prototyping of the even- tual product. Sometimes it’s necessary to invest some level of effort in creating a working prototype, an artifact delivering the core func- tion(s) of the full solution that you need to put into customers’ hands in order to permit a fair test. The key feature of the MVP is that the artifact is the simplest possible prototype, stripped down to the bare minimum required to accomplish the live test, with no addi- tional embellishments such that the fewest number of variables is un- der test at any time. Consider an MVP pretotype when: • You have learned all you can about market demand from simpler pretotypes (Fake Door, Pinocchio, Mechanical Turk, One Night Stand, or Impersonator), and further insight requires a deeper customer interaction with a functioning artifact. ___________________________________ These 6 models and their variants constitute a set of Lego blocks from which to begin experimenting with pretotyping11. I am constantly learning about new variants that may deserve their own label, and you may discover more along the way. My advice is to focus less on the label than on challenging your team to find a simpler pretotype, every time. Pretotyping@Work! 27! 10 The term MVP was coined by Eric Ries, author of The Lean Startup. 11 The PretoStorming Worksheet in Appendix 1 can be used to help you design preto- type tests, from isolating the “Do they want it?” questions to a thumbnail plan.
  • 29. 3. A WRENCH IN THE INNOVATION TOOLBOX Inventors schooled in innovation tools often ask two questions: • “How is pretotyping different to prototyping?”, and • “Isn’t pretotyping just another name for <<insert name of front- end innovation tool>>?”. In the first case, the confusion arises because Inventors have been trained to use “prototyping” as an umbrella term for any form of ex- perimentation between idea and finished product. Think about the mis- cellany of concept boards, schematics, moulded or carved shapes, half- built devices, and simulations you’ve encountered in your career: most will have carried the label “prototype”. It’s become a term that can denote any less-than-polished simulacrum of the finished it. In this context, I hope “pretotyping” can usefully isolate the ex- treme, hyper-simplified front-end of “prototyping”: ‣ A pretotype tests the question “Do they want it?”. The time hori- zon is hours or days, and the principal deliverable is revealed- preference demand data. ‣ A prototype tests the question “Can we build it?”. The time hori- zon is often months or years, and the principal deliverable is a working artifact that validates one or more performance attrib- utes. In the case of the second question, Inventors often think they already pretotype, under the guise of another label, such as Voice of Cus- tomer, Ethnographic research, Empathy interviews. These techniques can be useful in identifying problems with current offers or opportu- nities for new offers: in other words, they apply pre-idea. Some In- ventors believe they are already pretotyping when they apply post-idea but pre-prototyping techniques such as Focus Groups. These techniques are less effective than pretotyping because they don’t offer a true revealed-preference test to the customer. In summary: ‣ Pre-idea, customer insight techniques such as VOC can be useful in stimulating ideas for new products and services, by revealing customer frustrations, needs, or blocked ambitions. ‣ Post-idea, pretotypes gather authentic market demand data by pre- senting revealed-preference choices to customers and seeking com- mitments to use or buy. Other techniques like Focus Groups use an abstraction of the product idea, usually in the form of a con- cept presented to an existing customer, which can skew the analy- sis because it asks for opinions rather than commitments. Pretotyping@Work! 28!
  • 30. The following graphic sums up where I believe pretotyping belongs in the “front-end of innovation” (or FEI) toolkit12, as the quickest and best means of escaping Thoughtland: Notice that Focus Groups play no part in Demand Validation: simply put, I believe that pretotyping is a superior technology and displaces these opinion-trading marketplaces. Pretotyping@Work! 29! 12 This is a simplification for exposition, and innovation process folks will have some reflexive critiques about missing items. For example, I always recommend ad- ditional insight “lenses” to supplement customer insights (e.g., major trends analysis, orthodoxies or blind-spots analysis), and of course the true nature of the FEI is iterative.
  • 31. Invest Like a Grownup Investors, has this happened to you? A breathless Inventor team pre- sents a new idea: it’s going to be HUGE, it will REDEFINE THE INDUS- TRY, it will generate MASSIVE PROFITS! You are skeptical, but you give them some runway to investigate the market potential. Weeks later the team reports back. What have they done? The Inventors have mocked up a prototype, run focus groups, and built a great looking business case projection. Virtual champagne all around, and the team gets the next round of funding. Months later, after much more R&D and marketing effort the project collapses, while you offer insincere thanks for their efforts thus far. You knew it! Why does this happen? Because Inventors don’t bear the same risks as true entrepreneurs; it’s the company’s money, and frankly they all just love working in Thoughtland. And although your role includes sponsorship of innovation, in practice you are rewarded for shrewd stewardship of current businesses. You hedge by giving teams just enough rope to hang themselves with. How to get Inventors to think differently? Discuss the Laws of Fail- ure with them and reject the Thoughtland hype. Agree the demand evi- dence that would give you reasonable confidence to proceed and what pretotype(s) would deliver this data. Repeat. Pretotyping@Work! 30!
  • 32. 4. DON’T BELIEVE IT, PROVE IT! US Law presumes that a defendant accused of a crime is innocent until proven guilty. In this fashion, the Founding Fathers protected indi- vidual rights. The Laws of Failure means that the reverse should apply to ideas in Thoughtland: a new product is presumed a failure until it can be proved likely to be successful. In this fashion, the Funding Fathers (and Mothers) protect scarce innovation resources. At the risk of beating the analogy to death, Thoughtland opinions are hearsay, and asking “Would you...?” hypothetical questions call for speculation. What ideas need are evidence in the form of data. Pre- totypes deliver data. This chapter describes two metrics with which Inventors and Investors can progressively build confidence in new ideas based on data. RETURN ON PRETOTYPING INVESTMENT (RPI) RPI13 provides the first reassurance that the validation of market ap- peal for the new idea can be accomplished at low effort. RPI ex- presses the learning efficiency achieved by testing an idea using a pretotype experiment instead of a traditional prototype. Learning efficiency can be expressed in either time (speed of learn- ing) or money (cost of learning) units. Here’s the formula: RPI = ______________ _____________ Learning (Pro) Cost (Pre) X Learning (Pre) Cost (Pro) Where: Learning (Pre):! How much (%) you think you will learn from a given pretotype compared to the full product. Learning (Pro):! How much (%) you will learn from a prototype or final product - set to 100% for final product. Cost (Pro):! How much it would cost (time or $) to develop/test/ market a prototype or the final product. Cost (Pre):! How much would it cost (time or $) to create and test a given pretotype. Pretotyping@Work! 31! 13 Metrics Worksheets I and II in Appendix 1 can frame the discussion around set- ting targets for and tracking ILI and OLI for your pretotype tests.
  • 33. Let’s work an example to illustrate RPI, returning to our old friends, Webvan. Let’s pretend we are the Inventors and Investors behind Webvan. We have this terrific sounding idea, which looks like it will take some significant capital to develop: the business plan calls for $1B to fully build out our infrastructure. Yikes! If only we had more confidence in market demand. Before we can calculate RPI, we need to design the right pretotype. What “Do they want it?” questions should we ask (before we blow $100M+)? How about: • What % of people will use the web to order groceries? • How often would they use it? • Will people in cities use it more than people in suburbs? • What kind of products will they buy? • What’s the $ value of the average transaction? What pretotype design would give us good data on these questions? The simplest place to start would be a Fake Door campaign, but given that the technology enablement is a critical part of the Webvan solution, the following sounds better: 1. Create a high-quality website (polished front-end, no back-end). 2. Advertise locally in a major city (e.g., San Francisco) and a suburb (e.g., Palo Alto). 3. If/when orders come in, purchase food at existing stores. 4. Rent delivery trucks and hire temporary personnel to deliver food. 5. Run the experiment for 4 weeks. We should get a strong indication of demand from this test, but what proportion will it be of the learning we would get from building the full solution? To calculate RPI, we need to make some estimates on how effective this MVP/Mechanical Turk pretotype would be. It requires judgement, but this feels robust: let’s say 75%. Costing the pretotype similarly requires some judgement, but given the cost of the full solu- tion, we can be generous: $1M. Plugging these numbers into the formula gives us the Cost RPI on this pretotype: Pretotyping@Work! 32!
  • 34. RPI = .75 x 100 = 75 = 7500% cheaper RPI = ________ ________ X .75 $100M 1.00 $1M The estimates doubtless have a wide confidence interval, but there is clearly an enormous learning efficiency to this pretotype. The Inves- tors in Webvan should have been willing to stifle their Internet Bubble Fever in order to prove out the major elements of the idea before authorizing the full solution. But what about RPI in time terms? Let’s revisit the IBM Speech-to-Text example to explore time efficiency of pretotyping. The formula remains the same, but the Cost elements are calibrated in Time units rather than $. In calculating RPI, you should examine your expectations: what Learning and Cost effects is this preto- type likely to produce? Here’s my logic chain: 1. The default solution for IBM was to build a prototype, not the full product. The prototype might have told them, say, 80% of what the final product would have told them. 2. The pretotype would be a valuable test of basic user appeal, but would not shed light on more subtle factors such as how usage de- cays over time, whether usage varies by time of day, etc. Let’s say, 50% of what the final product would reveal. 3. However the Cost (i.e., Time) parameters look dramatically dif- ferent between (Pro) and (Pre). In that era, the Pro might take perhaps 5 years (60 months) for the hardware and software to be viable enough for a customer test. The Pre by contrast might take no more than 1 month to engineer. This logic gives us RPI variables as follows: Learning (Pre) = .50! Learning (Pro) = .80 Cost (Pro) = 60 months! Cost (Pre) = 1 month Plugging these numbers into the formula gives us the Time RPI on this pretotype: Pretotyping@Work! 33!
  • 35. RPI = .62 x 60 = 37.5x “faster learning” RPI = ________ _________ X .50 60 months .80 1 month Of course, any Investor might challenge my logic and offer different numbers, but the bottom line of RPI is that, under almost any condi- tions, the cost or learning rate efficiency is a) massive, and b) re- markably insensitive to less favorable estimates of the Pre’s perform- ance. Take the Webvan or IBM examples and halve the Learning (Pre) or double the Cost (Pre): in either case the argument for doing the preto- type test is still nearly impossible to refute. All you need is an in- formed estimate of the competing Cost (Pro) as your baseline, plus a logic chain for how a suitable pretotype will perform against it. Refer to Metrics Worksheet I in Appendix 1 to help you apply the RPI calculation to one of your early-stage ideas. INITIAL AND ONGOING LEVEL OF INTEREST (ILI/OLI) The Initial Level of Interest (ILI), is simply the % of a target group interested enough in it to give it a try, or: # invited to try ‘it’ # who’ve actually tried ‘it’ ILI = ________________________________ Calculating ILI requires forming a point of view on how many customers you want to expose the pretotype to, which tends to vary widely depend- ing on the nature of the final product and the volume of ultimate sales that will represent success. Clearly this target number of customers will be very different if the it is a new app (hundreds!) versus if it is a new packaging line for a factory (a dozen?). Equally, your view on what ILI - proportion of those invited who actually try it - is likely to depend on the nature of the product also. Think of ILI as tracking the behavior of a subset of your eventual tar- get market, as follows: Pretotyping@Work! 34!
  • 36. Small representative sample (~100-1,000) I (1000) Time Target market Let’s say your target group of customers to whom you’ll expose the pre- totype offer is 1,000: we call this number, ‘I’. Now say that, over the period when the pretotype offer is available, 741 (‘T’) of that 1,000 actually try it. Your ILI is: 741 / 1,000 or .741: I (1000) Time T (741) ILI = 741/1,000 = .741 Great start! It looks like a good % of your sample have taken the bait and tried your offer. Capturing an ILI is a good start, but as the folks behind late-night in- fomercials will tell you, you can sell anything once! To be sure you’ve got the right it, you need to see how many return for another try. In other words, you need to measure Ongoing Level of Interest, the % of those who initially tried it who continue to use/buy it, or: Pretotyping@Work! 35!
  • 37. OLI(t) = _________________________ # who tried ‘it’ # still using ‘it’ after time t Note that the numerator from the ILI calculation - the ‘T’ actually try- ing your it - becomes the denominator for the OLI equation. Tracking OLI over time typically follows this pattern: I (people invited to try innovation) T (people who actually try it) R4 (...4 weeks) R2 (...2 weeks) R1 (return/retained users after 1 week) OLI Time ILI To illustrate, let’s calculate ILI and OLI for our fictional Webvan pre- totype. Recall our MVP/Mechanical Turk pretotype design: what sort of ILI would encourage further investment? As with the RPI calculation, the essence of the exercise is to build a logic chain that sets expecta- tions for the pretotype-based validation of Webvan’s it (internet-based grocery ordering): 1. Given the scale ambition for the full solution, the pretotype should be exposed to a reasonably-broad cross-section of the ur- ban and suburban target communities. So let’s set ‘I’ at 10,000 people. 2. While the Thoughtland data on the Webvan it were overwhelmingly positive, a high ‘T’ is highly unlikely, given that it’s a pre- mium service, and not everyone will see the advertising. Let’s target a ‘T’ of 5%, or 500. We launch the pretotype, and let’s say that our initial response is 843, meaning that many people see the advertising, investigate the offer, and become customers. Not a response on the scale of the Thoughtland reac- tion to the Webvan idea by any means, but it beat our target. Pretotyping@Work! 36!
  • 38. I (10,000) Time T (843) ILI = 843/10,000 = .084 = 8.4% As Investors in Webvan, we should be encouraged by this initial result. But to be sure we are not seeing the infomercial effect, we decide to continue the trial for a few more weeks. If the number of return cus- tomers as a proportion of the 843 first-timers is high enough, we’ll know we have the right it. We need a target for OLI: let’s aim for 50% of the original 843. Fate, however, is not only a cruel mistress but also apparently a fickle shopper. The OLI data disappoints, with fewer and fewer of the 843 original customers returning over the next 2, 4, and 6 weeks: I (10,000) Time T (843) R2(54)R4(19) R6(6) OLI(2) = 54/843 = 6.4% OLI(4) = 19/843 = 2.2% OLI(8) = 6/843 = 0.7% Clearly the first-time experience did not encourage enough customers to return for second and subsequent trials. It may not always be clear precisely why, but the trend tells the story. For our current purposes we can conclude that the data show we do not have the right it in Webvan after all. Pretotyping@Work! 37!
  • 39. Listening to this example people often respond with: “But online grocery ordering and home delivery is a successful business. Look at Peapod, or Schwan’s”. This illustrates a nuance in defining the it under test: it describes a complete (if implied) business model of the offer under test. Webvan’s it was a nationwide service promising delivery in under 30 minutes in 26 major markets, a massive implied customer base and in- frastructure footprint: Webvan wanted to “own” premium grocery retailing in the US. This ambition colors the pretotyping process by setting an ambitious bar. Our hypothetical pretotype therefore spent $1M to build a high- quality website, and sought a very high ILI and OLI to confirm the proposition. Our test dismissed this it, but that doesn’t discount the possibility that under different business model constraints a similar it could be successful. For example, Tesco, a profitable UK bricks and mortar grocery retailer pretotyped online ordering by using their stores, employees and vehicles for fulfillment; they now consider Tesco.com simply another channel for reaching existing customers. In another case, Peapod was another pure-play online grocer that controlled its expansion by providing service only where their major stakeholder (Dutch international grocery outlet operator Royal Ahold) had existing distribution facilities. Investors can consider pretotyping a method for low-cost strategy model- ing, playing out different scenarios until the right mix of product fea- tures, execution facilities, marketing, pricing, and partnerships can be proved out. In the Webvan example, the Investors could choose to fold their tent after the first round, or rethink the business model and try another pretotype test. Pretotyping@Work! 38!
  • 40. 5. BUILDING CONFIDENCE INCREMENTALLY Pretotyping, and the RPI, ILI and OLI metrics that support it, are a practical illustration of Bayes’ Theorem. Thomas Bayes was an 18th cen- tury English mathematician and Presbyterian minister, and his theorem explains how a subjective belief should rationally change to account for (new) evidence: INITIAL BELIEF + NEW DATA = IMPROVED BELIEF Pretotyping is a rapid but structured search for new evidence on which we can base a change in our expectation for the likelihood of success. Bayes provided the mathematical formula by which probabilities can be adjusted for new evidence, and though the equation is powerful enough to govern critical aspects of modern life (e.g., GMail spam detection), we need not dive into the mechanics here. The key learning is that Inves- tors should aim to build confidence incrementally, and based on evi- dence. This implies many short, data-informed meetings with Inventors, the goal of which is to either downgrade or upgrade their shared belief about success. In most cases, the outcome of a pretotype test will be clear, and thanks to the First Law of Failure, emphatic: you’ve got the wrong it! In a few cases, the data will deliver an encouraging confirmation: you’ve got the right it! But how should Investors interpret ambiguous pretotype test results? This can of course be a test-hygiene issue: if a test tries to answer too many questions at once, it can be difficult to ascribe clear meaning to the results. Beyond this issue how do you handle test results that undershoot your target ILI and OLI, but overshoot the level of fiasco? RPI is your friend: the payback to pretotype experiments is so robust that you should run one or more additional pretotypes until you get a clear trend in the results. To recall and expand our idealized dialog between Inventors and Investors: 1. BOTH: Discuss and converge on a few “Do they want it?” questions. 2. INVENTOR: Design the simplest pretotype you can to answer those questions. 3. BOTH: Agree upper threshold (“Right it”) and lower threshold (“Wrong it”) target expectations for ILI, before the test. 4. INVENTOR: Run the test, confirm actual ILI. 5. BOTH: Agree whether ILI suggest continuation. If so, agree rea- sonable target expectation for OLI, and the appropriate repeat Pretotyping@Work! 39!
  • 41. pace (R=7 days, R=14 days, etc) and meet again after every R milestone to review progress. Your decision will be clear based on the OLI trend you see: Experiments Exp. 1 Exp. 2 Exp. 3 Right it Wrong it ??? Encouraging: keep investing! Experiments Exp. 1 Exp. 2 Exp. 3 Right it Wrong it ??? Discouraging: stop investing! Experiments Exp. 1 Exp. 2 Right it Wrong it ??? Ambiguous: keep pretotyping! Pretotyping@Work! 40!
  • 42. Discussion of this topic would not be complete without introducing the Dead Cat Bounce. This charming term is used by Wall Street investors to denote an encouraging uptick in an otherwise bear-market (i.e., down- ward) trend. The reference is to the fact that even a dead cat will bounce once if dropped or kicked hard enough. In fact, scientists have long known that almost any natural system can be stimulated to produce an involuntary response (think of the doctor’s reflex-testing hammer). A classic example from business history is the Hawthorne Effect, in which factory productivity increased in response to both positive and negative changes to lighting levels administered by the researchers. The relevance to pretotyping is that early-stage Investors can influence the outcome of the experiments they fund, so they must be vigilant to the risk of creating the conditions for a Dead Cat Bounce. Investors should set stretch yet achievable “right it” thresholds, provide enough resources - usually time - for the Inventor team to construct and run the pretotype test, then scrutinize the ILI data carefully before making your next go/no-go decision. Inventors will always want to try more tests, but stick to your guns and insist they “say it with numbers”. Pretotyping@Work! 41!
  • 43. 6. PRETOTYPING FOR ALL REASONS So far we have only discussed end consumer-facing products and services, and our case studies propose inviting a fairly large sample of potential customers to try the pretotype. For many Investors, however, the land- scape looks very different to this classic Business-to-Consumer (“B2C”) model, but I would argue that pretotyping method can be adapted to sup- port innovation within these different contexts well. INTERNAL OR PROCESS INNOVATION Studies14 have reported that, for many companies, most of their innova- tion resources go towards internal innovation, that is to say process changes, system introductions, quality initiatives. All of these inno- vations have the potential to lower cost or improve the end-customer’s experience, thus contributing indirectly to preserving or increasing revenue and profits. Pretotyping is a highly suitable method for testing the effectiveness (“success”) of an internal innovation (“it”) with a given group of em- ployee “customers”. With a new product or service, the related uncer- tainty under study is “Do they want it?”; with internal innovations, the uncertainty is a variant of “Will they comply?” (e.g., use the new proc- ess, switch to the new system, apply the training to their productive work, etc). So the key to applying pretotyping to internal innovations is to isolate the “Will they comply?” question, before choosing the right pretotype method and running the test. BUSINESS-TO-BUSINESS (“B2B”) Corporations operating in a Business-to-Business (“B2B”) environment typically sell components or sub-assemblies to other companies that then turn these into finished goods. Customers in this context are usually fewer in number but individually far more important to the company’s success. This raises the stakes for pretotyping new products and serv- ices: few B2B companies will be willing to jeopardize valuable customer relationships with a speculative Fake Door offer. The solution here is transparency and focus. B2B companies should begin by negotiating the business practice of pretotyping with one or more (preferably the most progressive) customers. This blunts the revealed preference nature of typical “blind” B2C pretotypes, but the relation- ship preservation value of this transparency is worth the sacrifice. The agreement should define the limits of the pretotyping activity, such as how many experiments per year will be conducted, and circumscribe the product categories and business processes that might be in scope. The Pretotyping@Work! 42! 14 E.g., Journal of Economic Behavior, Vol 50 (2003), Stephanie Rosenkranz, sug- gested that as much as 60% of innovation effort went to internal innovation.
  • 44. second adaptation is to limit the pretotype modes used to the four most partnership-friendly: Pinocchio, Mechanical Turk, One-Night Stand, and MVP. Fake Door and Impersonator are simply less practical in a B2B con- text. Another key difference in the B2B environment is that many innovations impose process changes on the customer’s part, and the “switching costs” of those changes can skew how receptive the customer is to the innova- tion. For this reason, pretotyping new processes can avoid often con- frontational negotiations between supplier and customer in which the supplying firm attempts to force the innovation on the customer, or in which the customer seeks to defray switching costs by changing the terms of doing business. GOVERNMENT TO BUSINESS (“G2B”) OR TO TAXPAYER (“G2T”) Public sector agencies can also pretotype services, from proposed new policies to tax regimes to the delivery of taxpayer-funded services like garbage collection. As in the B2B context, a degree of transparency is advisable, but citizens have generally been enthusiastic about the use of social networks, crowdsourcing platforms, and idea marketplaces to engage voters and taxpayers in the work of making policy. Pretotyping new policies, laws, or services would be the next step in interactive democracy. ___________________________________ CLOSING ARGUMENTS The Laws of Failure state that for any innovation, success is extremely rare. Pretotyping supports rapid, disciplined testing of breakthrough innovations, allowing Inventors and Investors to: • Invent Like A Startup: firms should experiment with lots of ideas, both the obvious ones (potential False Positives, like Webvan) and the crazy-sounding ones (potential False Negatives, like Twitter). • Invest Like A Grownup: firms should invest in breakthrough inno- vations based on evidence and data, not opinion or speculation. As evidence incrementally builds confidence, then investment should flow. Pretotyping changes how Inventors and Investors talk with each other, such that their mutual interest is efficiently gathering data, not trad- ing speculations. Pretotyping does not result in fewer failures, but faster failures. This conserves innovation resources so that the small number of “right its” can be identified and supported sooner. Embrace pretotyping in your business if you want to fail (fast)! Pretotyping@Work! 43!
  • 45. APPENDIX 1 - PRETOTYPING WORKSHEETS PretoStorming experiment design worksheet Pretotyping Metrics I - Calculating RPI and ILI Pretotyping Metrics II - Calculating OLI Pretotyping@Work! 44!
  • 46. Pretotyping@Work! 45!
  • 47. Pretotyping@Work! 46!
  • 48. Pretotyping@Work! 47!
  • 49. APPENDIX 2 - ABOUT THE AUTHOR Jeremy Clark is a growth strategy and innovation expert, helping com- panies to unleash innovation for over 20 years. As a consultant, he has coached business leaders across many sectors through innovation and growth strategy projects, and he has helped to create hundreds of millions of dollars in new wealth from innovative products and services.  Many of these are highly visible brands, others thrive as internal process innovations or B2B offers embedded in OEM customer solutions. Before becoming an independent consultant, Jeremy was a Principal at Strategos, the firm founded by management expert Professor Gary Hamel, and he continues to provide support to Hamel’s latest project, the Management Innovation eXchange (or MIX). Jeremy co-founded Pretotype Labs with Alberto Savoia in 2012 to introduce agile innovation tech- niques to complement more traditional approaches to mature company in- novation such as R&D labs and structured NPD processes. Jeremy received his MBA from the University of Chicago, and is a fre- quent speaker on strategy and innovation. Jeremy is an expert in corporate venturing, an approach that embeds entrepreneurial principles and methods within companies.  Increas- ingly, he helps companies to harness the power of social media to en- gage larger communities and customer groups in company innovation work. Contact: jeremy@fxxinc.com or visit www.pretotypelabs.com Pretotyping@Work! 48

Related Documents