Lean Startup as an Entrepreneurial Strategy: Limitations, Outcomes, and Learnings for Practitioners

John M. York^1,2*, Jonathan L. York³ and Philip Powell^4
*Correspondence: John M. York, Institute for the Global Entrepreneur at the Jacobs School of Engineering and the Rady School of Management, San Diego, USA, Tel: +18052382485, Email:

Keywords

Entrepreneurship strategy • Entrepreneurial outcomes • Hypothesis-driven entrepreneurship • Lean startup (startup) • Lean startup (start-up) limits (limitations) (pitfalls) (boundaries) • Minimum viable product • Practical entrepreneurial learnings • Startup practices

Introduction

It is a minority of cases in which entrepreneurs succeed because they can define the concept adequately at the beginning of their startup journey. Further, it is a rare circumstance in which these individuals achieve an acceptable product/market fit (P/MF) between the opportunity and their value proposition [1].

Such outcomes are due to the lack of customer input on research and development and the limited market research performed before developing the product or service [1]. This reality is because many entrepreneurs advance their business ideas forward withoutan understanding of their industries, competition, and customers. Consequently, they misread their markets, which leads to theintroduction of products that are either not needed or not simple enough for application [1]. Furthermore, many startups lack a structured process to discover and understand markets, identify customers, and validate hypotheses during the gestational stage of the firm [2]. As a result, customers do not engage with or purchase these products. Hence, these startups fail to identify and address critical customer challenges that lead to P/MF; this failure preempts firms from investment and scaling effectively [3].

The lean startup (LS) embodies a popular approach to help entrepreneurs address market uncertainty and improve their odds of success. EricRies’ book sales are over one million copies [4]. The tech startup community in Silicon Valley, the National Science Foundation (NSF) Innovation Corps™ (I-Corps™) program, and numerous universities use the methodology [5-7]. Corporations (e.g., Dropbox, General Electric, Intuit, and Proctor and Gamble) employ LS [8,9]. LS meetups globally engage 20,000 regular participants [10].

In considering the attention with LS, this paper aims to explore what limitations do exit with the methodology and/or its use, what is the impact of the methodology on performance outcomes, and what learnings can practitioners and educators use. The flow of this paper proceeds from a brief introduction to LS and discusses the evidence found to address each of these three core areas.

Research questions and search/review methods

This aim of this paper is to review the literature specific to addressingthree research questions: (1) what the limitations of this methodology and its use as an entrepreneurial strategy are?; (2) what are the outcomes associated with using this approach?; and (3) what learnings should practitioners consider when utilizing this approach?

To identify relevant literature concerning LS and its limitations, the authors engaged multiple sources. These included published peer-review papers, non-peer review documents including practitioner publications (e.g., Harvard Business Review, Sloan Management Review, Rotman Management Magazine), graduate theses, business publications, blogs, and books, andnon-peer-reviewed web content. Electronic databases reviewed included ABI/Inform, EBSCO, JSTOR, Google [and Google Scholar], ProQuest Dissertations and Theses, Science Direct, and Web of Science.

The search strategy began with the use of keywords relating to LS. These included “lean (or agile)”, “lean (or agile) entrepreneur”, “lean (or agile) entrepreneurship”, “lean (or agile) start*up (startup, startup)”, “lean (or agile) venture”. For the first question, the strategy added modifiers to “lean start*up*”, including “barrier*”, “challenge*”,“disadvantage”, “drawback*”, “issue*, ”hurdle*”, “limit*, ”limitation*”, “pitfall”, or “problem*”. For the second question, the strategy added the following terms, such as “failure”, “impact”, “new venture performance”, “outcomes”, “performance”, and “success” to the search string. There were no specific search strings used for the third question since learnings from the investigation would inform our response and recommendations to address it.

The broad search captured 300 citations that included LS in the title or abstract. The search concerning the first and second questionsled to 200 and 50 citations that included LS in the title or abstract to examine in more detail. From the search results, the choice of citations depended on the authors’ knowledge related to the field of entrepreneurship, startups, and LS practices to extract value from a source’s content. These included published peer-review papers and non-peer review documents (e.g., graduate theses, business publications, blogs, and non-peer-reviewed web content). A closer examination led to the identification of a limited number of publications that specifically looked at LS and its limitations to address the first question and LS and outcomes.

Due to the limited number of peer review papers drawn from the search, the review included citations from non-peer review pieces. Most of the publications involved those from advocates and practitioners describing the methodology. These works were predominantly in the non-peer review space and included trade publications and blogs describing the methodology. Furthermore, the review employed a “snowball” approach involving the identification of additional citations from relevant references from articles reviewed from the initial search and did not appear. The authors examined such articles for relevance based on the specific research questions and knowledge related to the field of entrepreneurship, startups, and LS practices to extract value from a source’s content. The authors reviewed these pieces and included those that provided relevant data to address the research questions.

Lean startup

LS involves a scientific methodology for developing businesses and products. Itaims to shorten the product development cycle by adopting a combination of hypothesis-driven experimentation, iterative product releases, validated learning, and customer feedback [11,12].

LS draws on the Toyota Production System and agile software development [13,14]. Furthermore, its foundation includes several academic theories bricolage, business model, creation and discovery, dynamic capabilities, effectuation, organizational learning, and real options [15-27]. Finally, this methodology stands on scientific literature support, rangingfrom moderate (for experimentation and minimum viable product [MVP]), to robust (for effectuation and iteration), to very strong (for customer involvement) [12].

Several components define LS. The first involves customer discovery, wherethe startup focuses on identifying the customer, his/her needs, repeatable businessmodel, and P/MF. Discovery involves direct customer conversations, with the entrepreneur“getting out of the building” (“GOOB”) to understand critical issues and confirm the customer’s problem or “job-todo” [8,28-30]. The entrepreneur’s job is to get inside the customer’s head to discover and validate the problem, then find out whether one’s proposed solution might work. Such insights can speed the construction and validation of an MVP and a scalable business model.

The next component involves experimentation [11,19,29]. Ries [29] fashions LS as a scientific approach using hypothesis testing to provide for validated learning to guide decisions. This phase involves the running of experiments and the “build-measure-learn” cycle [11,29].

Essential to this process is the minimum viable product, or MVP, to get the customer’s job done. This MVP enables the firm to launch sooner and reach early evangelists to get initial input of the product [11]. Ries defines it as the product version that can drive a “build-measure-learn” cycle turn with the most minimal effort and development time but requires extra work for one to measure its impact [29]. Also, the MVP should contain a “bare-bones” set of features and capabilities to measure its traction in the market [31]. Finally, it allows the firm to trial its riskiest assumptions and shortening the feedback time [32]. Tied to experimentation is innovation accounting. Having a metricbased evaluation helps to measure progress and validate learning. It defines actionable metrics linked to a specific business model [33]. Startups test their hypotheses and use quantitative metrics to evaluate progress. Examples include thresholds (e.g.,a Kickstarter target), web landing page engagement (e.g., click-through rates, sign-ups), A/B tests (comparison of two versions of a product or communication), and MVP responses (e.g., willingness to pay).

The final piece involves iterations and pivots in the product’s design and the firm’s business modelbased on learnings from experimentation.Due to these actions, scholars also characterize LS as an adaptive strategy [34,35]. Iterations require minor changes to the MVP or business model. Pivoting involves a more substantial course correction from the initial hypothesis and MVP to new ones around the product, strategy, and engine of growth. Learnings from customer interviews provide qualitative data, and hypothesis testing supplies quantitative data to drive these actions.

Several canvases support LS. These frameworks allow the entrepreneur to chart out hypotheses and changes related to value propositions, MVP characteristics, and business models. Osterwalder and Pigneur [36,37] provide the value proposition (VPC) and the business model canvases (BMC). The VPC defines the value proposition (and minimum features for an MVP) based on customer needs. The BMC outlines the business model based on nine pieces that define value creation/extraction and operations/ efficiency. Maurya [38] offers a third map, the lean canvas. It helps entrepreneurs to deconstruct their ideas into their essential assumptions and breaks it into product and market sections [39].

What are the Limitations of this Methodology and Its Use as an Entrepreneurial Strategy?

Exploration of this question led to the identification of several notable areas that highlight the limits associated with the methodology. Such limits include those that are (1) inherent to the methodology, (2) related to appropriate knowledge and utilization by practitioners, and (3) boundaries conditions (or relate to fit with specific business verticals). The following areas customer discovery, minimum viable product, experimentation, iteration/pivoting, and boundary conditions will examine various experiences that highlight such limitations (Figure 1).

Customer discovery

Essential to customer discovery is interviewing. However, this process is fraught with problems and biases. Poor implementation of the interview methodology and subsequent analysis can undermine customer discovery efforts.

In a recent Long Range Planning essay, Felin et al. [40] offer several challenges to customer discovery. They question the ideal timing, product type, and sectors for it, along with the emphasis of the customer in the early phase [40]. These scholars contest the assumption that the customer knows what s/he wants, as there may be hidden or unexpressed needs. They raise issues concerning what data in observations are most relevant or not [40]. They observe that available feedback can teach startups the wrong lessons and lead to both a myopic view and dangerous traps [40].

Two other academic groups identify issues with customer biases involved in the interviewing process that pose a significant risk to customer discovery [41,42]. York and Danes [42] explain that many entrepreneurs, who rely upon a subjective view and limited data, fail to obtain or notice available information critical for making a proper decision. Also, they identify multiple interviewing biases. These include selection (i.e., friends and family), confirmation (i.e., leading, confirmatory, and closed-ended questions), overconfidence (i.e., overestimating one’s knowledge, skills, and data), optimism (i.e., extreme positivity), representativeness (i.e., generalize findings from small samples), and acquiescence (i.e., respondents providing answers they think the entrepreneur wants to hear) [42].

Chen et al. [41] add to further perspectives on interview biases. For face-toface interviews, they cite issues with generating saliency (i.e., highlighting the most noteworthy points) and vividness (i.e., producing powerful feelings or defined images in one’s mind); providing inappropriate cues (i.e., misleading or inconsistent body language);andusing inappropriate analogies (i.e., making comparisons between two items to describe a point) [41]. Specific to consecutive interactions, these authors raise concerns around the contextual considerations related to recency (i.e., proximity in time from the analysis), primacy (i.e., the effect of rank, office, or being first and foremost), and contrast (i.e., state of being strikingly different from something else [41,43]. Their next considers issues with large samples, including the effects of over confidence, redundancy (i.e., duplication), and dilution (i.e., the state of diluting something such as a signal) [41,43]. Their final considersthat of biased processing (i.e., the irrational or illogically process of information) by the entrepreneur [41,44].

Croll and Yoskovitz [33] provide an additional perspective suggesting that interview subjects might also have their own cognitive biases due to different expectations and frame-of-reference. This point is critical because entrepreneurs need to interpret customer feedback with such insights in mind. Additionally, they reinforce Blank’s points regarding the need to conduct many interviews, far beyond the initial 15 that they recommend [28,33].

Furthermore, the ability to obtain a suitable customer sample and the right customers can be a challenge. In their case study in Indonesia, Nirwan and Dhewanto [45] notice the entrepreneur’s ability to access customers makes it difficult to capture customer feedback and confirm hypotheses. Chassagne [46] observes this barrier with Brazilian entrepreneurs as well. He observes that entrepreneurs encounter difficulties implementing the “get out of the building” phase. Suchproblems may represent a mindset, timing, or resource limitation that precludes the entrepreneur from generatinga reasonable sample translating to meaningful feedback and insights.

Interestingly, both Nirwan and Dhewanto and Gustafsson and Qvillberg [46,47] add that there is difficulty in honing in on an opportunity due to the high variation and complexity in customer discovery processes. Such observations might suggest that these entrepreneurs’ interviewing efforts might not have been enough to identify the real needs or that their biases around their product or business limited these engagements. Alternatively, they might indicate limited or no market opportunity at the outset.

Finally, Ng [32] observes that entrepreneurs tend to ask the wrong questions. She explains that they conduct poor interviews during discovery because they are focusing on selling the product, instead of investigating current customer behaviors and gaining insights to find an appropriate solution. Ng adds that they talk too much, ask leading questions, and fail to dig deeper. These observations emphasize that the interviewer’s goals should be to understand the customer, explore needs, and not to validate (or promote) a value proposition.

Minimum viable product

Felin et al. [40] challenge whether firms should engage customers in their early stages using the MVP. They contest whether the MVP interaction would provide a usable and reliable signal in the nascent product, strategy, and business model development process [40]. They question why customers would have a better sense of a future product’s viability and whether such interactions would generate transformative and novel products [40].

Heitmann [48] argues that bringing an inferior, unfinished product to the market (e.g., “buggy” software) leads to a considerable percentage of dissatisfied customers. He cites LeBoeuf [49] who indicates that 96% of dissatisfied customers will not share any feedback on the startup because of the incompleteness of the MVP. He continues that the adding and the testing of new features can lead to unnecessary testing loops that waste money and time. This author proposes that entrepreneurs focus upon the concept of a “minimum desirable product,” one to cause enough satisfaction and desire for the customer to stay interested and not abandon [48].

From an academic vantage, Frederickson and Brem [12] identify the problem of the entrepreneur stoically adhering to an idea, product, or theory. Such illustrates what Felin and colleagues [48] describe as more of a “supply-side” approach that does not require the customer perspective. Frederickson and Brem [12] explain that limited resources might set a boundary condition exploring alternative or broader solution spaces. They also identify this problem to exist with entrepreneurs who employ more of a “causal” rather than an “effectual” thinking approach from the start [12]. To this end, these scholars add that these entrepreneurs see the solution from the start, and, thus, limit their options and end up with an extremely narrow set of solution options [12]. Hence, these authors conclude that such a narrow perspective limits the entrepreneur’s ability to identify solutions that might address the customer and market need more effectively [12]. By doing so, such entrepreneurs are not adopting LS methods and implementing properly in their value creation efforts.

Other academics note problems in designing and developing the MVP. Ghezzi [18] reports this issue from a survey of 272 mobile startups, along with follow-up interviews. In this study, while 62% of survey respondents indicate the MVP as a vital concept, 82% express that the defining and designing of an MVP as one of LS’s disadvantages. In examining the verbatims from his qualitative interviews, this scholar identifies the complexity of what the MVP is as a factor, especially in more sophisticated spaces such as artificial intelligence [18]. Further, he learns from these interviews that the ability or inability to craft an appropriate MVP, to prioritize tests around it, and its use in the business-to-business setting as problematic [18].

Interestingly, Warberg and Thorup [50] share issues in the MVP development process. In examining software startups in Scandinavia, they identify several technical challenges associated with the MVP. They observe that LS devalues the proper architecture in the software (i.e., “junk code”). Further, it creates unnecessary waste in the software because of the need to rewrite and clean up software because of too simplistic code at the outset. Finally, they add that LS hinders the development of innovative solutions in the software. They note that the emphasis on rapidly launching a product can eclipse the emphasis on the overall quality or creativity of the product [50]. To illustrate this point, they quote a point from a blog by Cohn that emphasizes that with using a related software development methodology, scrum in agile, teams begin with a safer approach and never attempt any “wild ideas” that could translate to an innovative solution [50,51]. To this end, they suggest an emphasis on innovation as an essential activity to accompany the use of LS.

Other consultants reiterate the challenge of implementing the MVP in practice. In describing the case example involving ThingShare (aplatform for peer-to-peer video game renting), Kortmann(2012) adds to the above concerns andwonders whether his company “launched” the product too early. He explains that while his firm had invested time and expense going to market with more than an MVP, it short-changes its early adopters. He suggests the redefinition of a viable MVP-a product that did not need any more features, provides for revenue and profitability, and engages a critical mass of customers [52].

Furthermore, Ng [32] suggests that startups dismiss the need for building an MVP.Instead, she observes that they hadpreset ideas. In many ways, this practice is common, especially in the engineering and science space, and can be problematic. Finneran [53] raises similar concerns. He challenges the point of releasing an inferior product or service that customers would pay to enable the startup’s learning process. This consultant adds that his customers prefer a more polished product; they do not want to invest any of their time or efforts in evaluating an MVP.

Concerning the Indian experience, Rao [54] offers perspective on the MVP. Henotes that the MVP might not encompass the essential intellectual property protection needed because the firm has not finalized the product and, thus, cannot secure a definitive patent application [54]. Rao adds that Indian entrepreneurs engage demanding customers who werefamiliar with Western ‘readymade’ products and well on their way down the adoption and commercialization curve. He continues that such customers are not familiar with innovative early-stage products locally and, thus, reject an inexpensive MVP. Hence, this author finds that the entrepreneur ends up with a more developed product and enters the Indian market only after success abroad [54]. The cultural uniqueness or more the natural preferences of consumers in an emerging economy such as India’s might explain Rao’s observations. In Indonesia, Nirwan and Dhewanto [45] observe similar behaviors. They find that the MVP ischallenging to implement due to customer expectations, perceptions, and confusion, especially in a market with multiple competitors. While these authors acknowledge that MVP’s purpose is to create a minimum product to capture customer interest,they find that the startups do not want to create an inferior product in a market. Nonetheless, they observe that such firms cannot afford to go too far in developing a full product due to available capital. Interestingly, Chassagne [46] notices similar issues that force startups in Brazil to “run fat” rather than “lean” with an MVP due to the size of the market and the high level of competition. Thus, this Brazilian experience, along with those in Indonesia and India, suggests two vital considerations. These include (1) cultural issues involved with customers embracing the MVP; and (2) the perspective that firms in these countries need to get it right on the first launch.

Experimentation

The most problematic LS practice involves experimentation. Felin and colleagues [40] contest that a hypothesis must be more than just a guess and that LS distorts the development of meaningful hypotheses [40]. They argue for entrepreneurs to pursuea more scientific, theoretical, and logical approach [40,55]. Furthermore, these authors opine that some of the most valuable ideas might not lead to the type of experimentation in LS [40]. They raise concerns with the use of step wise experiments, which use early adopters and the rapid testing of ideas or products. They argue that this approach creates only incremental value. These authors highlight issues around experimental composition and design, along with what types would be most critical to lead to a break through product of value. They question whether startup founders can visualize the unknown future versus present realities. These scholars continue that this process of experimentation would not yield reliable and predictive information that would translate to a meaningful product or venture.

In his mixed-methods study, Ghezzi [18] highlights issues with experimentation. He observes that 52% in their survey indicate that the defining testing priorities and designing tests are a challenge [18]. He cites that 69% of the respondents specify that identifying and engaging early evangelists and trial users to test the MVP as a disadvantage [18]. To reinforce these survey findings, this scholar shares several verbatims from entrepreneur interviews. Some of the issues he identifies include that of getting agreement between the founder and his/er team on statements to test, prioritization of tests, designing appropriate tests around an MVP, tests around a purchase action, and holding off from willingness to pay inquiries [18]. He also shares perspectives around the amount of time and effort in getting the testing process refined enough to provide useful information and frustrations about not learning anything from an experiment [18]. He also finds a wide range of expenses in running tests ($19,000 to $180,000, mean $34,000) and as a percentage of raised capital (18% to 43%, 24%) [18]. To this end, he reports that survey participants ratethe use of LS with a “poor”overall satisfaction score of 2.8 (4-point Likert scale), which may reflect some of their frustrations with testing, along with the MVP.

Other consultants reinforce these views. Shafer [56] identifies several pitfalls that involved bias and ill-designed experiments. He first cites facilitator and observer bias concerning hypothesis development and testing. Another point that this consultant raises considers theambiguous results from open-ended experiments. Vlaskovits [57] adds that some environments are too complex and chaotic for meaningful hypotheses to be formed and tested. He explains that coming up with perfect experiments provides a great excuse not to take action because of the amount of effort needed to run a proper evaluation to provide meaningful data [57]. Ng [32] observes that a significant problem with experimentation is the testing of the wrong aspect. She notes that many come with the “I have an idea!” hypothesis. She explains that this mindset leads to a tunnel vision, in which the entrepreneur could not identify whether the guess was correct due to inherent bias. Ng adds that the forming of a wrong hypothesis was due to the entrepreneurs misunderstanding the problem and overlooking the root cause.

A related issue involves that of engaging early adopters as part of the experimentation process. Heitmann describes this effort as looking through a “keyhole” and observing only early adopters, which limits the breadth of options [48]. Thus, he feels this focus would miss the “early majority” segment, essential to scaling a business. Finneran [53] also notes that working with“early adopters” and “early evangelists” might be unrealistic as none of these individuals would give feedback on unpolished software to be the first users.

Nirwan and Dhewanto [45] observe that Indonesian entrepreneurs experience challenges in creating and validating the problem and then the solution. The inability to obtain enough of a sample makes it difficult for the entrepreneur to capture customer feedback and to confirm hypotheses. Schaefer [56] reinforces this point regarding experiments related to the lack of statistically significant effects due to small samples. Finneran [53] offers a similar concern in gaining an adequate number of customers to engage with the MVP early in the process.

Related to experimentation are concerns with innovation accounting. Ng [32] sees the inability of entrepreneurs to define a baseline metric to use for accountability during experimentation. Burgstone [58] challenges the use of innovation accounting metrics (e.g., views, likes, engagement of customers, traffic) instead of standard accounting practices.

Iteration and pivoting

Scholars and consultants recognize potential limits with iteration and pivoting. Heitmann [48] observes that without actual learning and change, the entrepreneur’s previous work was for naught. Ng [32] supports this point and adds that a common mistake involves discarding an idea without learning from the data and getting the whole team on the same page related to learnings and pivots.Vlaskovits [57] adds that it is hard to get entrepreneurs motivated to be resilient when, upon a pivot, one decides that one’s initial direction was not enough.

Heitmann [48] observes that taking the stigma away from failure detracts from the focus upon persistence. He also notes that sometimes, the entrepreneur gets stuck in a cycle of pivoting and fails to recognize the need to move on to generate revenue and scale. Kressel and Winarsky [59] reinforce this point by commenting that constant pivoting is like having a compass without a bearing because it is continuous and without a specific purpose. Hence, the concern here is that LS may be teaching entrepreneurs to think of success as merely the act of pivoting and iteration (i.e., the process) rather than a focus on delivering a final product and generating revenue (i.e., the outcome).

In practice, several other scholars offer additional insights related to challenges with pivoting. Gustafsson and Qvillberg [47] observe difficulty in pivoting due to the lack of big problems seen by customers. The issue here is whether the startup here had failed to pivot due to inadequate learning through poorly executed customer discovery or experiments, or rather due to the lack of applying the learnings. Nirwan and Dhewanto [45] observe the same challenge in pivoting due to the lack of a significant problem to address in their business-to-business case study. These authors indicate that this action isa challenge because it leads to an incremental product as the solution that customers show limited interest in a fiercely competitive marketplace. They also notice a further barrier involves the speed of iteration due to regulatory and administrative considerations. Finally, Lilac and colleagues [60] report from their survey of Croatian entrepreneurs that despite their knowledge of LS, these individualsdo not change their business model. Such observations reflect potentially a disconnect between knowledge and practice, or that the entrepreneurs remained fixed in pursuing their ideas despite their use of LS.

Frederikson and Brem [12] note a significant issue with pivoting. They explain that effectuation, an adaptativeprocess, restricts the breadth of solutions [12]. Such limits influence the entrepreneur to chart a direction within specific boundaries that determine the pivot direction [12]. Felin et al. [40] add that LS experiments might lead to a narrow view of opportunities (i.e., looking for one’s keys with a flashlight). Ladd [61] explains that LS might produce “false negatives,” translating to the entrepreneur rejecting good ideas because LS did not provide clear rules for defining go/no go, success (P/MF), stopping testing, and scaling.

Boundary considerations

One question explores whether all types of firms can use the LS methodology. Considering its Silicon Valley roots, LS fits with softwaredriven ventures that address a business-to-consumer market, especially when considering market uncertainty [29,62]. Bortolini and colleagues [34] add that the popularity of LS paralleled the “boom” period in the growth of mobile apps that began in the late 2000s. Investors Kressel and Winarsky [59] argue that LS makes sense for software- or web-related companies with modest startup operating expenses. Frederikson and Brem [12] explain that specific practices (e.g., experimentation, MVP, and iteration/pivoting) are most applicable to software development. Croll and Yoskovich [33] describe six digital models (e-commerce, the two-sided marketplace, software as a service, free mobile app, media, user-generated content) that use LS practices, particularly innovation accounting.

Interestingly, several established corporations employ LS. Ries highlights over thirty firms (startups and established) in his book [29]. Notable firms include General Electric (GE), Hewlett Packard, Intuit, Paypal, Proctor and Gamble, Telefonica, Toyota, and Zappos [11.12,29,63-67].

Nevertheless, it is crucial to consider what type of business might benefit (or not) from an adaptive strategy such as LS. For example, Andries and Debackere [68] reflect this consideration in their survival analysis of 117 firms from independent and large-firm new ventures in the biotech, automation, and environmental sectors. These authors note that some firms have barriers to shifting their business models due to significant investment needs for research and development and other organizational and inventory requirements [68]. They observe that not all industry sectors enjoy survival benefits with adaptation. Such need to consider the impact (and context of) sector maturity, technology advancement, dynamics or industry pace (rapid vs. slow), capital intensity, financial support, and even economic cycle (e.g., recessionary) that can influence the survival benefit with a business model adaptation strategy [68].

However, in some settings such as the material technologies space (e.g., chemical, advanced materials, semiconductor, silicon chips) LS may not apply well because such verticals must address technological uncertainty, along with legal/regulatory, financial, and operational risks. Harms et al. [62] underscore this point by explaining that materials and science-based ventures (1) operate under a high degree of technological uncertainty to resolve so they can develop the actual products in a specific time frame, and (2) often serve business markets. They observe that the close link of product and process innovation in such ventures make LS less suitable for resolving market uncertainty and create challenges for an MVP. Process changes impact the product (and vice versa).

Furthermore, in such firms, feedback loops may take too long and be too expensive. Iteration or pivoting on a product might also require the resubmission of intellectual property (IP) protection due to the changes in both products and manufacturing processes that a patent, for example, would cover. Any change would lead to firms returning to the starting point, costing significant firm time and capital in its development and commercialization processes.

Interestingly, in a case evaluation involving 69 semi-structured interviews and journal observations by employees in an early-phase firm with new manufacturing technology (e.g., heating process), Gustafsson and Qvillberg [47] reinforce some of the above considerations. They pose that the complexity of the manufacturing technology and process for the drying of sheet metal, along with the customers’ differing needs, significantly challenged the ability to prototype quickly and to provide a quality MVP. They identified multiple barriers to LS, including the customers’ emphasis on end-productreliability, the need for physical distribution channels, and the lack of a significant “customer problems” to address in the application segments they chose to explore,

The biotech and pharmaceutical industries further exemplify such challenges. This vertical involves a complicated business with many challenges that require a long time to market (approximately ten years) and significant investment ($2.5 billion) [69-71]. The use of LS during the drug discovery and development process may be problematic since firms cannot alter these products without a restart and may require new IP. Such efforts require time and capital. Patients in a clinical trial represent the only consumers able to receive the product before regulatory approval. Finally, various pieces (e.g., manufacturing, packaging, labeling, and supply chain) are subject to regulatory approval. As the firm advances a material product, it needs to consider other value chain partners (e.g., regulators, licensing partners, large purchasing organizations, and insurers). These players can influence product development and peel away financial value from the asset and the innovator firm. Thus, LS poses significant challenges for commercialization in these sectors and in any others where the firm needs to address multiple stakeholders and risks vis-a-vis an iterative approval or sales cycle. Interestingly, Eisenmann et al. [11] reinforce the above observation relative to the poor fit of LS in industries with long lead times and high demand. They explain that complicated business, which requires engineering and scientific breakthroughs or regulatory milestones to reach, are difficult, if not impossible, to launch timely a first-generation product and subsequent improvement for LS to offer value. The pharmaceutical industry provides such an example. These authors add in industries in which individuals must limit mistakes [11]. These comments counter the “learn from mistakes” mentality with LS. They highlight situations where a mistake would be intolerable, such as failure scenarios that a firm cannot fix post-launch, impacts a customer’s mission-critical activities, or society has low tolerance [11]. Industries such as health care and aerospace exemplify such areas. Another area they cite is an area where unmet demand is low, such as in the areas of alternative energy sources [11].

What are the Outcomes Associated with Using this Approach?

The second question explores the question about the outcomes associated with LS. Of interest relates to the influence of using this methodology on some type of performance-related outcome, or the extent to which a new venture meets its goals concerning market share, profit margin, return on assets, revenues, or other specific metrics [72,73].

Anecdotal experience

Most of the documented experiences involve anecdotal experiences (e.g., reports, examples in books, cases) [12]. The most notable examples involve the experiences of General Electric, the Startup Genome, and the National Science Foundation’s Innovation Corps™ (I-Corps™) program. A Harvard Business Review case description involving the multinational, General Electric, offers insight into the successful use of LS at the corporate level [74]. The article discusses the Fastworks program. It provides notable examples by highlighting how two divisions experienced significant success using the methodology [9,74]. The first involves the gas turbine group, which achieves a product development cycle that is two-years faster and 40% less expensive, along with $2 billion in revenues) [74]. The second describes how the appliance group improves its efficiency by halving the cost and doubling the rate of product development while increasing its sales growth rate two fold [9,74].

The Startup Genome project offers further unpublished insights [74]. In analyzing survey responses from 650+ web startups, this group found greater success with startups that pivot once or twice (raise two and a half times more funds, three and a half times more substantial user growth, and 52% less likely to scale prematurely) [74]. However, they do find that other factors, such as founder experience and team mix, do influence outcomes as well [74].

The I-CORPS ™’s program, which utilizes LS as its base process, represents another significant experience. Nnakwe et al. [6] highlight results as of March 2017 within a review paper on the I-CORPS™: 973 teams from 222 universities and leading to 320 startups (30% of teams) and $83 million ($259 thousand/team) in follow on funding. Venture Well [5] provides updated numbers: 1450 teams from 230 universities and resulted in 600 startups (41% of teams) and $210 million ($350 thousand/team) in follow on funding. Unfortunately, neither group offers a rigorous analysis in the empiric literature.

These experiences offer valuable insight into the influence of the lean startup methodology. Each offers examples of outcomes associated with methodology, either by the use of customer discovery or experimentation. However, these examples, among others, appear in non-peer review sources. More significant, they lack an appropriate methodologic rigor to either define the effect of LS clearly, to dissect its influence versus other internal and external confounders, and, finally, to account for any potential author biases in documenting these experiences.

Empiric experience

Several studies are beginning to shed some light on the impact of LS or LS-like practices (e.g., adaptation) on NVP (Figure 2). They represent a diversity of experiences. Such is that the variability in outcomes may represent differences in study populations, design, endpoints, and business sector.

Camuffo et al. [76] provide some of the most rigorous data from a randomized control trial involving 116 Italian startups (59 treatment, 57 control), and 16 data points over a year. The authors highlight that the treatment group progresses through more intensive training on frameworks for predicting performance and conducting rigorous hypothesis tests. These authors observe that these efforts translate into more pivots (P<0.05, linear regression) and dropouts (P<0.05, linear regression), along with a shorter time to revenue (P<0.05, Cox regression), versus the control group [76]. Such findings emphasize the importance of structured training and follow up with the methodology.

Ghezzi and colleagues [1,18] offer additional insights. The first involves a conference paper that describes a comparative case assessment of LS (two teams) versus business plans (two teams) with startups in the mobile space [1]. They find that teams using LS realize (1) shorter times for product development (3 and 4 mo. versus 8 and 15 mo.), shorter venture organization (3.5 mo. versus nine mo. and 1.5 yr.), and first customer acquired (1 and 2 wk. vs. two mo. and none); and (2) equity funding (2 LS, 1 BP) [1]. Their second study involves a comprehensive survey of 227 startups in the mobile space. In it, entrepreneurs cite several advantages with LS: (1) decreasing time and cost for startup testing (74%); (2) aligning customer and business idea (68%); (3) verifying and pivoting business model (52%); and (4) gaining financing (39%) [18].

Ladd et al. [20] share mixed experience involving 271 clean-tech teams (185 LS, 86 non-LS) using a bimodal endpoint (award/no award) to assess pitch competition performance at the end of an accelerator program. LS users represent 13%, and non-LS users are 7% of successes within the whole group, whereas only 19% of the LS users and 22% of LS users within each group are successful. However, teams validating their hypotheses fare three times better in the competition (P<.01), and customer discovery is significant in enhancing success (P<0.05) [20]. Unfortunately, the number of validated hypotheses and subsequent success and concurrent use of hypothesis testing does not correlate linearly, and customer discovery does not improve outcomes. However, by focusing on validating the customer segment, value proposition, and channel areas of the business model, the LS group outperforms those who did use the methodology by two-fold (P<.001). Eesley and Wu [35] provide another relevant study showing mixed effects. They compare the short-term and two-year performance of students randomized to adaptive or planning-based approaches (with and without diverse mentoring) in an entrepreneurship class taught as a MOOC. In the short-term, teams (n=942) using the business planning approach perform better in course grading by 0.552 points (P<0.05) than the students in the adaptive-only group. However, the diverse adaptive group can narrow the gap with an additional 0.538 points, which mitigated the advantage seen with the planning group. The two-year follow-on survey (n= 554) finds that those who used adaptive approaches fare better concerning revenue (P<0.5) and funding (P<0.05) [35]

Andries and Debackere [68] report the results of an investigation of the adaptation-performance hypothesis in 117 entities (65 independent new ventures and 52 business units of established firms). Drawing data from the annual CorpTech directory and defining adaptation as at least one significant change in one’s business model, they provide results from survival and multiple variate analyses (Cox). In their study, These authors report that firms adapting their initial business model (i.e., pivoting or an adaptive strategy) experience higher survival versus non-adapting firms (P=0.0892, Log-Rank test; P=0.0636, Wilcoxon test) over the 15 mo. analysis period [34,68]. However, they report that this benefit does not apply to all firms. Further analysis reveals that survival benefits vary with types of business. Adaptation benefits less mature, capital-intensive, and high-velocity industries versus more mature, stable industries [68]. Also, it benefits business units of established firms more favorably than in independent firms [68].

Nilsen and Ramm [77] report negative findings from a survey of 47 Norwegian high-tech startups in their thesis. Their survey includes information around the knowledge and use of LS and the company. With the firm-specific data, they calculate a success score based on several questions clustered to define this variable. They report that the respondents are knowledgeable about the methodology. However, these authors do not see the translation from knowledge to practice to success. First, the analysis finds no significant correlation (Pierson’s r) between knowledge and use of LS (r=0.093, p=0.535) (Nilsen and Ramm, 2015). More significantly, the analysis fails to identify a correlation between the use of lean and the success score (r=0.091, p=0.542) [77].

In examining these studies, several issues do appear. First, these reflect a limited sample of the experience. Second, this mix reflects a variety of methodologies, endpoints, industries, firm types, and results, depending on the researcher’s lens. Third, five Camuffo [76] Ghezzi [1,18], Ladd et al. and Nilsen and Ramm [77] directly evaluate LS, whereas two Andries and Debackere [68] Eesley and Wu [35] examine adaptation, which emulates LSpractices of iteration and pivoting. Fourth, four studies make a comparison, with three Camuffo [76] Ghezzi [68] Ladd [20] using LS as one of the groups. Fifth, two Camuffo [76] and Ladd [20] utilize a sample of over one hundred groups. Finally, one Camuffo [76] utilizes rigorous methods.

Interestingly, it is the work of Camuffo et al. [76] that stands outs. It highlights the importance of rigorous use of the “scientific approach.” Further, the study indicates that those startups that use LS rigorously can discard poor ideas early (dropouts), pivot to new ideas (pivots), and reach a successful outcome (revenue) earlier. Other academics, such as Felin et al. [40], laud this study as a positive example of using LS. However, while this study provides valuable peer-review evidence concerning the influence of LS on outcomes, it is limited due to the business sectors studied (e.g., Internet, furniture, retail), outcomes identified (dropout, pivot, time to revenue), and timeframe (one year). Accordingly, it set the path for further research to examine a broader spectrum of startups, outcomes (e.g., sustained revenues, growth, market share, venture investment, viability).

What Learnings Should Practitioners Consider when Utilizing this Approach?

This final question delves into the relevance of the observations and considerations that prior sections in this paper raise. Most importantly, it highlights the need to translate such learnings to practice for entrepreneurs (along with their mentors and teachers) to consider when using LS methods.

Education and implementation

The first learning relates to education and implementation. First, entrepreneurs need to build a strong foundation. This base includes a thorough understanding of LS principles,canvases, and customer discovery, interviewing, experimentation, and inventory accounting skills. Such skills are necessary for proper implementation. However, understanding the concepts might not be enough. Work with Norwegian tech startups indicates that knowledge of LS does not correlate with its actual use (r=0.093, p=0.535, Pierson’s r), creating a need to bridge this gap [77].

Camuffo et al. [76] underscore the need for this rigorous training to build a strong foundation. This group also emphasizes the need for rigorous implementation using the scientific approach [76]. However,it is not just the scientific approach but the consistent use of all the essential LS practices, not just individual pieces on an ad hoc basis period [76]. The entrepreneur needs to complete one’s assumptions and updates in the business model canvas (and value proposition canvas) correctly during one’s conduct of customer discovery and other business model development experiments. For example, a common misconception that entrepreneurs misinterpret is with the use of channels not as distribution channels but rather as media channels. Another involves including partners when one thinking about distribution channels.

Also, individuals need to understand how to define hypothesis statements, design appropriate experiments (including large enough samples), utilize proper metrics, and interpret (and apply) results. The observations by Ladd et al. [20] reinforces this point, both related to the positive outcomes (proper hypothesis testing translates to success) and negative findings (no difference in pitch results, which may be due to non-rigorous use of LS in the experimental group).

Further, one needs to engage in customer discovery properly and conduct experiments to address set hypotheses, with unaided feedback, rather than framing customer or market responses by sharing the completed product, rather than an MVP or no product at all. Such efforts will support an objective assessment of the customer need, response, and market, to be able to reduce uncertainly effectively.

Control for influences

The second learning considers accounting for both internal and external factors. Such might confound outcomes or influence the understanding and implementation of LS concepts. Considering internal factors, the entrepreneur needs to control for one’s cognitive biases in conducting customer discovery and experiments [68] (Chen et al. 2015; York and Danes 2015). Also, one needs to be aware of both entrepreneur and customer interpretations of questions and responses from interviews and in the design and results of experiments [33,41]. Controlling for the various influences on LS use andimplementation.

For external factors, one needs to account for cultural considerationswith MVPs use and customer interviews [45,46,54]. Entrepreneurs need to ensure they are selecting a business that is in a dynamic market that has a defined technology, not necessarily an application-or service-based business versus those with a base technology or product that has substantial IP, regulatory, or capital requirements [62,68]. This consideration is essential since entrepreneurs in these business sectors have multiple risks beyond the resolution of customer and market considerations, translating as P/MF.

Use in an appropriate business sector or use other mitigations strategies in conjunction with ls

The third learning involves ensuring thatone is using LS in the correct business space. It makes sense in terms of engaging the market quickly by running a rapid experiment to achieve P/MF, as in software and applicationsbased businesses to reduce market uncertainties.

Harms et al. [62] underscore this need in their paper examining LS and materials ventures. The primary focus of LS’s actions is to reduce market uncertainty. However, if a firm is venturing in a more complex business sector (e.g., biotechnology, chemicals, materials ventures, pharmaceutical), it needs to address other risks. Such uncertainties to mitigation include those involving technologic, legal/regulatory, financial, implementation/ operational, and time to market risks. Recognizing the need to manage these other uncertainties in such sectors is essential, even if the entrepreneur use LS to mitigate market uncertainties. Hence, one must recognize the need to address these risks and utilize appropriate strategies to mitigate them.

Furthermore, one needs to consider the appropriate strategy for moving forward the entrepreneurial venture based on the business sector, business model, competition (and competitive advantage), and relationship to a sector’s value chain. One helpful strategy tool is the entrepreneurial strategy compass, a framework that consists of several strategies for startups. One needs to consider whether the firm is (1) engaging a market rapidly, (2) fit in within a value chain, (3) develop a new value chain, and (4) have an IP approach. Another helpful strategy includes a stage-gate system for evaluating the ability to achieve technological or regulatory milestones in sectors involving significant technological, regulatory, financial investment, or time-based risks [62]. Furthermore, there is value in using a business plan in sectors (or points in development) where there are limited market (or technological) uncertainties, or that the entrepreneur has them resolved [62].

Focus on what investors seek

The fourth considers the focus on what the investorsseek. While LS centers on addressing customer needs, testing products, and validating a repeatable business model, it does miss a few vital elements that investors may seek. Kressel and Winarski [59] emphasize several critical success parameters that LS might not adequately address. The first involves a substantial market opportunity with swift growth potential. Another is an outstanding team that can implement. The next considers a differentiated technologic or business solution (i.e., competitive advantage) that exceeds what is offered by competitors. The final piece points to a value proposition that delineates the firm’s value, strategy, and implementation/operational efforts.

Remember to focus on meaningful outcomes

The final area relates to being oriented to economic performance and critical commercial milestones. LS tends to focus on interim metrics that tie in with the experimentation process [29,33]. While this practice is valuable, entrepreneurs need to consider the actual commercial outcomes tied with new venture performance rather than interim metrics as leading indicators. That means the entrepreneurs need to set milestones that include customer acquisition, customer trial and acquisition, confirmation of the business model, revenue, time to revenue, profitability, growth, and not just winning pitch competitions and gaining investor resources. Work by Camuffo [76] and Ghezzi et al. [1] provide excellent examples of meaningful endpoints to which the entrepreneur should incorporate into one’s dashboard.

Conclusion

This paper examines three relevant questions concerning the lean startup methodology. These focus on the (1) the limitations of this methodology and its use as an entrepreneurial strategy; (2) the outcomes associated with using this approach are; and (3) the learnings practitioners need to consider when utilizing this approach. To this end, there are several conclusions to draw.

First, this analysis finds that limitations do exist. Observations that emanate from case studies, consultants, practice pieces, and scholars raise concerns with several essential components. These issues relate to not only elements within the framework, such as customer discovery, experimentation, and a minimum viable product, but also the proper use of these pieces in practice. Furthermore, some of the considerations related to proper implementation might be related to cultural considerations. To this end, entrepreneurs and educators should critically examine LS methodology, consider the business space in which it adds the most value, and take vigilance to ensure that entrepreneurs are rigorously employing this approach.

Second, concerning outcomes associated with the use of LS, the literature is not equivocal. This finding is due to the diverse methods, populations, endpoints, and business sectors. These also reflect a mix of anecdotal and a limited number of peer-review studies. However, work by Camuffo et al. offers a glimpse of the potential of LS and its use. It highlights that rigorous educational and coaching efforts by academics and mentors and strict implementation by entrepreneurscan lead to significant differences in discarding poor ideas, the number of pivots, and the realization of revenue (and earlier). Still, more work is needed to see whether such observations apply to business sectors beyond that studied (furniture, Internet, and retail) and to longer-term, more sustainable endpoints.

Third, this discussion, based on the evaluation of evidence from the first two questions, offers several practical learnings for entrepreneurs (and their mentors and teachers)to consider when deciding to utilize this entrepreneurial approach. These include (1) education and implementation, (2) consideration of internal and external influences, (3) application and use within appropriate business sectors; (4)focus on what investors seek; and (5) focus on meaningful outcomes. Practitioners should consider some of the recommendations offered when utilizing this methodology to optimize their experience and outcomes from this method. To this end, entrepreneurs, educators, and mentors should consider the recommendations offered to ensure the optimal use of this methodology in practice and educational programs.

In closing, there is a significant need for further research. Quantitative work, which employs rigorous controls, may address the outcomes question to offer greater clarity and dissect the impact of the methodology versus that of its implementation (along with associated influences). Qualitative work can help to dissect the underlying factors that influence the methodology use and contextual factors within and outside of the startup. Such work will help to define the real impact of LS on startup success more clearly. More importantly, these findings will aid educators and mentors to help entrepreneurs understand and implement LS, along with other appropriate strategies, to enhance their abilities to achieve positive long-term, sustainable outcomes.

References

1. Ghezzi A, Cavallaro A, Rangone A and Balocco R. “A Comparative Study on the Impact of Business Model Design and Lean Startup Approach versus Traditional Business Plan on Mobile Startups Performance.” Proceedings of the 17th International Conference on Enterprise Information Systems 3 (2015): 196-203.
2. Trimi S and Berbegal-Mirabent J. “Business model innovation in entrepreneurship.” Int Entrepren & Manag J 8(2012): 449-465.
3. Andreessen M. “On product/market fit for startups.” (2015).
4. Schwabel D. “Why Companies Need to Create an Entrepreneurial Culture.” (2017).
5. Venture Well. I-Corps™. (2019).
6. Nnakwe CC, Cooch N and Huang-Saad A. “Investing in Academic Technology Innovation and Entrepreneurship: Moving Beyond Research Funding Through the NSF I-CORPS™ Program.” Technol Innov 19(2018): 773-786.
7. National I-Corps™ grants. “NSF I-Corps Sites Program (I-Corps Sites) Program Solicitation NSF 14-547.” (2014).
8. Blank S. “Why the Lean Start-Up Changes Everything?.” Har Bus Rev 91 (2013): 65-72.
9. Lashinsky A. “Five Big Companies Practicing the Startup Way.” (2018).
10. Roush W. “Eric Ries, the face of the Lean Startup movement, on how a once-insane idea went mainstream.” (2011).
11. Eisenmann T, Ries E and Dillard S. “Hypothesis-driven entrepreneurship: the lean startup.” Har Bus School Background Note 812-095(2011): 1-23.
12. Frederiksen DL and Brem A. “How do entrepreneurs think they create value? A scientific reflection of Eric Ries Lean Startup approach.” Int Entrep Manag J 13(2017): 169.
13. Krafcik JF. “Triumph of the Lean Production System.” Sloan Manag Rev30(1988): 41.
14. Ohno T. “Toyota Production System: Beyond Large-Scale Production.” Productivity Press Cambridge Mass (1988).
15. Alvarez SA and Barney JB. “Discovery and creation, alternative theories of entrepreneurial action.” Strateg Entrepren J 1(2017): 1-26.
16. Baker T and Nelson RE. “Creating something from nothing: resource construction through entrepreneurial bricolage.” Admin Sci Quart 50 (2005): 329-366.
17. Fisher G. “Effectuation, causation, and bricolage: a behavioral comparison of emerging theories in entrepreneurship research.”Entrepren Theor & Prac 36 (2012): 1019-1051.
18. Ghezzi A. “Digital Startups and the Adoption and Implementation of Lean Startup Approaches: Effectuation, Bricolage, and Opportunity Creation in Practice.”Technol Forecast Soc Change 146 (2018): 945-960.
19. Kerr WR, Nanda R and Rhodes-Kropf M. “Entrepreneurship as Experimentation.” J Econ Persp 28(2014): 25-48.
20. Ladd E. “Routines of New Venture Conceptualization: Evidence and Extension of an Entrepreneurial Dynamic Capability.” Electronic Thesis (2015).
21. Rappa M. “Business Models on the Web: Managing the digital enterprise.” (2001).
22. Read S and Sarasvathy DS. “Knowing what to do and doing what you know: effectuation as a form of entrepreneurial expertise.” J Priv Equity 9(2005): 45-62.
23. Sarasvathy SD. “Causation and effectuation: toward a theoretical shift from economic inevitability to entrepreneurial contingency.” Acad Manag Rev 26(2001): 243-263.
24. Shane S and Venkataraman S. “The promise of entrepreneurship as a field of research.” Acad Manag Rev 25(2000): 217-226.
25. Teece DJ, Pisano G and Shuen A. “Dynamic Capabilities and Strategic Management.” Strateg Manag J 18(1997): 509-533.
26. Yang X, Sun SL and Zhao X. “Search and execution: Examining the entrepreneurial cognitions behind the lean startup model.” Small Bus Econ 52 (2019): 667-669.
27. Zott C and Amit R. “Business model design and the performance of entrepreneurial firms.” Organiz Sci 18 (2007): 181-199.
28. Blank S. “The Four Steps to the Epiphany-Successful Strategies for Products that Win.” K&S Ranch (2005).
29. Ries E. “The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses.” Crown Business New York (2011).
30. Christensen CM, Hall T, Dillon K and Duncan DS. “Know your customers jobs to be done.” Har Bus Rev 94 (2016): 54-62.
31. Moogk DR. “Minimum viable product and the importance of experimentation in technology startups.” Technol Innov Manag Rev 2 (2012): 23-26.
32. Ng G. “Five Pitfalls of Running Lean Startup Experiments.” (2014).
33. Croll A and Yoskovitz B. “Lean Analytics: Use Data to Build a Better Startup Faster.” O’Reilly Media (2013).
34. Bortolini RF, Nogueira CM, Danilevicz A, de MF and Ghezzi A. “Lean Startup: a comprehensive historical review.” Manag Dec (2018).
35. Eesley C and Wu Lynn. “For Startups, Adaptability and Mentor Network Diversity can be Pivotal: Evidence from a Randomized Experiment on a MOOC Platform.”MIS Quarterly Forthcoming (2019).
36. Osterwalder A, Pigneur Y and Clark T. “Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers.” Wiley Hoboken NJ (2010).
37. Osterwalder A, Pigneur Y, Bernarda G, Smith A and Papdakos T. “Value Proposition Design.” Wiley Hoboken NJ (2015).
38. Maurya A. “Running lean: Iterate from plan A to a plan that works.” O’Reilly Media (2012).
39. Mullen S. “An Introduction to Lean Canvas.” (2016).
40. Felin T, Gambardella A, Stern S and Zenger T. “Lean startup and the business model: experimentation revisited.” LRP 53(2019): 101889.
41. Chen T, Simon M, Kim J and Poploskie B. “Out of the Building, Into the Fire: An Analysis of Cognitive Biases During Entrepreneurial Interviews.” New Eng J Entrepren 18(2015): 59-76.
42. York JL, Danes JE. “Customer Development, Innovation, and Decision-Making Biases in the Lean Startup.” J Small Bus Strat 24(2014): 21-39.
43. Dictionary.com. (2020).
44. Kazmi S. “Managing Common Financial Risks Green PEJ (Ed.). Enterprise risk management: a common framework for the entire organization.” Elsevier, Butterworth-Heinemann Amsterdam, Netherlands (2016): 151-159.
45. Nirwan MD and Dhewanto W. “Barriers to Implementing the Lean Startup Methodology in Indonesia Case Study of B2B Startup.” Soc & Behav Sci 169(2015): 23-30.
46. Chassagne GCE. (2015) “An exploration of the lean startup approach among Brazilian digital startups.” Thesis presented to São Paulo School of Business Administration of Fundação Getulio Vargas, as a requirement to obtain the title of Master in International Management (MPGI) (2015).
47. Gustafsson A, Qvillberg J. “Implementing Lean Startup Methodology An Evaluation.” Master’s Thesis at the Chalmers University of Technology (2012).
48. Heitmann J. “The Lean Startup. A Pragmatic View on Its Flaws and Pitfalls.”MA Thesis. The University of Twente Student Thesis (2014): 1-8.
49. LeBoeuf M. “How to Win Customers and Keep Them for Life, Revised Edition, Berkeley Publishing Group, New York, New York. (2000)
50. Warberg N and Thorup N. “Preventing challenges in Lean Startupmethodology.” Copenhagen Business School, Master’s Thesis (2015).
51. Cohn M. “My Primary Criticism of Scrum.” (2014).
52. Kortmann P. “The Problem with a Lean Startup: The Minimum Viable Product.” (2012).
53. Finneran J. “The Fat Startup: Learn the lessons of my failed Lean Startup.” (2013).
54. Rao AS. “Minimum Viable Product (MVP) for Product startup: An Indian perspective. Presented in Forum for Knowledge Sharing IX Annual Conference.” Conference Theme: Technology: Corporate and Social Dimensions, Venue-National Institute of Advanced Studies (NIAS), Bangalore (2014).
55. Popper KR. Objective Knowledge: An Evolutionary Approach. Clarendon Press, Oxford, UK (1967).
56. Shafer H. Innovation Games: A Lean Startup Secret Weapon Presentation.” Innovation Games (2014).
57. Vlaskovits P. “Lean Startup: Pivot, Persevere, or Portfolio?.” (2019).
58. Burgstone J. “What’s Wrong with the Lean Start-up.” (2012).
59. Kressel H and Winarsky N. “Why the Fail-Fast Approach Is Not Right for Breakthrough Ventures.” Har Bus Rev (2015).
60. Lalic M, Calopa MK, Horvat J. “Lean startup in Croatia-does the Croatian startup scene develop effective enterprises?.”Tem Jl 1(2012): 200.
61. Ladd T. “The Limits of the Lean Startup Method.” Har Bus Rev (2016).
62. Harms R, Marinakis Y and Walsh ST. “Lean startup for materials ventures and other science-based ventures: under what conditions are it useful?.”Translat Mat Res 2(2015): 035001
63. Chesbrough H, Apruzzese S and de Olano Mata M. “Telefonica: A lean elephant.” Har Bus Rev (2016)
64. Furr N and Dyer P. “The Innovator’s Method: Bringing the Lean Start-up into Your Organization.” Har Bus Rev Press, Boston, MA (2014).
65. Nelson C. “The diesel engine MVP.” The Lean Startup Conf, San Francisco (2014).
66. Owens T and Obie F. “The Lean Enterprise: How Corporations Can Innovate Like Startups.” Wiley Hoboken NJ (2014).
67. Kuhn K. “How HP shipped faster much faster.” The Lean Startup Conf (2014)
68. Andries P and Debackere K. “Adaptation and Performance in New Businesses: Understanding the Moderating Effects of Independence and Industry.” Small Bus Econ 29(2007): 81-99.
69. Mehta SS. “Commercializing Successful Biomedical Technologies: Basic Principles for the Development of Drugs, Diagnostics, and Devices.” Cambridge University Press, Cambridge, England (2011).
70. Mullen R. “Cost to Develop New Pharmaceutical Drug Now Exceeds $2.5B. Scientific American (2014).
71. Vedd RR, Fountaine HD, Liu D and Wu A. “FDA drug approval and its relation to a pharmaceutical company’s stock price.” J Finan Account 24(2019): 1-42.
72. Kawakami T, MacLachlan DL and String fellow A. “New Venture Performance in China, Japan, and the United States: The Impact of Formalized Market Information Processes.” J Prod Innov Manag 29(2012): 275-287.
73. Moorman C. “Organizational market information processes: cultural antecedents and new product outcomes.” J Mark Res 32(1995): 318-335.
74. Power B. “How GE Applies Lean Startup Practices.” Har Bus Rev (2014): 13-15.
75. Marmer M, Hermrmann BL and Berman R. “Startup Genome Report 01, A new framework for understanding why startups succeed.” (2011).
76. Camuffo A, Cordova A, Gambardella A and Spina C. “A Scientific Approach to Entrepreneurial Decision Making: Evidence from a Randomized Control Trial.” Manag Sci 66(2019): 503-1004
77. Nilsen GT and Ramm NA. “Lean Startup: A success factor? A quantitative study of how the use of the Lean Startup framework affects the success of Norwegian high-tech startups.” MSc Thesis in Innovation and Entrepreneurship University of Oslo (2015).