Skip to main content
Contact
Back to blog

The Innovation Stack

Larry Schmitt

Innovation Partner

“Where is the Life we have lost in living?
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in the information?”

– The Rock by T.S. Eliot

Highlights

  • The two dominant models of innovation are the process/activity model and the human/proficiency model. A third model – the knowledge/flow model – offers an alternative perspective and approach to designing and building an innovation system.
  • The ‘Innovation Stack’ reveals the knowledge/flow model and provides a way to understand the fundamental ‘Jobs to be Done’ by a next generation innovation system
  • The innovation stack is analogous to a ‘full-stack’ in software. Software stacks are used to develop complex software applications.
  • Using the stack analogy, we can identify the significant knowledge transformation components of innovation to see where we can apply new technologies that augment the human capacity for innovation.

Examine a diagram of a corporate innovation system and what you typically see are a series of boxes or diamonds organized in phases with arrows connecting the various components. There are funnels, cycles, gates, transitions, etc. that depict the various components and how the objects of innovation (ideas, concepts, opportunities, etc.) flow through the system. These types of diagrams depict a structural perspective of an innovation system.

Examine a different diagram of the same system and you will see people with various proficiencies and mindsets taking on various innovation roles and engaged in activities to perform specific jobs that require certain skills. These types of diagrams depict a human perspective of an innovation system.

Both of these perspectives are useful and important, but there is a third perspective, equally important and useful, that is often overlooked, and that is the knowledge perspective. The knowledge perspective depicts the knowledge gathering, transformation, transferring and processing that happens during the act of innovating. The term ‘knowledge’, as used here, refers to a hierarchy of data-information-knowledge-understanding-insight[1] (we’ll leave out wisdom for now).

Every innovation system and every innovative activity is based on the accumulation, transformation and interpretation of knowledge within and among individuals, teams and organizations. Most of the time innovators, their teams and their organizations perform these knowledge actions tacitly and sub-consciously. They may be aided, nudged and directed by the processes, methods and tools they use, but rarely are the underlying knowledge actions made explicit.

For example, what is it that happens mentally when an innovator or innovation team forms a ‘point-of-view’ about a possible new thing? What happens when they try to convey that point of view to others? What knowledge activities are involved when discovering new needs and desires? How does an innovator mentally project a potential new innovation into an uncertain future to gain insight into how it may fit?

Whether you are searching for information using Google doing interviews and ethnographic research, designing the conceptual outline of a new thing, projecting how people could experience the new thing, building spreadsheets to analyze value potential, putting together a presentation for management, or any of a thousand other innovation activities, you are performing an act that involves knowledge of some form or another. There are literally hundreds, if not thousands, of these knowledge activities happening throughout an innovation project and they are not captured in either type of diagram depicting the process or the human perspectives mentioned above.

The Innovation Stack – A New Perspective

The Innovation Stack provides a knowledge perspective for innovation. It defines a taxonomy and framework for explicitly revealing the individual and shared knowledge processes that take place during the act of innovating. The following illustration shows the high-level components of an innovation stack.

  1. Learn: involves gathering and probing to reveal relevant information. Hearing or seeing new information solicits additional inquiries which results in additional information – and on and on.
  2. Synthesize: consists of pattern finding and model building. Patterns reveal entities and influences. Models organize these to enable systems thinking about future states.
  3. Create: includes forming new concepts and maturing concepts with additional insight and refinement. It is both thinking of a new thing and then thinking about the new thing to shape and develop it.
  4. Decide: is comprised of evaluating and selecting in the face of uncertainty. It means having the correct measure of a new thing and then choosing among alternatives in the right way.
  5. Socialize: results in understanding and telling. The tellers understanding of the new thing gets told and discussed in a way that causes both the audience and the teller to understand better.
  6. Future: means foresight and scenario building. A future perspective infuses all other components and is the implicit context in which every innovation activity is undertaken.

Futuring – A Universal Innovation Competency

Thinking about the future is unique to humans. It is also critical to innovation. The ability to create models of how the future could unfold informs every action an innovator takes. An innovator constantly tests their ideas against the futures they envision. The more explicit and plausible these futures are, the better

An innovation system calls upon the services the innovation stack provides. It is what the innovation team and individuals directly interact with. Every innovation effort is a journey of discovery. The system enables the innovator and team to do this well. The stack provides the services the innovation system needs.

These stack components interact in complex ways. There is not just a flow from bottom (data) to top (insight). Instead, each component is called upon as it is needed. For example, as an innovator and their team learn, they are simultaneously synthesizing (forming models) creating (developing new concepts) socializing (discussing with peers/execs etc.) and doing additional learning (probing and gathering) and making decisions. All of these activities interact in complex ways that are similar to the services a software stack provides – hence the term ‘Innovation Stack’.

Stacks and Stacks

An Innovation Stack conceptually organizes the knowledge gathering and transformation actions that happen while an individual or team is ‘doing’ innovation. A useful analog is the concept of a ‘full-stack’ used by software developers to create complex software systems. Software stacks are used to develop a vast array of complex software systems, from telecommunications and energy network management with millions to billions of nodes and transactions to vast social networks with billions of participants and daily interactions to Massively Multiplayer Online Games (MMOGs) with millions of players and complex worlds.

In the software world, the concept of a ‘stack’ is an organizing principle that has great utility. There are protocol stacks, application stacks, hardware/firmware stacks, game stacks etc. all of which have the common attribute of organizing a set of sophisticated services that a developer uses to create a complex, dynamic, software system (e.g. a game, an enterprise application, a retail platform etc.).

The Stack in Full-Stack

A software ‘full stack’ consists of a selected set of compatible components, some of which are listed below. A ‘full-stack’ employs tools from multiple levels to implement a complex system.

An example full-stack used by developers to create complex, multiplayer games might consist of a Linux powered Apache server, a Node.js/Express.js backend framework, MongoDB as the database, an Unreal geometry and physics engine, and Angular.js UX services. All of these are accessed by the program which is written in JavaScript.

These components perform services that are used to construct a virtually unlimited set of functions for a software program whose operation can be highly complex and non-deterministic.

The stacks used by developers to create these applications make it possible for them to tap into very powerful services developed and refined by others. Without these stacks, it would be impossible for developers to deal with the level of complexity of the system they wish to create.

The stack organizes these services hierarchically and functionally. The advantage of a stack architecture is that it provides a modularity of both functionality and interfaces. Modularity allows stack components to be implemented, enhanced and used independently of one another while at the same time being compatible.   A developer can call on the various components of the stack to perform specific operations at the appropriate time. This makes it possible to conceptualize and implement very complex, dynamic systems that would otherwise be virtually impossible to implement.

A stack does not dictate (nor constrain) the flow of data, information and knowledge through the system or the order in which things are executed. This is up to the software application that calls on the stack components it needs when it needs them. The knowledge flow is determined by the player’s interaction with the application that uses the services provided by the stack components when they are needed. There is no regimented flow from bottom to top. The flow is much more dynamic and organic based on the player’s actions.

An Innovation System is Like a MMOG

Interestingly, the software that is most like a corporate innovation system are the Massively Multi-Player On-Line Games (e.g. MMOGs like World-of-Warcraft, Overwatch, etc.). In an MMOG, players undertake journeys through complex, non-deterministic, worlds in which the world itself is affected by their, and others, actions and they must learn what works and what doesn’t. These games are arguably the leading edge of many new software technologies (including AI) and can provide insight into how the future design and construction of innovation systems may unfold.

Like an MMOG, an innovation system enables players (i.e. the individuals and teams who innovate) to undertake non-deterministic journeys through a VUCA world that is constantly growing and transforming in non-deterministic ways. These journeys take place with other independent actors, both real and artificial. These journeys involve the discovery and transformation of data and information into knowledge, insight and understanding as the players start from ignorance and work toward revelation in pursuit of a goal. The critical capabilities are learning, creating, evaluating and selecting (decision making), knowledge and insight are the means and new business success is the objective

The many traits that Innovation Systems share with MMOGs are:

  • They are very complex software systems that utilize AI to perform basic and critical functions (this is not yet true for innovation systems but it will be – see below on the future of innovation systems).
  • The player (innovator) is operating in a complex world of which they can see only a small part at a time and they are required to explore and learn about the world.
  • There are independent actors (individuals, teams, organizations) in the world. They can be partners, competitors, antagonists etc. and they can be real or artificial.
  • The player’s journey in the world is non-deterministic, depending on many factors that happen in-the-moment based on the player’s and other’s decisions.
  • Exploration and discovery are inherent and necessary. Gathering information and knowledge, transforming that into understanding and insight are critical to achieving your goals.
  • The game can be played over and over again with an unlimited set of potential journeys available to the player.

Every innovator and innovation team has thought to themselves at some point in an innovation journey, ‘Is this really going to work?’. Will it work technically? Will it work in the market? Will it work in our organization? Will it work as a business? The answers to these, and a myriad of other questions, depends on how their specific version of the innovation system (i.e. their ‘game’) is designed, built and played.

This is not to say that an MMOG and an Innovation System are equivalent. The journeys that innovators undertake happen in the exponentially more complex real world. Innovators must individually and collectively discover and interpret the current and plausible future states of this world in order to gain the knowledge and insight that drives creativity, invention and entrepreneurship.

The innovators, like the MMOG players, can be excellent, but if they are saddled with a poorly designed innovation system (the game), they will fail. Therefore, like an MMOG, designing and building a good innovation system is critical. And the way that modern MMOGs are designed and built relies heavily on the concept of a software ‘stack’.

The Innovation Stack – A Deeper Dive

Innovation faces a problem. The level of complexity in the world is increasing at an accelerating pace. The introduction of intelligent software is solving some forms of complexity, but also creating other forms of it. The creation of an innovation stack architecture is one way to address this increasing complexity. It will let developers of innovation systems tap into a variety of sophisticated and powerful services provided by intelligent software assistants.

A more in-depth depiction of an innovation stack is shown in the following table. This is certainly not the only stack that can be imagined but it is one that depicts an innovation system architecture and practice that has been successfully applied in numerous and varied corporations for nearly 20 years.

These innovation stack components identify and modularize the knowledge operations taking place in an effective innovation system. By modularizing the stack components and interfaces between them, it is possible to a) assess how a specific innovation system is supported by each of these, b) design specific stack elements to address particular knowledge operation requirements and c) define the distinctive human-AI collaboration that will be needed to augment each component.

The innovation stack is a coordinated and mutually functional organization of subsystems and services that an innovation system is built from. It consists of the language, infrastructure, libraries, API’s, etc. … that let a company build their innovation system. The elements of the stack are modular, shared, and persistent.

The innovation system created using the stack let’s an innovation team play the innovation game. The game lets innovators follow an infinite number of paths through the landscape of constantly increasing and changing data, information and knowledge. The Innovation Systems created using the stack are highly variable, customized and idiosyncratic to the organizations leadership, culture, and operating system.

In this analogy, playing the innovation ‘gameplay’ consists of identifying a nascent opportunity and taking it all the way to realization and scale. This involves exploring new domains, uncovering a multiplicity of opportunities, gathering evidence, testing and choosing among them, developing the chosen ones and eventually taking one or more all the way to realization. These new opportunities will either live or die, not just in the market, but within the organization itself.

Since innovation is all about learning, knowledge, creativity, design and testing (experimenting), the elements of the innovation stack are oriented to these tasks. Each of these tasks must operate at different levels of uncertainty and ambiguity. When a ‘game’ starts, very little information or knowledge can be available and what is available is highly ambiguous and uncertain. If the game is played properly, new knowledge and insight is developed that reduces ambiguity and uncertainty.

Other Innovation Stacks

There have been others who have employed the innovation stack metaphore. Steve Blank has written about an innovation stack that describes different levels of innovation maturity. Chris Dixon has used the ‘full-stack’ metaphor to essentially describe startup vertical integration. J.D. Meier has written about an innovation stack based on Gary Hamels work that describes a taxonomy of types of innovation. None of these, or a few other examples, is analogous to what software developers refer to as a ‘stack’ or ‘full-stack’.

Each play is highly organized by the rules and structure of the game but highly unconstrained in its gameplay. There are an unlimited number of different innovation journeys possible that depend, in this case

, on the real world circumstances and the skill, strategy and tactics of the players in navigating the world, uncovering new information, confronting situational circumstances and being creative in building their concepts.

Like a software stack used by MMOG developers, the innovation stack determines the sophistication of the innovation system you can build. Without a stack, all interactions and complexities of the game would need to be designed and built from scratch. The complexity of doing this is astronomical and would severely limit the resulting system. No modern game development company would ever consider creating a game without using a full-stack of services just as no innovation system developer should consider creating a system without a full-stack of innovation services.

The developers of this system are those within the company who create the organizations, processes, methods, tools etc. the company employs to innovate (these are often not explicitly designed but are ad-hoc assemblages of historical experiments).

Innovation Stacks and the Future of Innovation Systems

The future of innovation will be one in which humans will be assisted by intelligent software assistants or ‘cognitive collaborators’. This will happen within the next decade. It is already beginning. Innovation will be accomplished through the cognitive collaboration of humans and intelligent software. Humans alone won’t cut it. But how this transformation will play out is uncertain. There likely will not be a single event to point to. Nor will a single holistic system for cognitive collaborative innovation emerge. Instead it is likely that a suite of specialized, intelligent tools will be organized around the knowledge operations outlined above.

The modularization of the innovation stack allows developers to focus on specific, software intelligence capabilities instead of trying to design a general-purpose innovation intelligence all at once. This is indeed what is happening, although not in any organized or coherent manner.

There are many existing AI-based tools, with more being introduced every day, but it is difficult to discern what their respective strengths and weaknesses are. There is a lot of hype to go along with real progress. The accompanying chart gives an assessment of the current state of AI software for each of the stack components. Dark green shows the current state of AI software in general. Light green represents the level achieved in specific, niche areas.

Every one of these components will be affected by the digitalization and application of AI-based machine learning, deep learning, big data analytics and other cognitive augmentation technologies. The early stages of this are visible in the many new software systems being introduced to address various pieces of the innovation stack. What has not happened yet is the development of a comprehensive ‘full-stack’ from which a complete innovation system can be built.

There are some who are now saying that AI is becoming a commodity. Anyone can access deep-learning, natural language processing, pattern recognition etc. from any number of providers. The true differentiation is believed to be the access to the data available for both training (learning) and for analysis. The larger, better and more proprietary your data access, the more advantaged you will have.

This is indeed true for many current applications of AI. For the most commonly used AI systems today, the pattern recognizers, the semantic searchers, the entity identifiers, data does make a difference and AI technology is becoming a commodity. It is easy for anyone, using the AWS or Azure AI toolkit, to spin up an AI-based application that competes with the best-of-breed.

DeepMind’s latest AI system—AlphaStar—has beaten two professional players in Blizzard’s real time strategy game StarCraft II. … this milestone … is more impressive than the breakthroughs achieved by IBM’s DeepBlue and DeepMind’s AlphaGo

Unlike classic board games such as chess and go, StarCraft II embodies the complexity, random nature, and chaos associated with real world problems. Games like chess are turn-based, limited in scope, simple and fully visible on a board. StarCraft II is a full-blown war simulation requiring the player to make real time decisions on resource allocation, construction, unit control, and battle. Unlike chess players who always play on the same board, StarCraft II players face different maps each season, making their moves unlimited in scope.

Arguably, AlphaStar is the most important step forward thus far toward the goal of artificial general intelligence—the holy grail of AI.

But there are whole swaths of intelligence that are critical to innovation (and other areas) that are currently beyond the capabilities of current AI technology (at least that technology that is being commoditized) . These gaps are listed in the chart above and below:

  • Creativity/Invention
  • Model building and running
  • Future projection/Scenario building
  • Evaluation and selection
  • Storytelling

As the excerpt from a recent report from ARK Invest makes clear (see callout),

Just as AI been introduced into limited domains and situations and has slowly expanded its influence, so will the same occur in the innovation space. The ‘hand-crafted’, ramshackle, innovation system is at risk. By making these innovation cognitive activities explicit, the Innovation Stack not only provides a framework for assessing a company’s current innovation system, it can also be used to design and assess the nascent AI tools that are now becoming available to innovators.  It provides a perspective not only for current innovation systems but also for what they could evolve into as these new AI-based tools come into existence.

[1]   What has become known as the DIKW hierarchy (Data, Information, Knowledge, Wisdom) was formally articulated by Russell Ackoff in his address accepting the presidency of the International Society for General Systems Research 1989. It has been enhanced in various ways over the years as it has here with the addition of the understanding and insight layers.