Innovation is not a fixed-rate process and can be influenced by our ability to exploit the existing stock of knowledge, especially if that knowledge translates into broader distributions of available product complexity.

However, before we build products we need to assess existing knowledge. If we are lucky enough to have already the right data we can use data analytics for that purpose. Modern tools for data mining, management, retrieval and processing allow for fairly complex, real-time analytics on vast amounts of data.

For all that data to be useful we need to process it in a way that provides us with business insights on which we can act. Before we embark on any big data adventure it is vital to formulate meaningful problem statements. Fortunately, once we know exactly what we want, advances in cost-effective algorithms make the data analysis a trivial exercise that can be performed on a laptop or using a cloud based service, provided the company has employees with appropriate data science skills.

Simple. Yet, the majority of firms don’t use data analytics in their decision-making process, and those that do, not necessarily profit from it.

A recent study revealed that 59% of firms fail to use advanced analytics, despite possessing the data to reap benefits from analytics, and even when analytics is used, there are substantial variations in the return on using analytics.

The outcomes of the data analysis exercise should augment our decision-making. Data analytics outcomes should therefore be linked to company’s organizational structure.

The paper by Wu, Lou and Hitt [2] describes the how the way organizations function, the decentralization of innovators across the company structure, affects the analytics-innovation relationship.

Measures of firm innovation aside, the authors develop a measure of decentralization of innovation within the company “by noting the relationship between named inventors on a firm’s patent“. At first glance this is similar to the work of Mori and Sakaguchi on collaborative knowledge creation [1].

Mori and Sakaguchi note that the amount of “differentiated knowledge” in a collaboration influences the average collaborative output. Wu et al., on the other hand, find that data analytics can offset informational asymmetry and support innovation in decentralized structures. They acknowledge however that “broad participation of inventors with different skills, perspectives, and expertise” is a bonus. The outcomes of data analytics could be perhaps thought of as “differentiated knowledge”.

Analytics can mitigate these disadvantages to some extent by automatically aggregating and analyzing information to uncover trends and common patterns among innovations from divergent sources, and thereby facilitate the transfer and use of knowledge across organizational boundaries (including within and between the firms). Through the discovery process, analytics can also expand the set of problems that existing technologies can address by altering scientific communities’ conceptual framing of problems.

(Sadly, the effects of decentralized structure, access to differentiated knowledge, and improved information management have not been disentangled in Wu’s et al. paper.)

Overall the effect of data analytics on innovation in publicly traded firms was found to be small. Data analytics demand was found to be positively, albeit only weakly, correlated with decentralized innovation structure while firms have shown some increase in productivity when data-analytics skills were available in the company. Data analytics complementarity to decentralized innovation was found primarily in the facilitation of combination of existing technologies which are new to the overall market and those new to the firm.

All these results add to the “productivity paradox”. Technological improvements are in general expected to foster innovation and increase productivity. However, in many instances this is not so simple. In the past, advances in information technology did not translate directly into high productivity gains. Similarly, advances in data analytics seem also not add much to the productivity.

[1] Tomoya Mori, Shosei Sakaguchi, Collaborative knowledge creation: evidence from Japanese patent data, Aug 6 2019, arXiv:1908.01256v1

[2] Lynn Wu, Bowen Lou, Lorin Hitt, Data analytics supports decentralized innovation, Management Science, (2019) forthcoming.

Abstract. Data-analytics technology can accelerate the innovation process by enabling existing knowledge to be identified, accessed, combined, and deployed to address new problem domains. However, like prior advances in information technology, the ability of firms to exploit these opportunities depends on a variety of complementary human capital and organizational capabilities. We focus on whether analytics is more valuable in firms where innovation within a firm has decentralized groups of inventors or centralized ones. Our analysis draws on prior work measuring firm-analytics capability using detailed employee-level data and matches these data to metrics on intrafirm inventor networks that reveal whether a firm’s innovation structure is centralized or decentralized. In a panel of 1,864 publicly traded firms from the years 1988–2013, we find that firms with a decentralized innovation structure have a greater demand for analytics skills and receive greater productivity benefits from their analytics capabilities, consistent with a complementarity between analytics and decentralized innovation. We also find that analytics helps decentralized structures to create new combinations and reuse of existing technologies, consistent with the ability of analytics to link knowledge across diverse domains and to integrate external knowledge into the firm. Furthermore, the effect primarily comes from the analytics capabilities of the noninventor employees as opposed to inventors themselves. These results show that the benefit of analytics on innovation depends on existing organizational structures. Similar to the IT–productivity paradox, these results can help explain a contemporary analytics–innovation paradox—the apparent slowdown in innovation despite the recent increase in analytics investments.