Themes and Observations Themes and Observations
Organizational culture and change management
Organizational change management is a common theme across all case studies, with variations depending on the maturity of the AI integration within the company and the level of comfort with the introduction of new processes or technologies. The subject organizations collaborated with both internal and external stakeholders to ensure trust and buy-in and used various strategies to manage change.
Organizational buy-in
At Zymergen, a young company with AI/ML central to its founding mission, employee buy-in was generally assumed from the beginning. At Tata Steel Europe and Axis Bank, teams needed to approach changes in culture and practice more proactively with relevant internal stakeholders. In all three cases, executive support or sponsorship and early investment in employee training and awareness-building helped to engender trust internally and led to broader organizational buy-in. For instance, Tata Steel Europe recognized the value of bringing plant operators into the ML model development and began holding “office hours” in which they and project managers could discuss the analytics models with data scientists. Data scientists at Zymergen also found that upfront and frequent involvement of physical scientists in the machine learning model was critical to building trust.
The Critical Importance of Technical Explainability
Another important element that affected the adoption of AI within these organizations was the explainability of the AI models and techniques. For example, helping operators at Tata Steel Europe understand the early use cases and visualize the models was especially important to ensure their buy-in. Visualization and explainability also played an important role in Zymergen’s implementation of its experiment recommendation engine, as scientists wanted to understand why the model recommended certain strain improvements. Explainability was also critical to AI implementation in the Axis case, where the bank needed to be confident that the chatbot would not mislead customers, that regulatory compliance would be met, and that the chatbot would not complete fraudulent transactions.
Accordingly, transparency, explainability, and trust of AI systems played an important role in driving ML adoption and deployment at all three organizations studied, perhaps in part due to the types of problems and processes where AI was used. AI was often integrated in heuristic-driven processes where workers previously relied on years of formal education (e.g., Ph.D’s. in biology) or on-the job experience (e.g., steel plant operators), which could have created friction due to a sense of pride in acquired domain knowledge. Implementation was often most effective if there was close collaboration between developers, subject-matter experts, and AI systems.
External stakeholders
Managing change for external stakeholders required more attention for Zymergen and Axis Bank than for Tata Steel Europe. Zymergen needed to convince its customers of the value of its differentiated approach, and Axis Bank was required both to comply with domestic regulations and to drive customer adoption of its chatbot. The visibility of benefits and financial impact were key drivers for Zymergen, as illustrated by its adoption of performance-based revenue elements in contracts with clients. Meanwhile, Axis Bank stress-tested its chatbot through third-party agencies and ran extensive reviews to ensure accuracy and security to build trust with both regulators and consumers. In addition, Axis Bank ran customer campaigns to drive adoption and showcase the value of its AI Chatbot.
Impact on productivity and business results
Artificial intelligence’s impact on the broader economy and productivity has long been a topic of debate. Contemporary economists have also posed questions about the possible recurrence of a past computing technology “productivity paradox,” or the question of why the recent advancements in AI appear not to have translated into measurable productivity gains at the macroeconomic level. Although ongoing AI and broader digitization efforts have not yet produced clearly measurable productivity gains for the broader global economy For more, see “Is the Solow Paradox Back?”, McKinsey Quarterly, June 2018. , productivity improvements at the micro-level were reported to us across all three subject organizations in this case study project. While the case studies are limited to three organizations, the micro-trends reported on productivity and labor would be consistent with the previous Solow Paradox, where macroeconomic benefits emerge in the decade or two after the technology was deployed.
One hypothesis is that recent progress in machine learning will be subject to large diffusion and implementation lags before effects are seen at the level of entire economic sectors. ”Erik paradox.” Interestingly, the authors suggest that the economy could be in the midst of a broader economic transformation due to technological changes. The lags in the measurable effects of AI could be due to the time required to invest in and “build the stock” of the new technology as well as the “complementary innovations” required to enable AI, such as cloud-based computing, organizational changes, and new business models. “] The AI applications at all three subject organizations seem to have generated observable productivity gains, although their contribution to wider economic and labor effects may be harder to isolate.
Integration with business strategy
In all three cases, AI/ML-related applications were part of a broader business strategy driven by automation and digitization. The broader strategies often involved particular attention to digital enablement (e.g., a shift to cloud-based storage), organizational restructurings, or workforce training. For instance:
- Tata Steel Europe has positioned its Advanced Analytics Program as part of its ongoing broader digitization effort.
- Axis Bank launched the AI Chatbot within a broader set of customer service automation initiatives across the bank to address its growing need for improved customer service.
- Zymergen coupled a machine learning-based recommendation engine with its automated wet lab operations as a key point of differentiation.
These strategies suggest the importance of complementary innovations, both technical and organizational, to enable productivity gains observed from AI-related implementations.
Timeframe of transformations
Finally, we observed that executives and managers want quick results, be they financial benefits or operational improvements. In the Tata Steel Europe case, executives prioritized use cases with higher value potential and lower implementation complexity to realize some financial benefits early on. Similarly, Zymergen offers performance-based contracts to its clients where the contract payout depends on results demonstrated and benefits realized through the strain improvement program. We also observed that elevated executive attention led Axis Bank to release versions of its AI chatbot early on and improve its performance through ongoing iterations rather than doing more internal development and delaying its release until a later date.
Productivity
The business results stemming from AI take various forms in the three case studies. Tata Steel Europe experienced productivity gains through improved production yield, reduced raw material expenses, and enhanced product quality. TSE also increased productivity overall by delivering roughly 13 percent higher EBITDA (a common proxy for cash flow) through its AI-related initiatives, achieved with the same production staffing levels. Axis Bank reports it was able to handle its growing customer service volumes with fewer customer service agents, driven by a migration in volumes from human-enabled to automated customer service channels, implying an increase in the labor productivity of the overall customer service function. We do not have a measure of hours worked to estimate the increase in labor productivity precisely. Lastly, in the Zymergen case, management reported higher labor productivity, driven by a high degree of automation in the wet lab (reportedly allowing 10 times higher experimentation throughput) and accelerated project durations (eight to ten years was reduced to three to five years) compared with conventional R&D labs.
Return on investment
Case subjects also reported that AI/ML-related applications generally offered and generated a higher return on investment (ROI) compared to other more capital-intensive alternatives. As an example, Tata Steel Europe reported higher ROI in the first year of AI implementation compared to regular capacity investments, such as an investment in new plant equipment. Similarly, the payback period for a particular piece of plant equipment is four to six years, whereas Advanced Analytics use cases at Tata Steel Europe usually break even in one to two years. Zymergen offered accelerated strain improvement processes to its customers that may allow them to postpone or forgo large capital investments in downstream operations (e.g., fermentation manufacturing facilities). This evidence suggests that adoption of AI-related technologies could implicate broader trends of capital efficiency or contribute to shifts in organizations’ investment patterns.
Impact on labor and the workforce
Debate around the consequences and implications of AI-related technologies is often closely related to its potential impact on the workforce. Many say that broader adoption and advancement of AI will lead to profound workforce changes, suggesting that productivity gains from AI-related technologies may manifest themselves through labor reductions and growing inequality. Some have argued that inequality could increase with the proliferation of AI in the long term. While we do not address this question, please see Joseph Stiglitz and Anton Korinek’s paper for more: “Artificial Intelligence and Its Implications for Income Distribution and Unemployment,” December 2017. In this limited sample set of case studies, we examined whether productivity gains from AI-related integrations came at the expense of jobs, or whether the impact was more nuanced. We also explored what types of workforce skills may be required to adopt AI-related technologies and the challenges associated with the introduction of new technologies.
In the three cases, we observed varying degrees and forms of labor impacts, both direct and indirect, demonstrating that labor effects are often multi-layered and can extend beyond the core organization implementing AI. The case studies also demonstrated that organizational and societal contexts play a role in how AI-related initiatives impact labor. We noted distinct labor characteristics and dynamics in each case depending on how AI was integrated within the company’s operations. For instance, Tata Steel Europe operates in a mature and highly unionized industry with a relatively inflexible labor market. At Axis Bank, the impact of AI on labor was often indirect, as the majority of its customer service operations were outsourced and thus were not conducted by direct employees of the bank. Zymergen started out as an AI-native company and communicated the importance of automation to its workforce from the very beginning through its mission statement – though its business model may have downstream impact on its customers’ R&D teams, shifting to new types of work or potentially leading to lower hiring rates in the future. As these cases demonstrate, the social and economic impact of AI and other automating technologies goes beyond the immediate sites of implementation and extends externally across supply chains, partners, and customers.
Workforce reductions
While no direct AI-related workforce reductions were observed at Zymergen or Tata Steel Europe in the short-term, we did observe direct workforce reductions in one case. It is not clear what the net-impact of AI on jobs will be in the near future. The McKinsey Global Institute estimates that “total full-time-equivalent-employment demand might remain flat, or even that there could be a slightly negative net impact on jobs by 2030,” yet demand for new types of jobs may increase, as seen with the advent of the personal computer in the late 20th century. Axis Bank specifically aimed to automate its human-enabled customer service offerings and reduce the size of its outsourced customer service team. However, Axis pointed out that while the size of the customer service team may be further reduced, the team is unlikely to be completely eliminated because of the technical challenges and operational complexity posed by multiple languages in India. Similarly, while Tata Steel Europe’s workforce size has not changed as a result of the Advanced Analytics program, it is likely to change in coming years. With an aging workforce and maturing AI-related technologies in production, voluntary attrition of the production crew reaching retirement age in the coming five to fifteen years could potentially lower the overall size of the production workforce, even without direct displacement of particular workers.
Workforce composition
A common theme across all three cases is the changing composition of the workforce and the addition of new skill-sets and profiles to the organizations (e.g., data scientists, automation engineers). Zymergen’s core team This only includes scientists and research associates and does not account for data scientists, automation engineers, and lab technicians that support teams with their services. of researchers and scientists is 50 percent smaller than that of a conventional R&D lab. Zymergen, however, has a higher number of support staff partially dedicated to each team, compared to a conventional R&D lab. Support staff are composed of data scientists, software engineers, automation engineers (about 25 percent of Zymergen’s workforce) or lab technicians and personnel to run the wet lab (about 8 percent of Zymergen’s workforce). At Tata Steel Europe, upon initiation of the Advanced Analytics Program, the company built a team of data scientists and engineers through internal retraining. Although not as significant, the implementation of the AI chatbot led to re-tasking of key IT personnel at Axis Bank as well.
Workforce hiring
Shifts in the composition of a workforce also trigger changes in companies’ hiring needs, increasing the demand for hybrid profiles with interdisciplinary expertise (e.g., data science and steel or biology). Zymergen’s target hiring profile is different than that of a conventional R&D lab; for example, entry-level research associates are expected to have experience in biology and have data science capabilities, such as proficiency in Python or R. Similarly, Tata Steel Europe reminds its employees that it is in the business of making the best steel, not developing the best AI models. Hence, it looks for internal talent with a deep understanding of the steelmaking process who can be trained in data science skills.
Workforce ramifications outside case organizations
As noted, labor implications are rarely limited to the core subject organization, and impacts usually cascade externally to customers and vendors. Axis Bank’s implementation of its chatbot resulted in temporary business gains for third-party vendors such as technology developers and certification agencies, while leading to a reduction in the number of personnel assigned to its account at its third-party customer service provider. Zymergen’s strain improvement programs could be considered akin to an outsourced R&D lab, which, if proven successful, could displace a customer’s internal R&D workforce or create lower hiring demand for scientists and downstream manufacturing labor.
Technical implementation overview
Processes being replaced by AI
The technical advantages of AI/ML technologies allowed all three organizations to replace or enhance conventional processes that had been performed less effectively or efficiently by humans in the past. In particular, AI-related technologies helped these companies process high volumes of data at a faster pace to uncover patterns more reliably, more quickly, or more cost effectively than a human could. For instance, Zymergen leveraged AI in order to better understand the complexity of the microbial genome during fermentation. Tata Steel Europe analyzed thousands of variables throughout its production processes to optimize the material recipes that determine the chemical properties of steel. Axis Bank implemented an AI chatbot capable of handling thousands of customer inquiries simultaneously across multiple banking products with 24/7 availability.
Data strategy and infrastructure
Subject organizations had differing approaches to generating their training data for the core machine learning systems. Zymergen took a clean slate approach Zymergen is an “AI-native” company that was founded in 2013. As such, the company started its data storage in the cloud. All data infrastructure could be built with a clean slate and modern toolchains, making data exportation and analysis on cloud systems easier than it might be for an incumbent (such as Tata Steel Europe). The latter might be dependent on proprietary or embedded on-premise systems that were installed without these objectives in mind. to its infrastructure by investing early on in standardizing and accelerating the speed of its data collection processes at the wet lab. This created high total data acquisition costs, including high upfront investment costs and higher ongoing experiment costs (e.g., raw materials) because they ran more experiments as part of their learning process. Tata Steel Europe had to significantly improve its data infrastructure and systems to consolidate data in the cloud and improve overall data quality. Axis Bank has also put significant effort into generating its data, for instance assembling the chatbot’s training data manually, because open conversational NLP Natural Language Processing, a popular subfield of AI datasets are not applicable to banking domains, and in any case need to be customized for an organization’s products and business practices. Today, Axis Bank faces ongoing challenges relating to the limited standardization and varying quality of the data generated through new chat conversations from users. To address this challenge, the Digital Banking team at Axis Bank has conducted manual reviews of the chatbot during each release cycle, and it has also received a weekly list of questions the chatbot couldn’t answer. These approaches by subject organizations illustrate the costs of building and maintaining conversational AI systems over time.
A related issue: More sophisticated models and techniques did not always generate better results at subject organizations, since they often required much larger data sets to be effective. This proved to be a challenge for all three organizations, which are relatively new in their experiences with AI/ML applications. More complex models were also more difficult to explain to the end-users (e.g., scientists, plant operators) because of the “black-box” phenomenon whereby the reasons for models’ recommendations weren’t fully transparent.