Information Systems

9 – 1 1 8 – 0 1 6

R E V : J U L Y 3 1 , 2 0 1 8

Professor Srikant M. Datar and Research Associate Caitlin N. Bowler prepared this case. It was reviewed and approved before publication by a company designate. Funding for the development of this case was provided by Harvard Business School and not by the company. The citation review for this case has not yet been completed. HBS cases are developed solely as the basis for class discussion. Cases are not intended to serve as endorsements, sources of primary data, or illustrations of effective or ineffective management. Copyright © 2017, 2018 President and Fellows of Harvard College. To order copies or request permission to reproduce materials, call 1-800-545- 7685, write Harvard Business School Publishing, Boston, MA 02163, or go to www.hbsp.harvard.edu. This publication may not be digitized, photocopied, or otherwise reproduced, posted, or transmitted, without the permission of Harvard Business School.

S R I K A N T M . D A T A R

C A I T L I N N . B O W L E R

Data Science at Target

In May 2017, Paritosh Desai became Target’s Chief Data & Analytics Officer (CDO), thereby taking full responsibility for the continued development of its data science and analytics capabilities on which the company was banking its retail transformation. Desai joined Target.com in August 2013 as VP of Business Intelligence, Analytics & Testing (with the intent to build out a team of Data Scientists and Engineers) to explore how the retailer could use its relatively small but thriving e-commerce arm to drive sales and win customers. At that time, Target did not have a centralized data science operation. The company was awash in rich, fine-grained data that could provide any number of benefits if analyzed or harnessed in service of its customers and business units.

Starting from a coffee shop in Silicon Valley, Desai’s first move was to bring on two trusted engineers to help inventory all that Target had. Over the next four years, Desai developed Target’s existing analytics group (several hundred strong to start) by integrating data scientists into the organization. By 2017, the group had grown to a force 900 strong, including 150 data scientists, nearly 50 Ph.D’s, and individuals trained in computer science, operations research, mathematics, statistics, physics, and other quantitative disciplines. The group operated out of corporate headquarters in Minneapolis, Minnesota, as well as offices in Sunnyvale, California, Pittsburgh, Pennsylvania, and Bangalore, India. 1 For a company with 1,800 stores and over 300,000 employees, this was the beginning of a long journey, one that would build over Desai’s four years of intense learning.

Target Corporation Target was a nationally branded general merchandise retailer that sold products through more than

1,800 brick and mortar stores across the United States and, since 1999, through digital channels. In the mid-1990s the retailer differentiated itself from Wal-Mart, its primary competitor, a by developing an “upscale” image through focus on fashion and beauty, lifestyle, and entertainment. It collaborated with well-known designers, such as Michael Graves, to create small, but high profile and price-accessible lines 2 that established the retailer as having a savvy aesthetic sensibility. 3 The focus on design helped to drive consistent sales and growth; between 1996 and 2008; “Target’s revenues increased at an average annual rate of 12%, operating margins improved from 5.4% to 8.6%, and the stock returned

a Wal-Mart followed a low-price strategy.

D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

118-016 Data Science at Target

2

795%.”4 By 2008, Target was one of the world’s most admired companies in the general merchandise retail category.

Like many companies, Target struggled in the period from 2008 to 2013 following the financial crash in the fall of 2008. With no stores outside the U.S., its performance was closely tied to the U.S. economy and its positioning as a unique and middle-high retailer left it particularly vulnerable to softened consumer demand. Though Wal-Mart suffered, Target suffered more. Between May 2008 and March 2009 Wal-Mart’s stock price fell 16%, while Target’s fell by half, reflecting the challenges and headwinds these retailers would face in the next few years.5 Target had to find new ways to succeed.

Why Data Now?

Over its 90-year history, Target had developed into a particularly skilled merchandiser. It used several store formats to fulfill different strategic objectives for attracting customers. It also advertised to customers across a range of channels, from traditional print ads, direct mailers and circulars, to digital formats. Although overall sales had declined in 2013, online sales were a bright spot for the company; from 2012 to 2013 these had grown by nearly 30%. As Target.com, its e-commerce arm, continued to grow its online business, it also grew an enormous database on customers and their purchase history. For customers with Target.com accounts, it had data about on-site browsing history, returns, and non-purchase behavior, such as purchases abandoned in the middle of checkout, which contained important signals into how customers experienced Target.com.

Desai joined Target.com in August 2013 as Vice President of Business Intelligence, Analytics & Testing to transform the role analytics generated through Target.com could play for the larger Target enterprise. This was a complicated project that could have profound implications for the operations and relationships between and within the company’s different business units and functional areas, especially those between Target.com and its brick and mortar stores. Desai planned to focus on four areas:

1. Data science applications. Where were the opportunities to deploy data science for the biggest impact? There was no shortage of areas to consider. On the sales and retail side there was e- commerce, marketing and assortment, and pricing and promotions. On the operations side there was supply chain management, field operations (planning and scheduling), internet-of- things, and information security. He needed to learn what questions the business units had. Could these questions be answered with data? If so, how?

2. Business Insights (BI) & Reporting. The vision for analysts in business insights was to blend the art of retailing with data tools improved through access to the full breadth of Target’s data on a daily basis.b Improved tools could inform what strategies to pursue and surface operational bottlenecks in the complex organization.

How would data be collected, processed, analyzed and reported in a way that supported managers’ decision making? This included visualizing data and designing reports for use by BI analysts and business unit managers, building the infrastructure to deliver them, and designing the organization to effectively produce and use the system.

b Target’s data met the “four V’s” criteria that differentiated “big data” from regular data: volume, variety, velocity, veracity. Source: IBM Big Data & Analytics Hub, “The Four V's of Big Data,” IBM Website, http://www.ibmbigdatahub.com/infographic/four-vs-big-data, accessed October 2017. D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

Data Science at Target 118-016

3

3. Data architecture, management and governance. To do analytics with the volume of data Target had would require a major, multi-faceted engineering effort. This would be the fundamental building block for doing any kind of advanced data related work. This would include:

a. A period for basic exploration to understand Target’s data.

b. “Big-Data” engineering. This would include scale engineering in terms of hardware and software and require acquisition of talent.

c. Data management and governance. This would include development of standards and policies for how data would be made available reliably, like a utility, with appropriate quality and freshness and timeliness standards.

4. Experiment Design & Testing. A major purpose of building data science capacity was to cultivate a more detailed understanding of customer behavior and preferences. With data, Target managers could design offerings to match those preferences and, ideally, drive an increase in sales and repeat business. In practice, this would require a learning approach across different parts of the organization. Experimentation design and testing would be a key tool to facilitate effective learning from data insights.

The Enterprise Data, Analytics, and Business Intelligence Group Desai established the Enterprise Data, Analytics, and Business Intelligence group (EDABI) early in

his tenure. This group was tasked with developing the hardware systems and software to bring machine learning and sophisticated analytics to data science projects across Target. These systems would support the work of BI analysts who were on the front lines engaging with data as it came in, interpreting it, and working with managers to develop novel ways of using data.

He structured the Data Science & Engineering group within EDABI as an equal partnership between engineers focused on building and maintaining the hardware systems needed to store and serve petabytes of data and the data scientists who developed models and algorithms to identify patterns in the data and build predictive models. 6 One of the EDABI engineers described the unique set up this way: “Paritosh had a vision that data science and engineering be at the same level. He believed this would only work if people were working together.”

The team drew on individuals with different areas of expertise and with a range of professional experiences. Team members and their titles within one of five functional areas are listed below.

o Business Intelligence & Analytics: Mark von Oven, Vice President; Andy Miner, Senior Director; and Ben Schein Business, Senior Director;

o Testing & Measurement: Meghna Sinha, Senior Director;

o Data Science: Shalin Shah, Principal Data Scientist; Ramasubbu Venkatesh “Venky,” Senior Director, Sayon Majumdar, Director Data Science

o Data Engineering: Samir Shah, Senior Director

o Digital Experience and Product Management: Andy Feierfeil, Senior Director

EDABI could only succeed through cooperation, communication, and close coordination across these distinct disciplines. D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

118-016 Data Science at Target

4

Desai fostered a culture of entrepreneurship within EDABI. Data science at the scale he envisioned was new to Target, so experimentation would be critical to discovering where it would fit and how it could be woven into business practices. He initiated adoption of the “agile development” model when he joined in August 2013 and shepherded the approach across the entirety of the EDABI organization by early 2015. Team members lived by the mantra develop-test-measure and brought that iterative approach to experimentation to all their engagements. This included a rigorous focus on crafting testable, measurable metrics and a clear thought process about how a metric would facilitate action. Communicating this approach to partners within the business was an ongoing activity.

The data scientists on the EDABI team first set their sights on Target.com and developing web- based and mobile customer experiences that fed and utilized the personalized recommendation engines they were building.

Online digital personalization

“The days of measuring sales in units are over. We need to apply data science to get at what is under the surface. This is an attribution problem.” (Meghna Sinha, Senior Director of Testing & Measurement)

Using Target’s online data to solve the attribution question for its online customers was a priority for Desai. By understanding customers’ behavior on Target.com, the EDABI team sought to maximize touchpoints on the site and, ideally, design ways to better serve customers at those moments and thereby facilitate more purchases and increase sales.

Building a mobile experience.

The promise of data generated on web and mobile platforms assumed that customers could access and use them so easily that they would return, time after time. Only through repeated use by individual users could they collect the longitudinal data that allowed for accurate prediction and recommendations.

From his experience leading Data Science at Gap, Inc. Direct, Desai knew that users of mobile apps experienced frustration in spans of milliseconds. Therefore, the most important engineering requirement was for users to get a response from the page or app in milliseconds, consistently; they had to load quickly and seamlessly. From a user experience perspective (driven by algorithms developed by data scientists), the most important requirement was that the site help guests find what they were looking for; the experience had to be relevant and curated. For example, if a customer who searched for “sneakers,” the site needed to return a list of sneaker-like shoes to be relevant. If the customer had purchased a certain brand of sneaker in the past, the site might lead this new list with a sneaker from that brand to meet the standard of curated.

Measurement and testing.

To truly learn from the data, EDABI had to institute standards around measurement practices that, eventually, data science practitioners around the company would adopt. Defining what, exactly, constituted success was an important decision. For instance, only if a customer searched for a product and bought that product in the same session would they consider it a statistically significant purchase; this was called a “specific conversion.” The goal was to carefully measure the attribution of the feature (the purchased item), which required that they only counted those purchases that were exposed to recommendations and the purchase was one of the recommended products. If they did not exercise care, they would be unable to quantify the incremental impact of the feature. D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

Data Science at Target 118-016

5

With sharply defined metrics in place, the team could engage in extensive A/B testing to determine whether interventions on the site actually worked. They experimented with how they surfaced these recommendations (for example, on various parts of the page or varying places on the website) and also had dozens of personalization algorithms to deploy for different customers. For instance, a product manager could try a different banner design, feed that banner to two different groups of customers, and track which banner was associated with higher conversion rates, then adopt that design. A/B testing allowed them to learn what was optimal for both guests and Target.

Understanding the context in which customers were browsing the site or how they came to the site was also important, particularly if they converted. Had a customer arrived via email, google search, or a TV ad? Was the customer browsing on phone or laptop? This context knowledge added additional layers through which to evaluate their purchase (or non-purchase) behavior and, potentially, could inform what recommendations to make. As Andy Miner, Senior Director in Business Intelligence and Analytics, said, “If you don’t rely on context then predictive ability is more like Netflix home example.”

The reality was that customers’ preferences changed. Miner's description that “last week did not always inform this week,” was a sobering reality. Also, individuals did not always purchase for themselves, and therefore might exhibit dramatically different purchase behavior over weeks or months. These were challenges the team was addressing in its Journey 2.0 initiative. The vision was that clicks would have a very specific impact on the predictive model in real time, so that recommendations were unfolding in real life. (This relied on sophisticated cache-ing technology.) From an engineering perspective it was a challenging project, but Desai believed it was worth the investment to build web and mobile platforms that provided a level of relevant recommendations that would keep customers coming back again and again.

There was always the risk that recommendations could go wrong. How many times would a customer return to an app that recommended something irrelevant to the customer? Andy argued, “Experimentation is the cost of success.”

Other opportunities

Personalization was just the requisite first step that could enable a range of other possibilities down the road. With detailed customer data, Target might be able to show manufacturers what was and was not working with the products they sold. Could Target convince them to work together on product redesign based on data? Were there ways to shape the relationship between Target’s brick and mortar stores and its digital platform to change the way it used space? These were all questions to consider in the years ahead.

Bringing Data Science to the Business Units Target was a large corporation organized around business units and functional capabilities, such as

marketing, finance, and digital. Its merchandising businesses covered broad categories: Style with Apparel & Accessories, Beauty and Home Furnishings, so-called “Hard Lines” like Electronics & Entertainment, and Groceries and Household Essentials. Each unit had support from experts from the different capability groups. These experts sat with business units and contributed to managers’ decision making discussions; however, they reported to managers within their own capability areas. Target.com was another arm Target operated. There were no shortage of projects to take on. The question for Desai was where and how to deploy his team’s modest resources within such a large company where there were far more questions than his team had the capacity to answer. D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

118-016 Data Science at Target

6

Merchandising was Target’s core capability and Desai envisioned data being used in support of decision making around this key activity. With access to petabytes of data, Desai directed his team to find BI analysts and managers asking questions with the potential to generate value if answered more accurately. At first, utilizing “big data” was not required. This was a bit of a surprise to some of the team.

Desai’s team reached out to business units, each with its own BI analytics team, in search of questions. Having worked within a particular unit for a while, alongside business managers, these analysts knew the nuances of the business unit well. They had worked on the key questions of what products to promote and how to promote them over several annual cycles. Over a six-month period, the team found analysts and managers with questions and worked closely with them to fully understand them. They learned a lot.

Unclear connection. In regards to promotions, managers had to determine what products to promote, to whom, through what channels, through what discount/promotion mechanism, and by how much. c Managers reviewed reports all week that compared sales data in a variety of cuts: day-to- day, week to week, year over year. They went to their analysts with specific product-level questions. They consumed curated data constantly. Yet they did not always clearly articulate the connections between data, metrics, and business impacts.

In some cases, it was not clear that the impact a question and answer would have on the business had been fully considered. For example, given a particular promotion strategy on dish detergent, how would the promotion strategy impact incremental sales? Would customers have bought detergent anyway without the promotion? If they developed a new metric, would a manager be able to act differently based on the answer? These were very difficult questions to answer, but they had a direct impact on the business.

In other cases, it was not clear that managers’ questions were rooted in testable hypotheses. Was there a clear, logical link between how the question was quantified, the action it would enable, and measures of impact and success? If a manager got an answer, made a decision, and acted, would she be able to evaluate whether that action was more effective than the original alternative?

In almost all cases, it was not possible to know whether decisions that were good for a business unit or store would have a positive impact on Target as a whole. Past analysis could suggest that a promotion on dish detergent would increase store sales by 25%, but there was no way to quantify the effect this promotion might have in sales in other stores and hence the incremental value that decision would have for Target as a whole. There was plenty of data, but there was no clear framework in place to help managers think at an enterprise-level.

Barriers for analysts. Analysts sometimes faced structural barriers to finding answers. A seasoned BI analyst might have intimate knowledge of a business, good working relationships with managers in the unit, and motivation to answer the questions that came to him accurately and quickly. Yet finding, processing, and aggregating the specific data he needed could be challenging and time consuming. If the question never changed and a tool existed that allowed the analyst to answer the question, then the process was simple. But retail was a constantly changing environment, questions often changed, and analysts could not always find ways to answer new questions within hard stop time constraints.

c For instance, would a promotion for a particular store be advertised in a circular, an insert, or via direct mail? Would the price be reduced or would there be another mechanism? Would the promotion apply to all customers of just REDcard holders (loyalty credit card)? D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

Data Science at Target 118-016

7

Analytics for validation. Sometimes managers used BI analysts to validate decisions rather than answer questions.

Drive to learn. As the team worked with BI analysts and managers in different units over time, the sophistication of the questions they asked them increased. An analyst would bring a new question with more potential for business impact, one that required an engineer’s expertise to develop a tool for answering it. Once the team hit its stride, the BI analyst might return an hour later to find that an engineer had “done stuff” and the data he needed to craft an accurate and useful answer was suddenly available. Wait time was measured in minutes, not days. Analysts saw the possibilities and got excited. The data science team could see that in the long run the analyst was just as much the customer as the business manager bringing questions from the front line.

This engagement with BI analysts and managers throughout the business units was an incredibly valuable experience. There were no easy answers going forward, but the team had found real insights for taking data science to the next level at Target.

Tooling up the Enterprise Desai had begun his work at Target on the premise that data and analytics done better could help

transform Target much faster. By mid-2014, Desai’s small team had forged important relationships with analysts and managers across a number of business units. EDABI had a handle on testing and measurements, had defined key metrics to accelerate digital business, and the product- recommendation personalization was almost ready. The team was about to start laying the foundation to collect and act on digital data in real time. There was much work to be done, but the potential for impact was visible at the executive level. Between Q1 2013 and Q2 2014 digital sales growth had increased from approximately 18% to about 30%, which was very promising given the struggling U.S. economy and flat brick and mortar sales growth. 7 In December 2014, leadership promoted Desai to Senior Vice President of Enterprise Data, Analytics, and Business Intelligence at Target Corporation.

The engineering team had built out the back end system for storing and serving up data. The team had invested in an application programming interface (API) so data scientists out in the organization could easily access the data and engage in experimentation for customer facing experiences as well as employee facing tools. The next challenge was to put analytics tools powered by this infrastructure in the hands of BI analysts and, possibly, business managers so they could pose and answer questions as part and parcel of their daily work flow.

Desai assembled a team of five and put the following question to them:

How do you create a system that empowers analysts to produce product grade insights? Consistently? In a dynamic environment?

Product grade referred to the various digital-products and features being rolled out (some for use by customers and others for use by BI managers and analysts). The team needed to understand the products’ stand-alone impact, the roles these products played in the larger ecosystem, and how well they worked together. This feedback would be very important for the product management team to refine its product roadmap, in terms of what to build and how to enhance the most relevant features. The goal was to streamline this feedback and speed up the response time so engineers received data and responses about how the products were functioning much faster (from weeks to days). D o

N ot

C op

y or

P os

t

This document is authorized for educator review use only by ANANTH CHIRAVURI, United Arab Emirates University (UAEU) until Jan 2023. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860

118-016 Data Science at Target

8

This was a complex proposition, but one Desai believed was critical to the success of data science at Target. The working corollary became: Design a system so flexible and easy to use so that “it was as easy as possible to have an analyst ‘say yes’ to a business manager’s questions.”

The team needed to make a series of strategic decisions about how to design the organizational infrastructure such that the data science IT infrastructure actually helped BI analysts and business managers meet both business units’ objectives as well as the objectives of the larger Target enterprise. Making strategic design decisions about IT infrastructure, data science tools, data flows, and governance would also be part of this process. Balancing tensions between and across the various systems would be a formidable challenge.

Centralized versus decentralized organization

All organizational structures existed to manage the inherent tensions between the fine-grained, local knowledge managers detected at the peripheral boundaries of an organization, and the broader perspective of executives seated at an organization’s center, enabled by aggregated and summarized data reporting that flowed to them from across the organization.

Because one of its strategic functions was to enable enterprise-wide analytics, Desai decided the data science team would start as a mostly centralized service. The primary organizational question then became: how to treat the cohort of BI analysts who would be the primary customer for whom the team would design the tools? Should they continue to be embedded in business units? Or should they sit closer to the data scientists? How should the team think about these tradeoffs? Could analysts’ location within the organization evolve over time? If so, how? What might that evolution look like?

Structured versus flexible tools

Desai’s team started building and introducing small tools to individual BI analysts at first, for the sake of outreach, cultural development, experimentation, and learning. But the window for small-bore projects was closing. The ability to scale effective tools to meet need throughout the business was a significant issue the team needed to consider. An ideal outcome would be a tool that busines

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.