Leveraging Search Technologi Google is the worlds premier search engine with more than 60,000 searches made every second, whic

165

CHAPTER 6

Search, Semantic, and Recommendation Technology

C H A P T E R O U T L I N E

Case 6.1 Opening Case: Mint.com Uses Search Technology to Rank Above Established Competitors

6.1 Using Search Technology for Business Success

6.2 Organic Search and Search Engine Optimization

6.3 Pay-Per-Click and Paid Search Strategies

6.4 A Search for Meaning—Semantic Technology

6.5 Recommendation Engines

Case 6.2 Business Case: Deciding What to Watch— Video Recommendations at Netflix

Case 6.3 Video Case: Power Searching with Google

L E A R N I N G O B J E C T I V E S

6.1 Describe how search engines work and identify ways that businesses gain competitive advantage by using search technology effectively.

6.2 Explain how to improve website ranking on search results pages by optimizing website design and creating useful content.

6.3 Describe how companies manage paid search advertising campaigns to increase awareness and drive sales volume.

6.4 Describe how semantic Web technology enhances the accuracy of search engines results and how businesses can optimize their websites to take advantage of this emerging technology.

6.5 Describe how recommendation engines are used to enhance user experience and increase sales on e-commerce websites.

Introduction Every day, over 1.5 billion people around the world use what seems to be a simple tool to find information online—a search engine. We sometimes take for granted that behind a relatively simple user interface, an increasingly complex set of search engine technologies are at work,

166 C H A P T E R 6 Search, Semantic, and Recommendation Technology

helping us find the information we need to do our jobs, conduct research, locate product reviews, or find information about the television shows we watch. Because most search engine services are free, people are not generally aware that “Search” has become a multibillion- dollar- a-year business. More importantly, the way search engines work and how they rank-order the links displayed when we conduct a search have huge implications for millions of other busi- nesses. Because consumers typically don’t look past the first few pages of search results, hav- ing your business appear at the top of a search results page can make a big difference in how much traffic your website gets. In this chapter, you will read about how search engines work and how they determine which websites are listed at the top of search results. You will also read about the strategies companies use to increase their presence on search results pages includ- ing search engine optimization (SEO) and pay-per-click (PPC) advertising.

Semantic technologies are increasingly being used by search engines to understand Web page content. In this chapter you will read about the ways that search engines are using semantic technology to improve performance, increasing relevant pages and decreasing the number of irrelevant pages that appear in search results.

Finally, you will read about recommendation engines. These tools attempt to anticipate online information you might be interested in. Netflix uses recommendation engines to sug- gest movies you might like to watch and news organizations use them to recommend stories you might want to read on their websites. Amazon credits its recommendation technology for increasing sales by suggesting products that customers might want to buy.

Business managers need to understand search and recommendation technologies because their influence in directing potential consumers to business websites is already significant and expected to grow substantially in the future.

Case 6.1 Opening Case

Mint.com Uses Search Technology to Rank Above Established Competitors

Company Overview Mint is a popular, Web-based personal finance service that makes it easy for users to keep track of bank, credit card, and other financial accounts using a computer or mobile device. Customers can also use the service to create budgets and monitor progress toward financial goals. Since it began in 2006, the company has grown rapidly despite competition from more established companies. In 2009, Mint was acquired by Intuit, the maker of TurboTax and Quicken financial soft- ware. Today, over 20 million people use Mint’s free financial manage- ment service (Table 6.1).

The Business Challenge In the months leading up to the 2006 launch of Mint.com, a personal finance service, the leadership team faced a formidable challenge: How

to establish name awareness and brand equity in a market filled with established competitors, without spending a lot of money? Mint knew it would be competing in a market space already populated by familiar brands like Quicken Online and Microsoft Money Online. Since online platforms and communication channels tend to favor existing compa- nies with established audiences and reputations, the team knew they had to come up with a powerful strategy for overcoming the estab- lished brands.

Mint’s Content Marketing Strategy As a Web-based service, it was critical for Mint.com to rank high on search engine results pages (SERPs) when consumers used sites like Google or Bing to find information about personal finance services and related topics. Consumers are more likely to visit websites that appear at the top of SERPs. While the service was still in the beta (trial) stage of development, workers at Mint developed an aggressive strategy to optimize the brand’s ranking on popular search engines. Their strat- egy involved building the company’s Web presence on criteria used by

G eb

er 86

/E +

/G et

ty Im

ag es

Ia nD

ag na

ll C

om p

ut in

g /A

la m

y S

to ck

P ho

to

fil o/

D ig

ita lV

is io

n Ve

ct or

s/ G

et ty

Im ag

es

Introduction 167

TA B L E 6 . 1 Opening Case Overview

Company Mint

History Mint was launched in 2006 as a free, Web-based personal finance app by founder Aaron Patzer. In 2009, the company was acquired by the financial software company Intuit.

Growth Within two years of launch, Mint claimed over 1.5 million users. By 2012, the company claimed 10 million users and by 2016, the number of users rose to over 20 million.

Product lines Mint’s original service allowed users to track balances and transactions on credit card, investment, and bank accounts as well as to create budgets and establish financial goals. In addition, Mint now offers users a bill pay service and credit score monitoring.

Social technology Prior to the release of its flagship personal finance app, Mint created a large following of prospective users with MintLife, a blog that offered valuable advice targeted to young professionals.

Search technology Mint utilized an aggressive SEO strategy to rank highly on search results pages. Specific actions included the following: • Creation of useful personal finance content on its blog, MintLife • Use of targeted keywords in website content • Established audiences on popular social media sites such as Facebook and Twitter • Used various strategies, including sponsorship of third-party blogs, to generate links (or “backlinks”)

to Mint.com from other websites

Website Mint.com

search engines to determine SERP ranking. The strategy focused on the following:

• Increasing the number of other websites that linked back to Mint’s website (called “backlinks”)

• Creating interesting and useful content about personal finance topics that prospective customers would find helpful

• Identifying keywords and phrases used by prospective customers when searching for personal financial services, and creatively inserting these words and phrases into website content

• Regularly updating and adding to their collection of personal finance content

• Establishing a presence on popular social media sites, expanding their audience on those sites, and encouraging the audience to share links to Mint’s website content

Months prior to the launch of its personal finance service, Mint rolled out a personal finance blog called MintLife and quickly devel- oped a reputation for providing helpful financial advice targeted to young professionals. Blog posts on MintLife were creatively seeded with keywords and phrases the team had identified as likely to be used by prospective customers when conducting Internet searches for financial services. Mint also created landing pages on their web- site containing content optimized for keywords and phrases related to financial services. As search engines tracked this content, Mint began to lay a foundation for eventually being viewed by search engines as a credible authority for personal finance topics. New posts were regularly added to the blog, which further enhanced Mint’s ranking since search engines favor websites with lots of content (content depth) and regular updates. To further establish its position as a useful and authoritative site, Mint sponsored sev- eral third-party blogs and cultivated relationships with authors of established finance and money management blogs. Mint’s founder, Aaron Patzer, gave hundreds of interviews, resulting in print media and online articles about the start-up company. These and other actions resulted in more third-party websites posting links back to

Mint.com. These “backlinks” were tracked by search engines and resulted in additional increases to the site’s ranking on SERPs. Popu- lar search engines also track a company’s presence on social media and the extent to which users share information about the company and its products. Mint’s blog featured content in a variety of inter- esting formats: videos, podcasts, infographics, and so on. Users on social news sites like Reddit.com frequently shared and “upvoted” interesting infographics from Mint’s blog. Links to other types of blog content were shared by users on Facebook, Twitter, and other social media platforms. As a result, Mint’s expanding audiences on Face- book, Twitter, and other social media platforms further enhanced the new company’s SERP ranking.

Results In 2007, Mint launched its new financial services website into a market where it already enjoyed considerable name recognition and aware- ness. Within 2 years, the service acquired 1.5 million users and was purchased by Intuit for $170 million. The company continued its suc- cessful content marketing strategy, climbing to 10 million users in 2012 and over 20 million users today.

Questions 1. Why did Mint invest the time and effort to publish a financial

services blog almost two years before the launch of its service?

2. How did Mint use social media sites to increase its ranking on the search results pages of popular search engines?

3. Why did Mint use keywords and phrases associated with personal finance when creating content for its blog?

4. Why did Mint put so much emphasis on improving the rank of its website on SERPs?

5. Why did Mint use infographics, videos, and other types of rich media in its financial services blog?

Sources: Compiled from Sukhraj (2015), Bulygo (2013a), Obi-Azubuike (2016), Prince (2016), Greene (2016).

168 C H A P T E R 6 Search, Semantic, and Recommendation Technology

6.1 Using Search Technology for Business Success Search engines like Google, Bing, Yahoo, and others have traditionally been regarded as a consumer technology. But search technology has become an important business tool with many different uses and applications. In this section, you will learn how search engines work and the role they play in generating revenue and consumer awareness for organizations. You will also discover how businesses use enterprise search technology to unlock hidden content with their organizations. Finally, you will read about how search and Internet technology is evolving to provide more accurate and useful results.

How Search Engines Work People use the word search engine to refer to many different kinds of information retrieval (IR) services that find content on the World Wide Web. These services vary in significant ways. Understanding how these services differ can improve the quality of results obtained when con- ducting a search for online information. Listed below is a brief description of different IR ser- vices for finding Web content:

• Crawler search engines rely on sophisticated computer programs called spiders, crawlers, or bots that surf the Internet, locating Web pages, links, and other content that are then stored in the search engine’s page repository. The most popular commercial search engines, Google and Bing, are based on crawler technology.

• Web directories list Web pages organized into hierarchical categories. Originally, Web direc- tories were created and maintained by human editors who decided how a website would be categorized. Today, many Web directories use technology to automate new website list- ings. Web directories are typically classified as “general” directories that cover a wide-range topical categories, or “niche” directories that focus on a narrow range of topics. Examples of popular general directories include Best of the Web, JoeAnt, and LookSmart. Wikipedia maintains a list of general and niche Web directories.

• Hybrid search engines combine the results of a directory created by humans and results from a crawler search engine, with the goal of providing both accuracy and broad coverage of the Internet.

• Meta-search engines compile results from other search engines. For instance, Dogpile generates listings by combining results from Google and Yahoo.

• Semantic search engines are designed to locate information based on the nature and meaning of Web content, not simple keyword matches. The goal of these search engines is to dramatically increase the accuracy and usefulness of search results. Semantic search engines are described in more detail in Section 6.4.

Web Directories Before crawler search engines became the dominant method for finding Web content, people relied on directories created by human editors to help them find information. Web directories are typically organized by categories (for instance, see the categories listed on Best of the Web). Web page content is usually reviewed by directory editors prior to its listing in a category to make sure it is appropriate. This reduces the number of irrelevant links generated in a search. The review process, however, is very slow compared to the automated process used by crawlers (described in the following section). As a result, the listings in a Web directory represent a relatively small portion of the Web. Directories are particularly useful when conducting searches on a narrow topic, such as identifying suppliers of a specific type of product or service. Companies who need

Search engine an application for locating Web pages or other content (e.g., documents, media files) on a computer network. Popular Web-based search engines include Google, Bing, and Yahoo.

Spiders also known as crawlers, Web bots, or simply “bots,” spiders are small computer programs designed to perform automated, repetitive tasks over the Internet. They are used by search engines for scanning Web pages and returning information to be stored in a page repository.

Using Search Technology for Business Success 169

to identify vendors or suppliers may consult a niche Web directory created for just this purpose. For example, see the Web directory at business.com.

How Crawler Search Engines Work The two most popular commercial search engines on the Web, Google and Bing, are based on crawler technology. Behind the relatively simple interfaces of these two powerful search engines, a great deal of complex technology is at work (Figure  6.1). Because modern search engines use proprietary technology in the race to stay ahead of competitors, it is not possible to tell exactly how they decide what websites will appear in a SERP. While they each produce different results, it is possible to describe the basic process shared by most crawler search engines. The following description is based on publications by Grehan (2002) and Oak (2008).

1. The crawler control module assigns Web page URLs to programs called spiders or bots. The spider downloads these Web pages into a page repository and scans them for links. The links are transferred to the crawler control module and used to determine where the spi- ders will be sent in the future. (Most search engines also allow Web masters to submit URLs, requesting that their websites be scanned so they will appear in search results. These requests are added to the crawler control queue.)

2. The indexer module creates look-up tables by extracting words from the Web pages and recording the URL where they were found. The indexer module also creates an inverted index that helps search engines efficiently locate relevant pages containing keywords used in a search. (See Figure 6.2 for examples of an inverted index.)

3. The collection analysis module creates utility indexes that aid in providing search results. The utility indexes contain information about things such as how many pages are in a web- site, the geographic location of the website, number of pictures on a Web page, Web page length, or other site-specific information the search engine may use to determine the rel- evance of a page.

4. The retrieval/ranking module determines the order in which pages are listed in a SERP. The methods by which search engines determine website listing order varies and the spe- cific algorithms they use are often carefully guarded trade secrets. In some cases, a search engine may use hundreds of different criteria to determine which pages appear at the top of a SERP. Google, for instance, claims to use over 200 “clues” to determine how it ranks pages (Google.com, 2014).

Page repository a data structure that stores and manages information from a large number of Web pages, providing a fast and efficient means for accessing and analyzing the information at a later time.

Crawler control module a software program that controls a number of “spiders” responsible for scanning or crawling through information on the Web.

Page Repository

WWW

Text Utility

Indexer Module

Structure

Collection Analysis Module

Surfer–Client QueriesSpiders/ Crawlers

Crawler Control Indexes

URL Submissions

1. URL 2. URL 3. URL

. Query

Formulation

Results

Ranking ..

FIGURE 6.1 Components of crawler search engine (Adapted from Grehan, 2002).

170 C H A P T E R 6 Search, Semantic, and Recommendation Technology

5. Web pages retrieved by the spiders, along with the indexes and ranking information, are stored on large servers (see IT at Work 6.1).

6. The query interface is where users enter words that describe the kind of information they are looking for. The search engine then applies various algorithms to match the query string with information stored in the indexes to determine what pages to display in the SERP.

Each search engine utilizes variations and refinements of the aforementioned steps in an attempt to achieve superior results. The Web search industry is highly competitive and the proprietary advances in search technology used by each company are closely guarded secrets. For instance, even the first step in the process, crawling the Web for content, can vary greatly depending on the strategic goals of the search engine. Some search engines limit the number of pages scanned at each website, seeking instead to use limited computing power and resources to cover as many websites as possible. Other search engines program their spi- ders to scan deep into each website, seeking more complete coverage of each site’s content. Still other search engines direct their spiders to seek out websites that contain certain types of content, such as government sites or shopping (e-commerce) sites. Another decision that search engines make regarding spiders is the amount of resources directed at searching new websites versus devoting resources to exploring previously indexed pages for updates or changes.

One of the many challenges faced by large commercial search engines is storage. In the simplest sense, the crawler approach to search requires a company to store a copy of the Web in large data centers. In addition to the petabytes of storage required to maintain this copy of the Web, the search engine must also store the results of its indexing process and the list of links for future crawls.

Petabyte a unit of measurement for digital data storage. A petabyte is equal to one million gigabytes.

Document ID

1 To the heart, real love always endures.

Though passion may cool, love remains true. True love kindles the passion in my heart.

2

3

Content URL

Search Query:

Page Index

Inverted Index

ID Term Document: Position

1 1:3, 3:7

1:4

1:5, 2:5, 3:2

1:6

1:7

2:7, 3:2

2:1

2:2, 3:5

2:3

2:4

2:6

3:3

3:8

heart

real

love

always

endures

true

though

passion

may

cool

remains

kindles

heart

2

3

4

5

6

7

8

9

10

11

12

13

True love

Results Ranking (based on position)

True love kindles the passion in my heart.

Though passion may cool, love remains true.

Documents with both terms: 2 and 3

FIGURE 6.2 Search engines use inverted indexes to efficiently locate Web content based on search query terms.

Using Search Technology for Business Success 171

IT at Work 6.1

Google Data Centers Not only does Google maintain a copy of the Internet for its search engine services, it is also constantly updating a map of the entire planet for users of its popular Google Earth application. In addition, the company is making a full-text, searchable copy of over 129,864,880 known books, equal to 4 billion pages or 2 trillion words. And then there are applications like Gmail, serving roughly 425 million people and YouTube, where 300 hours of video are uploaded every minute! Add all this up, and Google is facing perhaps the biggest data storage challenge ever. So where does Google store all of these data?

Challenges: Energy, Performance, and Security Information collected by Google is housed on over 1 million servers spread across 12 different facilities worldwide. The facilities are large, factory-like installations containing row upon row of racked and stacked servers. Cooling systems, required to keep servers from overheating, are a significant component of any large data center (Figure  6.3). Google pioneered the software systems that connect the company’s servers and make it possible for various applications to access data stored on the machines. Unlike other companies that purchase servers from outside suppliers, Google builds its own. Based on its experience creating the hardware, software, and facil- ities necessary to power the company on a global scale, Google is recognized as a leader in data center operations.

The company’s data centers, including the servers, are built with energy efficiency, reliability, and performance in mind. As Google is a leading provider of Internet services, its data infrastruc- ture must keep up with growing consumer demand for speedy performance and reliability. A typical Google search delivers millions of pages of results in less than half a second. Consumer expectations for performance have grown so high that waiting more than a few seconds for an e-mail to load or a search to run can cause frustration.

More recently, Google has had to contend with revela- tions that the U.S. National Security Agency (NSA) breached its server network security. This follows cyberattacks in 2010 and 2011 by hackers suspected of being associated with the Chinese government. Protecting company data from criminals is a significant challenge in itself, but Google is understandably

frustrated by the fact that it must now fight off cyber-attacks from two world superpowers, one of which is its own government.

Environmental Impact Industrywide, data centers used 70 billion kilowatt-hours of electricity in 2014, representing a 4% increase from the amount used in 2012. Industrywide, data center energy use and the related environmental impact have become an issue of growing concern. Google is widely recognized as operating some of the most efficient data centers in the world, but many critics are disturbed by the industry’s overall level of energy consumption. According to some estimates, data centers account for about 2% of the world’s energy use and the fast rate of growth is cause for concern (see Figure 6.4). Google has taken an active approach to reducing its environmental footprint. Beginning in 2017, Google will source 100% of its energy needs for offices and data centers from renewable sources. See Google’s data center Web page https://www.google.com/about/ datacenters for additional information.

Google Data Center Statistics • Number of servers worldwide Over 1 million • Number of data centers Nine in North America, one in

South America, two in Asia, and four in Europe

• 2016 Capital investment in data centers Approximately $11 billion

• Data processing volume Over 100 petabytes a day • Average energy efficiency PUE* = 1.12 • Energy use Continual use of about 260 megawatts of

electricity, approximately 0.01% of global energy consumption

• Energy use comparisons Owns about 3% of servers world- wide, but only uses about 1% of data center industry energy

• Renewable energy Claims that 100% percent of its energy use comes from renewable sources

*PUE stands for Power Usage Effectiveness. A PUE of 2.0 means that for every watt of power devoted to computing, an additional watt is spent on cooling, power distribution, and overhead. The Data Center Industry average PUE falls between 1.8 and 1.89.

Sources: Jacobson (2010), Grifantini (2011), Newman (2011), Schneider (2011), Glanz (2011, 2012), Gallagher (2012), Venkatraman (2012), Anthony (2013), Miller (2013), Sverdlik (2016).

B lo

om b

er g

/G et

ty Im

ag es

FIGURE 6.3 Pipes pass through the chiller plant at the Google, Inc., data center in Changhua, Taiwan. Google doubled its spending plan for its new data center in Taiwan to $600 million amid surging demand from Asia for its Gmail and YouTube services.

© a

sh ar

ky u/

S hu

tte rs

to ck

FIGURE 6.4 New, large-scale data centers being constructed for companies like Google, Microsoft, and Facebook house thousands of servers and are creating concern among environmentalists over increases in energy consumption.

172 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Why Search Is Important for Business Search engines have become a part of our everyday life. They are free, easy to use, and become more powerful and effective every day. Most of us take them for granted and are generally unaware of the complex technologies that power these tools. For the average Web user, it may not be vitally important to understand how search technology is evolving. But for business managers, understanding the potential power of search technology is crucial and becoming more important every day. It has long been recognized that access to information is a competitive advantage. Search technology impacts business in each of the following ways:

• Enterprise search—finding information within your organization • Recommendation engines—presenting information to users without requiring them to

conduct an active search • Search engine marketing (SEM)—getting found by consumers on the Web • Web search—finding crucial business information online

Each of these important search technology applications are described in what follows.

Enterprise Search Enterprise search tools are used by employees to search for and retrieve information related to their work in a manner that complies with the organization’s information-sharing and access control policies. Information can come from a variety of sources, including publicly available information, enterprise information (internal records) found in company databases and intranets, as well as information on individual employee computers (Delgado, Renaud, & Krishnamurthy,  2014). Enterprise search tools allow companies to gain competitive advantages by leveraging the value of internal information that would otherwise remain hidden or “siloed.” Information can be inaccessible as a result of incompatible technol- ogies in various units, lack of coordination or cooperation between units, security concerns, and concerns about the cost of making information accessible (Thomas, 2013; Walker, 2014).

In most organizations today, a large portion of employees are “knowledge workers” (e.g., business analysts, marketing managers, purchasing agents, IT managers, etc.). Access to information has a significant impact on their productivity. Enterprise search tools allow workers to extract internal information from databases, intranets, content management sys- tems, files, contracts, policy manuals, and documents to make timely decisions, adding value to the company and enhancing its competitive advantage.

Structured versus unstructured data One of the challenges encountered by …

I teach college students how to succeed in their programs by leveraging the power of academic writing, without

cheating or cutting corners.

Hey there! I'm Chelsea.

I'm an online educator and academic writing expert.

Thank you for downloading this free guide on APA formatting 7th edition for STUDENT PAPERS!

Let's get started!

Pag

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.