GOOGLE’s Top Search Result?

In Google’s early years, users would type in a query and get back a page of 10 “blue links” that led to different websites. “We want to get you out of Google and to the right place as fast as possible,” co-founder Larry Page said in 2004.

Today, Google often considers that “right place” to be Google, an investigation by The Markup has found.

We examined more than 15,000 recent popular queries and found that Google devoted 41 percent of the first page of search results on mobile devices to its own properties and what it calls “direct answers,” which are populated with information copied from other sources, sometimes without their knowledge or consent.

When we examined the top 15 percent of the page, the equivalent of the first screen on an iPhone X, that figure jumped to 63 percent. For one in five searches in our sample, links to external websites did not appear on the first screen at all.

A trending search in our data for “myocardial infarction” shows how Google has piled up its products at the top. It returned:

• Google’s dictionary definition.
• A “people also ask” box that expanded to answer related questions without leaving the search results page.
• A “knowledge panel,” which is an abridged encyclopedia entry with various links.
• And a “related conditions” carousel leading to various new Google searches for other diseases.

All of these appeared before search results by WebMD, Harvard University, and Medscape. In fact, a user would have to scroll nearly halfway down the page—about 42 percent—before reaching the first “organic” result in that search.

Google’s decision to place its products above competitors’ and to present “answers” on the search page has led to lawsuits and regulatory fines. A number of websites said it killed their revenues—and their companies. Founders of both innovative startups and companies that had been around for a decade or more told The Markup that once Google started placing its product first, they didn’t stand a chance.

Travel research firm Skift wrote in November that the entire online travel industry is suffering. “The fact that Google is leveraging its dominance as a search engine into taking market share away from travel competitors is no longer even debatable.”

The choice to highlight its own products has been deliberate: Internal emails unearthed by the European Commission in an antitrust investigation show Google staffers discussing the need to place its comparison-shopping product at the top of the search results to garner traffic. An email the following year noted traffic to the retooled product had more than doubled from four million to 10 million visits, and “most of this growth is from improved google.com integration.”

Sally Hubbard, an expert on antitrust and technology companies with the Open Markets Institute, said Google’s decisions in search have huge implications. “Imagine you go to the library, and the card catalog is picking and choosing what book to get based on what makes the library the most money.”

Google makes five times as much revenue through advertising on its own properties as it does selling ad space on third-party websites.

In a written statement, Google spokesperson Lara Levin took issue with The Markup counting “answers,” related queries, and similar results as “Google” in our analysis.

“Providing feedback links, helping people reformulate queries or explore topics, and presenting quick facts is not designed to preference Google. These features are fundamentally in the interest of users, which we validate through a rigorous testing process,” she said. “Sometimes, the most helpful information will be a link to another website—other times, it will be a map, a restaurant listing, a video or an image.”

Levin called The Markup’s methodology “flawed and misleading.” She criticized it for being “based on a non-representative sample of searches” and said using Google Trends makes it more likely that results would include Google “knowledge panels” than a random sample would. However, Google has not publicly released a random sample of searches for research.

In response to Page’s 2004 quote about the company’s mission to get people “out of Google,” Levin said times have changed: “As a search engine, Google’s mission is to quickly direct searchers to great information, wherever that information is, as Page went on to explain. At that time, that generally meant to direct people from search results to websites. As search technologies have developed, that’s not always the best way to assist people.”

She did not answer questions about whether those changes present the search engine with a conflict of interest.

Nearly nine in 10 U.S. web searches are performed on Google.

The effects of placing its own products on the search page can be stark: In the nine years since Google Flights and Google Hotels launched, those sites have become market leaders. They garnered almost twice as many U.S. site visits last year as each of their largest competitors, Expedia.com and Booking.com, even though we found Google Flights doesn’t always show users all the options.

“Google makes the most money when, long term, they can addict searchers to their platform,” said Rand Fishkin, a search engine analyst and frequent Google critic who found that 12 percent of real-world clicks on the search results page go to Google properties. “If Google can train you, don’t go to Genius.com, don’t go to TripAdvisor, don’t go to the restaurant’s website, just come to Google—always come to Google—then they win.”

Levin said some regulators have defended the company’s practices, pointing to the Federal Trade Commission’s 2013 ruling that highlighting its products on search “could be plausibly justified as innovations that improved Google’s product”; and a U.K. High Court finding in 2016 that returning Google Maps for location queries was not an abuse of market dominance.

Although the FTC decided not to take legal action, as Google notes, it did require the company to change some practices. Google agreed to allow websites to opt out of having their content scraped for Google Flights, Google Hotels, and local business listings.

And at least some FTC staffers had concluded that Google’s boosting its own properties in search rankings “led to a significant decrease in traffic for the websites of many vertical competitors,” according to an internal FTC report, half of which was accidentally provided to The Wall Street Journal. At the time, Google responded, “Speculation about consumer or competitor harm turned out to be entirely wrong.”

Levin said the company “continues to engage” with regulators conducting myriad active probes of Google’s parent company, Alphabet. Fifty U.S. attorneys general are currently investigating its ads and search business for potential antitrust violations. The Federal Trade Commission is examining past acquisitions by tech companies, including Google, for harm to competition. Politico reported that the Department of Justice will file an antitrust lawsuit against the company soon. The European Commission, which has already issued three multibillion-dollar fines against Alphabet for antitrust violations since 2017, launched a preliminary investigation into Google for Jobs in 2019.

“Google helped build the free internet. And now they’re helping dismantle what they built,” said Chris Cummings, CEO of Curiosity Media, which owns the translation website SpanishDict.com.

The site provides free translations and dictionary entries, many written by linguists and translators, he said. It is ad-supported and needs web traffic to survive. For years, he said, it grew as Google grew. But then Google began giving the top spot in searches to Google Translate, which is automated and asks users for corrections.

“The big loss is for consumers,” he added, “because nobody thinks that Google Translate is the most accurate translator.”

He said data from the Google Search Console tool for websites shows SpanishDict gets as high as 80 percent or more of the clicks when it’s the top result but only 2 percent when Google Translate appears above it.

“If we’re only getting 2 percent of the click-through, there’s no business to run here. We only exist because there are still some queries where they don’t put their stuff at the top,” he said. He said Google’s actions “have affected our ability to invest in the future.”

Levin did not respond to questions about Google Translate or its effect on SpanishDict.

SpanishDict.com showed up three times in our sample, each time below either Google Translate or Google’s dictionary.

To determine the amount of space Google dedicates on the search page to direct answers and its own products, we built a custom scraper to gather all trending Google searches for two months, starting in November 2019. Then we built another scraper to run the searches as they would appear on mobile devices, where the majority of searches now occur. We wrote more than 1,000 lines of code to parse and analyze the resulting dataset, which contained more than 1 million rows.

We found that the majority of links to and results for non-Google sites were pushed down to the bottom-middle of the page, where data shows users are less likely to click.

We categorized search results into four types: Google, non-Google, ads, and AMP, a web format created by Google four years ago. AMP pages are hosted by Google but created and monetized by publishers.

AMP has been controversial, with some publishers and developers saying it gives Google too much control over the web. The company tells website owners that using it makes them eligible “for more prominent presentation of thumbnail images in search results and in the Google Discover feed.” Because AMP has some features of a Google result and some features of a non-Google result, we gave it its own category.

Levin objected to that decision, saying AMP results should be non-Google. “Those are outbound links to publishers and other web creators. To suggest otherwise is not factual,” she said. She also said our results may be heavy on AMP content because our sample, Google Trends, leans toward breaking news, which triggers “top stories” carousels. The news stories in those carousels were often delivered in AMP in our data.

Direct answers include “featured snippets,” which excerpt content from websites, and “knowledge panels,” which show summaries and facts drawn from the “knowledge graph,” Google’s fact database curated from other sources. They also include weather, sports statistics, and dictionary definitions. All of these appeared on the first search results page, typically at the top, without the need to click through.

Google acknowledged in written comments to Congress last fall that one major reason people end a search is because Google’s modules provided the answer on the search page.

In our sample, Google featured its dictionary definitions before Urban Dictionary, Cambridge Dictionary, Dictionary.com, Wikipedia, Merriam Webster, and Investopedia, among others. And searches for song titles typically returned a YouTube video in the top spot, followed by the lyrics, displayed in full on the search results page.

Levin said Google does not give preference to YouTube, its subsidiary. Recent tests by The Wall Street Journal found that it did.

The quantity of Google results in some searches in our sample was quite large. A search for the Shania Twain song “Man! I Feel Like a Woman,” which was trending during our study, returned the following on the first page: links to four YouTube videos in various positions on the page; a box labeled “about” with some hyperlinked facts that led to new Google searches; a box labeled “people also search for,” which led to a new Google search; and a “people also ask” box. Together, direct answers and results leading to Google products took up 67 percent of the first search results page for that query. Non-Google results took up only 22 percent of the page.

Even some of the “traditional” results that appeared after the Google results in that query were touched by its hand. Genius.com and Biography.com’s results were delivered in AMP.

Travel websites are among those who say they have suffered from Google Search’s preferential treatment of Google products. TripAdvisor, which laid off 200 workers in January, pointed to Google “siphoning off quality traffic that would otherwise find TripAdvisor” as its “most significant challenge.”

In queries for specific airlines that appeared in our sample, Google presented Google Flights at the top of the results page, before links to the airlines’ own websites. A search for “nonstop flight” also returned Google Flights in the top spot, above competitors.

And travel sites aren’t just Google’s competitors; they’re also its customers. Together, Expedia and Booking spent $5.8 billion on Google advertising in the 12 months ending in September 2019, according Skift, the travel research firm.

“When they compete against their advertisers … I think it’s bad practice,” Barry Diller, chairman of Expedia Group, said during an earnings call in February in which he called Google an “existential” threat.

In the industry, Google Flights is not seen as the best product. It did not crack the top 10 of Frommer’s 2020 ranking of airfare search engines, for instance.

The Markup found that Google Flights did not always display the cheapest fares or all available flights, even when those fares and flights showed up in ITA Matrix, which is powered by the same software Google acquired in 2010 and used to launch Google Flights.

For example, a search on Google Flights for a one-way flight from Billings, Mont., to London’s Heathrow on Feb. 19 showed the cheapest flight was $1,068. The same search on Orbitz turned up a flight for $772, while ITA Matrix offered $766.69.

And when The Markup searched for a one-way flight from Morgantown, W.Va., to New York City on Feb. 21, Orbitz produced a long list of results, including a three-hour-and-40-minute journey that combined flights from Southern Airways and American Airlines with a stop in Pittsburgh for $253.47. The same search on ITA Matrix showed the same flight at a different time for $242.40. But a Google Flights search showed nothing: “Aw snap, no results.”