What passage indexing and natural language processing mean for the future of SEO

Taken in totality, Google’s recent announcements on its expanding use of natural language understanding algorithms represent a major evolution in how it determines what content gets surfaced in the search results. In turn, these algorithms will impact how we approach content and optimization.

Ranking passages in the search results. Google is rolling out passage-based indexing, a change that enables it to identify individual passages on a page and process them as possibly being the most relevant for a given query — even if they aren’t part of the main theme of the page. (Note that Google doesn’t technically index passages separately.) The company expects that this will apply to 7% of search queries across all languages when it’s fully rolled out.

BERT goes from 10% to nearly 100% of all queries. BERT is a neural network-based technique for natural language pre-training that Google and Bing use to better discern the context of words. It now powers almost every English-language query on Google search — a dramatic increase from one in 10 queries when Google first announced the use of BERT last October.

This wider application of BERT should improve Google’s understanding of content and search intent. You can learn more about what BERT means for language processing in search by reading our resources FAQ: All about the BERT algorithm in Google search and A deep dive into BERT: How BERT launched a rocket into natural language understanding.

Misspelling improvements. One in ten search queries are misspelled, according to Google. To improve the search experience for those queries, the company is also applying advancements in language understanding to better model edge cases in spelling errors, such as when context may be required to identify a misspelling or in cases where words are drastically misspelled.

For SEOs who may have been attempting to optimize for misspellings, this change, which is expected to roll out by the end of this month, means that your efforts are better spent elsewhere.

The implications for SEOs and the direction of Google search

More freedom to focus on audiences, not search crawlers. The technology used to power passage indexation enables Google to identify pages that have one individual section that matches a query particularly well, even if the rest of the page is slightly less relevant.

“If anything, these updates move us toward a world where we can focus more on users and not worry as much about bots, from a content and ranking perspective,” said Dr. Pete Meyers, marketing scientist at Moz, adding that making your site friendly for GoogleBot to crawl it will still be important from a technical SEO perspective.

“As Google rolled out Featured Snippets and became more focused on topical authority, there was a movement toward much more focused content,” Meyers said. Search engines’ preference for topical authority and focused content may have cornered SEOs into a mindset in which users and search engines exist on equal footing as priorities when creating content.

“If the algorithm can understand the relevance of passages, we can hopefully relax a bit about this and not go overboard,” he said, “We don’t need a page for every question a visitor might ask, for example.”

The flexibility to move away from organizing content for search engines should afford SEOs more freedom to instead create content that matches the searcher’s intent, in the form that makes the most sense for the subject matter and the audience, whether that is long-form or laser-focused. 

What’s good, what’s bad and what to watch out for. Some may interpret the way Google is presenting passage indexing as another milestone of the zero-click search trend.

“The better display of passage-related information on Google SERPs means the probability of users clicking on the search result would reduce,” said Kaushal Thakkar, founder and managing director of 2020 Search Engine Land Award-winning agency INFIDIGIT, “Since the passage listed will provide additional information to the users on the SERP itself, not requiring them to visit the source page.” However, the overall trend of increasing query volume each year means that, for many businesses, any decrease in traffic may go unnoticed, Thakkar added.

“As an SEO, I’d pay more attention to search impressions data in Search Console,” said Hamlet Batista, CEO of RankSense, recommending that SEOs also monitor their clickthrough rates, engagement and the quality of their traffic as these algorithms impact search results. Unfortunately, at this time there isn’t a specific report in Google Search Console showing traffic from passage indexation, but you may see a rise in page impressions if those passages start ranking for queries.

“I wouldn’t abandon [keyword research], but adapt it to intent research,” he said, adding that the same intent can be expressed using various keywords. Google’s expectation is that only 7% of queries will be improved by passage indexation, which means that keyword research will remain an important part of SEO, at least for the foreseeable future.

“Glass-half-full SEOs see this as people searching more as [search] engines become more useful,” Batista said, noting a potentially positive outcome of these algorithm updates. In addition to potentially greater search volume, these updates may also lead to higher quality traffic as results would presumably be more relevant to users.

Looking to the future. These advancements also speak to the shortcomings of search engines in their current state, as well as the direction that Google is taking in order to address them.

“First of all, [passage indexation] illustrates one of the big challenges with search, which is the vast array of types of information that people might be looking for,” said Eric Enge, general manager at Perficient Digital, “Many times this is information that is so specific, yet we already see Google reporting that it impacts 7% of all search queries. Chances are, as this algorithm gets refined that 7% number will go up significantly.”

“In addition, this highlights the challenges we all face as SEOs (and digital marketers),” Enge said, referring to how businesses must create a broad range of content to answer users’ questions and address their needs, which are often more complex than we initially anticipate.

If search engines continue to prioritize focused content the way they have been, then providing a complete user experience while aiming for high visibility in the search results may result in tradeoffs. For example, it might be necessary to create a large amount of single-keyword-optimized pages to address your users’ various questions while still adhering to search engines’ preferences for focused content.

This may result in creating pages about very similar topics or an excess of content and pages that are hard to maintain and difficult for users to navigate, ultimately hindering the experience you were trying to improve in the first place.

If Google can continue to advance its natural language understanding, then it’ll be better equipped to gauge relevance, so SEOs won’t have to approach content in such a rigid manner. Google’s recent, and future, applications of natural language processing and AI will be aimed at removing those tradeoffs so that it can serve relevant results, no matter how obscure a query might be or where on a site that information lives.

“This is more confirmation that Google is working hard to provide users with any and all the information that they want,” Enge said, “As publishers of websites, it’s incumbent on us to do the same.”