15 lessons from 15+ years in search

My career in search started as a librarian. When I moved into SEO, the job was obscure enough that my landlord phoned my employer to confirm I wasn’t making it up. Now I advise luxury and enterprise brands on how AI systems read them. 

Here are some lessons to help you become a better SEO expert: orient, diagnose, measure, build, then survive.

First, work out what search actually is

Search keeps changing, but the underlying patterns are worth understanding before you optimize for them.

1. Learn the history so the reruns stop scaring you

Every “new” GEO debate is a rerun: cloaking, keyword-density panic, the death of SEO. I have survived that cycle many, many times.

Lesson: Pattern recognition is essential. Figure out the past SEO cycle that the current new big thing rhymes with.

Curious and in need of some dopamine? I mapped the whole thing once as a dinosaur timeline, Triassic to Cretaceous, GeoCities to RankBrain, in “The Age of Dinosaurs & Search Engine Bots.” 

2. Study the friction humans work around

Every search evolution corrects toward how people actually search. Boolean strings to natural language, desktop to mobile, typed to spoken, now screenshots and multimodal prompts. Humans never take the complicated path, and the next interface tends to embrace whatever their workaround is.

Cognitive science refers to a workaround like that as scaffolding, a temporary structure built around a system’s limits that you internalize until you stop noticing it is a workaround. 

The scaffolding is coming down. A 17-month Semrush clickstream study, credited to Luke Harsel, found that 65% to 85% of ChatGPT prompts match no term in a database of 27 billion keywords. 

Lesson: Never bank on humans following a complicated path; they’ll find a clever workaround at every step of the way.

3. Build for the intent, not the search box

A query is intent compressed into whatever shape the search bar can accept. Prompts add more words and more context than keywords because the search bar changed shape. People can now ask questions, add screenshots, and still get an answer. 

Focus on what the searcher wants to accomplish and how that creates value for your business. At the same time, pay attention to the shape of the search interface so you understand the constraints you’re optimizing within — or trying to overcome. 

4. Pick your bot philosophy and commit to it

Bots as middlemen, bots as audience, or human-centric but bot-aware. Whatever your bot philosophy, it will shape every deliverable you produce. Sure, you can trick a middleman, but tricks put you on a treadmill where every update could be the one that knocks you off

A model forms an opinion of your brand. You’re arguing your case to it directly, same as you’d argue it to a buyer. Persuasion compounds. I made that case in my Search Engine Land article titled “Brand depth determines what AI systems recommend.” 

Lesson: Being human-centric but bot-aware means focusing on the humans first while balancing bots also needing access, but without making the bots the primary audience. 

Something breaks. Diagnose before you touch it.

Before you try to fix a problem, make sure you understand what’s actually broken and where the real constraint lies.

5. Triage before you troubleshoot

A problem is something you can potentially solve. A situation is simply a state of affairs that may not have a practical solution.

Don’t confuse the two. If you treat a situation as a problem, your team can burn time trying to fix something that won’t change. If you treat a solvable problem as a situation, you may accept a constraint that you could actually remove. 

Lesson: Triage before you troubleshoot. 

Bonus lesson: Nothing is static. A problem can become a situation when every solution is exhausted or unacceptable, and a situation can become a problem again when a fresh perspective reveals a possible solution. 

6. Hyperfocus on the bottleneck

Eliyahu Goldratt’s theory of constraints taught me to look for the single bottleneck limiting a system. I learned this working as a technical SEO in dev centers. Find the constraint that is slowing everything else down.

Here’s a concrete example: If your pages aren’t crawlable, your content doesn’t matter. 

Lesson: Find the one thing choking throughput and focus on fixing that. Sometimes, that bottleneck could turn out to be you. 

7. Unplanned work will devour your roadmap

The Phoenix Project by Gene Kim, Kevin Behr, George Spafford names four types of work: 

• Business projects.
• Internal projects (campaigns).
• Changes.
• Unplanned work.

Lesson: Sort your work into business projects, internal projects, changes, and unplanned work. Then look at how much capacity unplanned work is consuming. 

8. Serve critical content in raw HTML, always

JavaScript: Just because you can use it doesn’t mean you should use it for critical content. That’s what I told a JS specialist in 2013 at Confoo in Canada. He agreed back then and has spent years since that conversation making it better (hi, Martin Splitt).

Every few years, the framework changes and we make the same mistake: relying on JavaScript to deliver critical content. Single-page apps, client-side catalogs, infinite scroll — the list goes on.

Latest lesson in the series: Serve critical content in raw HTML because JavaScript is still a struggle when it comes to AI search.

Then you have to prove any of it worked

Good decisions require good judgment about imperfect data, useful metrics, and when optimization stops creating value.

9. Be a data pragmatist

Data that captures human behavior isn’t perfect, nor does it have to be. It’s sampled, modeled, consent-gated, and increasingly affected by automated activity.

Clicks, scrolls, and submissions are traces of human behavior, not human behavior itself. As more agents interact with websites and search systems, those signals become harder to interpret.

The lesson: Question every vendor metric before you report it.

The real lesson: Get comfortable making decisions with imperfect data. Build custom metrics that answer meaningful business questions instead of accepting vendor metrics at face value.

10. Goodhart’s law: Don’t confuse the metric for the mission

GEO is speedrunning an encore: Teams are increasingly focused on citation counts and mention frequency.

Here’s a concrete piece of advice: “Citations are receipts, LLM visibility residue. Optimize for what earns the receipt, not the receipt.”

Lesson: If a measure becomes a target, it’s no longer a good measure. Here are a few examples:

• We targeted rankings and got doorway pages
• We targeted links and got PBNs.
• Word count became a proxy for depth of expertise, and we got 3,000-word ultimate guides or really long recipes with a life story on top.

Dig deeper: Brand depth determines what AI systems recommend 

11. Over-optimization leads to marketing enshittification

Refreshing the same blog post for the fourth or fifth time is like cutting the same salami into thinner and thinner slices. You’re not making a new salami. You still have the same amount you started with, but the slices get so paper-thin, nobody’s enjoying it. You’re optimizing a fixed asset and sometimes, that’s not enough. 

Lesson: At some point, optimization stops paying off. That means it’s time to create something new that doesn’t yet exist and thus hasn’t been optimized within an inch of its life.

12. Be curious beyond SEO — or you’ll sound like everyone else

An industry this young borrows all its durable ideas from older ones: 

• Content hierarchy was inherited from the printing press.
• HTTP status codes were originally meant to check if the coffee machine worked fine (that’s why we have code 418 = I’m a teapot).
• Goodhart came from economics.
• Theory of constraints came from manufacturing.

Lesson: A brain fed only SEO content can’t produce more than ultra-processed SEO content (the equivalent of a soggy Big Mac, but in text form). 

13. The more experience you gain, the harder it is to learn new things

The first 80% of any skill is cheap to acquire: Patterns repeat, documentation exists, and other experts solve things publicly.

The last 20% is edge cases, “It depends,” and the situations where the usual strategy stops working. It’s context-specific and hard to access. 

Lesson: Seek out people who are smarter than you if you want to keep improving.  

14. Consistency compounds

Being consistent compounds. That’s how you build a reputation with humans and in the model’s parametric memory.

Lesson: To be truly visible, you have to be a little more redundant than you’re comfortable with. 

15. Stop waiting for things to settle

Competition breeds complexity. Every iteration stacks another layer: rendering on top of crawling, entities on top of keywords, agents on top of retrieval. It never gets any simpler. 

Lesson for anyone operating in the search industry: Stop waiting for things to settle down. Being good at SEO is about staying functional, curious, and pragmatic. Stop looking for silver bullets and embrace the chaos.