Programmatic SEO for large catalogues, without the spam risk

7 min read

Programmatic SEO for large catalogues, without the spam risk

7 min read

Programmatic SEO means generating pages from templates and data rather than writing each one by hand: a page for every destination and month, every product type in every size, every town a marketplace serves. For a business with a large catalogue or inventory, it can match thousands of specific searches that no editorial team could cover.

It is also the quickest way to fill an index with thin, near-duplicate pages of the kind Google's spam policies were written for. The difference is not the template. It is whether each page has real demand and something on it that is unique and useful. This article sets out how we decide, build and test programmatic pages, and when not to.

Why it matters commercially

Done well, programmatic pages capture long-tail searches with clear purchase intent: black leather ankle boots in size 5, villas in Menorca with a private pool, used vans under £10,000 in Leeds. Each is small, but together they can add up to a meaningful share of category demand, and the searcher has already narrowed down what they want, so the page only has to show it.

Done badly, the risk is site-wide rather than page-level. Thousands of weak pages absorb crawling, dilute internal links and can cross into spam. Google's spam policies say sites that violate them may rank lower or not appear in results at all, and that violations are found by automated systems and, where needed, human review that can lead to a manual action.

What Google's scaled content abuse policy says

Google added the scaled content abuse policy in March 2024. It defines it as many pages generated for the primary purpose of manipulating search rankings and not helping users, typically large amounts of unoriginal content with little value, no matter how it is created.

The examples Google lists include using generative AI tools to create many pages without adding value, scraping feeds or search results to build pages, stitching together content from other pages without adding value, and pages that make little sense to a reader but contain search keywords. Doorway abuse is a separate policy covering pages created to rank for specific, similar queries, such as pages for many cities that funnel users to one page.

Templates are not on the list. Google's guidance on people-first content asks whether you are using extensive automation to produce content on many topics, and whether you are producing lots of content in the hope that some of it performs. The test is purpose and value, not production method.

Where it tends to work

The businesses with the strongest case share two things: structured data that changes by combination, and searchers who express their needs as combinations.

Marketplaces and aggregators: listings by category and location, where supply, prices and reviews genuinely differ from one town or category to the next.
Travel: destination, property type and feature pages backed by live availability, prices and guest reviews, rather than recycled destination copy.
Large retail catalogues: product type pages by size, colour, material or fit, where the indexable set reflects real stock and real search demand, not every filter the platform can generate.

Generative AI can help with parts of the build, such as drafting short summaries from the underlying data for an editor to check. It cannot supply the thing that makes the page worth indexing. If the data is thin, more generated text makes the page longer, not more useful, and that is the pattern the policy examples describe.

How we run programmatic SEO

We treat a programmatic page set as a product launch with gates, not a content upload. This is the sequence.

Prove the demand pattern. We start with keyword data, not the database. If searchers combine attributes in a consistent pattern (product type plus size, destination plus feature, service plus town) and enough combinations have measurable demand, there is a case. If only a handful do, write those pages by hand.
Prove the unique data. Each page needs information that differs meaningfully from its neighbours and helps the searcher choose: live listings and prices, availability, specifications, reviews, local supply or first-party statistics. If the only thing that changes between two pages is the place name in the heading, they are doorway pages in waiting.
Set a quality threshold for indexing. We define the conditions a page must meet before it can be indexed, such as a minimum number of live products or listings, a minimum number of reviews, or a data point not available on the parent page. Pages below the threshold can still exist for users, but carry noindex and stay out of the sitemap.
Design the template around the decision. Lead with what the searcher needs: what is available, at what price, what others think and how the options compare. Add written guidance only where it genuinely varies by page, and mark the data up with structured data; our guide to product pages and schema for AI answers covers which markup matters.
Control indexing and crawling. Decide which combinations are indexable, give each a single canonical URL, and keep the rest out of the crawl. For filter-based pages, Google's faceted navigation guidance recommends a consistent filter order and a 404 for combinations with no results. Its crawl budget guide, written for sites with over a million unique pages or over 10,000 that change daily, asks owners to remove duplicate content and return 404 or 410 for removed pages, so crawl budget planning is part of the build.
Build internal links that reflect the hierarchy. Programmatic pages should sit in a browseable structure, from category to sub-category to combination, with links between close siblings. One of Google's doorway examples is substantially similar pages that sit closer to search results than to a clearly defined, browseable hierarchy, so internal linking is part of compliance as well as discovery.
Launch a sample, not the catalogue. We release a few hundred pages across a range of demand levels, submit them in their own sitemap, and track indexing, impressions, clicks and engagement for six to eight weeks against a comparable set we held back. If Google declines to index much of the sample, a full rollout would fare no better.
Roll out in waves and prune. Expand in batches only while the indexed share and performance hold. Remove or noindex pages that fall below the threshold, for example when stock runs out or demand disappears.

How to measure it

Programmatic pages need their own reporting segment, so their effect is visible separately from the rest of the site.

Indexed share: the proportion of submitted programmatic URLs that Google has indexed, checked in Search Console for the page set's sitemap.
Clicks and revenue per indexed page, by template and by demand band, compared with the rest of the site.
Crawl activity: whether Googlebot spends its time on the pages that matter, from Search Console crawl stats or server logs.
Cannibalisation: whether the new pages compete with existing category pages for the same queries.

Report upwards in commercial terms: incremental non-branded clicks and revenue from the new page set, against the cost of building and maintaining the data. The same approach we use for SEO forecasting sizes the opportunity before the build is approved, so the result can be judged against a number agreed in advance.

When it does not fit, and the mistakes we see

No unique data. If the pages run on the same feed every competitor and affiliate already uses, they add nothing new.
Location pages for a business that has no presence or difference by location. That is the doorway pattern Google describes.
Using AI to reword the same paragraph on every page so pages look unique. That adds words, not value, and matches the scaled content abuse examples.
Indexing every combination the database can produce, including empty and near-empty ones.
Launching everything at once, so there is no chance to learn what works before the whole index is affected.
Neglecting maintenance. Stale prices, expired listings and out-of-date availability lose users' trust quickly and waste crawling.

Questions to ask your team or agency

What search demand exists for these combinations, and how many have measurable volume?
What is on each page that a searcher could not get from the parent category or a competitor?
What is the indexing threshold, and what happens to pages below it?
How big is the test sample, what will we measure, and what result would stop the rollout?
Who maintains the data, and how often does it refresh?

Our playbooks for marketplaces and aggregators, travel and hospitality and e-commerce cover the page types that tend to earn their place in each sector. We plan, build and test programmatic page sets as part of our organic search work.

Frequently asked questions

What is programmatic SEO?

Programmatic SEO is the practice of generating large numbers of search landing pages from a template and a structured dataset, such as one page per product type and size or per destination and feature. It suits marketplaces, travel sites and large catalogues where searchers combine attributes in predictable ways. It works when each page has real demand and data that differs meaningfully from its neighbours.

Is programmatic SEO against Google's guidelines?

Not in itself. Google's scaled content abuse policy targets many pages generated mainly to manipulate rankings rather than help users, no matter how they are created. Templated pages built on unique, useful data for real searches are not the target. Pages that differ only by a keyword or place name, or that reword the same text at scale, risk breaching the scaled content or doorway abuse policies.

How many programmatic pages should you launch at first?

Start with a sample rather than the full set. We usually launch a few hundred pages across high, medium and low demand, in their own sitemap, and compare indexing, impressions and clicks with a held-back set over six to eight weeks. If Google indexes and ranks the sample, roll out in waves; if it does not, fix the template or data first.

How do you stop programmatic pages being thin content?

Set a threshold each page must meet before it is indexed, such as a minimum number of live listings, reviews or a data point unique to that page, and noindex pages below it. Lead the template with the data the searcher needs to choose, add copy only where it varies by page, and prune pages when stock or demand disappears.