Duplicate Pages in Online Stores

Thousands of pages in Search Console marked "duplicate" or "crawled, currently not indexed". Let us work through where they come from in stores and what harm they actually do.

The main sources

• Filters. Every combination is a separate URL. Three groups with five options each produce hundreds of pages, near-identical in content.
• Sorting and pagination — the same set of products in a different order.
• One page under several addresses: with and without index.php, with and without a trailing slash, a product under different categories.
• Language versions that are not correctly linked to each other.
• Utility parameters: campaign tags, session identifiers.

What the harm actually is

A widespread myth is the "duplicate content penalty". There is no such penalty. The damage is more specific and more mundane:

1. Crawl budget is spent for nothing. The crawler works through endless filter combinations instead of product pages. New items reach the index weeks late.
2. Link equity is diluted. Instead of one strong category page you have a hundred weak ones, and none of them reaches the top.
3. The wrong page appears in results. Instead of the category, a random filtered view is shown — with an empty or odd selection of products.

What is done about it

Canonical URLs. A filtered page points at the main one, so the signals collect in a single place.

Indexing rules. Filter combinations are closed off from the index. Importantly: blocking crawling and blocking indexing are different things, and confusing them produces the opposite of the intended effect.

Exceptions for valuable combinations. Some filters have genuine demand — "men's winter jackets", for example. Those pages should be opened up instead, and given their own title and description.

Pagination control, so that pages 2..N do not compete with the first.

What to expect

The technical changes are quick; re-indexing takes weeks. There will be no sharp traffic jump the next day — this is clearing obstacles, not adding acceleration.

The hands-on work is the duplicate removal and canonical setup service. If the current state has not been diagnosed yet, start with a technical audit. Indexation of filtered pages is closely tied to filter configuration, and language duplicates to hreflang.