Why Google Crawls But Doesn't Index Your Pages — And What Small Businesses Can Do About It
If you run a small business website, you have probably seen this in Google Search Console: "Crawled – currently not indexed" on a large share of your pages. Not blocked. Not errored. Google just fetched the page, looked at it, and decided not to keep it. We run frontagevalue.com ourselves, so we watch the same numbers — and this status is the most common question small site owners bring to us.
Here's what that status really means, why Google has been indexing less since 2022, and what actually moves pages into the index — including what we do on this site.
What "crawled but not indexed" actually means
"Crawled – currently not indexed" is not an error message. It means Google's crawler fetched the URL successfully, but the indexing system chose not to store it. The page wasn't broken and nothing is blocking it — Google simply decided the page didn't clear its quality bar, or that it wasn't worth the crawl budget.
That decision has gotten dramatically stricter. Back in 2022, Google announced a shift in mission — from indexing the whole web to indexing "all the knowledge on the web". The Helpful Content update that followed was the first big wave of deindexing, and the threshold has kept climbing since. SEOs in a recent r/TechSEO thread described it as an ongoing purge: one site owner with an authority domain registered in 2001 saw his indexed pages drop sharply in the last year; a 20-year-old artist website reports less than half its pages indexed now, while Bing and Yahoo still show nearly everything. You are not alone, and your site is probably not broken.
Why Google is indexing less
Four patterns come up again and again — in the thread, in Google's own statements, and in our checks of other small sites:
1. JavaScript-heavy pages. If your content only appears after JavaScript runs, the crawler may fetch the page, see little useful text, and move on. One site owner in the thread moved from a client-side-rendered app to a server-rendered framework and said indexing "skyrocketed since then."
2. Duplicate-ish or thin content. Template pages, near-identical product descriptions, tag archives and filter combinations all look the same to Google — and now it keeps only the best version of each. If half your pages are variants of each other, the variants lose.
3. Publishing volume. Mass-produced content is the clearest signal of a factory site. SEOs in the thread explicitly warned against churning out hundreds of posts a day — Google treats volume as a smell, and pages from such sites are the first to be culled.
4. Crawl budget. Google spends a limited amount of time on each site. If the crawler wastes that time on low-value URLs, the pages you actually care about can be left waiting — sometimes for weeks.
What actually works — our checklist
There is no guaranteed fix — the thread's most experienced commenter was blunt about that. But the same fixes kept being repeated by people who saw real movement:
1. Serve real HTML. Static or server-rendered pages put your full content in the raw response. This site is built that way — every word of this post is in the HTML before any JavaScript loads. If you're on a JS framework, switching to server rendering or static generation is the single most cited fix.
2. Make every page earn its place. Merge or prune pages that don't offer something unique. A shorter site where every page answers a real question beats a long site full of filler.
3. noindex what shouldn't rank. Tag pages, filter combinations, thank-you pages, internal search results — if you don't want them in results, say so explicitly. That keeps the crawler focused on pages you do want.
4. Publish less, publish better. One strong, genuinely useful post outperforms ten rushed ones — and it's the only volume strategy Google doesn't penalize.
5. Update, link, prove. Refresh old pages, earn internal and external links, and make the trust signals visible — a real about page, a contact page, named humans behind the business (Google's E-E-A-T criteria).
6. Request indexing, then wait. Search Console's "request indexing" button works sometimes — not always, and not instantly. Indexing can take days to weeks. Don't spam the button.
7. Don't depend on Google alone. Every commenter who reported recovery also mentioned alternative channels. We submit every new post to IndexNow, which Bing, Seznam and Yandex pick up in hours — Google ignores it, but other engines respond. A small business can't afford to be hostage to one algorithm.
Reading your own Search Console report
The Page indexing report groups URLs into a handful of statuses. Knowing what each one means tells you whether you have a problem at all:
| Status | What it means | What to do |
|---|---|---|
| Indexed | Google stored the page and can show it in results. | Keep earning links and updates. |
| Crawled – currently not indexed | Google fetched the page but chose not to store it — a quality or crawl-budget decision, not an error. | Improve uniqueness and value; request indexing; consider merging or pruning. |
| Discovered – currently not indexed | Google knows the URL (often from your sitemap) but hasn't crawled it yet. | Check sitemap and internal links; wait, or request indexing. |
| Excluded | Blocked by robots.txt, noindex, or an explicit removal — usually intentional. | Verify it's intentional; remove the rule if you want the page indexed. |
Screenshots and other non-HTML files also appear under Excluded — that's normal.
What we do on this site
We face the same numbers, so we apply the same checklist: static HTML pages, one thoughtful post at a time, no thin or duplicated pages, an about page with real people, and an IndexNow ping on every publish. We don't chase volume, and we don't pretend Google owes us anything.
That's the honest bottom line. Google's index is smaller and pickier than it was five years ago, and no trick guarantees entry. But the pattern is consistent: serve plain HTML, make each page worth keeping, earn links instead of generating pages, and build traffic channels you control. Do that, and the pages that matter will find their way in — on Google when it makes sense, and everywhere else sooner.
Google 检索了你的页面却不收录？——小型商家可以这样应对
如果你经营一个中小企业的网站，很可能在 Google Search Console 里见过这个状态： 「已检索（目前未编入索引）」——出现在相当大比例的页面上。不是被封禁，也不是出错。Google 只是抓取了页面、看了一眼，然后决定不收录。我们自己运营 frontagevalue.com，每天也看着同样的数据——而「已检索但未收录」正是小型网站主最常问我们的问题。
下面说说这个状态到底意味着什么、为什么 Google 从 2022 年起收录得越来越少，以及真正能让页面进入索引的做法——包括我们在这个网站上是怎么做的。
「已检索但未收录」到底是什么意思
「已检索但未收录」不是错误提示。它的意思是：Google 的爬虫成功抓取了网址，但索引系统选择不保存它。页面没有问题、也没有任何东西在阻止抓取——Google 只是认为这个页面没达到它的质量门槛，或者不值得消耗抓取预算。
这个决定的门槛已经明显变高了。2022 年，Google 宣布使命转变——从「索引整个网络」改为索引「网络上的所有知识」。随之而来的 Helpful Content 更新是第一波大规模清理，此后门槛一路提高。在最近的 r/TechSEO 讨论串里，从业者把它形容为持续性的清理：一位拥有 2001 年注册的权威域名、运营 19 年博客的站长，过去一年被收录的页面明显减少；一个运营了 20 年的艺术家网站，现在被收录的页面不到一半，而 Bing 和 Yahoo 仍然几乎全部收录。你不是一个人，你的网站大概率也没坏。
Google 为什么收录得越来越少
在讨论串里、Google 自己的表态中，以及我们检查其他小型网站时，有四个模式反复出现：
1. JavaScript 过重的页面。 如果你的内容要等 JavaScript 运行后才出现，爬虫抓取时可能只看到很少的可用文本，然后离开。讨论串里一位站长把客户端渲染的应用改成服务端渲染框架后，索引量「此后暴增」。
2. 重复或单薄的内容。 模板页面、大同小异的产品描述、标签归档页、筛选组合页——在 Google 眼里它们长得都一样，现在它只保留每个主题最好的那个版本。如果你一半的页面都是彼此的变体，输的是变体。
3. 发布量。 大量生产的内容是「内容工厂」最明显的信号。讨论串里的 SEO 从业者明确警告不要每天批量发布几百篇文章——Google 把数量当作一种风险信号，这类站点的页面最先被清理。
4. 抓取预算。 Google 在每个网站上花的时间是有限的。如果爬虫把时间浪费在低价值 URL 上，你真正在乎的页面可能一直排不上队——有时一等就是几周。
真正有效的做法——我们的检查清单
没有保证有效的办法——讨论串里最有经验的那位留言者也直言不讳。但那些真正看到变化的站长反复提到的做法高度一致：
1. 输出真实的 HTML。 静态生成或服务端渲染的页面会把完整内容放进原始响应里。这个网站就是这么做的——本文每一个字都在任何 JavaScript 加载之前就出现在 HTML 中。如果你在用 JS 框架，改成服务端渲染或静态生成，是被引用最多的修复方法。
2. 让每个页面都值得被收录。 合并或删掉没有独特价值的页面。一个每一页都在回答真实问题的短网站，好过一个塞满凑数内容的网站。
3. 给不该收录的页面加 noindex。 标签页、筛选组合、感谢页、站内搜索结果——如果你不想让它们出现在结果里，就明确说出来。这样爬虫会把精力集中在你想收录的页面上。
4. 少发，发好的。 一篇扎实、真正有用的文章胜过十篇赶工的文章——这也是唯一不会被 Google 惩罚的数量策略。
5. 更新、链接、证明。 定期刷新旧页面、积累内外部链接，并让信任信号可见——真实的关于我们页面、联系页面、有名有姓的人（这就是 Google 的 E-E-A-T 标准）。
6. 手动请求收录，然后等待。 Search Console 的「请求编入索引」按钮有时有效——但不是总是，也不是立刻。收录可能需要几天到几周，不要反复点。
7. 不要把命运押在 Google 一个渠道上。 每个报告恢复的人都会提到替代渠道。我们每发一篇文章都会提交 IndexNow，Bing、Seznam 和 Yandex 几小时内就能收录——Google 不理会它，但其他搜索引擎会响应。小企业不能被一个算法绑架。
读懂你自己的 Search Console 报告
页面编入索引报告把网址分成几种状态。看懂每种状态的含义，才能判断你到底有没有问题：
| 状态 | 含义 | 该怎么做 |
|---|---|---|
| 已编入索引 | Google 已保存该页面，可以在搜索结果中显示。 | 继续积累链接和更新。 |
| 已检索（目前未编入索引） | Google 已抓取页面但选择不保存——这是质量或抓取预算的决定，不是错误。 | 提升内容的独特性和价值；请求收录；考虑合并或删除。 |
| 已发现（目前未编入索引） | Google 知道这个网址（通常来自你的 sitemap），但还没抓取。 | 检查 sitemap 和内部链接；等待，或请求收录。 |
| 已排除 | 被 robots.txt、noindex 或明确移除规则阻止——通常是有意的。 | 确认是否符合预期；想收录就去掉对应规则。 |
截图等非 HTML 文件也会出现在「已排除」里——这是正常的。
我们在这个网站上是怎么做的
我们面对同样的数据，所以执行同一份清单：静态 HTML 页面、一篇一篇认真写、没有单薄或重复的页面、有真实人物介绍的关于页、每次发布都提交 IndexNow。我们不追求数量，也不认为 Google 欠我们什么。
这就是诚实的结论。Google 的索引比五年前更小、更挑剔，没有任何技巧能保证收录。但规律是一致的：输出纯 HTML、让每个页面值得保留、用链接和内容赢得位置而不是批量生成页面、并且建立你自己能掌控的流量渠道。做到这些，重要的页面自然会进来——该进 Google 的时候进 Google，而其他渠道会更快。