Noindex on Small Blogs: When the Tag Beats robots.txt

Noindex is a “do not show this URL in Search” request after Google can fetch the HTML. It is not a robots.txt Disallow line, and it is not a clever way to hide page 2.

This page is Tag-Then-File. It is not robots.txt for small blogs (the file). It is not pagination SEO. It is not why Google isn’t indexing (the full block ladder). XML sitemaps should list URLs you still want in Search.

Allow the URL to be crawled, add noindex on thin or private-but-public pages you must keep live, and drop those URLs from the sitemap. Do not Disallow crawl if you need Google to see the tag.

Official: Block Search indexing with noindex.

Table of contents

1. Tag-Then-File
2. URLs that deserve noindex
3. URLs that do not
4. WordPress without folklore
5. Verify without folklore
6. FAQ
7. One URL, one decision

Tag-Then-File

        TAG-THEN-FILE
  Need it live for humans?  YES
  Want it in Search?        NO  → noindex + allow crawl
  Want it gone as a URL?    → 301 instead
  Want it uncrawlable?      → robots.txt (different owner)

If the page should not exist, 301 it. noindex keeps the address.

URLs that deserve noindex

On a small blog, typical keep-live / hide-from-Search cases:

• Form thank-you pages
• Internal search result URLs (?s=)
• Thin date archives you still use in the UI
• Staging or “preview” if it leaked to the public web (better: auth lock)
• Attachment pages that only repeat the post

Confirm with View Source: name="robots" content includes noindex.

URLs that do not

• Your actual articles
• Topic hubs you want ranked
• Category page 2 as a blanket rule (pagination)
• Everything “to save crawl budget” on a 200-URL site

Crawl budget anxiety is mostly for giant sites. A small blog’s problem is mixed signals, not Google running out of patience.

Crawled – currently not indexed is a coverage status, not a reason to noindex good URLs.

WordPress without folklore

In Yoast or Rank Math, per-post noindex is a toggle, not a robots.txt edit. Site-wide “discourage search engines” in Settings → Reading is a blunt hammer for development—turn it off on the live host.

Do not add a header noindex and a plugin index on the same URL.

After you noindex, remove the slug from the XML sitemap. Then wait for a crawl. URL Inspection can show whether the tag was seen.

Verify without folklore

Paste the thank-you URL into Search Console Inspection. You want the HTML Google fetched to include noindex. If Inspection still says the URL is indexed a week later, the tag never shipped (cache, wrong template, plugin override) or Google has not recrawled.

Do not Disallow the URL in robots.txt at the same time “to be extra sure.” Extra sure often means Google never reads the tag.

A thin author archive you noindex should still be reachable for humans who click a byline. If nobody should open it, 301 the archive to a real hub instead.

FAQ

What does noindex do?

A robots meta tag (or X-Robots-Tag) with noindex asks search engines not to show that URL in Search results after they crawl it. The page can still be live for humans. Google’s documentation: Block Search indexing with noindex.

Is noindex the same as robots.txt Disallow?

No. robots.txt can discourage crawling. If you block crawl, Google may not see a noindex tag on that URL. To remove a URL from Search while keeping it public, allow crawl and noindex. The robots.txt file owner is a different CashPilot URL.

Should I noindex page 2 of a category?

Not as a default. Pagination should use unique URLs and self-canonicals, not a blanket noindex on every page 2. That walkthrough is the pagination post.

When should a small blog use noindex?

Thank-you pages, pure search-result URLs, thin author archives you cannot redirect, staging if it is accidentally public, or thank-you screens after a form. Do not noindex your best articles to “save crawl budget.”

Does noindex help rankings of other pages?

It keeps junk out of Search. It is not a ranking booster for the rest of the site. Google does not publish a crawl-budget gift for noindexing your blog.

How do I noindex in WordPress?

Most SEO plugins have a “allow search engines / noindex” toggle per post, page, or archive type. Confirm with View Source: robots meta containing noindex. Labels vary by plugin version.

Can I noindex and also put the URL in the XML sitemap?

Do not. Sitemaps should list URLs you want indexed. Mixing noindex and sitemap inclusion sends a mixed signal. Sitemap hygiene is the XML sitemap owner.

Where is Google’s official noindex help?

Block Search indexing with noindex. Pair it with robots.txt docs only after you understand crawl vs index.

One URL, one decision

Pick a thank-you page. Toggle noindex. Confirm source. Remove it from the sitemap. Leave your real posts indexed.