# CarpForum.co.uk - robots.txt
# Public forum content (forums, topics, members, articles) is open to crawl.
# Account, PM, moderation and one-shot action pages are blocked - no content to index.
# /Shared/ is deliberately NOT blocked: it holds the 301 redirect stubs that pass
# ~20 years of old Google-indexed URLs onto their new-site equivalents.
#
# PATHS HERE ARE CASE-SENSITIVE. Site links were standardised to capitalised filenames
# 2026-09-20 (/Profile.asp, /LogIn.asp, ...) so these rules match what's actually linked.
# Before that, "Disallow: /Profile.asp" sat here while every link said "/profile.asp", so
# the rule had never matched anything. If a lowercase form is known to be indexed already,
# it needs its OWN line - hence /settings.asp below alongside /Settings.asp.
#
# Profile.asp is deliberately NOT disallowed, even though it has nothing for a crawler.
# It blocks guests, so a crawler only ever got a contentless 200 - 13,718 URLs were sitting
# in Search Console as "Soft 404". Disallowing them would NOT remove them: a page Google
# is forbidden to fetch is a page whose noindex it can never read. So Profile.asp now sends
# <meta name="robots" content="noindex"> to guests (see Profile.asp / Includes/CanonicalURL.asp)
# and stays crawlable until Google has actually dropped them.
# ONCE THAT COUNT IS NEAR ZERO, add "Disallow: /Profile.asp" back here to stop the crawling.

User-agent: *
Allow: /

Disallow: /Admin/
Disallow: /Admin.NET/
Disallow: /LogIn.asp
Disallow: /LogOut.asp
Disallow: /Settings.asp
Disallow: /settings.asp
Disallow: /ProfileEDIT.asp
Disallow: /PMInbox.asp
Disallow: /PMTopic.asp
Disallow: /PMTopicCREATE.asp
Disallow: /PMTopicREAD.asp
Disallow: /PMUnreadCOUNT.asp
Disallow: /TopicMANAGER.asp
Disallow: /TopicCREATE.asp
Disallow: /TopicEDIT.asp
Disallow: /MessageEDIT.asp
Disallow: /MessageDELETE.asp
Disallow: /MessageLIKE.asp
Disallow: /MessagePOST.asp
Disallow: /TopicREADNEXT.asp
Disallow: /PasswordReminder.asp
Disallow: /Search.asp
Disallow: /EmailTest.asp
Disallow: /PromoGO.asp
Disallow: /PromoSTATE.asp

Sitemap: https://www.carpforum.co.uk/Sitemap.asp