There are too many website owners displaying their email address in the “name [at] tld [dot] com” format, or something like that. Whatever happened to the good old mailto link with the email address being displayed how it’s supposed to? I’ll tell you what happened — spam happened.

Many site owners are afraid of displaying their email address, as it should be displayed, on their site. Since spammers can use scripts to scan websites, the same way google does, to look for email addresses. I’ve been seeing more and more contact forms and weirdly formatted addresses to combat the issue, but there’s a better way.

The robots.txt file in the root of your website is where you control what these bots can do and what they can’t. So if you’ve got your contact info on a certain page, then don’t allow bots to view that page. Bingo.

Now, the robots.txt file isn’t a standard and there’s nothing that guarantees that spam bots will follow the rules, but we can take this a step further.

As described over on the spamcop forums you can create  bot trap. Basically, you create a script that redirects to a 403 page. On the redirect page you can output anything: random email addresses, addresses that you want to monitor for spam, etc. Then you make sure legit bots that follow the rules of your robots.txt file don’t get caught in the trap by putting a “Disallow: <your bot trap>” in the robots.txt file. To make sure the bad bots get caught in the trap you put an anchor link inside an HTML comment as the first and last lines of each page. When the bad bots see the link they will follow it and then they get the 403 error.

This isn’t bulletproof, but I think it helps just to disallow your contact page from being monitored by legitimate bots like Google. I don’t think you want your contact page showing up in Google cached pages or Google search results anyway.

Resources: The Web Robots Pages (How to use robots.txt)