Check for 404s and 500s, before they ruin your SEO
Posted by multippt
Most new browsers have a habit - they will not announce error pages. That’s all as a side-effect of people wanting to have a custom “file not found” error message. While this is not a big problem, it’s always a good idea to check for such “page” errors that may affect your search engine optimization.
How search engines treat 404s
Search engines like Google have to deal with missing pages, and they identify such pages via that famous 404 error code (i.e. file not found). So, how does Google treat missing pages? It will remove missing pages from its index. Not a good thing if your “error page” turns out to be a working page that was wrongly producing the error.
This usually affects PHP scripts, and poorly made .htaccess files. So, if you are a person who uses plain HTML and don’t use anything server-related (which almost 80% of everyone is using), then you need not ever worry about this problem. Though, I’m not implying that everyone should downgrade from PHP to HTML, it’s just that PHP has that lovely ability to spoof error codes, hence the error messages.
Similarly, web-pages may spurt out a similar error - 403. This error means that the page is forbidden from being viewed, pretty common for files given the wrong permissions. Like 404, Google will not index such pages. Of course, think using error codes for stopping search engines (you got the robots.txt file for that) - they are meant to be used for… errors.
How search engines treat 50xs
Google is pretty lenient when it comes to error 50x. Error 50x messages indicate that there is something wrong with your hosting. If something goes wrong on the server-side, you get 50x. Google will still check on your site, but it will slow down on indexing your site (after all, a site with a malfunctioning server ain’t going to update much right?).
Why aren’t browsers showing the errors?
It’s a sad thing, but yes, web browsers will not indicate error codes to you (they are not meant to be “seen” anyway). So, if you have a main page that gives error 501, you will not know that it got that problem. In a bid to satisfy everyone’s desire for a customized error page, all browsers will display the page as “normal” regardless of the error given out (except for 30x errors which means redirection).
Detecting errors
Google webmaster has a nice way of telling you it can’t crawl/index something. Though, it has a crude way of telling you that. Fortunately, it’s just sufficient enough if you want to do a little check up on the page if it is really a missing page. Unfortunately, if it is a genuine missing page, you’ll have to go link-hunting for that broken link (not the scope of this article, but you can always use a sitemap generator to do that).
Why do errors ruin SEO
Errors are errors. People shouldn’t be seeing them at all. Similarly, you will not want search engines to face errors like that as well. An error is treated differently by all search engines, but the ultimate result is that the page producing an error will be given a “penalty”, well sort of. Most problematic pages are rarely indexed, let alone appearing in search results. Remember that nice Google bomb that featured a “page not found” page appearing for “Weapons of mass destruction”? Google has fixed that problem by simply removing error pages from the index.
—
Ideally, your site should not have a single error response (unless the visitor deliberately stumbles into the unknown). If it does, fix it, otherwise you may have a few missing pages that will not appear in search engines.
