HTTP 404 - File not found is a browser error message that we've all grown to know and hate. On a number of search engines, it gets to a stage where we feel that all their links will return this result? How is this happening, who is to blame and how do we fix it?
On reviewing my server logs, I had noticed around 1% of all requests from my site will return this HTTP error code. One of the reasons for it was a stupid mistake I made around 6 months ago. I wasn't happy with the naming of a couple of my files, so I renamed them without considering the consequences:
- The files had been on my site for a few days
- During that time a couple of search engine bots had crawled through the pages. A search engine bot is a software program that scours web sites for content and returns the results to a search engine database. The search engine interface feeds off this to return links and descriptions to surfers when they have entered their search criteria
- Since I changed the names of the files after the bot went through, the pages in their original state no longer "existed".
- The search engine query results reflect the database entries, pointing to the wrong filename, visitor clicks on the result - 404.... aaaaaaaaargh!
It would appear that some search engines only update their results once or twice a year, such is the hugeness of the task of spidering the web these days. Last month, the percentage of requests to my site that resulted in this HTTP error code was 2%, the previous month to that - slightly higher. While I am happy that this is reducing, I am kicking myself that I didn't think things through six months ago - we live and we learn.
404 errors may also be caused through a malformed browser request (user error - wrong URL typed into address bar)
Save yourself the shame and embarassment (and homicidal thoughts directed at you from frustrated potential visitors) - plan your site carefully before you promote to search engines.
If you do or have found yourself in the same situation as me, there is something that you can do about it (dependant upon your hosting service set up). Instead of a visitor being directed to those rather horrible "file not found" pages, you can create custom error pages. Here is an example:
http://www.tamingthebeast.net/aaaargh
The above link is incomplete which triggers a 404 response on my server.
By implementing custom error pages, you have a good chance of retaining the visitor, especially if you include the standard navigation buttons. It also acts as a means of apologising to the visitor for the inconvenience.
It isn't just 404 error messages that you can apply this to. There are a number of error code returns that you could customise, all with the goal of alleviating visitor stress and encouraging them to further explore your site. A listing of http error codes can be viewed here:
http://www.tamingthebeast.net/educat/ errcode.htm
Creating custom error pages:
-You may want to check with your hosting service first before creating custom error pages as certain hosting configurations may not allow you to create custom error pages.
-first design and publish the pages to your web space. You'll only really need to design a couple for the more common errors, for file not found (404) or unauthorised/forbidden (403, 401). Your custom error pages should have a brief summary of what went wrong and an encouragement for the visitor to try again or explore a different area of the site. The best custom error pages are those that match the site's other pages in navigation and layout.
-After publishing the pages, you'll need to edit the.htaccess file in the root directory of your server based web (not your local copy). Use the Edit utility (set to ASCII transfer mode) in your FTP program to view the file. The.htaccess file contains a number of settings to control who can access the contents of a specific directory and how much access they have. It can also be used to create a "URL Redirect".
-If you have a FrontPage based web, be especially careful, as the.htaccess file contains other settings as well.
-If you don't find a.htaccess file, you can create your own, but once again, check with your hosting service first.
-Add the following lines to the end of the file (examples provided as a guideline alter path and file names to point towards your error pages)
ErrorDocument 404 http://blah/blah/404.htm
ErrorDocument 403 http://blah/blah/403.htm
ErrorDocument 401 http://blah/blah/401.htm
Custom error pages are simple to produce, help you to increase your site traffic and encourage better visitor/customer relations.
Michael Bloch
michael@tamingthebeast.net
http://www.tamingthebeast.net
Tutorials, web content and tools, software and community.
Web Marketing, eCommerce & Development solutions.
Copyright information....If you wish to reproduce this article, please acknowledge "Taming the Beast" by including a hyperlink reference to the website (http://www.tamingthebeast.net) & send me an email letting me know. The article must be reproduced in it's entirety & this copyright statement must be included. Thanks. Visit www.tamingthebeast.net to view other great articles FREE for reproduction!
About the Author
Michael is an Australian Information Technologies trainer and web developer. Many other free web design, ecommerce development and Internet articles, tutorials, tools and resources are available from his award winning site; Taming the Beast.net (http://www.tamingthebeast.net)