How to Prevent Directory Listing

I am finding that an increasing number of hosting companies appear to be configuring their servers to allow the file listings of web site directories to be viewed by users. I don’t know why they are doing this. It can be a serious breach of security that may allow a hacker to gain information about a site that can be used to hack the site. It is always a good idea to take steps to prevent the listing of directory files.

To start, let’s define the problem so that you can verify whether or not a site is configured to prevent directory listing. You can test this by entering the domain name and a directory name for any directory that exists in your site. For example, if you have a directory named “images”, and you enter your full domain name and the the directory name, like the following:

http://www.domainname.com/images/

Doing this should generate a browser error page or perhaps will be captured by your site’s error page. However, many servers are configured to simply list the contents of the directory, such as in the following example for a directory called “confidential”.

First of all, confidential information such as the documents listed above should never be placed on a web site. It is very difficult to protect documents like this if someone happens to stumble upon their location, or a hacker finds the information due to weak web site security. I define the issue of being able to list directory contents as one of weak web site security.

There are advanced methods, such as password protecting directories and web pages that should be used if you need to prevent users from entering some sections of a site or if you wish to protect confidential information. This article simply addresses the issue of how to prevent directory listings, which can allow hackers to gain easy access to areas of your web site that are essentially unprotected.

Part of the methodology for securing a web site includes using file and directory names that are not easy for someone to figure out. When the contents of web site directories can easily be listed and viewed, that level of security is removed.

If your site is on a Unix or Linux server, and your hosting company allows server overrides through the site’s .htaccess file, you can add the following to the .htaccess file.

IndexIgnore *

This directive tells the server to ignore requests to display the file index for a directory. Some hosting companies allow this method to be used, while others do not. Be very careful when making any changes to a site’s .htaccess file. If a directive is not allowed or it creates a conflict or there is a typo in the directive, your site could crash with a server error 500. Before you make any changes to a site’s .htaccess file, make sure that you have a backup copy of the original file that you can quickly copy back to the site if you run into an error.

There is another method that is commonly used to prevent a directory listing and when used properly does not generate errors and can be used on any web site. Why does this work? All web servers are configured to display a default page for a directory if a default file exists. That is how your home page is found when someone simply enters a domain name for the URL for a web site and the home page is displayed. A server is configured to search a list of default file names and if it finds a match, it displays the page.

The default file name list for a typical Linux server looks like this:

index.htm
index.html
index.php

Microsoft servers add Microsoft-specific default names, such as default.htm, default.html, default.asp, etc.

If no such file exists on a server configured to prevent listings an error is normally generated. But when a server is configured to allow a directory listing, the directory index is displayed rather than generating an error.

The method used to prevent directory index from being displayed is to provide a default file that will be displayed. This can be used in any directory on a web site except those directories that already utilize a default file. This includes the root directory. Never place one of these files in the root directory, never overwrite a default file that already exists and never place a default file in a directory where another default file already exists.

The best way to determine if you need to protect a directory is to simply use the method shown at the top of this tutorial to see if you can list the file contents of a directory. If you can list the files, cut-and-paste the following code into a pure text editor such as Notepad and save the file using one of the default file names. I typically use index.html for the default file name.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>403 Forbidden</title>
<meta name="robots" content="noindex,follow">
<meta name="robots" content="noarchive">
</head>
<body>
<center>
<br><br><br><br><br>
<h1>Forbidden</h1>
You do not have permission to access this directory.
<br><br>
<a href="/">Click here</a> to return to the Home Page.
</center>
</body>
</html>

Aside from displaying this file as the default page for a directory, the file accomplishes a few other things. First, the two meta tags will prevent the page from getting indexed by search engines if they happen to find the page. Second, it provides a path to your home page. This can be used by both human users and spiders.

Note: Whenever you create a web page it is important to use a pure text editor such as Notepad or an editor designed to create web pages. Never use Word or a word processor to create web pages. The files that word processors create contain formatting codes and other invisible information that can create problems with web servers.