This article describes a search engine optimization technique known as cloaking. While there are some legitimate uses for this technique, search engines consider cloaking to be an unethical practice, because the technique can be used to artificially boost a site’s rankings in search engines. The use of cloaking, or a technique that is mistaken for cloaking, will likely get a site banned (completely removed) from a search engine database if the technique is detected.
Cloaking is a substitution technique that detects search engine spiders and substitutes a Web page designed especially for that spider. Specific search engine spiders can easily be detected through either the IP address or the HTTP User Agent signature. The substitute pages served to the spiders are frequently loaded with keywords intended to give the real Web page a significant boost in rankings. Special spider pages rarely resemble the actual Web pages that a user would see. The technique is called cloaking because the page a spider sees is a substitution for the real page a user sees, thus the real page is cloaked.
Search engine companies consider cloaking to be a highly unethical practice, and if a cloaking technique is detected, the site will likely be heavily penalized or banned. Once a site has been banned from a search engine’s database, it can be very difficult to get the site reinstated. It is probably the most egregious violation from a search engine’s perspective and is considered to be a black hat SEO technique.
Identifying Cloaking
Most cloaking techniques are identifiable and can be reported to a search engine by a competitor. Because a cloaked page that a search engine sees has different content and is likely to have a different structure from a page a user sees, is easy to identify by comparing the copy of the page that is cached (stored) by the search engine to the page that a user sees when viewing the page. You can view the cached copy of a page in either Google or Yahoo. On the search results page listings for most sites there is a link called "cached". This will display the cached copy, which can be compared to the current page. Be aware that differences may reflect changes to a page, especially if the cached copy is old. But if the cached copy is current and differs significantly from the current Web page, the chances are good that the site employs cloaking techniques.
Google and other search engines continue to improve their methods for detecting true cloaking, but there are several mostly innocent techniques used by designers and developers that can be mistaken for cloaking techniques. These methods can result in a site being penalized.
The methods to avoid
- Any type of JavaScript redirection. In the summer of 2004, Google targeted and banned numerous sites using JavaScript redirects. Most of these sites were customers of a particular search engine optimization company that employed a cloaking technique using JavaScript. To date, only Google has developed the ability to read JavaScript, and Yahoo is currently picking up some JavaScript links.
- 302 redirects from one page to another. A 302 status code indicates a temporary redirection. Most redirection and forwarding techniques generate 302 status codes. These types of redirections can be easily included in ASP, PHP and any server-side dynamic programming language. If you need to perform an automatic redirection from one page to another, it should ideally be a 301 permanent redirect. There are situations where you have to redirect a page due to form validation (verifying user input) and other common e-commerce requirements. These pages are generally safe from penalties because search engine spider cannot fill in or submit forms and therefore will never see the validation page.
- Meta tag redirects. These are a bit of a gray area. If you need to use a meta tag redirect, it’s best to set it up with at least a five second delay. Never set the delay to zero.
The 301 Permanent Redirect
There is only one type of redirection that is recognized as legitimate by almost all search engines. This method is called a 301 Permanent Redirect. The 301 refers to the status code sent to the requester (a spider in this case) through the HTTP header that accompanies each Web page request. A 301 status code tells the search engine algorithm that a permanent URL or page name change has taken place, which should cause the search engine to change the URL currently with the page to the new URL at the redirected location. The 301 redirect is most commonly used when a site owner changes the site’s domain name, but it can also be used with individual pages whose names have changed.
If you need to redirect a user from an old page to a new page, make sure that you set up the redirection properly.
Remember that cloaking is considered to be a very unethical practice that is a black hat SEO technique. It can be very successful in boosting a web site’s rankings with search engines. The problem with techniques like this is that they only work until you get caught. If you get caught using cloaking, your web site may be permanently banned form a search engine. When a site gets banned, sometimes the only remedy is to register another domain name and start all over again. This not something that you want to risk with an e-commerce site.