301 redirect on IIS To Fix Domain Issues In Windows

Redirecting Requests to Files, Directories, or Programs (IIS 6.0)
You can redirect requests for files in one directory to a different directory, to a different Web site, or to a file in a different directory. When the browser requests the file from the original URL, the Web server instructs the browser to request the page from the redirected URL.

An advanced capability of the redirect method allows you to redirect all requests for files in a particular directory to a program. Generally speaking, you should also pass any parameters from the original URL to the program, which you can do by using redirect variables.

iis redirect Important

You must be a member of the Administrators group on the local computer to perform the following procedure or procedures. As a security best practice, log on to your computer by using an account that is not in the Administrators group, and then use the runas command to run IIS Manager as an administrator. At a command prompt, type runas /User:Administrative_AccountName“mmc %systemroot%\system32\inetsrv\iis.msc”.

>Procedures

To redirect requests to another directory or Web site

1.

In IIS Manager, expand the local computer, right-click the Web site, virtual directory, or directory within a Web site to which you want to add a redirect, and click Properties.

2.

Click the Home Directory, Virtual Directory, or Directory tab.

3.

Under The content for this resource should come from, click A redirection to a URL.

4.

In the Redirect tobox, type the path to the destination directory or the URL of the Web site. For example, to redirect all requests for files in the /Catalog directory to the /NewCatalog directory on another Web site, type http://sitename/NewCatalog. The redirect must be to a fully qualified URL.

5.

Set the required flags as specified below, and then click OK. More than one flag is allowed.

To redirect all requests to a single file

1. In IIS Manager, right-click the Web site or directory, and click Properties.

2.

Click the Home Directory, Virtual Directory, or Directory tab.
3. Under The content for this resource should come from, click A redirection to a URL.
4. In the Redirect to box, type the URL of the destination file. The redirect must be to a fully qualified URL.
5. Under The client will be sent to, select The exact URL entered above check box to prevent the Web server from appending the original file name to the destination URL.

6.

Click OK.

You can use wildcards and redirect variables in the destination URL to precisely control how the original URL is translated into the destination URL.

To redirect requests to a program

1. In IIS Manager, right-click the Web site or directory, and click Properties.
2. Click the Home Directory, Virtual Directory, or Directory tab.
3. Under The content for this resource should come from, click A redirection to a URL.
4. In the Redirect to box, type the URL of the program, including any redirect variables needed to pass parameters to the program.For example, to redirect all requests for scripts in a Scripts directory to a logging program that records the requested URL and any parameters passed with the URL, type /Scripts/Logger.exe?URL=$V+PARAMS=$P. $V and $P are redirect variables.
5. Under The client will be sent to, select The exact URL entered above check box to prevent the Web server from appending the original file name to the destination URL.
6. Click OK.

Redirect Reference (IIS 6.0)

Redirecting a client request is one way to ensure that users get the correct page, if your site is under construction or has changed identity. You can redirect client requests to a directory on the same Web server or to a different URL.


H3>Configuring URL and Wildcard Redirection

You can configure redirection to a URL and wildcard redirection programmatically. Redirection variables and flags allow you to specify targets and behavior by changing the HttpRedirectmetabase property.

The following example shows the format of a simple redirection string.

http://DestinationURL[,Flag[s]]

Where DestinationURL can include the following redirect variables to pass portions of the original URL with the destination URL. You can use more than one variable in a single redirection string.

Variable Description Example

$S

Passes the suffix of the requested URL to the new URL. The suffix is the portion of the original URL that remains after the redirected URL is substituted.

If the EXACT_DESTINATION flag is not set, the resulting destination URL will have the name of the requested file appended as a folder name, as well as the file name itself.

If the virtual directory that is mapped to the following Web site:

http://fabrikam/Scripts

is configured to be redirected to the following exact URL:

http://fabrikam/NewScripts$S

A new request for the following URL:

http://fabrikam/Scripts/Program.exe

Would be redirected automatically to the following URL:

http://fabrikam/NewScripts/Program.exe

$P

Passes parameters (such as querystring parameters) in the original URL to the new URL, without the question mark (?).

If the virtual directory that is mapped to the following Web site:

http://fabrikam/Scripts

is configured to be redirected to the following URL:

http://fabrikam/NewScripts$P

A new request for the following URL:

http://fabrikam/Scripts/File.asp?var1=5&var2=6

Would be redirected automatically to the following URL:

http://fabrikam/NewScriptsvar1=5&var2=6

$Q

Passes parameters (such as querystring parameters) in the original URL to the new URL, including the question mark (?).

If the virtual directory that is mapped to the following Web site:

http://fabrikam/Scripts

is configured to be redirected to the following exact URL:

http://fabrikam/NewScripts$S$Q

A new request for the following URL:

http://fabrikam/Scripts/File.asp?var1=5&var2=6

Would be redirected automatically to the following URL:

http://fabrikam/NewScripts/File.asp?var1=5&var2=6

$V

Passes the requested URL, without the server name and without any parameters. To include parameters, use the $P or $Q variable with the $V variable.

If the EXACT_DESTINATION flag is not set, the resulting destination URL will have the name of the requested file appended as a folder name, as well as the file name itself.

If the virtual directory that is mapped to the following Web site:

http://fabrikam/Scripts

is configured to be redirected to the following exact URL:

http://contoso$V

A new request for the following URL:

http://fabrikam/Scripts/File.asp

Would be redirected automatically to the following URL:

http://contoso/Scripts/File.asp

Redirection Using Wildcards

You can use redirect wildcards to match any number of characters in the original URL. Begin the destination URL with an asterisk (*) and a semicolon (;), and separate pairs of wildcard characters and destination URLs with a semicolon. The following example shows the format of a redirection string that uses wildcards.

*; Wildcard1; Destination1[; Wildcard2; Destination2][, Flags]

Where WildcardN can include an asterisk (*) to match any number of characters in the original URL and pass them to DestinationN by using the $0 through $9 variables defined in the following table.

“Variable” “Description” “Example”
$0 through $9 Passes the portion of the requested URL that matches the indicted wildcard character. Wildcards must start at $0 for each Wildcard;Destination set. The EXACT_DESTINATION flag must be appended to the redirection string. If the virtual directory that is mapped to the following Web site:http://fabrikam/Scriptsis configured with the following redirection string:*; /Scripts/*_in.asp; /NewScripts/$0.dll; /Scripts/*_out.asp; http://contoso/NewScripts/$0.dllA new request for the following URLs:http://fabrikam/Scripts/data_in.asphttp://fabrikam/Scripts/data_out.aspWould be redirected automatically to the following respective URLs:http://fabrikam/NewScripts/data.dll

http://contoso/NewScripts/data.dll

!

Does not allow a redirect.

This variable is useful when you want to exempt files or subdirectories from redirection when a parent directory is configured to be redirected.

If the virtual directory that is mapped to the following Web site:

http://fabrikam/Scripts

is configured to be redirected, but the following Web site:

http://fabrikam/Scripts/Commerce/Accounts

needs to be exempt, programmatically set the redirection string for the exempted folder to the following

*;!

For example, to redirect all requests for /Scripts/Filename.stm to a single file called Default.stm, and to redirect all requests for /Scripts /Filename.htm to a single file called Default.htm, use the following redirect request for the /Scripts virtual directory:

*;Filename.stm;/Default.stm/Filename.htm;/Default.htm

Using Redirection Flags

The following flags augment redirect strings of either URL or wildcard format. More than one flag is allowed. Flags are appended programmatically to the redirection string in HttpRedirect, or configured by selecting check boxes in IIS Manager.

Flag User Interface Check Box Description
EXACT_DESTINATION The exact URL entered above Indicates that the value provided for the destination should be considered an absolute target location.
CHILD_ONLY A directory below this one Alerts IIS that redirection should occur only once because the destination is in a subdirectory of the original URL. This flag avoids loops. Also, this flag instructs IIS to redirect requests only if they are to subfolders or files of the original URL.
PERMANENT A permanent redirection for this resource Indicates that this redirection is permanent for this resource.
TEMPORARY Clear the A permanent redirection for this resource check box” Indicates that this redirection is temporary for this resource.

How to Move Pages on Windows Servers

Most Windows servers will be running Microsoft’s Internet Information Server (IIS), and pages can be moved using ASP or ASP.NET code, depending on what the server is running (ask your web host). To move a page using ASP, set up the page at its new location then place the following code at the top of the old page:

Move a page using ASP
<%@ Language=VBScript %>
<%
Response.Status=”301 Moved Permanently”
Response.AddHeader “Location”, “http://www.domain.com/new-page.asp”
%>

Make sure that code is the first thing at the top of the page. As for the rest of the page, you can either remove its content or leave it—search engines won’t care one way or another. However, some of your human visitors (remember them?) will have auto-redirection turned off in their browsers.

This means they won’t be redirected, and will see the old page instead of the new one. So you may want to include a link to the new page, along with a short message telling them that the page they’re on is outdated and has been moved to a new location.

Similarly, you can move a page using ASP.NET, but beware there’s currently a glitch in the way ASP.NET 2 handles redirects. That glitch should be fixed soon. To redirect a page using ASP.NET, set up the new page at its new location, then place the following code at the top of the old page:
<script runat=”server”>
private void Page_Load(object sender, System.EventArgs e)
{
Response.Status = “301 Moved Permanently”;
Response.AddHeader(“Location”,”http://www.domain.com/new-page.asp”);
}
</script>

For both of these redirects leave the old pages in place for as long as possible, since it usually takes several months for search engines to switch over to only crawling the page at its new location. They’re a bit slow that way.

Get Found Now – a division of: On Top Of It, Inc

How to Use 301 Redirects For SEO On All Server Platforms

This article will show you how to use the 301 redirect to PERMANENTLY move anything from an entire website to another domain, or a single web page’s URL from on to another. You can move a URL from inside of the same site between directories or sub domains as well as between 2 completely different domain names. This is very handy for SEO and as Google has recently stated, it will also move Page Rank and placement if you use the redirect 301 command correctly.

APACHE Webserver: 301 Redirects

What’s a 301 Redirect?
The number 301 identifies a specific 3-digit server header code that is generated by a web server when a browser or search engine spider tries to download a web page. The 301 code says the page has been permanently moved to another location. Browsers and spiders then know to “redirect” themselves to any new address that you specify in your .htaccess file.

What’s a .htaccess File?

A .htaccess (pronounced “dot-H-T-access”) file is a simple text file that resides in the root directory of your web site where, typically, your home page also resides. The .htaccess file serves a variety of different functions and can live in other directories as well but, for the purpose of doing a 301 redirect, you should place a .htaccess file in your root directory. If one doesn’t already exist, you can create it using any plain-text editor (like NotePad, or TextPad), then name the file .htaccess (that’s right, no extension – just a period followed by htaccess).

If your site’s root domain already contains a .htaccess file, you can simply scroll to the end of whatever code is already there and append your 301 redirect code at the bottom of the file, starting on a new line. While this may sound complicated, it’s actually very, very simple to do. If you can upload files to and from your web server, then chances are you’ll have no trouble managing (i.e. altering or creating and uploading) your .htaccess file(s).

There are Four Primary Reasons to Use a 301 Redirect:

  1. To change primary domain names an event that commonly occurs whenever a company changes names or switches to a more keyword-friendly domain.
  2. To ensure that SE spiders only index either the www or non-www version of your site. Since each version is technically a different URL, to have both versions available for indexing dilutes PageRank, as it is being split between multiple sites.
  3. To catch traffic from ancillary domain sources without presenting duplicate content to the engines and running the risk of being penalized.For instance, it’s good online strategy to protect your trademark or other proprietary product or service names by purchasing related, but different, top-level domain names. Examples would include .net or country-specific extensions (such as .co.uk), or even common misspellings. And, whenever such domains lend themselves to type-in traffic, you’d be well advised to activate them – but use the 301 redirect to seamlessly send that traffic from these ancillary domains to your primary domain.
  4. To prevent any of your ancillary domains from being displayed in the search results in lieu of your primary domain.

Changing Your Domain Name

If ever you should need to change your domain name, proceed as follows:

  1. First, don’t do it unless absolutely necessary. Chances are very good you will lose ranking, inbound links, and traffic at least temporarily. The steps outlined here can help you minimize, but not entirely eliminate, these negative repercussions.
  2. If you are, however, convinced that you must change your domain name, be prepared to leave your old site up for at least three months and possibly longer depending on the length of time it takes for your site logs to show all traffic to the old site has stopped. Of course, you’ll also want to be certain that all major engines who had indexed your old site have spidered and re-indexed your new site.Your new site logs will show when you are no longer receiving referral traffic from the old site. Your old site logs will show when you are no longer receiving referral traffic from the search engines.
  3. When making the switch, be sure to leave the content up on the old site. You’ll want to copy the old site to its new location, as opposed to just moving the site to a new location. The difference is that when you copy a site, you leave the original in its old location. If you take the old site’s content down and just replace it with a 301 redirect, the spiders may see the old pages as empty content and simply delete those pages from their index.While it’s true that you run a small risk of being snagged by the duplicate content filter, it’s a minor risk compared to the risk of having your site dropped entirely out of the listings. Besides, since both Google and Yahoo recommend the 301 as the preferred method for permanently redirecting to a new site, you’ve given yourself a defendable reason for the duplicate content to co-exist at least until your logs show that traffic to your old site has ceased.Also bear in mind that the duplicate content penalty is usually applied to only one of any two identical pages, so people should still be able to find your site perhaps just not under your preferred domain at first. Still, that’s much better than being dropped altogether if the 301 redirect doesn’t work due to a search engine glitch beyond your control.Critical point: If the content of your old site appears gone to the indexing spider, then you’re out of the listings no traffic equals no money and it’s cheap beer and hot dogs for you and your friends come Superbowl-party-time (ouch!).
  4. Contact everyone who links to you and ask them (beg them, pay them?), to change their link to your new site. Monitor this process to make sure they follow through. Sure, this can be a headache if you have a lot of links but, if people don’t change them, all those links won’t do you much good and they’ll all be broken links when you finally take the old site down (a fact you might want to use as leverage while gently persuading the referral site owners to make the necessary link changes after all, they probably don’t want “broken links” on their site). Unfortunately, there isn’t an effective shortcut to getting other sites to update their links to you. That’s just one of the reasons why we say don’t change your domain name unless it’s absolutely necessary. The more incoming links you have, the more challenging it is to switch to a new domain name.
  5. Once you’re sure you want to make the change, use a 301 redirect. On an Apache server (running UNIX) a 301 redirect is very easy to do. Just append the following statement on a separate line at the end of your .htaccess file:
    redirect 301 / http://www.newdomain.com/
    By placing the above statement in the .htaccess file within your website’s root directory, you’ll be redirecting all traffic from the old site to whatever site you specify in http://www.newdomain.com/.

    Important to note: This simple one-line command only works if you are maintaining the same exact structure of the old site on the new site. Every page on the old domain will be redirected to its equivalent page on the new domain. However, if you have some pages on the old domain that are not going to be on the new domain, this single line command would redirect users and spiders to some non-existent pages… the ones you removed. Therefore, bear in mind that your site move will be easiest on you if you just copy the new site structure exactly the same as the old site.

    Of course, once you’ve established your site under the new domain name and have phased out the old domain name, you can then start rearranging the site structure. In other words, don’t try to change your domain name and restructure your site at the same time unless you’re either a professional or a masochist.

    If you only want to redirect portions of the old site to the new site, such as a single directory, you can place the following text within a single line in the .htaccess file:

    redirect 301 /directoryname/ http://www.newdomain.com/directoryname/

    This is useful if you want to test out the redirect on part of your site before redirecting the whole thing.

    You can also redirect individual pages:
    redirect 301 /directoryname/pagename.html http://www.newdomain.com/directoryname/pagename.htmlRemember: the command above needs to all be on a single line in order to work.

  6. If you’re listed in directories such as DMOZ.org (aka, the Open Directory Project [ODP]), then changing your domain name presents an additional challenge. It’s not so much that changing domain names within DMOZ is a problem, it has more to do with the fact that many other directories, including the Google directory, get their directory listings from DMOZ. That means that, depending on how frequently they sync up their listings with DMOZ’s, it could be quite a while before those other directories list your new domain. As we’ve suggested, moving a domain can hurt your rankings and the lack of directory listings for the new domain (resulting in a temporary link popularity deficit) is one of the reasons why this is true.Regardless, to change your domain name in DMOZ, go to the category in which you are listed and click on ‘Update URL’ at the top right. You’ll be asked to enter both your old and new URLs. A DMOZ editor will then review your site and change your listing. Nearly all directories have an Update URL feature, so that’s where to look for a starting point in updating your domain information within the various directories.By the way, don’t attempt to list your new domain along with your old domain within a directory. They strongly object to duplicate listings and your new domain definitely won’t make it past the directory’s editors.
  7. Once you’ve set up your redirect, use a server header checking tool (such as: Check Headers) to verify the server is correctly processing your redirect. You’ll want the site you’re redirecting from to return a 301 Moved Permanently header and the site you are redirecting to to return a 200 OK server header.Here’s a sample 301 redirect to give you an idea of what a server header checking tool will reveal when using a 301 to consolidate www and non-www URL’s, giving preference to the www version (notice the relevant portion of the response highlighted in yellow).Move a page using ColdFusion<cfheader statuscode=”301″ statustext=”Moved permanently”>
    <cfheader name=”Location” value=”http://www.domain.com/new-page.html”>

    Move a page using Java Server Pages (JSP)

    <%
    response.setStatus(301);
    response.setHeader(“Location”, “http://www.domain.com/new-page.html”);
    response.setHeader(“Connection”, “close”);
    %>
    If You’re On a Windows Server

    The examples below all are specific to Unix- or Linux-based web servers. It’s also possible to use redirects on a Windows-based web server. This is easiest if you use scripting in a .asp page (which is one good argument for using .asp for all the documents on your site, even if you don’t put scripting in them).

    Here’s an example of a .asp 301 redirect (save to yourfilename.asp):

    <%@ Language=VBScript %>
    <%
    Response.Status = “301 Moved Permanently”
    Response.addheader “Location”, “http://www.newdomain.com/newurl/”
    Response.End
    %>

    How to Change Domain Names on Windows Servers

    Since the code outlined above must be put on each page, you don’t have the ability to bulk redirect whole directories with this technique the way you do with Apache. However, if you have Admin access to your IIS server, you can easily redirect whole directories or even whole sites using the admin interface. That process is laid out step-by-step on the Microsoft website.

    1. In internet services manager, right click on the file or folder you wish to redirect.
    2. Select the radio titled “a redirection to a URL”.
    3. Enter the page that the page will be redirected to
    4. Check “The exact url entered above” and the “A permanent redirection for this resource”.
    5. Click on ‘Apply’.

    How to Fix the www vs. non-www Problem on Windows Servers

    The Microsoft tutorial above is also helpful when it comes to fixing the www vs non-www issue on Windows servers. It all has to be done through the admin interface.

    Essentially, it’s the same as changing a domain name except you’re redirecting from http://domain.com/ to http://www.domain.com/.

    The .NET tutorial site xoc.net has a good short tutorial.

    To perform the redirect, suppose that the content for www.xoc.net comes from c:inetpubwww-xoc-net:

    1. Open Internet Services Manager and create a web site for xoc.net. Make the content directory for the site be the same directory, c:inetpubwww-xoc-net, as www.xoc.net.
    2. Test getting content from http://xoc.net. This should work, but won’t redirect yet.
    3. Select the xoc.net web site in Internet Services manager and enter the properties.
    4. In the Home Directory tab, change the option button “When connecting to this resource the content should come from” to “A redirection to a URL”.
    5. Specify the URL as http://www.xoc.net.
    6. Check the checkbox that says “A permanent redirection for this resource.”

    Similarly, you can move a page using ASP.NET, but beware there’s currently a glitch in the way ASP.NET 2 handles redirects. That glitch should be fixed soon. To redirect a page using ASP.NET, set up the new page at its new location, then place the following code at the top of the old page:

    <script runat=”server”>
    private void Page_Load(object sender, System.EventArgs e)
    {
    Response.Status = “301 Moved Permanently”;
    Response.AddHeader(“Location”,”http://www.domain.com/new-page.asp”);
    }
    </script>

    #1 Server Response: http://sampledomain.com/
    HTTP Status Code: HTTP/1.1 301 Moved Permanently
    Date: Wed, 15 Dec 2004 23:09:18 GMT
    Server: Apache/1.3.29
    Location: http://www.sampledomain.com/
    Connection: close
    Content-Type: text/html; charset=iso-8859-1
    Redirect Target: http://www.sampledomain.com/
    #2 Server Response: http://www.sampledomain.com/
    HTTP Status Code: HTTP/1.1 200 OK
    Date: Wed, 15 Dec 2004 23:09:23 GMT
    Server: Apache/1.3.29
    X-Powered-By: PHP/4.3.8
    Set-Cookie: PHPSESSID=be8ce4fee0c29c968c8382be1fdfa3bd; path=/
    Expires: Thurs, 16 Dec 2004 08:52:00 GMT
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    Pragma: no-cache
    Connection: close
    Content-Type: text/html

    From the example above, we see that, when the spider initially requested the page, the server read the .htaccess file which told it to respond with a 301 Moved Permanently response code. After which the spider redirected itself to the new domain at http://www.sampledomain.com/. There the spider received a 200 OK response code, telling it that the page was successfully located. This is the combination of responses you’ll be looking for when you setup your own 301 redirection.

    Most web hosts pre-configure their servers to point both the www and non-www version of your URLs to the same page by default. This is good because someone forgetting to type the www will still be directed to your site rather than getting a 404 not found response code (one of those other 3-digit server codes).

    However, pre-configuring the server to default all pages to the www version can also be a liability. That’s because this allows people to link to your site with or without using www. The end result being that search engines may index some of your pages with the www, and some of your pages without. This effectively splits your site into two different sites in the eyes of the search engines and dilutes your incoming link popularity (think PageRank).

    Ideally, you’ll want all of your listings and links to use one or the other (we mildly prefer the www but consistency is really the only important factor).

    Worth noting is that both Google and Yahoo use a redirect to consolidate their traffic to the www version of their URL. For example, entering http://google.com/ will redirect you to http://www.google.com/.

    To ensure that search engines only index the www versions of your site’s pages, place the following code into a .htaccess file in your root directory
    <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /
    RewriteCond %{HTTP_HOST} !^www.your-site.com [NC]
    RewriteRule ^(.*)$ http://www.your-site.com/$1 [L,R=301]
    </IfModule>
    Remember to replace www.your-site.com with the www version of your domain name.

    To make sure search engines only index the non-www versions of your pages, use the following:
    <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /
    RewriteCond %{HTTP_HOST} !^your-site.com [NC]
    RewriteRule ^(.*)$ http://your-site.com/$1 [L,R=301]
    </IfModule>
    Of course, replace your-site.com with the non-www version of your domain name.

    In case you’re wondering why the code used to merge your www and non-www pages is a bit more complicated than the code used to change your domain name, it’s because the www and non-www versions of your site both point to the same place on the server (they are considered aliases of each other). This means that anything you put in your .htaccess file applies to both types of pages. If you were to use the simpler redirect that we used to change domain names, the redirect would get stuck in an endless loop – redirecting back and forth between the two types of pages. Eventually your users would just see an error message telling them they had exceeded the number of redirects permitted.

    A 301 redirect can also solve a similar problem that occurs with URLs such as

    http://your-site.com/

    and http://your-site.com/index.html.
    Both of these addresses point to your homepage, but search engines still see them as two different pages, thereby splitting your page’s importance. You can add the following code to your
    .htaccess file to make sure search engines only index http://your-site.com/:

    RewriteRule ^index.html$ / [R=301,L]

And Microsoft can give you more info on how to redirect pages using their IIS Server. More about domain and page redirect utilizing IIS

Owning Multiple Domain Names

Another use for a 301 redirect involves protecting your trademark or other proprietary keyword-friendly or misspelled domains by purchasing and activating such applicable domain names

Use mod_rewrite To Deal With Database Driven URLs

Perhaps you’re familiar with that little bit of Apache server magic known as mod_rewrite. You may already know that it can take your ugly, unwieldy, dynamic URLs that look like:

http://yoursite.com/index.php?item=muumuu&color=blue&size=large

…and turn them into static-looking URLs that are easier on the eyes:

http://yoursite.com/muumuu/blue/large

or even:

http://yoursite.com/m/b/l

Clearly, such rewritten URLs offer a more friendly appearance to real humans. They’re more descriptive and they’re easier to remember. What’s more, they tend to be about 30% to 50% shorter than URLs containing database driven dynamic characters and, on a super-large site, you may even save a little bandwidth.

But the real question is:

Do rewritten URLs make a difference when it comes to getting your pages indexed and ranked higher in the search engines?

Well, the answer is, they do make a difference. In fact, rewritten URLs can make a big difference.

While it’s true that Google has come a long way in parsing dynamic pages, just because the Google spider isn’t choking on your dynamic URLs anymore doesn’t mean it isn’t avoiding indexing them, and it also doesn’t mean it isn’t still giving preference to static URLs.

Google will accommodate dynamic URLs to a point, but it doesn’t like them. We’ve seen dynamic sites go from 2,000 pages indexed by Google to over 10,000 pages indexed after using mod_rewrite to switch from dynamic to static-looking URLs. In addition, these pages get indexed more quickly and more frequently than their dynamic counterparts.

How can I tell if mod_rewrite is set up on my server?
The easiest way to tell if you have mod_rewrite installed is simply to ask your system administrator.If you are the system administrator, or if you’re looking for some of the more technical details about getting mod_rewrite running, please have a look at the official mod_rewrite documentation.

In our experience, search engine spiders always prefer static URLs over dynamic URLs, and if you have a large, dynamic site that you want to get fully – as well as quickly and frequently – indexed by the search engines, then rewriting your dynamic URLs is a must.

A Simple Example

Despite what you may have heard, it’s actually fairly easy to get started with some basic URL rewriting. Here’s an example:

First, you’ll need to edit (or create if it doesn’t already exist) an .htaccess file in whichever directory you will be rewriting your URLs. An .htaccess file (note the period at the beginning of the file name) is an Apache server configuration file that lets you make changes to your server setup on a per-directory basis. For more info on creating and editing your .htaccess file, check out the official .htaccess guide, as well as this excellent tutorial.

Now, say you have a dynamic URL that looks like:

http://www.yoursite.com/index.php?item=muumuu&color=blue

…and you would like to have a more search engine (and user) friendly URL such as:

http://www.yoursite.com/muumuu/blue

To accomplish this, simply add the following two lines to your .htaccess file:

RewriteEngine On
RewriteRule muumuu/blue index.php?item=muumuu&color=blue

(Note: there is a space between the words blue and index.php in the above line)

The first line turns the mod_rewrite engine on. The second line consists of three parts:

  1. RewriteRule — This part lets the server know that you’re about to specify a rule to rewrite your URL.
  2. muumuu/blue — This is the way you want the dynamic part of your URL to appear to spiders and other visitors to your web site.
  3. index.php?item=muumuu&color=blue — This is the way the dynamic part of your URL appears currently.

And voilà! …you’ve just hidden your dynamic URL from any search engine spider.

Not so fast, there, pardner…

“Just a minute?” we hear you saying,

“This is all well and good for someone who has just a few dynamic pages, but I have over 10,000 dynamically generated pages! It could take me weeks to write all those rules.”

Right you are, and this is where the voodoo magic of regular expressions comes in.

The term regular expression refers to a type of miniature programming language which is extremely effective at recognizing and matching patterns in text. You can find a nice intro here.

Unfortunately, the awesome power of regular expressions comes with a fairly steep learning curve. Listed below, however, are some pre-fabricated formulas that should apply to many sites. These should help you get started with mod_rewrite even if you’re a regular expression novice.

By the way, you can get pretty far along the right path if you just know the following details about regular expression syntax:

. Matches any one character
? Matches the preceding character zero or one times
* Matches the preceding character zero or more times
+ Matches the preceding character one or more times
^ Matches the beginning of a line
$ Matches the end of a line

Now, let’s go back to our first example, but this time, instead of writing a specific rule for muumuu and blue, let’s make our rule flexible enough to handle any word as a value for the variables item and color in our dynamic URL. Open up that .htaccess file again and add the following:

RewriteEngine On
Options +FollowSymlinks
RewriteBase /
RewriteCond %{REQUEST_URI} !index.php
RewriteRule ^([^/]+)/([^/]+)/?$ /index.php?item=$1&color=$2

Looks like gibberish, doesn’t it?

Don’t worry, we can handle it. Let’s walk through it line-by-line:

  1. The first line, as we already know, simply turns on the mod_rewrite engine.
  2. The second line contains the phrase:

    Options +FollowSymlinks

    This line, primarily for those servers running Windows operating systems, tells the server to follow symbolic links. If you’re on a Unix or Linux-based server, then symbolic links are already enabled, but it doesn’t hurt to include this line in your .htaccess anyway – just in case they’ve been turned off.

  3. Next we come to the third line:

    RewriteBase /

    This sets the base URL. Setting it to “/” is usually the same as setting the base to your domain root which, in our example, is http://www.yoursite.com/.

    It’s important to understand that when we rewrite URLs we are rewriting the stem, not the root. In other words, when mod_rewrite rewrites a URL such as:

    http://www.yoursite.com/index.php?item=muumuu&color=blue

    … it will modify the part of the URL that looks like

    index.php?item=muumuu&color=blue

    while ignoring the part that looks like http://www.yoursite.com/.

  4. The fourth line consists of three parts:1) RewriteCond — This is the rewrite condition. In other words, this line needs to be true in order for any URL rewriting to occur. If this condition is not satisfied, the immediately following rewrite rule is ignored and mod_rewrite continues on processing any rules after it.2) %{REQUEST_URI} — This is an Apache-specific variable which refers to the page that is being requested. In the case of the URL http://www.yoursite.com/index.php?item=muumuu&color=blue the page being requested is index.php.3) !index.php — In English, this is equivalent to saying “is not index.php”. In regular expressions, the “!” character is also know as the “not” character.So what this line really says is “if the page being requested is not index.php, then process the next mod_rewrite rule”. We need to make sure that we don’t run mod_rewrite if index.php is called without any arguments, because this could leave us stuck in an infinite loop where we are just replacing index.php with itself over and over again.
  5. The last line also consists of three parts:1) RewriteRule — We already know that this part lets the server know that you’re about to specify a rule to rewrite your URL.2) ^([^/]+)/([^/]+)/?$ — This is the challenging part (No, it’s not a complicated smiley). It’s a regular expression that introduces something we haven’t seen before – namely, the “[ ]” symbols, which define a character class. A character class is a way to separate characters from each other in a regular expression. The rules are slightly different within a character class, namely, the “^” symbol no longer means “beginning of line”, but instead means “not” – similar to the “!” symbol we saw earlier.3) The other new concept we are seeing is the parenthesis. This is not only a grouping technique but also has the added power of being a back reference. What this means is that any text that matches an expression between parenthesis can be saved and referred to later by the symbols $1, $2, $3, and so forth. In other words, the text matching inside the first pair of parenthesis is placed in the variable $1, the text matching inside the second pair of parenthesis is placed in the variable $2, and so on.We can break this regular expression apart piece-by-piece:
    ^ Look for the beginning of a line.
    ( Opening parenthesis indicating the start of a back reference.
    [ Opening bracket indicating the beginning of a character class.
    ^ "Not" symbol (since it's inside a character class).
    / This represents an actual forward slash, such as you would find in a URL.
    ] Close the character class.
    + Match the preceding character or group of characters one or more times.
    ) Close the back reference.

    So what the expression ^([^/]+) really says is “find the beginning of a line, followed by as many characters as you can find that are NOT forward slashes, and store them in the variable $1 for later use”.

    If you have a line like:

    muumuu/blue

    …then the expression ^([^/]+) would have the effect of storing the word muumuu in variable $1.

    Back to our regular expression:

    / Look for a forward slash following the last chunk of text we matched.
    ([^/]+) We explained what this means earlier. It matches the word blue in the example above, and stores it in the variable $2.
    / Followed by another forward slash.
    ? This question mark means “match the preceding character 0 or 1 times.” Since it comes directly after the last forward slash, this means that the last forward slash is optional (i.e. it can show up once, or not at all).
    $ Finally, look for the end of a line.

    Now let’s look at the last part of line three, which looks like:

    /index.php?item=$1&color=$2

    This is the way your dynamic URL looked originally, except that in the place of the back reference variables $1 and $2 you will actually find your current GET variables (such as muumuu and blue). What will happen is the text that was captured by the back references in the regular expression will be inserted into your dynamic URL at the point where the $1 and $2 variables are found.

    When a request is made for a page like:

    http://www.yoursite.com/muumuu/blue

    …mod_rewrite will recognize that this URL is actually pointing to the address:

    http://www.yoursite.com/index.php?item=muumuu&color=blue

    …and will send your visitor (be they human or search engine spider) to the correct address, without them ever seeing what goes on behind the scenes. This is part of the beauty of mod_rewite. The URL is not being redirected – it is actually being rewritten on the server by mod_rewrite. No visitor to your site will ever see your dynamic URLs, or even be able to tell that they ever existed.

We Never Said It Was Easy

While it’s true this is a rather complicated example, and especially so if you’re new to regular expressions, our goal here is to give you some practical examples that can be applied to real life situations instead of the “toy” examples that are often used in most tutorials.

Additionally, the example we’ve used can easily be modified for any number of variables. For example, if your dynamic URLs have between one and three variables, depending on the circumstance:

http://www.yoursite.com/index.php?item=muumuu

http://www.yoursite.com/index.php?item=muumuu&color=blue

http://www.yoursite.com/index.php?item=muumuu&color=blue&size=large

… you could add the following to your .htaccess:

RewriteEngine On
Options +FollowSymlinks
RewriteBase /
RewriteCond %{REQUEST_URI} !index.php
RewriteRule ^([^/]+)/?$ /index.php?item=$1
RewriteCond %{REQUEST_URI} !index.php
RewriteRule ^([^/]+)/([^/]+)/?$ /index.php?item=$1&color=$2
RewriteCond %{REQUEST_URI} !index.php
RewriteRule ^([^/]+)/([^/]+)/([^/]+)/?$ /index.php?item=$1&color=$2&size=$3

… to get URLs that looks respectively like:

http://www.yoursite.com/muumuu/

http://www.yoursite.com/muumuu/blue

http://www.yoursite.com/muumuu/blue/large

Caveat emptor (A warning):

Bear in mind that rewriting your URLs increases the amount of processing done by your server, so your pages might load slightly slower, and, if you have a really high-traffic site, the load on your server might be unacceptably high.

The danger with mod_rewrite is that the search engine spiders won’t realize that you have a dynamically generated site, so they will crawl your site just as fast as they would a static site. Your system may be overwhelmed and you could experience a server slowdown or even crash if you don’t have adequate processing power.

We highly recommend that you experiment with mod_rewrite on a test server before taking it live with your main site. We have done our best to make sure that these examples work in a variety of circumstances but, considering all the vagaries of different server configurations, you may find that rewrite rules that work on our servers might not work on yours.

Voodoo?

Whether or not you agree with the fellow who commented: “this is damn cool voodoo, but voodoo nonetheless,” we consider it a valuable tool – especially useful for large, dynamically-generated sites.

Knowing that Google, for one, loves large sites and knowing that you can double or quadruple (or more) the number of pages that search engines will index, that should be enough incentive to, at least, experiment with mod_rewite. The reward waiting in the wings is an almost certain boost in your site’s overall search engine rankings once you get all of your database-driven dynamically-generated pages indexed and then working together in sync.

Get Found Now – a division of: On Top Of It, Inc

Search Engine Optimization ABCs and FAQs

META-TAGS
META-TAGS are the part of your website’s HTML code that allows a search engine to easily identify your site’s content. In the infancy of search engine optimization, meta-tags were the primary vehicle marketers and coders used for attracting search engine traffic. Today, search engines examine more than meta-tags; they take into account site content, internal link structure, and link popularity. Yet nonetheless, meta-tags remain the cornerstone of a successful search engine marketing program. Three primary meta-tags help search engines determine the relevance of a website for a specific search phrase: the title tag, the description tag, the keyword tag.

The TITLE TAG.
First, there is the Title Tag. Your Title Tag must describe exactly what your webpage contains. It has to contain the keywords — descriptive words that describe what your website offers or does — for which you hope to be found. This Title Tag will be the first thing your user sees so it must be clear and simple to read.Keep the Title Tag 70 characters or less. Make sure that it employs words and phrases that describe the content of the page as accurately as possible. Make each and every Title Tag is unique and relevant, that every word in this tag is contained somewhere in the tag’s page.

TITLE TAG HINTS.

Designing Search-Friendly Web Pages:

There are two important points to remember about search engines:

1. The first is…The quality of your site counts. Search engines make their money through advertising. Showing ads to their users is their profit model, and the more users they have, the more money they make. The way search a search engine gets more users is by giving them the best search results. This means that if your site is the most useful site to customers in your category, then search engines want to rank you highly. Indeed, their livelihood depends on it.

2. The second thing to remember is…Search engines are computer programs. More precisely, search engines run a program called a spider than downloads your web pages, reads the text and links on those web pages, and decides what to do with your pages based on that information. This is known as crawling. Search engine spiders are computer programs that crawl web pages.

Employ a Flat Directory Structure
In general, the flatter your site’s directory structure, the better your chances of getting more of your pages spidered. In other words, pages that are several sub-directory levels deep will often get spidered less frequently. For example, consider the depth of the web page at the following URL:

http://www.yoursite.com/content/articles/2005/05/pages.htm

Closer inspection revels that the file page.html is six directory levels deep.

A page that deep should generally not expect to be easily indexed or rank highly in any search engine. Expectations maybe made for popular sites with lots of incoming links, but any web page buried that deep puts itself a ranking disadvantage.

It is not uncommon to use directories to organize the structure of a site. That is because a flat directory structure-one that places all the pages only tow or three levels deep-is very difficult to logically organize. If the structure of your site dictates that you must utilize deep sub-directories, then it becomes even more important to utilize a strategically placed site map.

Use a Site Map
A site map is a web page that links to every other page within your site. By creating a good site map and linking to from your homepage, you ensure that each [age on your site is only one click away fro your site map and only two clicks away from your homepage. This is the optimum structure in terms of making web pages easy for the search engine spider to find.

As you know, search engines spiders find new pages by following links fro the pages that are already in their index. Thus, if you want a spider to crawl a web page, you have to make sure that it’s linked to from a page the search engine already knows about.

However, unless you have a very small site, linking to all your pages from your homepage would look messy and unprofessional to your customers. The site map allows us to accomplish our objective of getting all of our pages indexed without cluttering our home page with links. By placing a link on your homepage to the site map, and then a link form the site map to all of the rest of your important pages, you make all your site’s pages easy to find and index.

The search engines themselves will tell you to use a site map to ensure that all your web pages get indeed. In fact, Google has its own site map program. Our SEO Tune Up covers the basics of good Search Engine Optimization practices.