Google Webmaster Center Updates
If you are using the Google Webmaster Center then you have seen some changes recently to both the interface and the features. A lot of industry people were expecting a Google update to Page Rank a couple of weeks ago. I believe they are waiting to update PR until they have finished rolling out the latest Webmaster Center upgrades as well as their latest Universal Search updates.
If you follow the Web 2.0 and Search Engine Optimization philosophy than this should get you excited. Both Universal Search and the addition of RSS subscriber tracking to the Webmaster Center are steps in the right direction for Google and could truly benefit Social Network Marketing SEO.
How Google’s Webmaster Center Updates Can Help Your Site’s Placement
Before we get started, if you are unsure of what we are talking about below are some earlier posts that explain why all this is important.
- The Changing Face Of Google – Universal Search is explained in this article.
- Search Engine Optimization and Social Network Marketing - SEO and Web 2.0 Optimized social network marketing is covered in this post.
- RSS Real Simple Syndication Explained - RSS syndication is explained here.
- RSS Syndication Services FeedBurner – this post explained how FeedBurner works. It was written prior to Google’s purchase of FeedBurner which is talked about in my Web 2.0 Podcast. Also of note is the updated post about FeedBurner RSS to Email service.
If you are not using Google’s Webmaster Center you aren’t getting the most out of your site. If you are than you have an edge over your competitors and have a ton of useful information about how GoogleBot spiders your site. Tomorrow I will be posting a video tutorial that will be a visual guide on how to get going with Google’s Webmaster Center.
Changes To Google’s Webmaster Center
The first thing you will notice is the look of the center is different. The dashboard is basically the same giving you a quick glance at vital stats about your site including errors or any crawl issues your site might have. The first major change is the new navigation element at the left. Below is a shot of the new nav:
As you can see there is now a blue box with your main page choices. The page that you are on is highlighted a deep blue. As you navigate around the different main sections, each one has subcategories on the left in the nav, as well as links and short descriptions in the body of the page.
The choices are Diagnostics which give you the option of looking at the crawl stats for either a normal web crawl or a mobile web crawl. If your site isn’t set up for viewing over a mobile phone then all you need be concerned with is the web crawl.
The next section Statistics has some of the most important info within the Webmaster Center. The next figure shows the options you will see when on the Statistics main page.
The best part of the statistics is the Page Rank meter which is under Crawl Stats:
The most important part of this is the Page Rank Yet To be Assigned. The longer your bar, the better. Right before this blog obtained its PR, the Page Rank Yet To be Assigned was about half way. Once the Page Rank update came a few months back, my blog had gotten a 3 PR up to all the pages and posts up to 1 week prior to the update. I was happy to see the blog got the same PR that my site was awarded. Even though this domain is 6 years old, the site is less than a year old and is doing pretty good considering.The moral of this is, the higher your Page Rank Yet To Be Assigned your site is, the better you are going to do in the next update.The next upgrade is very cool. They added a link to your subscriber stats. This shows how many RSS subscribers you have through Google. Now before you get too excited, they have a few issues that will need to be resolved. Right now they are pulling the feed location out of the auto discovery RSS header if you haven’t added an RSS feed as a sitemap. OK so what is the big deal? If you are like me you are using FeedBurner to manage your feeds. Even with feed control plugins such as Feed Locations or FeedBurner’s Feed Smith, the self discovery will show the default www.blog/feed location that is default in WordPress as opposed to the feedburner.com feed that users will actually subscribe to.
Another problem is you can’t add a feed that isn’t in your domain to the sitemap section of the Webmaster Center. This means you can’t add the FeedBurner URL. I have asked them to make an exception for FeedBurner and I believe their response will be favorable since they no own FeedBurner. There is a big difference between the 16 Google subscribers and the 150 FeedBurner subscribers I actually have. And this isn’t showing the hundreds of RSS to Email subscribers that I have through FeedBlitz. On the bright side, FeedBurner is now giving me credit for those FeedBlitz RSS to Email subscribers in my control panel, but that has yet to show up in my overall subscribers number in their analytics.
The next section is also very important. Links can give you a very clear picture of how many inbound links Google is giving your site credit for.
It will show you all the pages on your site that has inbound links and give you a number of how many inbounds each page has. You can click on the link and it will show you what sites are linking to you, at least in the eyes of GoogleBot. Below is a shot of that feature in action:
A great upgrade would be to eliminate internal links from your own site. Google will count links from your own site if you use Fully Qualified Domain Name (FQDN) linking structure like I do. That means that your internal links are not relative i.e.
(a href=”/contactus.html” mce_href=”/contactus.html” )Contact Us(/a) this is a relative link
(a href=”http: //w w w.website.com/contactus.html”)Contact Us(/a) this is a FQDN link.
Other than SEO, FQDN linking can protect your site from being stolen in its entirety by scrapers. If you have all relative links someone can rip your whole site and put it up on their domain in 5 seconds!
By using Google’s link tool, you can see if your reciprocal link partners have kept your link on their site. It is also a great way to see if your purchased links are showing up. If you are unsure about the different types of inbound links there are, read my post on Linking Strategies: How To Get Inbound Links. I know there are a lot of people who are against paid links, but if the site and content are relevant, then it is just another form of advertising. I am totally opposed to paid linking that puts links on sites that have nothing to do with the content on either site.
Along with external or inbound links, Google also shows you your internal linking structure. This is useful to see how often certain areas of your site is crawled. They give you the date the link was last spidered which can indicate how important certain pages of your site are to Google. With more emphasis being put on Universal Search having fresh content is more important than ever. If you haven’t added a blog or other types of news type content management to your site, now is the time. If you are unsure how to get going with Web 2.0 and Social Network Marketing visit our Managed Content Syndication Services page and find out how we can get the ball rolling and take your Internet Presence to another level!
The next section of the Webmaster Center is for Sitemaps. These are XML sitemaps that tell Google about your site. If you don’t have an XML sitemap you better get one quick! WordPress has a plugin that will create one, but if you have a non-blog website I recommend that you use Sitemap Writer Pro, go ahead and use my discount code and save some $$$, enter this code when you buy it: MS130807EN and save 30%. It’s a great tool and it is the only sitemap writer program that can crawl a dynamic site with over 10,000 URL’s and NOT crash.
The final section of the new Google Webmaster Center is Tools. I could write a whole post just about how to use the tools section, but I am going to focus only on what is different. The best change is more information about your site’s robots.txt file. The link above is geared towards WordPress robots.txt, but also talks about regular robots files and links to my other posts about it. This tool is good to see the exact time that Google crawls your site. This can give you a heads up when last minute changes are needed. The first thing a spider does is scan your server site header then look at the robots.txt. It is so important to have duplicate content and private areas of your site disallowed. Any server action that can put a spider into a loop, like a cgi-bin application for a shopping cart, will force the search engine spider to leave your site!
Some other new goodies in the Tools section are the enhanced images section and remove URL’s. I don’t have a lot of info on the enhanced images section, but will be delving into it as I build and optimize my wife’s new Art site. She is starting a company called Original Oils Only which is about collecting art. It will need a lot of images that we are going to want spidered and indexed so I will keep you all updated on how the enhanced images can benefit your search engine placement.
The remove URL’s is a huge new feature. Yahoo’s Site Explorer released a similar tool months ago which is very good for getting URL’s out of the index that have either been eliminated, redirected or are leftovers from an older site. By using the web crawl diagnostics to see what problems the spiders have when crawling your site, you can now do something about it and request that they are removed. Obviously this can be very dangerous, so read everything carefully and pay attention to what you are doing. Last month I noticed that Google had made changes to its verification process, tightening up security on your site’s verification process. It would be very bad if a hacker or one of your competitors was able to remove URL’s off of your site.
The other features within tools have been around since the beginning and will be covered in my Google Webmaster Center video tutorial I will be releasing soon. As always leave a comment or question below and I will get back to you as soon as humanly possible!
WordPress Robots.txt For SEO
A couple of weeks ago I wrote an article about changing WordPress permalinks and how best to avoid duplicate content. I have gotten a lot of questions about the robots.txt portion of WordPress SEO. It is important to understand that the robots.txt file must be in your top directory. There is some speculation about this and the use of wildcard (*) within the robots.txt file. Spiders or bots are different, but you can look at this page on Google’s Webmaster help site which outlines their policy on robots.txt. Here is a quote directly from Google on placement of the robots.txt:
The robots.txt file must reside in the root of the domain and must be named “robots.txt”. A robots.txt file located in a subdirectory isn’t valid, as bots only check for this file in the root of the domain. For instance, http://www.example.com/robots.txt is a valid location. But, http://www.example.com/mysite/robots.txt is not.
OK so now what do you do?
SEO Strategy With The Robots.txt For WordPress
We all know that duplicate content is bad. WordPress in its very nature is a dupe content nightmare. There are many ways that the same page can have different URL’s and our job is to limit what the spiders crawl, without affecting the sites usability.
Not all spiders are the same, in fact it is unclear if they all support the wildcard, but Google does. The question is, do you optimize primarily for Google? I do. Yahoo does not have as strict enforcement of their content guidelines as Google does.
Below is how I set up a robots.txt on a test site that has excellent placement for specific keywords. I will keep everyone informed how the progress of the site in the SERP’s goes. Even though Google has eliminated Supplemental listings, they still retain a basement for dupe pages. My test site had a bunch or pages in the sup index, so we will see if the WordPress SEO robots.txt file has helped.
User-agent: *
Disallow: /cgi-bin/
Disallow: /z/
Disallow: /stats/
Disallow: /dh_
Disallow: /wp-content/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: */trackback/
Disallow: /cgi-bin/
Disallow: /adlogger/
Disallow: /ads/
Disallow: /mint/
Disallow: /*?*
Disallow: /20*
User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: */trackback*
Disallow: /*?*
Disallow: /z/
Disallow: /wp-*
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.txt$
Disallow: /*/feed*
Disallow: /20*
Website Usability and Web 2.0
Website usability has been changed dramatically with the addition of Web 2.0 and social network marketing sites coming into the mainstream. I have been getting a lot of heat lately over some of my website usability posts, especially the one about keeping content above the fold. Sure I admit that this isn’t as significant when you are talking about blogs, but don’t think that people are going to be scrolling on your site looking for specific products or other information. The fact is you have to lay out your site so that the content and information that make you money is put right in front of your visitors.
I have looked at hundreds of Google Analytics site overlays since I wrote that post and links that are further down on the page just don’t get clicked on with any regularity. This also has a lot to do with the fact that most of my clients had me design their site or work closely with their designers. I stick to the website usability rules I learned in reading Jakob Nielsen’s books because they have always and still do work for me.
Usability and Social Network Marketing Web 2.0 Websites
For most of us social network marketing and Web 2.0 mean blogs and blogging systems. I use WordPress as my platform of choice and have had a great deal of success. I am running a couple of TypePad sites now to test another system, but so far I still think WordPress is the best.
Most website usability rules get thrown out the window just in the very nature of blogs. They are long and require a lot of scrolling. If you look at my blog you can see that I don’t use the more option and don’t use summaries on the home page. Most people believe that summaries are the way to go, but I wanted to run a test. My blog was set up on the same day as I set up one of my data recovery clients blogs. They both are updated frequently and they run different types of set ups. The reason for this is so I can have an opinion that is based upon facts, not conjecture.
The statements that I make about usability are also based upon facts and statistics. I know that everyones website is different, but the most basic rules of usability should still be followed. When you are creating a blog, it looks very clean and easy to follow the first couple of months. The issues start when you are 6 months into the blog and have hundreds of posts. If you don’t properly plan out your blog, it will be very hard for your visitors to find stuff. I use some good plugins that help me keep the site organized and are trying out a few new ones.
At SES in San Diego there was a lot of talk about tagging, Ultimate Tag Warrior and tag clouds. At this time I use the Technorati tagging system because I have concerns about duplicate content and actually have the /tag/ folder disallowed in my robots.txt file. I still think that UTW is a good idea and I am implementing it on a few other sites. As Web 2.0 gets older, website usability plugins will become more prevalent. I am in the process of a major overhaul of this site including a theme change. I will keep all of you apprised of what I have learned and I will explain why I am setting the site up the way I am. The first thing I did was change my permalink structure. This was and still is a major pain in the ass, but it had to be done. You can read about it in my changing the permalinks on an existing site post.
Related Posts:
5 website usability rules that won’t ever change
Web 2.0 Explained
Website usability and the Home Page
ASK XML Sitemap Submission
For those of you who don’t know about XML sitemaps for your website please read my XML Sitemap Guide to learn how to create and manage your sitemaps. Google and Yahoo have webmaster tools for you to upload and manage your sitemaps. Live search will have a similar portal pretty soon. If you haven’t heard about it, read my post Live Search New Webmaster Portal in Beta for more info.
That will take care of the top 3 major engines, but what about ASK? I get 10% of my traffic from ASK and it is due to the fact that I ping them with my sitemap whenever I change the site. No they don’t have a webmaster center, but you can submit sitemaps to ASK one of 2 ways:
- Through the robots.txt file – Once you have prepared the sitemap, add the sitemap auto-discovery directive to robots.txt as follows: SITEMAP: http://www.the-URL-of-your-sitemap-here.xml. The sitemap location should be the full sitemap URL. Read my post Search Engine Optimization: Robots.txt for more info.
- Alternatively, you can also submit your sitemap through the ping URL: http://submissions.ask.com/ping?sitemap=http://www.the-URL-of-your-sitemap-here.xml. My article on Search Engine Optimization and Social Network Marketing has a great overview of XML-RPC.
Make sure you follow the sitemap guidelines at sitemap.org when preparing your sitemap.