With the attention that Microsoft, Yahoo! and Google are getting for agreeing on a standard sitemaps protocol, I figured it was probably time I got one in place which meets standard found on sitemaps.org. 

Though I’m sure I would have had a lot of fun using the Google Sitemap Generator Python scripts recommended by Google Webmaster Tools, I opted to go with yet another WordPress Plugin instead. 

Sitemap Generator for WordPress v2 was a single file download and a single click installation.  There’s a beta release of the next version available, but I stuck with version 2.

Though the plugin pings Google and let’s them know about the new sitemap.xml file, I still visited Google Webmaster Tools and set things up manually as well. 

I recently wrote about Google not indexing my site. One can find johnnycoder.com through Yahoo! and MSN, but I’d really like for Google users to find me. Well, I thought I found a backdoor into getting indexed yesterday…

As you may already be aware, Google now let’s us build Custom Search Engines. The idea is you can create a search engine which reflects your knowledge and interests by specifying the websites that you want searched. ( For example, I’m aware of a couple .NET-related implementations already: Haack Attack the Web from Phil Haack and .NET Search Engine from Gavin Joyce. )

Anyway, I created my own search engine and included only johnnycoder.com as a searchable site. I figured (read: hoped) that Google would opt to index the included sites. I mean, this is the launch of a new product. It’s time to shine, right?

Wrong. Google does pull back a few johnnycoder-related results via my search engine, but they are only pages which have been indexed through other sources (Technorati, Jon Galloway, etc). Oh well.

Though my experiment failed, there is a silverlining. Custom Search Engine home pages display some (if not all) of the included sites. Find a custom search which reflects your interests (see Phil and Gavin above) and you will most likely find sites to include in your list of favorites.

If you haven’t played around with the custom search already, check it out. Also, here’s a good example on how to integrate your custom search engine with Firefox 2.0 or IE 7.

Are you indexed by Google? Well, JohnnyCoder.com isn’t. Here’s the thing. The URL was submitted a while ago, but it hasn’t made a lick of difference. But I shouldn’t be surprised since Google is very up-front about how they handle submissions.

We add and update new sites to our index each time we crawl the web, and we invite you to submit your URL here. We do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about when or if they will appear.

As you can see, Google doesn’t make any promises.

So, what else can I do? Well, I’ve started sending influential thoughts Google’s way. When I’m feeling particularly positive, I hold my breath and check if my site’s status has mysteriously improved. This is done by executing the following Google site search. Of course, the outcome is always the same:

Your search - site:www.johnnycoder.com - did not match any documents.
Suggestions:

• Make sure all words are spelled correctly.
• Try different keywords.
• Try more general keywords.

And as you can see, along with promises, Google doesn’t make any suggestions. (No helpful ones at least.)

I haven’t given up on Google yet. I’m sure they will start indexing my site when they are good and ready. After all, they are the search kings and they must know what they are doing. In fact, I’m sure it’s not them. It must be me and what I’m not doing. But I really didn’t know what to do so I have started to do some research. I found that most sites provide teasers. It’s helpful information, but not too helpful — if you know what I mean. The sites may say to submit your URL to Google (as I have already done.) Otherwise, their advice is to leave search engine submission and optimization to the experts and purchase their service for only $99 annually.

At least Submitside.com collected search engine “Add URL” and put them all in one place. They also recommend submitting only to the following: Google, Yahoo! and MSN. And only if you are plenty of time should one submit to these smaller engines — Rex, Infotiger, Netsearch Voyager, ExactSeek, LinkCentre, HotRate, TrueSearch, WhatsNu, LookSeek Susy (South Africa), Walhello.com, Dirs.org and Wisenut. They also list a number of engines which one should NOT bother with since many providers actually “power” others and submissions would be redundant.

Most everyone is an agreement. Submitting your site is not enough. Indexing will take up to 8 weeks (if at all) and this alone is not enough to start getting visitors. Being in the top search results depends mostly (but not only) on popularity of your site and a new site is not popular at all. So you need a different strategy.

One approach is to increase the number of links to your site from more popular sites. I suppose this could be done through the use of Technorati or by simply adding comments which link back to your site on other peoples’ blogs. Or by simply begging a popular site owner to add a reference to your site.

Arguably, the best strategy is to “buy” traffic through paid listings. This unique form of search engine advertising means that you can be guaranteed to appear in the top results for the terms you are interested in within a day or less. If you wish to build visibility quickly it is agreed that this is the way to go.

SearchEngineWatch provided a really nice breakdown of this information. They suggest that any site owner established a search engine submission budget. This is true whether you’re running a commercial web site, a hobbyist site in your own time or a site for a nonprofit organization. They suggest you first submit to Yahoo’s human-compiled directory for one year. This is a flat rate of $300 annually. They believe this provides the most bang for your buck merely because this is potentially one of the best links you can gain to influence crawlers. If you need to be listed fast, however, they suggest one use paid placement programs as well. A $50 budget for Overture (which is now Yahoo! Search Marketing) and a $25 budget for Google will last about a month. SearchEngineWatch does claim you can get listed without spending a penny, but if you want to be seen in as many places as possible, as quickly as possible, you will want to use the combination of submissions which they outlined above.

So far only about half of the articles I’ve read are even interesting, but I have learned the difference between directories (like Yahoo!) and crawlers (like Google, Ask and MSN.) And I’ve learned about paid listings like Overture and Google AdWords and I have learned a bit about how search engines work when it comes to page rankings and who feeds who. I’m learning, but not even dangerous yet. Unless someone has an extra $375 to fund my search engine submission budget?

Have you had a chance to look at Google Code Search? It truly will help you find function definitions and sample code by giving you one place to search publicly accessible code through the use of regular expressions and language and filename filters, but this is taking a backseat to what Code Search can really do…like discover vulnerabilities in the indexed code and make you laugh.

How’s about this? Discover what coders are cursing about. Find complaints about stupid users. Realize that you aren’t the only coder who hates their job. And find out who is coding drunk or who begs forgiveness from the coding gods. This is just the fun stuff. Find “real” vulnerabilities at kottke.org.

Undoubtedly, you have noticed the obnoxious Google Ads sprinkled about the site. I must explain.

1. Though nearly all legitimate bloggers have sites plastered with Ads, this is not my attempt to conform and become a real blogger overnight.

2. This is not a shameless attempt to make a few pennies each month. Granted, some individuals have made money through AdSense, but their blogs actually have a steady flow of traffic. Of course, nobody (except for you) reads this blog and I can’t in good conscience start clicking these links myself.

3. This is not a quick way to clutter up an otherwise clean site.

No, this is merely an experiment. Google Ads are all over the place and I wanted to know just how easy (or hard) they are to implement. As it ends up, they are easy. Even a guy like me can figure them out.

Account Setup - Yes, I actually setup an account though it isn’t active yet. After providing my contact information and my domain, I was informed that it takes 1-2 days to complete the registration/authorization process. I’m sure once the account is activated, I’ll have the opportunity to select which type of ads are appropriate for my site, but that’s it for now.

Web Updates - About 10 minutes into the 2-day waiting period, I got a little antsy so I started to look around for clues on how to incorporate the Ads on this site. As it turns out, viewing the source of a page hosting an Ad provided all the information I needed. Ads are included using only a few lines of Javascript code:

<script type=”text/javascript”><!–
google_ad_client = “MY_ID_GOES_HERE”;
google_ad_width = 200;
google_ad_height = 90;
google_ad_format = “200—90_0ads_al_s”;
google_ad_channel =”";
google_color_border = “F9F9F9″;
google_color_bg = “F9F9F9″;
google_color_link = “55AAE1″;
google_color_url = “333333″;
google_color_text = “333333″;
//></script>
<script type=”text/javascript”
xsrc=”http://pagead2.googlesyndication.com/pagead/show_ads.js“>
</script>

So, I modified some of the formatting options (size, style), borrowed someone else’s AdSense ID and dropped the code on my page. It worked (after a couple of additional formatting tweaks) and now I’m just waiting for my own ID to show up in 1-2 days. (I wonder if I will still want the Ads up by then…)