Search Engine Optimization and Writing for the Web

6264368250_cd6cb54cbc_bGet These Spiders Off Me!

My development team is constantly haranguing me about my writing style. “You can’t just write for people,” they say. “ You also need to write for robots.”

“And who are these robots of which you speak?”

Web crawling bots (or spiders) are computerized algorithms sent out by the major search engines, Startpage, Google, Bing, Yahoo, and the like, that scour the web and index all they find. This makes it possible for the user to receive 12 million results in 0.24 seconds!

But there is a downside…

The first three results of any web search get 84% of the action. This leaves only 16% for everyone else.

Keyword Search – The Parsers’ Tale

Where you appear in the search results depends upon where you are indexed in the search engine database. This is determined mostly by three interrelated criteria:

  • Keywords
  • Links
  • Popularity

When a spider comes across a webpage, it parses the document for keywords, starting with the URL, the actual address of the page itself (this is why http:/ is better than http:/ It then looks at metadata, the description of the website, then the actual text itself. If a term is bolded, it carries more weight.


If there are links to other pages it will follow them, and parse those as well.

Finally, is the page popular? How many hits does it get? How many other websites have a link that points to yours? All is taken into account. After all, what good is a Web without Spiders?

Now that we have the robots sorted, next up is style, and: