You know, EVERYBODY seems to worry about SEO.
(Well, everybody that doesn’t rely solely on paid traffic worries about SEO that is…but you know what I meant.)
The biggest concern you see so many IMers have – esp anybody that does any sort of niche marketing – is their SERP ranking. Where are they ranked in Google, and how can they improve said ranking?
But you gotta walk before you can run.
And before you worry about where you rank in Google, you gotta make sure Google knows your content even exists. You’ve gotta make sure Google knows about you so that your content will even show up in Google in the first place.
This is called getting your content ‘indexed’.
I gotta be honest – I didn’t know people even had a problem with getting their content indexed. And from what I see on some of the IM forums out there, there are those that do. Now, I don’t mean just the people created a zillion and four pages on an adsense site. I mean even peeps just putting up a small 10-15 page niche blog have trouble getting their content indexed.
That blows my mind.
See, I’ve never had any trouble with it, but I think that’s likely because I approach it slightly differently than most others do. A lot of people just create content, ping it, and think that’s gonna be enough to get the content indexed. But that’s not necessarily the case.
First of all, you gotta remember what ‘pinging’ is. A ‘ping’ is (more or less) like sending up a signal flare to the search engines. It tells them that you’ve got something new on your site, and that they should send their little spiders over to crawl your site, recognize that you’ve got new content, make note of it, and list it in the search engine.
Seems simple enough, right? Problem is that pinging isn’t always enough to get your content indexed.
Why not? Hell if I know. Don’t ask me to explain why Google does half the shit they do. All I know is that I wanna get my content indexed, and pinging alone doesn’t always do it.
Know what most people do if pinging alone doesn’t work? They ping it again.
And again.
And again.
And again.
And…you get the idea.
Not a smart move, in my opinion.
More than likely, when you ping your content, you’re not just pinging it once. If you use something like pingler or pingomatic or whatever, you’re only submitting your url one time, but these sites actually turn around and submit your url to a whole crapload of services that send out these pings to the search engines. So for every time you submit your url, you might actually have 10, 20, or even 50+ pings sent out for you.
What happens next is anybody’s guess (tech gurus and Google-nerds feel free to comment below and enlighten us all if you know), but here’s how I kinda think it works…
NOTE – this is only my theory. I could be totally wrong and be jacking it all up. If you follow my recommendations based on the following scenario, it’s because you’re choosing to believe I know what the hell I’m talking about…when I might very well not. Buyer beware (even though this is free info), caveat emptor, your mileage may vary, and all that other good stuff…
I kinda think that search engine spiders run all over the damn internet, crawling webpages as quickly as they can. Because popular sites get crawled often anyway, and new content is being put up around the intrawebz at a rate of 1.3 billion pages per minute (I totally made that up – but it’s a lot), they get behind in their work. Sure, those spiders are just programs, but they still can’t be everywhere at once. So, they have little “to-do” lists they carry around, telling them which site they’ve gotta hit up next.
When you send out a ping, you get added to their little “to-do” list. They might not get to you ASAP, but it’s their intent to get to you as soon as they can.
Well imagine that spider is kinda like a parent with their list of shit they have to get done, and you’re like a little kid that wants mommy & daddy to help you out with their new toy. You go and ask mommy & daddy, and they tell you that they’re busy right now, but they’ll help you as soon as they can. Then mommy & daddy get busy with other stuff (like going to work, making dinner, and the next-door neighbors’ barking-ass dog), and they don’t get around to helping you. It’s not that they didn’t want to, it’s that they just couldn’t get around to it – life got in the way.
At this point, you can do one of two things. You can either try to be cool about it, and hit up mommy & daddy again, telling them you’re not trying to bug ‘em, but can they help you out when they get time. OR, you can be a little brat and throw a tantrum like the spoiled little punk you are.
If you do the first, mommy & daddy will likely be cool right back, and either help you out then because they feel bad about not getting to you (and if not then, then pretty quickly). Or if you do the second, you can totally piss off mommy & daddy and instead of helping you, they tell you you’re *not* getting any help, ground you for being a brat, and then throw away your new toy, just to show you who’s boss.
Damn kids.
Anyway, my point is this – I have it in my head that search engine spiders, devious little bastards that they are, do the same thing. If you ping them once, they wanna get over to you and crawl your content. But sometimes – for whatever reason – they just won’t. So you can either be cool and try to ping them again and see if they’ll be able to help you out. Or, you can be a little bitch about it and ping your content several times/day.
To which at this point, I have it in my head that the spiders give you the virtual middle finger, and either keep your content so far down the list that they don’t come see you days or weeks on end, or just avoid you completely, because some Google assclown has written an algorithm the spider carries around that now deems you a pain in the ass and they don’t wanna deal with you.
What I like to do is attack it from a TOTALLY different point of view.
Instead of relying on a signal flare of a ping and yelling “COME LOOK AT ME! COME LOOK AT ME!” while waving my arms like a lunatic, I’d rather act like a cattle rancher and just herd the spiders that are already out there, tricking them to come and crawling my content.
Now you need to know – when the search engine spiders hit your page, they’ll not only crawl the whole thing, but they’ll also follow any links that are on it. (This is one of the reasons that internal linking and sitemaps are so important for SEO.) So when they get done crawling the content, they’ll follow a link to another page on your site, then to another page, and to another page, etc until it’s crawled the whole site. At the same time, if you’ve got out-bound links (i.e. – links to other websites), those spiders will leave your site and go start crawling that site.
This is what we want to take advantage of, acting like a cattle rancher.
To get herds of cattle from one place to another, you don’t see ranchers standing where they want the cattle to come, shooting their pistols in the air, yelling “COME HERE COWS!” At the same time, you don’t just have ranchers where the cattle start at, just shooing them away into any old direction.
Instead, what they do is go along with the cattle, guiding them where they want to go.
Now, you would think the cattle should be in charge of this situation, right? I mean, there’s only a few ranchers herding possibly several hundred head of cattle. And if the cattle get spooked, take charge is what they’ll do – they’ll stampede.
BUT, if the ranchers can keep their cool, and keep the cattle calm, they can easily herd them almost wherever they want them to go without too much effort.
If you want to get your content indexed, then going out and herding search engine spiders the way ranchers herd cattle is the way to do it.
You do this by building quick and easy backlinks at a few very highly trafficked and popular sites. These will be sites that get a ton of use, and have spiders coming to and crawling them virtually non-stop. Think social bookmarking sites. Web 2.0 sites. Social media sites. Search engine spiders are all over these sites all the time.
Now, remember what I said before about how a spider will follow a link off your page and onto another one if you link to it? Well what you do is get links back to your new content on these sites so that the spiders that are already naturally on these high-volume sites already leave those sites and come to yours naturally – you’re herding them like cattle.
I have a little system I use. First, I build some quick and easy social bookmarks. Then, I submit the RSS feed of my new content to a few aggregators. Then, I like to use a posterous network I’ve setup to syndicate a short blog post (think 100 words – like a short email) with a link back to my new content. This will post it not only on posterous.com, but syndicate it to blogger.com, wordpress.com, tumblr, etc. My posterous network also has social media built into it, linking back to the posterous.com entry. I will then go back and do a quick round of social bookmarks (use an automated solution) to the posterous.com entry as well.
After that, I ping it all just once. (Though it’s not totally necessary, it still helps.)
Take a look at what you’re doing there – you’re guiding (herding) search engine spiders:
- from social bookmarking sites to your content
- from RSS aggregators to your content
- from web 2.0 sites to your content
- from social media sites to web 2.0 sites to your content
- from social bookmarking sites to web 2.0 sites to your content
See what’s happening here? You’re out there herding search engine spiders that are already on these sites regularly, and like a rancher calmly herding cattle, you’re calming herding those spiders to your new content…and they don’t even realize it.
The result? My new content is generally indexed in just a few (usually 6-8) hours. Sometimes it’s a little as 4. Now an then it takes a little longer, like 10-12. Hell, I can setup an absolutely brand new site with new content and have the whole thing indexed in usually 24 hours. Usually much less.
And at the same time, I’m getting a few backlinks to my site – and additional backlinks never hurt.
Try herding spiders to get your content indexed, and you’ll find you outperform incessant pinging hands-down.
Tags: content indexation, google, google spider, ping, search engine spiders, seo, SERP
Leave A Reply (No comments So Far)
No comments yet