Hi Folks,

I'm looking for some input with regards to the use of a robots.txt file to block search engines from trolling and indexing image directories.

Personally, I've found very little value in letting the search engines index image directories on my server. For the most part, it just ends up sucking up a lot of bandwidth (by the bots) and provides a handy tool that people can use when trying to find images for their own sites.

I can see how the indexing of images could help some sites (ie. a site that sells artwork) but outside of that ... has anyone ever noticed any significant perk for having your images indexed?

Also, are their any downsides to blocking the engines from accessing your images, besides the obvious of not being listed in image specific directories?

I don't see any reason to allow bots to index your image directories.  In fact one of the most basic facts you hear about SEO is that the search engines can't read pictures.  I say block em, and let your customers use the bandwidth.

dude having images on your site is a killer way to make some $$$

I have a few pages on my site that are really just nothing but pictures of famous women,  Anna Kournakova, Maria Sharapova, Morgan Webb, etc...  Put you a CPM pop up on a page like that, and even if it doesn't rank especially well you can make $50-$100 a month for basically nothing.  I make that much in a week if Maria Sharapova is on television over the weekend.  Best thing is that these pages are not even really related to my site at all, so someone surfs in looks at the picture and gets my pop up, I collect my dime and they leave, or not - I don't worry about them clicking on anything, or buying anything, or trying to sell them anything - I already got paid.  And if they browse around my site for a while and like it, fine -- if not - who cares ?

I say don't block em.  Bandwidth is super cheap, if bots are wasting your bandwidth on images then you have a host that sucks.

And another thing, for some sites like especially sites about art, or popular culture, or entertainment, I think people are more likely to search for images in some cases than they are to search for a keyword or phrase.  If that fits you, you'll be glad you've got images in image search.

My site is currently #4 for my keywords on Google image search, but if you search for practically any song name, album name, band member, or any phrase related to my subject matter I've got several images ranking on the first page.  I get more targetted traffic from google image search by far than I get from regular google.  And even the untargetted off-topic traffic that I get I can make pay off because I can put CPM ads or popups on the page(s).

So again I say don't block em.  All you've got to lose by blocking them is a few measly pennies worth of bandwidth, and you've got a lot to gain if you figure a way to capitalize on it.

From a SEO point of view, there is no harm in blocking images from bots. It will help you save your server bandwidth and offers your site visitors a better experience (Fast Loading of Pages).

You can block the images folder using ROBOTS.TXT file. And add the following content in it.

User-agent: *

Disallow: /images/

It also offers a way to protect your images from public piracy.

indianwebseo wrote:

From a SEO point of view, there is no harm in blocking images from bots. It will help you save your server bandwidth and offers your site visitors a better experience (Fast Loading of Pages).

It also offers a way to protect your images from public piracy.

Confused here, could you elaborate?
How does the robots file make the pages load faster and protect from piracy?

I understand there are reasons  some people want to keep images from appearing in images.google.com etc.  Here's how:

Put the images in a seperate directory and disallow that directory in your robots file.
You can also put the pages in that directory if you don't want them to show, (or a different directory and disallow that one too).
You can also add a meta tag

<meta name"robots" content="noindex">

  or even

<meta name"robots" content="noindex, nofollow">

on the page holding the image.  I'm pretty sure that's the tag but it's been a while since I used it.

ColoEagle wrote:

indianwebseo wrote:

From a SEO point of view, there is no harm in blocking images from bots. It will help you save your server bandwidth and offers your site visitors a better experience (Fast Loading of Pages).

It also offers a way to protect your images from public piracy.

Confused here, could you elaborate?
How does the robots file make the pages load faster and protect from piracy?

Ha ha ha ha ha... Pirates must obey the mighty robots.txt M8TY.. ha ha ha..  (no not really)..

Ahhhh, selling artwork on the internet.  I have some expeirence in this area.

Here's the key, put the full images in a password protected area the the search engines and non-paying people can't get to.  Give the public a medium quality version of the images for sale with a very prominent watermark.

Remeber if they can see it, they can take it.  So don't let them see it until they've paid.  Either by subscription (download so many , or unlimited per time period) or per image.

Let the search engines index the "preview" images.  It'll just increase business.  Look at how everyone else does it:
example

This is the only thing you can do to signifigantly reduce piracy.

Give a url so we can check it out will ya?

ColoEagle wrote:

How does the robots file make the pages load faster and protect from piracy?

I'm guessing it's because if the host is busy serving images to picture collectors, your resources: bandwidth, ram , clock cycles and all else are diverted from serving your site. The reason you and I don't "get it" is because the traffic numbers and proportion of bandwidth consumed would need to be waaaay further along the curve before it's going to be an issue. With a  very busy site, though, I imagine everything is an issue.

"Protect from piracy", I suspect relates to the fewer places your images/thumbs are displayed, the less likely they are to be lifted. If they're being displayed on google/images or whatever, the only thing that can happen, is they get "borrowed" because conversion is going to be way down on someone elses site.

Personally, I agree with M1 elsewhere, that any traffic is good traffic, even if they have to clip the URL to find you.

For a lot of webmasters, I think subtle watermarking of your greatest works with your site name would pretty well cover image theft. On the other hand, my robots.txt blocks the images folder, database, adminsection and so on

From a SEO point of view, there is no harm in blocking images from bots.

yes there is

It will help you save your server bandwidth and offers your site visitors a better experience (Fast Loading of Pages).

it will save you pennies worth of bandwidth and it won't offer your site visitors a better experience at all

It also offers a way to protect your images from public piracy.

no it doesn't, it doesn't offer any protection from piracy at all

Mutilated1 wrote:

From a SEO point of view, there is no harm in blocking images from bots.

yes there is

OK, I'm curious, how does blocking SEs from spidering your image folder have a harmful effect on your SEO, if the images themselves aren't your business?

indianwebseo wrote:

From a SEO point of view, there is no harm in blocking images from bots. It will help you save your server bandwidth and offers your site visitors a better experience (Fast Loading of Pages).

You can block the images folder using ROBOTS.TXT file. And add the following content in it.

Code:

User-agent: *

Disallow: /images/

It also offers a way to protect your images from public piracy.

That would not work unfortunately, I've tried that.
The SEs index and cache the images from the web page crawled, they do not crawl image folders. So unless you disallow the web page the images will be indexed and cached.
A good way for them to not show up in image searches is to give them nonsense names and alt text. Bad for seo though.
I assume you want your pages indexed?
You can add the tag <META content="noarchive" NAME="robots"> or
<meta name="robots" content="nocache,noarchive"> for the pages with the images you do not want searchable, while still keeping the pages indexed they just won't be cached or "ripped off" if you prefer.
I have done this on a couple of sites of mine and so far it has worked ok - pages with this tag also seem to be crawled more often by the spiders. Maybe just a coincidence, I don't know.
Also the watermark is a good idea but anybody thats halfway decent with photoshop could get around that.
I guess basically once your images are out there, they're fair game. Not that its right or legal just the way it is.
With one click if you have FF downloadthemall plugin anyone can steal your stuff.
IMO, but as honest as I can be.
Dan

OK I won't post quotes from everybody here. and not in any particular order.

So far my knowledge / understanding of the robots text file is exactly with everybody else. I was just interested and still am on indianwebseo's elaboration and his reasoning for making these statements.

The piracy issue is what really caught my eye, how is a file on a website going to stop me from copying an image if I wanted to?

I am with M1, what's wrong with having your images indexed. Traffic coming in is a good thing. unless I am missing something.

yes, texxs your meta tags are correct.
We also have to remember that there are robots out there that do not honor the robots.txt file or the noindex, nofollow tags.

Yes, all you have to do is right click and save an image right from the page even if it is a background image placed with CSS. Even a no right click java script is useless to all except the average casual surfer.

Watermarks are IMO probably the best image theft deterrent you can have. Even if someone is good enough with a photo program it will at least make them question the worth of the image because of the work they would have to go through to have a clean image.

DMX, you may be correct, we may not be getting high enough volume of traffic to experience this. LOL

Steven_A_S wrote:

Mutilated1 wrote:

From a SEO point of view, there is no harm in blocking images from bots.

yes there is

OK, I'm curious, how does blocking SEs from spidering your image folder have a harmful effect on your SEO, if the images themselves aren't your business?

I guess if you want to turn away potential visitors, thats your business.

Personally I'm pleased if someone visits my site for whatever reason.

Turn away traffic if you want to, no skin off my nose.

I'll prefer to make a few bucks from mine.

Yes everything you said is correct ColoEagle.
The thing is when you spend hours creating a graphic or pay big bucks for a professional photographer and some scraper comes along and steals your hard work and $$ it really sucks.
Forrest Gump "that's about all I have to say about thaat"
Using graphics to draw in visitors is a good thing, unfortunately some people use the SE's image search to bypass the hard work or investment.
BTW although hardly used live.com's image search is awesome!! If you are in to that kind of thing.
Dan

By "watermarking of your greatest works with your site name would pretty well cover image theft", I didn't mean so no one can take your images. They can take your images. Right click, screen capture, browser cache, source code image path all come to mind. All I meant is, make a reasonable effort to protect your stuff and move on to something else.

I would like to think that most developers adept with photo shop or PSP would make their own (or ask). Maybe in four years google will have developed the "Duplicate Image Filter".

'Scuse me I must go start a thread about how the DIF effects TBPR.....

Hey darchell  don't let our "back and forth" scare ya.  It's a great way for you to get several points of view and advice from several expeirenced people.  It may sound contradictory at first but read it all twice and you'll see we're all kinda saying the same thing.

Some of you guys might find this interesting.

I bet you didn't know that Google and yahoo don't actually download your images when they spider your pages.  Its true.

They both have a separate bot with a separate user agent that will come along and pick up the images of interest at a later time.

A bot is not like a web browser, what it spiders is never going to be rendered on a screen anyway, so there is no reason for them to load your images at that time at all.  Instead what they will do is keep a list of the images that look promising according to whatever algorithm or criteria they are currently using, and then they'll send a separate bot to go look at the chosen images at a later time.

I'm sure some of you know it alls won't believe me, but you can look at your raw server logs and see for yourself that Google, yahoo, and MSN aren't loading your images when they look at your pages.  Instead you'll see a unique user agent come along and pick up 1-n images all at once, and the user agent will have something like "google-imagebot" or "yahoo-imagesearch" in it.

The proof is that the regular spider will get the page, and then days or weeks later at a completely unrelated time the image spider will come and look at some of the images.

Strangely, the alt text in the image tag and the filename of the image don't make a single a bit of difference at all.  I know that goes against the conventional wisdom, but its true.  If you really want to optimize your images for search, it won't hurt to give them alt text and a choice filename, but whats more important - really the only thing thats important is the words around the image.

Makes sense to me. Why have the bot, which has fifty million others sites to visit, camp out collecting images when this way seems more efficient. Sort of like, tanks to claim the territory and infantry to hold it.

The number of things I don't know amazes me each and every day.

DMX wrote:

Makes sense to me. Why have the bot, which has fifty million others sites to visit, camp out collecting images when this way seems more efficient. Sort of like, tanks to claim the territory and infantry to hold it.

The number of things I don't know amazes me each and every day.

I agree 100% with Mutilated BUT image bots are not the only way SE's get your graphics. They are for the most part cached and displayed. If you have any doubts look at the cached page of your website.
If it shows only text Mutilated is correct, most times though you will see some of your pictures show, which indicates a deep crawl.
And yes you may do an image search for Easter Bunny and come up with alot of women dressed in sarran wrap. So surrounding text does have an impact.
My point was if you give the graphic file a totally unintelligible name it is less likely to show up in a search. I am in no way an expert on this subject but one thing I DO know is how spiders and search engines work. IE what they index and what they don't. Trust me on this one.
Dan