Has anyone else been hit by this "new" bot. It has been ravaging one of my sites and I was wondering if I should block it in robots.txt.
According to Microsoft "We are using the MSRBot web crawler to collect data from the web for further study."
http://research.microsoft.com/research/sv/msrbot/
What?
Any advice would be appreciated. I don't know if I want my data collected for further study, it's not a BW issue.
Thanks again, Dan

Interesting -- I've never heard of this until you posted it -- thanks!

But, in reading what you posted, this new bot pulls your robots.txt and collects the data anyway ... however; they also give you the code to stop that, by placing it in the top part of your site(s).

Unless it consumes lot of bandwidth
I don't think you should stop it.

webgaya.

Digging a little deeper into this subject, you can use this nifty tool to check your "GET"; "POST"; & "HTTP" on your site(s).

Any chance you could give us an IP address/es for MSRBot please?

My logs only show IP's so id need it to know if he's been hitting me hard.

On a slightly different note, Google has hit me with 1666 page views in the last 2 days. Giving me 326 pages in the index. On friday, I had none as ive just changed my domain tdl. just gonna go check me SERPs

Sure. It is just kind of funny. I have a smallish 100 page website and it has hit me thousands of times. I don't get it.
So my server logs are full of this bot, without sitting for hours going through my logs I found this one although there might be more.
209.249.11.3
Dan