Sitemapper 4.6 not excluding...

User 291040 Photo


Registered User
1 post

Using Sitemapper 4.6 on Windows XP

I can't seem to stop some of my subfolders being indexed. I've doing a site spider of http://www.frequencycast.co.uk , and am trying to exclude certain folders from the spider. An example of one I want to get rid of: /blogs/2008/, and any folders below it.

Under Tools > Options > "subfolders to ignore", I've tried various combinations:

http://www.frequencycast.co.uk/blog/2008
http://www.frequencycast.co.uk/blog/2008/
http://www.frequencycast.co.uk/blog/2008/*

... but these subfolders keep appearing in the results.

What am I doing wrong?

Pete
User 462238 Photo


Registered User
144 posts

Did you try inserting the Meta between head tags

<META NAME="ROBOTS" CONTENT="INDEX,N0 FOLLOW">
User 355448 Photo


Ambassador
3,144 posts

Pete,

You might also try entering the ignore as /blog/2008/ and see if that works.

Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.