Sitemapper is only spidering a small minority of the links on my

Home » Forums » Sitemapper » Sitemapper is only spidering a small minority of the links on my


Registered User
1 post

I have just recently used Sitemapper to create and send a site map to Google for one of my websites www.campchef.com. This was completely successful with the site containing about 1000 links.

I now am moving on to the other four sites for which I need to generate Sitemaps. When I run Sitemapper against these other sites I am only getting a couple of the static pages. For instance, www.outdoorcooking.com only generates eight links when I would expect it to be easily over 1500. I am not using javascript, iFrames, or Flash to create the site menus. Simply a php backend process to create the links out of available product categories from the database. Also, in the options tab, I have left the defaults and not specified any directories or files to ignore.

Please let me know what I could be doing wrong.


Senior Advisor
1,332 posts

You might try a site such as this one:

http://www.delorie.com/web/lynxview.html

and see whether the spiders are actually able to navigate your links. Might be something easy to overlook that's preventing the mapping. Seeing the site from the "eyes" of a spider might help pinpoint any such problems. "You can't be a real country unless you have a beer and an airline - it helps if you have some kind of a football team, or some nuclear weapons, but at the very least you need a beer." -- Frank Zappa

Visit Spinland Studios: http://www.spinland.biz


Registered User
2 posts

OK so I'm having the same problem. I used delorie and found that the spiders can't find my links. Unfortunately this is because I used the CoffeeCup flash menu builder for the links from my main page. Is this a problem that can be fixed or am I just stuck with the problem that menus build using your program can't be spidered.


Senior Advisor
3,144 posts

Joshua,

If the sitemapper has a problem, so will Google. Also anyone with a screen reader or without flash will have a problem.

One suggestion is to have a menu similar to the CSS menu that is part of the CC HTML Editor upgrade that will be released soon. Bill R.
First Baptist Church
Brighter Day Charities
Roberson Family


Senior Advisor
1,398 posts

Joshua.

For some time now some search engines and Adobe have been working together for quite some time to make flash searchable, but it could be a long wy off just yet.

You could try this.

make a page and call it internl links , or somthink like that.
At the bottom of you front page put a text link to it.
Then run site mapper, hopefully it will find all the links.

If it does save you site map.
Next change the link at the bottom of the page to the proper sitemap and remove the old links page.
Run site mapper again and hopefully it will find all your links gain.
Jim
---------------------------
http://www.freebits.co.uk/


Ambassador
46 posts


You could try this.

make a page and call it internl links , or somthink like that.
At the bottom of you front page put a text link to it.
Then run site mapper, hopefully it will find all the links.


This worked for me.

My menus are run by javascript and Sitemapper would display my links. This suggestion turned out to be a solution for me. Thanks. GL Joe (P. Joseph Larson)

granitelabor.com


Registered User
3 posts

The sitemapper is really slow on my site http://www.bay-e.co.uk
last time I ran it, it took 4 days to complete. This is a php site and it need to generate around 30,000 links.

Do you know how to speed this up ?

Currently running version 5.0 Build 193


VP of Product Development
25,042 posts
Online Now

Charles London wrote:
The sitemapper is really slow on my site http://www.bay-e.co.uk
last time I ran it, it took 4 days to complete. This is a php site and it need to generate around 30,000 links.

Do you know how to speed this up ?

Currently running version 5.0 Build 193

Have less links ;) 30,000+ pages in a sitemap is really an over kill. Maybe try reducing the types of files it is trying to index.
~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_~
Scott A. Swedorski - VP of Product Development
CoffeeCup Software Inc.
Phone: 678.495.3480

"Want to update your website from anywhere? Check out our new free service, S-Drive Streams. Visit http://www.coffeecup.com/sdrive for more information."


Senior Advisor
2,037 posts

30,000 links. Dear God man, that's a lot of links.:D The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com


Registered User
3 posts

Yes it's a Big store. 32,000 items in fact. The pages are generated on the fly.

Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.