Sitemapper is only spidering a small...

User 1911703 Photo


Registered User
1 post

I have just recently used Sitemapper to create and send a site map to Google for one of my websites www.campchef.com. This was completely successful with the site containing about 1000 links.

I now am moving on to the other four sites for which I need to generate Sitemaps. When I run Sitemapper against these other sites I am only getting a couple of the static pages. For instance, www.outdoorcooking.com only generates eight links when I would expect it to be easily over 1500. I am not using javascript, iFrames, or Flash to create the site menus. Simply a php backend process to create the links out of available product categories from the database. Also, in the options tab, I have left the defaults and not specified any directories or files to ignore.

Please let me know what I could be doing wrong.
User 597929 Photo


Registered User
1,332 posts

You might try a site such as this one:

http://www.delorie.com/web/lynxview.html

and see whether the spiders are actually able to navigate your links. Might be something easy to overlook that's preventing the mapping. Seeing the site from the "eyes" of a spider might help pinpoint any such problems.
"You can't be a real country unless you have a beer and an airline - it helps if you have some kind of a football team, or some nuclear weapons, but at the very least you need a beer." -- Frank Zappa

Visit Spinland Studios: http://www.spinland.biz
User 486573 Photo


Registered User
2 posts

OK so I'm having the same problem. I used delorie and found that the spiders can't find my links. Unfortunately this is because I used the CoffeeCup flash menu builder for the links from my main page. Is this a problem that can be fixed or am I just stuck with the problem that menus build using your program can't be spidered.
User 355448 Photo


Ambassador
3,144 posts

Joshua,

If the sitemapper has a problem, so will Google. Also anyone with a screen reader or without flash will have a problem.

One suggestion is to have a menu similar to the CSS menu that is part of the CC HTML Editor upgrade that will be released soon.
User 126492 Photo


Ambassador
1,524 posts

Joshua.

For some time now some search engines and Adobe have been working together for quite some time to make flash searchable, but it could be a long wy off just yet.

You could try this.

make a page and call it internl links , or somthink like that.
At the bottom of you front page put a text link to it.
Then run site mapper, hopefully it will find all the links.

If it does save you site map.
Next change the link at the bottom of the page to the proper sitemap and remove the old links page.
Run site mapper again and hopefully it will find all your links gain.
Jim
---------------------------
User 129028 Photo


Ambassador
46 posts


You could try this.

make a page and call it internl links , or somthink like that.
At the bottom of you front page put a text link to it.
Then run site mapper, hopefully it will find all the links.


This worked for me.

My menus are run by javascript and Sitemapper would display my links. This suggestion turned out to be a solution for me. Thanks.
GL Joe (P. Joseph Larson)

pj-larson.com
User 500165 Photo


Registered User
3 posts

The sitemapper is really slow on my site http://www.bay-e.co.uk
last time I ran it, it took 4 days to complete. This is a php site and it need to generate around 30,000 links.

Do you know how to speed this up ?

Currently running version 5.0 Build 193
User 103173 Photo


VP of Software Development
0 posts

Charles London wrote:
The sitemapper is really slow on my site http://www.bay-e.co.uk
last time I ran it, it took 4 days to complete. This is a php site and it need to generate around 30,000 links.

Do you know how to speed this up ?

Currently running version 5.0 Build 193

Have less links ;) 30,000+ pages in a sitemap is really an over kill. Maybe try reducing the types of files it is trying to index.
Learn the essentials with these quick tips for Responsive Site Designer, Responsive Email Designer, Foundation Framer, and the new Bootstrap Builder. You'll be making awesome, code-free responsive websites and newsletters like a boss.
User 562592 Photo


Registered User
2,038 posts

30,000 links. Dear God man, that's a lot of links.:D
The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com

User 500165 Photo


Registered User
3 posts

Yes it's a Big store. 32,000 items in fact. The pages are generated on the fly.

Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.