Maximum size of website

User 424004 Photo


Trial User
8 posts

Hello,

Can sitemapper create sites with anywhere from 200,000 to 500,000 pages? Can it send multiple queries at a time? Can it divide the maps into 20,000 pages? compress them too? Can it make also some for search engines other than Google?

Thanks!
Frederic
User 562592 Photo


Registered User
2,038 posts

For your first question: no, I do not believe it can.
For the rest: I believe so.
The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com

User 424004 Photo


Trial User
8 posts

What is the biggest site it can crawl? :rolleyes:
User 424004 Photo


Trial User
8 posts

the site is www.petitesannoncesgratuites.com and there are a lot of pages....
User 562592 Photo


Registered User
2,038 posts

I am not sure what CC would tell you, but I recall a poster having problems with his site at aprox. 30,000 links.
The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com

User 424004 Photo


Trial User
8 posts

I tried for making a sitemap A1 sitemap Generator. So far it did find over 350,000 pages and keeps going! I am not sure how many pages my site has but is seems many..... So far so good! wait and see, fortunately the program can check a few links at a time otherwise it would take forever.
User 562592 Photo


Registered User
2,038 posts

Fred, let us know what you end up with. That would be good information to know. :D
The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com

User 424004 Photo


Trial User
8 posts

So far now I am at 482,000 pages..... A1sitemap Generator is still climbing..... Wao, that is a lot of pages.... All other sitemap program I did try did not go any close to that..... It seem to be like a nice company too, their tech guy Thomas helped me when I needed. I do run 4 threads at a time (I can adjust it higher or lower) but do not want to overload the server. I am doing the sitemap from overseas .... The server is in USA..... and the data for the site is in France :) I will keep you informed of the progress. Maybe there are much more pages than I expected !
User 424004 Photo


Trial User
8 posts

515,000 pages and rising....
User 562592 Photo


Registered User
2,038 posts

Wow!
The philosopher has not done philosophy until he has acted upon the mere conviction of his idea; for proof of the theory is in the act, not the idea.

My Web Development Company: http://www.innovatewebdevelopment.com (Created with Coffee Cup Software).

My Personal Website: http://www.EricSEnglish.com


Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.