I am not sure that this is something that CC can do, however, I would be happy to be proved wrong. However, this site has an online generator that can be freely used
http://www.1-hit.com/all-in-one/tool-ro … erator.htm
http://www.webtoolcentral.com/webmaster … generator/
It realy depends on what function you wanted your robot.txt to perform as to what you should be looking at. Is it to try and stop nasty people expoliting your site, or perhaps as a promotion tool. You may wish to look at this site as a source for further info http://webseodesign.com/resources/robots-txt-files.htm
I hope that some of this helps.
Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.