Txt generator that may help you make this file. Be aware that if your site employs subdomains and you wish to have sure pages not crawled on a certain subdomain, you'll need to make a individual robots.txt file for that subdomain. For more info on robots.txt, we recommend this guidebook http://dominicktmewn.onesmablog.com/Facts-About-Web-Design-Revealed-40093105