Forum Archive - Question about installation step 5

Question about installation step 5
Dec 18, 2011 5:53 pm by cyberkeyhan
Hi, I followed installation steps and installed the pro version, it works fine. I have created sitemap.txt with the languages I want to be indexed, but I didn't understand the step 5 in the readme file, "modify robots.txt.... 1- I don't find any robots.txt file on the root or anywhere else on my site, should I create one myself? 2- Except the sitemap: yourdomain.com/sitemap.txt what else should be inside? I don't know for example what should be DISALLOWED and what allowed from security aspect. Should I put a(*) for User-agent, allowing all user agents or limit it for example to only Googlebot? Is it correct to put the following in the file: User-agent: * Disallow: Sitemap: http://yourdomain.com/sitemap.txt Best Regards
Re: Question about installation step 5
Dec 19, 2011 10:53 am by Edvard
You can just create a blank robots.txt file and put this: Sitemap: http://yourdomain.com/sitemap.txt

SIMILAR TOPICS

Pre Sales Question ?Oct 18, 2017 6:37 amReplies: 1Post by: jackB2010
Question about SSLJul 9, 2017 8:30 amReplies: 3Post by: Travelaroundtheworld
Multiple question (login, authorization to edit...)Feb 20, 2017 12:47 pmReplies: 9Post by: José Soares
SSL wildcard installation on your serverFeb 3, 2017 7:50 amReplies: 1Post by: partychemicals
PHP strict standards errors / PHP 7 Compatability questionOct 6, 2016 10:53 amReplies: 1Post by: furnivat

Try GTranslate with a free 15 day trial