# Applies to all robots User-agent: * # Stop these folders from being trawled Disallow: /bin/ Disallow: /common/flash/ Disallow: /common/images/ Disallow: /common/js/ Disallow: /common/style/ Disallow: /Components/ Disallow: /hidden/ Disallow: /StockTicker/ Disallow: /shared/common/ Disallow: /shared/email/ Disallow: /shared/hidden/ Disallow: /shared/indexes/ Disallow: /shared/xml/ # Location of sitemap Sitemap: http://www.mmgroupltd.com/sitemap.xml # Number of secs to wait between successive requests Crawl-delay: 5 # Maximum rate is one page every 5 secs Request-rate: 1/5 # Only visit between 3:00 AM and 8:45 AM UT (GMT) Visit-time: 0300-0845