DirectoryBot
About DirectoryBot
DirectoryBot is Samonte Digital’s web crawler. It helps us build a directory of local businesses.
What it reads
DirectoryBot visits a business’s public website and reads its contact details, its links to social media pages, and whether the website is working.
It keeps only what the business publishes: business name, address, phone numbers and email addresses, social media links, and whether the site works. Some business contact details, such as an owner’s name or email, can be personal information; our privacy policy explains how we handle it.
It follows robots.txt
DirectoryBot obeys robots.txt. To block it from your whole site, add this to your robots.txt file:
User-agent: DirectoryBot
Disallow: /
DirectoryBot identifies itself with the user agent DirectoryBot/1.0 (+https://samontedigital.com/bot).
Corrections and removal
To correct your business’s details or have them removed, email connect@samontedigital.com.