About the Ducted bots
Information for site owners and administrators who see our automated requests in their server logs.
What this is
Ducted (https://ducted.app) is a product feed management tool. To keep merchants’ feeds up to date, we run automated fetchers that retrieve files our customers have explicitly configured. If you spotted one of our User-Agent strings in your logs, this page explains what it does and how to control it.
Each activity has its own User-Agent string, and every one of them points back to this page.
Feed Fetcher
User-Agent
Mozilla/5.0 (compatible; DuctedFeedFetcher/1.0; +https://ducted.app/bot-info)
What it does: Retrieves the specific product feed file(s) a merchant has configured in their Ducted account (for example an XML, CSV, or JSON feed URL). When a merchant sets up a project by pasting their own website or store address (or re-checks it later), it also reads that site's homepage once, to recognise the platform the site runs on and the business address its structured data states, and tries the few data endpoints that platform is known to publish.
What it does not do: Does not scrape product pages, does not crawl or follow links, and fetches nothing beyond the feed URL(s) a merchant has configured and, during that one-off setup check, the homepage and platform data endpoints of the site the merchant named.
Typical request pattern: Requests are made on the merchant's chosen refresh schedule (at most twice a day, and daily on most plans), plus the occasional one-off fetch when a merchant saves or tests a feed. A setup check of a pasted address is a handful of requests, at most ten, once. Ducted requests only the configured feed URL - typically a single request per refresh. The very first fetch of a newly added feed may arrive twice, once from each region below, so we can determine which one your server accepts; after that it is one request from one region.
Geocoder
User-Agent
Mozilla/5.0 (compatible; DuctedGeocoder/1.0; +https://ducted.app/bot-info)
What it does: Looks up a business address a merchant enters, or one found on their website, to place it on the map (its latitude and longitude for their feeds). It only queries OpenStreetMap's Nominatim service and, for US addresses Nominatim cannot place, the US Census Bureau geocoder.
What it does not do: Does not contact merchants' or anyone else's websites, and sends nothing but the address being looked up.
Typical request pattern: At most one request per second to Nominatim, one lookup per address, and only when a merchant saves an address or asks for its map position. Every answer is cached, so the same address is not looked up again.
Where our requests come from
Our feed fetches currently come from these addresses:
We add addresses rarely and announce changes rather than rotating silently. If you intend to allowlist us by IP, please contact us first so we can confirm the current set and tell you before it changes - matching on the User-Agent above is the more reliable option.
Why you may see us from more than one region. Some feed providers only serve certain countries, so we may retry a request from another region to reach a feed our customer has asked us to collect. Our identity never changes with the address: the same User-Agent and the same headers are sent from every one of them. We do not use residential or rotating proxy services, and we do not disguise ourselves as a browser.
Verifying that a request is really ours
A User-Agent string can be copied by anyone, so don’t rely on it alone. The address list above is the authoritative check - it is the one we keep current. As a second, stronger confirmation, the addresses we fetch from carry forward-confirmed reverse DNS, the same method used to verify Googlebot:
# 1. reverse-lookup the connecting address dig -x <ip> +short # a hostname ending in .ducted.app # 2. resolve that hostname forward again dig +short <hostname> # must return the address you started from
If both steps agree, the request genuinely came from us - that cannot be forged without control of our DNS. A lookup that does not match is not proof of the opposite: check the address list above, since we may have added a route whose reverse DNS is not in place yet. If something claims to be us from an address that is not listed, please tell us - we would want to know.
How to block or throttle us
We respect your control over your own servers. You can block or rate-limit our requests by matching the User-Agent string above, or by the addresses listed here - no special handling is required on your end.
If our requests are causing you any trouble, please reach out before blocking so we can adjust our behaviour or remove the affected feed. If you are a feed provider and one of your merchants has asked you to allow us through, allowlisting the User-Agent above is all that is needed.
Contact
Questions, complaints, or opt-out requests: admin@ducted.app.