Honeypot

A honeypot is a decoy system or resource designed to attract suspicious activity so defenders can detect and study it. It looks usable or valuable to an intruder or automated client but is instrumented for observation and isolated from real assets.

Network defenders deploy honeypots as fake services, credentials, files, form fields, API routes, or links. A web crawler may encounter a trap link that ordinary visitors cannot reach through visible navigation. Following it can reveal indiscriminate crawling, ignored directives, or an attempt to explore hidden paths. Other traps use URLs that expand without limit, which can consume a crawler’s queue and resources.

A honeypot does not need to contain a vulnerability or respond with realistic data. Its value comes from the low likelihood of ordinary interaction, which makes access a useful signal. Defenders must still isolate the decoy, minimize stored sensitive data, and treat observations carefully because scanners and misconfigured clients can trigger it without malicious intent.

Responsible crawlers stay within their intended scope, honor robots.txt and site terms, avoid hidden or unbounded URL spaces, and stop when access is denied. They should not try to identify and bypass defensive traps.

Related terms: web crawler and IP ban.