SpiderFoot OSINT Tool: Automated Reconnaissance Framework

What is SpiderFoot?

SpiderFoot is an open-source OSINT automation tool built in Python 3 that automates reconnaissance across a wide array of public data sources. You give it a target — a domain, IP address, email, phone number, username, or even a Bitcoin address — and it chains together over 200 modules that feed each other data to build a complete external profile. It ships with a web-based interface for interactive exploration and a CLI for scripted or headless operation, storing all findings in SQLite so you can query them directly with custom SQL.

Installing SpiderFoot

Install SpiderFoot by downloading the latest stable tarball from GitHub, extracting it, and installing Python dependencies with pip. The stable release path: `wget https://github.com/smicallef/spiderfoot/archive/v4.0.tar.gz && tar zxvf v4.0.tar.gz && cd spiderfoot-4.0 && pip3 install -r requirements.txt`. For the bleeding-edge development version, clone the repository directly with `git clone https://github.com/smicallef/spiderfoot.git` then run pip install. Docker users can build from the included Dockerfile for a containerized deployment. Python 3.7 or higher is required, and many modules will need their respective API keys configured in SpiderFoot's settings before use.

Running SpiderFoot via CLI and web UI

Start the web UI with `python3 ./sf.py -l 127.0.0.1:5001`, then open that address in a browser. From the web interface, click 'New Scan', enter your target (domain, IP, email, etc.), select which module categories to run, and click 'Run Scan'. For CLI-only operation, use `python3 sf.py -s <target> -t <target_type>` where target type is one of: IP, DOMAIN, SUBDOMAIN, HOSTNAME, NETBLOCK_OWNER, ASOWNER, EMAILADDR, PHONE_NUMBER, USERNAME, BITCOIN_ADDRESS, or PERSON_NAME. The `-M` flag limits which modules run, and `-o` controls output format (JSON, CSV, or GEXF). Run `python3 sf.py -h` for the full flag reference.

Key modules and capabilities

• ✓Over 200 modules that feed each other via a publisher/subscriber model, maximizing data extraction from each target
• ✓Dual interface: a web-based UI for interactive investigation and a CLI for scripted or automated scanning
• ✓37 pre-defined YAML correlation rules to pattern-match findings into actionable leads
• ✓SQLite backend storing all scan data for custom SQL queries and historical analysis
• ✓TOR integration for dark web searching and anonymity during reconnaissance
• ✓Exports scan data as CSV, JSON, or GEXF for ingestion into other tools like Maltego or Gephi
• ✓Calls external tools like DNSTwist, Whatweb, Nmap, and CMSeeK to extend built-in capabilities
• ✓Correlation engine fully configurable via YAML — write your own rules or extend the 37 pre-defined ones

Offensive and defensive use cases

• •Red team reconnaissance: spider a target's entire external footprint — subdomains, exposed buckets, misconfigured DNS, associated cloud assets — before an engagement begins
• •Threat intelligence: correlate findings across targets to identify patterns like shared infrastructure, reused Bitcoin wallets, or leaked credentials indicating compromise
• •Attack surface monitoring: use the CLI in a cron job to periodically rescan critical domains and detect new subdomains, certificates, or exposed services before attackers do
• •Due diligence: investigate an individual's digital presence by correlating usernames, emails, names, and social accounts across breach databases and dark web sources
• •Incident response: rapidly assess what an attacker could have discovered about your organization from public sources to understand your exposed blast radius
• •Bug bounty recon: automate subdomain enumeration and screenshotting across a large scope to identify in-scope targets faster

SpiderFoot alternatives

Who should use SpiderFoot?

SpiderFoot fits security consultants and internal red teams who need to map a target's external attack surface quickly before an engagement, threat intelligence analysts who want to correlate findings across targets without building bespoke pipelines, and bug bounty hunters who need automated subdomain enumeration across large scopes. It is less suited for casual use — if you just need a quick email lookup or subdomain check, a simpler single-purpose tool will get you there faster without the overhead. Defenders who want to understand their organization's public footprint will also benefit from periodic scans, especially if they lack dedicated attack surface monitoring infrastructure.

Frequently asked questions

Related repositories