Threat Intel Tools
The following is a list of tools and services that our analysts have come across during their day to day work for Cyberfive. Each time we find a useful tool or service we will add it to the list for reference (just because a tool is on our list does not mean we use it – we just think it deserves a mention as a useful resource for anyone learning OSINT).
For now, where possible, we are only listing tools useful for OSINT recon work and not direct pentesting (it’s difficult to draw a line in some areas and there are so many good pentesting tools out there).
Each has been flagged depending on whether they are free* or paid and whether they have an API available. If you think we are missing something or our information is incorrect or out of date then just send us a message via our contact form and we will rectify it.
Each category has been broken down into ‘Online Services’ or ‘Tools’, the latter being options that can be hosted and run internally (and therefore private – always good when conducting OSINT).
* free does not mean they can be used commercially – read the terms & conditions of all tools if you plan to use them for a commercial purpose.
(F) = Free
(P) = Paid
(A) = API
Infrastructure & IoT Search Engines
Online Services
Shodan – The original search engine for internet-connected equipment. (F)/(P)/(A)
Censys – Another search engine for internet-connected equipment. (F)/(P)/(A)
ZoomEye – A Chinese search engine for internet-connected equipment. (F)/(P)/(A)
Fofa – Another Chinese search engine for internet-connected equipment. (F)/(P)/(A)
BinaryEdge– Another search engine for internet-connected equipment but also includes data from Torrent downloads, DB leaks and Honeypots. (F)/(P)/(A)
Thingful – A search engine for the ‘Internet Of Things’ with data sourced across multiple datasets (currently in beta). (F)/(A)
PurplePee – Relational search tool for website HTTP headers, DNS records, SSL certificates and open TCP ports as well as ASN whois information. (F)/(A)
Tools
Ivre – Ever wanted to build your own Shodan service? Well now you can. (F)
Dark Web Search Engines
Online Services
Torch – One of the oldest dark web search engines claiming to have indexed over 1 million pages. (F)
http://xmh57jrzrnw6insl.onion.to (clear web access via Tor2Web)
NotEvil – Probably the closest you’ll get to a ‘google-like’ dark web search engine. (F)
http://hss3uro2hsxfogfq.onion.to (clear web access via Tor2Web)
Ahmia – On a mission to become the premiere search resource for the dark web and indexes I2P results as well as TOR. (F)
http://msydqstlz2kzerdg.onion/search/
https://msydqstlz2kzerdg.onion/i2p/ (I2P search)
https://ahmia.fi/search/ (clear web)
https://ahmia.fi/i2p/ (clear web I2P search)
Candle – Another dark web search engine. (F)
https://gjobqjj7wyczbqie.onion
https://gjobqjj7wyczbqie.onion.to (clear web access via Tor2Web)
Beacon – A privately built, commercial dark web search engine. (P)
https://www.echosec.net/darknet
OnionLand – A clear/dark web hosted search engine for the TOR network. (F)
https://onionlandsearchengine.com
http://3bbad7fauom4d6sgppalyqddsqbf5u5p56b5k5uk2zxsy3d6ey2jobad.onion (v3 service)
I2PSearch – A clear/dark web hosted search engine for the I2P network from the team behind OnionLand. (F)
Tools
FreshOnions – GitHub project to build your own TOR scraper (and the same project that powered the now defunct ‘FreshOnions’ search site that was hosted at http://zlal32teyptf4tvi.onion. (F)
Bug Bounty Indexes
Online Services
Open Bug Bounty – A not-for-profit open bug bounty program intended to make the Internet a safer place (used to beXSSposed.org). (F)
https://www.openbugbounty.org/
Hacker One – Commercially run bug bounty platform delivering individual company bounty programs. Has a searchable list of recent activity. (F)/(P)
https://hackerone.com/hacktivity
Bounty Target Data – Data dump provided hourly of all in-scope and out-of-scope domains for HackerOne and BugCrowd programs. (F)
https://github.com/arkadiyt/bounty-targets-data
BugHeist – Find bugs, win points and prizes – Says it all. Has a searchable list of latest activity. (F)/(P)
Tools
None identified yet.
Reputation & IoC Lookups
Online Services
Apility – Reputational database for IP address or hostname searches. (F)/(P)/(A)
Artists Against 419 – Searchable database of known fake/fraudulent scam websites. (F)/(A)
https://db.aa419.org/fakebankslist.php
Malc0de – Searchable database of malicious domains, IPs and URLs. (F)
Project Honeypot – Searchable database of IP addresses and associated malicious event behaviour. (F)
https://www.projecthoneypot.org
HoneyDB – Searchable database of data gathered from a network of honepots. (F)/(A)
Scumware – Searchable database of malicious domain names and URLs. (F)/(P)/(A)
URLHaus – Searchable database of domain names, URLs, IP addresses and file hashes. (F)/(A)
https://urlhaus.abuse.ch/browse/
VoIPBL – Searchable database of IP addresses associated with VoIP attacks. (F)
Blocklist – Searchable database of IP addresses and ASN numbers for malicious activity. (F)/(P)/(A)
http://www.blocklist.de/en/search.html
AlienVault OTX – A commercially backed TI exchange with comprehensive indicator searches. (F)/(P)/(A)
ThreatConnect – Another commercially backed TI exchange with comprehensive indicator searches. (F)/(P)/(A)
AbuseIPDB – Reputational database for IP address or hostname searches. (F)/(P)/(A)
BrightCloud – URL/IP lookup which provides reputational data, proxy categorisation and whois data. (F)/(P)/(A)
https://www.brightcloud.com/tools/url-ip-lookup.php
Desenmascara.me – Brand protection database for identifying fake websites. (F)/(P)/(A)
FortiGuard lookup – Proxy categorisation lookup tool (for FortiGuard devices). (F)/(P)/(A)
https://fortiguard.com/webfilter
Google Safe Browsing – Look up a website/URLs current status. (F)/(P)/(A)
https://transparencyreport.google.com/safe-browsing/search
HashDD – Lookup service for IP, Domain, URL and file hash information. (F)/(P)/(A)
IBM X-Force Exchange – Lookup service for IP, Domain, URL, application, vulnerability and file hash information as well as keyword searching. (F)/(P)/(A)
https://exchange.xforce.ibmcloud.com/
Joe Sandbox URL Analyzer – Online sandboxing service with search functionality across the dataset. (F)/(P)/(A)
IsItPhishing – Online search tool for URLs, brand names and subnets to identify phishing pages. (F)/(P)/(A)
Kaspersky OpenTip – Online sandbox service with search function across the dataset for file hash, IP, domain name or URL. (F)/(P)/(A)
https://opentip.kaspersky.com/
Norton Safe Web – Reputational database for domain name or URL. (F)/(P)/(A)
PhishTank – Search URL, brand or ASN in a database of known phishing websites. (F)/(P)/(A)
https://www.phishtank.com/phish_archive.php
Malware Domain List – Lookup IP, domain name or URL. (F)
http://www.malwaredomainlist.com/mdl.php
MalwareURL – Lookup IP, domain name or URL. (F)/(P)/(A)
https://www.malwareurl.com/listing-urls.php
McAfee TrustedSource – Check a domain name/URL for the proxy categorisation according to McAfee. (F)/(P)/(A)
https://www.trustedsource.org/en/feedback/url
MxToolbox – Check SMTP servers (IP/FQDN) against multiple blacklists. (F)/(P)/(A)
https://mxtoolbox.com/blacklists.aspx
RiskIQ – Search a multitude of different IoCs against a comprehensive TI dataset. (F)/(P)/(A)
Pulsedive – Search IP, domain names and URLs and augment with additional information (whois, ssl etc). (F)/(P)/(A)
Reputation Authority – Reputational database for domain name or URL. (F)/(P)/(A)
http://www.reputationauthority.org/
Scamadviser – Interesting lookup service to identify if a website is likely to be a scam or not. (F)/(A)
SecurityTrails – Search IP, keywords, domain names and URLs and augment with additional information (whois, DNS etc). (F)/(P)/(A)
Talos Reputation Lookup – Search IP, domain names and URLs for reputational information. (F)/(P)/(A)
https://talosintelligence.com/
Trend Micro Site Safety Center – Search IP, domain names and URLs for reputational information. (F)/(P)/(A)
https://global.sitesafety.trendmicro.com/
Unmask Parasites – Looks up the URL in the Google Safe Browsing database and performs a scan for illicit and/or hidden content. (F)
http://www.unmaskparasites.com/security-report/
urlscan.io – Online sandboxing service with search functionality for IP, domain names, file hashes, ASNs and URLs for reputational information. (F)/(P)/(A)
URLVoid – Search URLs for reputational information. (F)/(P)/(A)
IPVoid – Search IPs for reputational information. (F)/(P)/(A)
VirusTotal – Online multi-product A/V scanner service with search functionality for IP, domain names, file hashes and URLs for reputational information. (F)/(P)/(A)
https://www.virustotal.com/gui/home/search
ThreatMiner – Search a multitude of different IoCs against a comprehensive TI dataset. (F)/(A)
WebPulse Site Review – Proxy categorisation according to Symantec. (F)/(P)/(A)
https://sitereview.bluecoat.com/
Zscaler Zulu URL Risk Analyzer – Risk scoring reputational lookup for URLs. (F)/(P)/(A)
zveloLIVE – Proxy categorisation according to zveloLIVE. (F)/(P)/(A)
Tools
None identified yet.
Defacement Databases
Online Services
Zone-H – One of the longest running defacement databases. (F)
Tools
None identified yet.
Data Collection Frameworks
Online Services
SpiderFoot-HX – Automating OSINT collection so you can focus on analysis! Cloud-hosted or on-premise. (P)/(A)
Tools
Recon-NG – A full-featured reconnaissance framework designed with the goal of providing a powerful environment to conduct open source web-based reconnaissance quickly and thoroughly. (F)
https://github.com/lanmaster53/recon-ng
TheHarvester – Use it for open source intelligence (OSINT) gathering to help determine a company’s external threat landscape on the internet. The tool gathers emails, names, subdomains, IPs and URLs using multiple public data sources. (F)
https://github.com/laramies/theHarvester
SpiderFoot – Automating OSINT collection so you can focus on analysis! (F)
X-Ray – XRay is a tool for network OSINT gathering, its goal is to make some of the initial tasks of information gathering and network mapping automatic. (F)
https://github.com/evilsocket/xray
ReconDog – Tool for collecting technical information about target domains using a variety of 3rd party APIs so collection is performed passively. (F)
https://github.com/s0md3v/ReconDog
ReconSpider – ReconSpider is an OSINT Framework for scanning/collecting IP Address, Emails and Websites. Results are displayed in a dashboard for analysis. (F)
https://github.com/bhavsec/reconspider
DataSploit – Another OSINT recon framework for automating data collection and correalation with 3rd party data sources. (F)
https://github.com/dvopsway/datasploit
FinalRecon – Another OSINT recon framework this time more geared towards website recon. (F)
https://github.com/thewhiteh4t/FinalRecon
Aquatone – A tool for visual inspection of websites across a large amount of hosts and is convenient for quickly gaining an overview of HTTP-based attack surface. (F)
https://github.com/michenriksen/aquatone
Web Vulnerability Scanners
Online Services
WPSec – WordPress vulnerability scanner. (F)/(P)/(A)
Is It Hacked? – Vulnerability scanner aimed at Joomla, Drupal, WordPress, Magento, Prestashop, Laravel, Codeigniter, Symfony and more. (F)/(P)/(A)
Sucuri SiteCheck – Scan a website for malware, vulnerabilities, out of date software and other security issues. (F)/(P)/(A)
Tools
None identified yet.
Reputation & IoC Lists
Online Services
NetScout – Commercially supplied set of threat intelligence feeds. (P)/(A)NetScout – Commercially supplied set of threat intelligence feeds. (P)/(A)
https://www.netscout.com/product/atlas-intelligence-feed-aif
Blackweb – Opensource aggregation of freely available threat intel feeds which are then optimised and presented for use by Squid proxies. (F)
https://github.com/maravento/blackweb
Clean-MX – Spam/phishing intelligence feeds, available only with prior registration. (F)/(P)/(A)
https://support.clean-mx.com/clean-mx/viruses
IntelStack – Feed aggregation from multiple freely available sources and then optimised for Bro/Zeek use. (F)/(A)
Team Cymru Bogon List – List of IP addresses and networks which should never appear in the global routing table. (F)/(A)
https://www.team-cymru.com/bogon-reference.html
DShield – List of suspicious and malicious IP addresses. (F)/(A)
https://www.dshield.org/ipsascii.html
FireHOL – Free aggregation service for multiple other feeds. (F)
Malc0de – Domain and IP lists for malicious activities identified in the last 30 days provided in BIND format, WIndows DNS format and simple IP list. (F)
Malware Domains – Blocklist for domains associated with or actively delivering malware. (F)
http://www.malwaredomains.com/wordpress/?page_id=66
Malware Domain List – Another blocklist for domains associated with or actively delivering malware. (F)
http://www.malwaredomainlist.com/hostslist/hosts.txt
OpenPhish – Lists identifying domains associated with phishing attacks. (F)/(P)/(A)
https://openphish.com/phishing_feeds.html
URLHaus – Feeds of malicious activity based on ASN, Country and domain name. (F)/(A)
https://urlhaus.abuse.ch/feeds/
VoIPBL – Blacklists of malicious VoIP activity in fail2ban format. (F)/(A)
Blocklist – Ip and domain name blacklists covering a wide range of attack activity. (F)/(P)/(A)
Tools
None identified yet.
Indicator Management
Online Services
None identified yet.
Tools
None identified yet.
Online Review Checkers
Online Services
Review Meta – Online tool for spotting fake Amazon product reviews. (F)/(A)
The Review Index – Online tool for spotting fake Amazon, Steam or Flipkart reviews. (F)
Fakespot – Online tool for spotting Amazon, Steam, BestBuy, Sephora, Walmart, TripAdvisor and Yelp reviews. (F)
Tools
None identified yet.
Metadata Tools (File)
Online Services
MetadataRemover – Website that allows you to upload documents, see what metadata they hold and then republish the document without the metadata present (API coming soon). (F)
Tools
OutlookExport – A tool for exporting metadata from a users Outlook mailbox. (F)
https://www.codetwo.com/freeware/outlook-export/
FOCA – A tool that searches for common document formats on a specific website, downloads those files and then automatically extracts the metadata contained within. (F)
https://github.com/ElevenPaths/FOCA
Metagoofill – Another tool for locating documents from a specific domain, downloading them and extracting their metadata. (F)
https://github.com/kurobeats/metagoofil
ExifTool – A platform independent application and library (ported to a number of different languages) capable of extracting metadata from a wide variety of formats. (F)
Anonymouth – A java application created to identify and remove authorship and grammatical style indicators which may give-away the identity of the author through their writing style. (F)
https://github.com/psal/anonymouth
PDF-Redact Tools – PDF Redact Tools helps with securely redacting and stripping metadata from documents before publishing. (F)
https://github.com/firstlookmedia/pdf-redact-tools
EXIFPurge – A small portable application to remove EXIF meta data from multiple images at once. (F)
GooFile – Application for identifying particular document filetypes associated with a domain. (F)
https://code.google.com/archive/p/goofile/
Spiderpig – Metadata harvester written in Ruby. (F)
https://github.com/hatlord/Spiderpig
Metadata Tools (Image)
Online Services
Jeffrey’s Image Metadata Viewer – Website that allows you to upload image files to see what metadata they hold. (F)
http://exif.regex.info/exif.cgi
FotoForensics – FotoForensics provides budding researchers and professional investigators access to cutting-edge tools for digital photo forensics. (F)/(P)
Photo Forensics – Online toolset for forensic analysis of individual digital images. (F)
https://29a.ch/photo-forensics/
Reveal Image Verification Assistant – This tool features a multitude of image tampering detection algorithms plus metadata analysis, GPS Geolocation, EXIF Thumbnail extraction and integration with reverse image search via Google. (F)
http://reveal-mklab.iti.gr/reveal/index.html
TinEye – This tool provides ‘reverse’ image searching capabilities. Upload an image of your choice and let TinEye tell you where else the image appears on the web. (F)
Tools
ExifTool – A platform independent application and library (ported to a number of different languages) capable of extracting metadata from a wide variety of formats. (F)
EXIFPurge – A small portable application to remove EXIF meta data from multiple images at once. (F)
IRFanView – Application for viewing image files and associated metadata. (F)/(P)
Ghiro – Opensource automated forensic processing and analysis platform for digital images. (F)
Invid Verification Plugin – Browser plugin to save time and be more efficient in fact-checking and debunking tasks on social networks especially when verifying videos and images. (F)
https://www.invid-project.eu/tools-and-services/invid-verification-plugin/
VGG Image Classification (VIC) Engine – VIC is a web application that serves as a web engine to perform image classification queries over an user-defined image dataset. (F)
http://www.robots.ox.ac.uk/~vgg/software/vic/
VGG Face Finder (VFF) Engine – VFF is a web application that serves as a web engine to perform searches for faces over an user-defined image dataset. (F)
http://www.robots.ox.ac.uk/~vgg/software/vff/
VGG Image Search Engine (VISE) – VISE is a standalone application can be used to make a large collection of images searchable by using image regions as query. (F)
Typo-Squatting Tools
Online Services
DNSTwister – Online implementation of the excellent “DNSTwist”. (F)/(P)/(A)
Tools
DNSTwist – Probably the best known typo-squatting detection tool. (F)
https://github.com/elceef/dnstwist
URLCrazy – Very similar to DNSTwist, written in Ruby and requires a license for commercial use. (F)/(P)
https://github.com/urbanadventurer/urlcrazy
CatPhish – Another domain permutation generator. (F)
https://github.com/ring0lab/catphish
PunyDomainCheck – Another domain permutation generator although this one focusses on homograph attacks (aka punycode). (F)
https://github.com/anilyuk/punydomaincheck
URLInsane – Another domain permutation generator which hopes to have the combined functionality of URLCrazy & DNSTwist in one tools. (F)
https://github.com/cybint/urlinsane
DomainFuzz – Another domain permutation generator written for NodeJS. (F)
https://github.com/monkeym4ster/DomainFuzz
EvilURL – Another domain permutation generator but aimed at homograph attacks (aka punycode). (F)
https://github.com/UndeadSec/EvilURL
Exploit Databases
Online Services
Exploit-DB – Exploits for Pen Testers – Need we say any more?. (F)
Tools
None identified yet.
Malware Analysis Solutions
Online Services
Intezer – Not your run of the mill sandbox, Intezer examines malware based on a ‘DNA concept’ (code and behavioural comparison to other samples) in order to identify malicious behaviours and attribute these to known strains or families of malware. (F)/(P)/(A)
Hybrid Analysis – Online sandboxing service powered by Crowdstrikes ‘Falcon’ product. (F)/(P)/(A)
https://www.hybrid-analysis.com/
AMAaaS – Online sandboxing service specifically aimed at analysis of Android malware. (F)
BinaryGuard – Online ‘bare-metal’ sandboxing service. Bare metal services don’t rely on using virtual machines but instead use actual live machines which are quickly restored to a known good state after the analysis has been run. (F)/(P)/(A)
Iris-H – Online static-analysis service aimed at various document formats. (F)/(A)
Cape Sandbox – Online sandbox analysis service aimed at executables, URLs and PCAPs. Also includes a static analysis service too (F)/(A)
Comodo ‘Valkyrie’ – Online sandbox analysis service. (F)/(A)
Pikker.ee – Online sandbox analysis service powered by Cuckoo. (F)
Sndbox – Online sandbox analysis service. (F)/(P)/(A)
ViCheck – Online malware analysis service for files, URLs or phishing emails (forward to sample(@)vicheck.ca). (F)/(P)/(A)
AnyRun – Online sandbox analysis service. (F)/(P)/(A)
JoeSandbox – Online and appliance based sandbox analysis service. (F)/(P)/(A)
https://www.joesandbox.com/ (free)
https://www.joesecurity.org/ (Commercial products)
OpSwat – Online and appliance based sandbox analysis service and multi-product A/V scanner. (F)/(P)/(A)
VirusTotal – The original multi-product A/V scanner – No list would be complete without VirusTotal. (F)/(P)/(A)
Jotti – Another online multi-product A/V scanner. (F)/(A)
VirScan – And another online multi-product A/V scanner. (F)/(A)
Tools
Cuckoo – The original open-source sandbox analyser solution. Cuckoo supports analysis under Windows, Linux, MacOS and Android. (F)/(A)
Malice – An open-source version of VirusTotal/OpSwat for multi-program A/V scanning. (F)/(A)
https://github.com/maliceio/malice
Yara – An open-source malware identification tool based on a comprehensive signature creation capability. (F)/(A)
https://github.com/VirusTotal/yara
IRMA – An open-source malware identification framework built on multi-program A/V scanning. (F)/(A)
https://github.com/quarkslab/irma
Internet Archiving Databases
Online Services
Archive.org AKA The WayBack Machine – Probably the most well-known internet content archive. Internet Archive is a non-profit library of millions of free books, movies, software, music, websites, and more. (F)
Archive.is – A time-capsule for web pages. (F)
CachedPages.com – View the cached versions of a website. (F)
CachedView.com – View the cached versions of a website, just like CachedPages.com. (F)
ScreenShots.com – Premium service supplied by DomainTools. (P)
Tools
BlackWidow – Webpage preservation tool. (F – 30day trial)/(P)
http://softbytelabs.com/wp/blackwidow/
WayBack Pack – Tool to download the history for a particular website as archived by the WayBack Machine. (F)
Data Analysis & Visualisation Tools
Online Services
Maltego – The daddy of OpenSource data analysis and visualisation tools. Available in the cloud (yours or theirs) as well as on-prem solutions. (F)/(P)/(A)
Tools
Maltego – The daddy of OpenSource data analysis and visualisation tools. Available in the cloud (yours or theirs) as well as on-prem solutions. (F)/(P)/(A)
General Search Engines
Online Services
None identified yet.
Tools
None identified yet.
Specialist Search Engines
Online Services
None identified yet.
Tools
None identified yet.
URL 'Unshorteners'
Online Services
None identified yet.
Tools
None identified yet.
