Integrating SpiderFoot, Nmap, and Metagoofil

When you're running a red team operation, you need to have an intimate understanding of your target. The trifecta of SpiderFoot, Nmap, and Metagoofil can provide you with a comprehensive picture of your target’s content, structure, and technical details. Let’s dive into how to combine these tools effectively.

First, let's start with SpiderFoot. This tool is an OSINT powerhouse, automating the collection of a wide range of data about your target. You kick things off by running SpiderFoot to gather initial intel:

    spiderfoot -s targetsite.com -o results.xml

This command directs SpiderFoot to focus on "targetsite.com", compiling an extensive array of information and saving it into an XML file. SpiderFoot will scour various sources, pulling in data about domains, IP addresses, email addresses, and more. This dataset forms the baseline of your reconnaissance.

Once you’ve got this preliminary data, it’s time to drill down into the network layer using Nmap. Leveraging the IP addresses and subdomains discovered by SpiderFoot, you can perform a comprehensive network scan:

    nmap -A -oN nmap_scan.txt -iL spiderfoot_ip_list.txt

Here’s the breakdown: -A enables aggressive scan options including OS detection, version detection, script scanning, and traceroute. The -oN nmap_scan.txt flag ensures that the results are saved in a readable text file. The -iL spiderfoot_ip_list.txt argument specifies a list of IPs extracted from the SpiderFoot results. This detailed scan will uncover open ports, services running on those ports, and potential vulnerabilities.

Next, we move to Metagoofil, which specializes in extracting metadata from publicly accessible documents on your target’s site. This is where you unearth details like usernames, software versions, and document creation paths:

    metagoofil -d targetsite.com -t pdf,doc,xls -o metagoofil_output/ -f metagoofil_results.html

This command directs Metagoofil to focus on "targetsite.com", targeting document types such as PDFs, DOCs, and XLS files. The results are saved in a specified directory and compiled into an HTML report. Metagoofil’s insights can reveal critical details about the internal structure and software environment of your target.

Now, let’s put all of this together into a cohesive workflow. You start with SpiderFoot to gather broad intelligence, then use Nmap to perform a detailed network scan, and finally employ Metagoofil to extract specific metadata. Here’s a streamlined approach:

    Run SpiderFoot to gather comprehensive data on your target.

    Extract IP addresses and domains from SpiderFoot’s output.

    Feed these IPs into Nmap for an in-depth network scan.

    Use Metagoofil to pull out metadata from the target’s documents.

To make this process even more efficient, you can automate the extraction and handoff of data between tools using scripting. For example, here’s how you can extract IPs from SpiderFoot and pass them to Nmap:

    grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' results.xml > spiderfoot_ip_list.txt

    nmap -A -oN nmap_scan.txt -iL spiderfoot_ip_list.txt

This script uses grep to identify IP addresses within the SpiderFoot XML output and saves them into a text file for Nmap to use. This automation saves time and ensures you don’t miss any critical data points.

By seamlessly integrating SpiderFoot, Nmap, and Metagoofil, you create a robust pipeline of intelligence gathering. SpiderFoot provides the initial reconnaissance, Nmap delivers a deep dive into the network infrastructure, and Metagoofil offers granular insights from document metadata. This multi-faceted approach ensures that you’re fully equipped with detailed, actionable intelligence about your target.