Introduction to Spiderfoot
Spiderfoot is a versatile and powerful open-source reconnaissance tool that can gather intelligence about a target. This tool is designed to automate various OSINT (Open-Source Intelligence) tasks, making it an essential tool for security professionals and penetration testers.
One of the most significant advantages of Spiderfoot is its ability to gather information from a wide range of sources. The tool can gather information from search engines, social media platforms, WHOIS records, DNS records, and more. This makes it possible to gather a significant amount of information about a target with minimal effort. Additionally, Spiderfoot allows you to customize the search modules to suit your specific needs, which further increases its effectiveness.
Another great feature of Spiderfoot is its user-friendly interface. The interface is intuitive and easy to use and allows you to view the results in a variety of formats, including HTML, CSV, and JSON. This makes it easy to share the results and integrate them into other tools.
Some of the targets we can collect information about using Spiderfoot include;
- Domain names
- IPV4 address
- IPV6 address
- Sub-domain/Hostname
- Subnet
- Bitcoin address
- E-mail address
- Phone number
- Username
- Official names
- Network ASN
Requirements
- Active internet connection.
- PC running on Kali Linux (You can install Kali in Virtualbox).
- Have Docker installed on your PC.
- Target
Install SpiderFoot
In this guide, we will be using dockerized Spiderfoot. We first download the tool from its GitHub repository using the below command.
git clone https://github.com/smicallef/spiderfoot.git
We then navigate into the newly created folder containing the tool’s files and using the dockerfile supplied with Spiderfoot, we create an image that will be used to run the container using the below command.
docker build -t spiderfoot .
Once the image creation is over, we can check if the image has been created by running the below command. As shown in the image below, we can see Spiderfoot image has been created.
docker images
In order to connect to it from outside the container, we need to map a port on the server that runs the container to that port using the below command. The -p argument will achieve that. As you can see we are mapping port 5009 here on the local server to port 5001 which is the port Spiderfoot is listening to within the container.
docker run -p 5009:5001 -d spiderfoot
After mapping the tool successfully we can connect to it via the port as shown below.
python3 ./sfcli.py -s http://0.0.0.0:5009
After connecting, we can use our favourite web browser to access Spiderfoot GUI by visiting http://0.0.0.0:5009 from where we can be able to add new and view already completed scans.
Using Spiderfoot to perform a scan
To perform a scan using Spiderfoot, we have to provide the domain name of our target as shown in the image below. The tool allows different ways to perform reconnaissance. Depending on the type of reconnaissance, we can choose to scan by use case, the required data or by the modules available on the tool. In this guide, we will be using an online version of OWASP juice-shop.
Once we have provided the required information, we can run the scan to start reconnaissance. A new task is added as shown in the image below.
When a scan is started, Spiderfoot scours the internet to find any information related to our target. Spiderfoot also has an automation feature that can automatically scan a target's IP address range, gather information about a target's DNS records, and even search for specific keywords on various websites. This automation feature saves a lot of time and effort for the user.
Reporting
Spiderfoot has the process of report creation and generation automated. From the information collected, the tool is able to organize the information and present it in a different form for it to be easily understood by the users.
By clicking on the name of the project, we can open a specific project to view the details of the scan. With each project, we can view the summary report, view the correlations, browse the elements and view a graph of the project's report.
Summary report
On the summary page, we can view different information discovered about our target. Spiderfoot fetches the information found on the internet and is related to the target. Different kinds of information have different colour codes depending on their level of severity.
Browse the elements
Under the Browse tab, we can be able to browse through the different information collected by Spiderfoot to view the specific details of what has been discovered by the tool. The elements are categorized according to their type. From the Browse tab, we can also view the unique and total data elements.
Graph
The tool has a graphical interface from where the user can view how each of the discovered elements relates.
Additionally, Spiderfoot has a built-in reporting feature which makes it easy to present the results of your reconnaissance to others. The report can be customized and exported in various formats such as PDF, HTML, and CSV.
Conclusion
In conclusion, Spiderfoot is a powerful and versatile reconnaissance tool that can gather intelligence about a target. Its ability to gather information from a wide range of sources, user-friendly interface, and automation feature makes it an essential tool for security professionals and penetration testers. With the tool, you can easily gather all the information you need to carry out a successful reconnaissance mission.