Raw sockets manipulation and sniffing
Network sniffers enable you to capture and analyze packets moving in and out of a target machine, making them useful both before and after exploitation. While existing tools like Wireshark or Python libraries such as Scapy (which we’ll cover in the next chapter) can often meet your needs, knowing how to quickly build your own custom sniffer to view and decode network traffic offers additional flexibility and control is a good ninja move to know if you want to work with higher levels tools.
What is a raw sockets
A socket is an endpoint for sending or receiving data across a network. Normally, when you write applications that use sockets, you rely on higher-level protocols like TCP or UDP, which manage the data transmission. These protocols ensure that data is properly ordered, checksummed, and delivered.
However, raw sockets provide low-level access to the underlying network. With raw sockets, you bypass the usual TCP/IP stack and handle data directly as it comes from the network interface. This means you receive the raw network packets (including headers like IP and ICMP) and are responsible for processing them yourself.
Basic sniffer
Let's put our hands in the code and see how the head's of the famous network packets by coding a simple sniffer who Create a raw socket and bind it to the our desired host, read the packet and save it to a json file.
import socket
import logging
import json
from datetime import datetime
# Host to listen on (the IP address of your machine)
HOST = '0.0.0.0'
# Set up logging
logging.basicConfig(filename='sniffer.log', level=logging.DEBUG,
format='%(asctime)s - %(levelname)s - %(message)s')
def save_packet_to_json(data):
"""Save captured packet data to a JSON file."""
timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
filename = f"packet_{timestamp}.json"
packet_data = {
"timestamp": timestamp,
"data": data
}
with open(filename, 'w') as json_file:
json.dump(packet_data, json_file, indent=4)
logging.info(f"Packet saved to {filename}")
def main():
sniffer = None # Define sniffer variable initially as None
try:
# On Linux, use IPPROTO_ICMP for ICMP packets (as it's common for network sniffing)
socket_protocol = socket.IPPROTO_ICMP
logging.info("Running on a non-Windows system, using IPPROTO_ICMP")
# Create raw socket and bind it to the HOST
sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
sniffer.bind((HOST, 0))
logging.info(f"Socket bound to {HOST}")
# Include the IP header in the capture
sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
logging.info("Socket option IP_HDRINCL set")
# Read one packet
logging.info("Waiting for a packet...")
packet, addr = sniffer.recvfrom(65565)
logging.info(f"Packet received from {addr}")
# Save packet to JSON file
save_packet_to_json(packet.hex())
except Exception as e:
logging.error(f"An error occurred: {e}")
finally:
if sniffer: # Check if sniffer is not None before closing
sniffer.close()
logging.info("Socket closed")
if __name__ == '__main__':
main()
Now you can run this script then open an other VM/shell on the same network and ping your local machine. You should see this king of outpout in the log file :
You can also on the same machine ping some domain like
www.google.comif you do not have some firewall or ant malware system installed on your machine who blocks the packets operations.
2024-09-04 23:49:21,276 - INFO - Running on a non-Windows system, using IPPROTO_ICMP
2024-09-04 23:49:21,276 - INFO - Socket bound to 192.168.0.71
2024-09-04 23:49:21,276 - INFO - Socket option IP_HDRINCL set
2024-09-04 23:49:21,276 - INFO - Waiting for a packet...
2024-09-04 23:50:23,185 - INFO - Packet received from ('192.168.0.71', 0)
2024-09-04 23:50:23,186 - INFO - Packet saved to packet_2024-09-04_23-50-23.json
2024-09-04 23:50:23,186 - INFO - Socket closed
2024-09-04 23:54:22,625 - INFO - Running on a non-Windows system, using IPPROTO_ICMP
2024-09-04 23:54:22,626 - INFO - Socket bound to 0.0.0.0
2024-09-04 23:54:22,626 - INFO - Socket option IP_HDRINCL set
2024-09-04 23:54:22,626 - INFO - Waiting for a packet...
2024-09-04 23:54:52,033 - INFO - Packet received from ('192.168.0.46', 0)
2024-09-04 23:54:52,033 - INFO - Packet saved to packet_2024-09-04_23-54-52.json
2024-09-04 23:54:52,033 - INFO - Socket closed
2
Note
If you do not have firewall or any security on your machine, in your first window where you executed your sniffer, you should see some garbled output that closely resembles the following:
and the json result look like:
{
"timestamp": "2024-09-04_23-54-44",
"data": "4500005401ab00003f01f838c0a8002ec0a8004708007a9c9541000066da18b400007d9008090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637"
}
As you can see, we’ve captured the initial ICMP ping request but the informations are not very obvious 😂
Decoding IP layer
In its current state, our sniffer captures all IP headers, along with any higher-level protocols such as TCP, UDP, or ICMP. However, this data is in binary format, which, as we've seen, is difficult to interpret. To make sense of it, we need to decode the IP portion of the packet to extract useful information, such as the protocol type (TCP, UDP, or ICMP) and the source and destination IP addresses. This will provide a foundation for further protocol analysis.
To better understand how to decode incoming packets, it helps to examine the structure of a typical packet on the network. Refer to figure below for an illustration of the IP header layout.
We’ll decode the entire IP header (excluding the Options and Data field) and extract the protocol type, source, and destination IP addresses. This requires directly working with binary data and developing a strategy to parse each section of the IP header using Python.
In Python, there are two common ways to convert external binary data into a usable data structure: the ctypes module or the struct module. The ctypes module is a foreign function interface that allows Python to interact with C-based languages, enabling the use of C-compatible data types and functions from shared libraries. In contrast, the struct module is primarily focused on converting between Python values and C structs represented as byte objects. Essentially, ctypes handles binary data types while offering additional functionality, whereas struct is specialized for handling binary data alone.
Both approaches are commonly used in tools found online. In this section, we’ll explore how to use struct on IPv4 header from the network.
import ipaddress
import os
import socket
import struct
import sys
class IP:
def __init__(self, buff=None):
# Unpack the IP header from the buffer
header = struct.unpack('<BBHHHBBH4s4s', buff)
self.ver = header[0] >> 4 # Extract version (4 bits)
# The typical way you get the high-order nybble of a byte is to right-shift the byte by four places, which is the equivalent of prepending four zeros to the front of the byte, causing the last four bits to fall off
self.ihl = header[0] & 0xF # Internet Header Length (4 bits)
self.tos = header[1] # Type of Service
self.len = header[2] # Total Length
self.id = header[3] # Identification
self.offset = header[4] # Fragment Offset
self.ttl = header[5] # Time to Live
self.protocol_num = header[6] # Protocol
self.sum = header[7] # Header checksum
self.src = header[8] # Source IP
self.dst = header[9] # Destination IP
# Human-readable IP addresses
self.src_address = ipaddress.ip_address(self.src)
self.dst_address = ipaddress.ip_address(self.dst)
# Map protocol numbers to protocol names (ICMP, TCP, UDP)
self.protocol_map = {1: "ICMP", 6: "TCP", 17: "UDP"}
self.protocol = self.protocol_map.get(self.protocol_num, str(self.protocol_num))
def __repr__(self):
return (f"IP(version={self.ver}, ihl={self.ihl}, tos={self.tos}, len={self.len}, "
f"id={self.id}, offset={self.offset}, ttl={self.ttl}, protocol={self.protocol}, "
f"checksum={self.sum}, src_address={self.src_address}, dst_address={self.dst_address})")
def sniff(host):
# Define the socket protocol based on the OS
if os.name == 'nt':
socket_protocol = socket.IPPROTO_IP
else:
socket_protocol = socket.IPPROTO_ICMP
# Create a raw socket
sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
# Bind to the host and set the socket option to include the IP headers
sniffer.bind((host, 0))
sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
# Enable promiscuous mode for Windows
if os.name == 'nt':
sniffer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
try:
while True:
# Capture raw packets
raw_buffer = sniffer.recvfrom(65535)[0]
# Extract the IP header from the first 20 bytes
ip_header = IP(raw_buffer[0:20])
# Print protocol and source/destination IPs
print(f"Protocol: {ip_header.protocol} {ip_header.src_address} -> {ip_header.dst_address}")
except KeyboardInterrupt:
# Disable promiscuous mode on Windows if interrupted
if os.name == 'nt':
sniffer.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
sys.exit()
if __name__ == '__main__':
if len(sys.argv) == 2:
host = sys.argv[1]
else:
host = '192.168.0.71' # Default host, change if necessary
sniff(host)
Now you can ping your machine (if you have some security installed) and you will see this kind of output :
The format you see here : source IP -> target IP demonstrate that we have ...
Ping kind reminder
Since we used ping a lot in this tutorial I think a king reminder will not hurt. Ping uses the ICMP protocol which doesn't have ports like the TCP and UDP protocols (see the picture bellow).
If you need to see if Ping is disabled on a Linux system, you can check:
cat /proc/sys/net/ipv4/icmp_echo_ignore_all- 0 means Ping is enabled. (The system will respond to pings) - 1 means Ping is disabled (The system will not respond to pings)
TCP/IP & ICMP
When data is sent over the internet, it is broken up into packets. These packets consist of various headers (like IP, TCP, or UDP headers) and the payload (actual data). If something goes wrong with the delivery of these packets (e.g., a route is unreachable, or a device is down), ICMP is used to send error messages back to the sender.
ICMP messages are encapsulated inside IP packets. This means an ICMP message has its own header and is then placed inside an IP packet for transmission. Example of an ICMP packet encapsulated in an IP packet:
-
ICMP Header: This is typically 8 bytes long and includes fields like Type, Code, Checksum, and other fields that vary depending on the type of ICMP message.
-
ICMP Payload: The content of the ICMP message, which can be different depending on the type of ICMP message (e.g., error information, echo reply data, etc.).
ICMP stands for Internet Control Message Protocol, and it's an integral part of the TCP/IP protocol suite. It is primarily used for diagnostic and control purposes, helping manage and report errors in network communications.
ICMP does not directly transmit data like TCP or UDP. Instead, it works at the network layer (Layer 3) of the OSI model and is designed to communicate control information between devices (such as routers, computers, etc.). ICMP is typically used to: - Report network errors. - Test network connectivity (like using ping). - Perform diagnostics, such as discovering the path that packets take across a network (traceroute).
ICMP’s Role in the TCP/IP Stack
The TCP/IP stack (Transmission Control Protocol/Internet Protocol) is the set of protocols that govern how data is sent and received over the internet. ICMP is part of the IP layer (Internet Protocol) in the TCP/IP model and works closely with IP to report issues or provide feedback about data delivery problems.
ICMP is neither connection-oriented (like TCP) or connectionless (like UDP) in the sense that it does not facilitate the transmission of data between devices but helps manage the data flow in a network.
TCP/IP Stack Layers
In case you do not have the OSI model in your head all the time 🤓
- Link Layer (Network Interface): The physical connection between devices (Ethernet, Wi-Fi, etc.).
- Internet Layer (Network Layer): Responsible for routing packets across networks (IP, ICMP).
- Transport Layer: Ensures reliable or unreliable transmission of data (TCP for reliability, UDP for speed).
- Application Layer: Contains protocols that interact with the application (HTTP, DNS, SMTP).
ICMP belongs to the Internet Layer and works hand-in-hand with IP. It is used primarily for error reporting and diagnostic messages between network devices, such as routers and computers.
Key Features of ICMP
-
Error Reporting: ICMP helps report errors when IP packets cannot be delivered. For example, if a router can't forward a packet due to network congestion or an unreachable destination, it will send an ICMP error message back to the sender.
-
Diagnostics: ICMP is used in common tools like
pingandtracerouteto help troubleshoot network issues. These tools use ICMP to determine if a host is reachable and how long it takes to get there. -
Control Messages: ICMP can also send control messages to adjust the flow of data and manage network congestion.
Common ICMP Messages and Types
ICMP messages are typically classified by types and codes, which specify the nature of the message. Some common ICMP message types include:
- ICMP Echo Request (Type 8) and ICMP Echo Reply (Type 0):
- Used by the
pingcommand to check the connectivity between two devices. The Echo Request is sent by the client, and the Echo Reply is sent back by the destination if it's reachable. -
Example:
ping google.comsends ICMP Echo Requests, and Google responds with ICMP Echo Replies. -
ICMP Destination Unreachable (Type 3):
- This message is sent when a packet cannot be delivered to its destination. There are several codes for the different reasons why a destination might be unreachable (e.g., Code 1: Host unreachable, Code 3: Port unreachable).
-
This is the message type your script uses to detect live hosts when it sends UDP packets to closed ports.
-
ICMP Time Exceeded (Type 11):
-
Used in
tracerouteto show how far packets travel through a network. It indicates that a packet was dropped because its Time-To-Live (TTL) expired, meaning it was in the network too long without reaching its destination. -
ICMP Redirect (Type 5):
-
Sent by routers to inform a host to use a different route for reaching a destination.
-
ICMP Source Quench (Type 4): (Obsolete)
- Used to ask a sender to slow down its data transmission. This was primarily for congestion control, but it’s no longer used.
Building a socket scanner from scratch
Now let's begin the real deal 😎
Let's code a scanner.py script who detect live hosts in a specified subnet by sending UDP packets (a magic message to be precise) and listening for ICMP Destination Unreachable messages. This script will be a custom network scanner that uses raw sockets to send out UDP packets with a specific payload (the magic message) and sniffs for ICMP responses.
The script will be working as follows:
- UDP Probes: The script sends UDP packets with a "magic message" to every IP address in a subnet (
192.168.0.0/24in this case). - ICMP Response Handling: If a target host is alive but does not have the UDP port open (which is likely for a high port like
65212), it sends back an ICMP Destination Unreachable message. - Magic Message Check: The script checks whether the ICMP message contains the magic message (
"PYTHONRULES!"), ensuring that the ICMP message corresponds to one of the UDP packets the script sent. - Host Up Detection: If a valid ICMP message with the magic message is detected, the script marks the corresponding host as "up."
import ipaddress
import os
import socket
import struct
import sys
import threading
import time
# Subnet to target
SUBNET = '192.168.0.0/24'
# Magic string we'll check ICMP responses for
MESSAGE = 'PYTHONRULES!'
class IP:
def __init__(self, buff):
header = struct.unpack('<BBHHHBBH4s4s', buff)
self.ver = header[0] >> 4
self.ihl = header[0] & 0xF
self.tos = header[1]
self.len = header[2]
self.id = header[3]
self.offset = header[4]
self.ttl = header[5]
self.protocol_num = header[6]
self.sum = header[7]
self.src = header[8]
self.dst = header[9]
# Human-readable IP addresses
self.src_address = ipaddress.ip_address(self.src)
self.dst_address = ipaddress.ip_address(self.dst)
# Protocol mapping
self.protocol_map = {1: "ICMP", 6: "TCP", 17: "UDP"}
self.protocol = self.protocol_map.get(self.protocol_num, str(self.protocol_num))
class ICMP:
def __init__(self, buff):
header = struct.unpack('<BBHHH', buff)
self.type = header[0]
self.code = header[1]
self.sum = header[2]
self.id = header[3]
self.seq = header[4]
def udp_sender():
"""
Sprays out UDP datagrams with the magic message to the specified subnet.
"""
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sender:
for ip in ipaddress.ip_network(SUBNET).hosts():
sender.sendto(bytes(MESSAGE, 'utf8'), (str(ip), 65212))
class Scanner:
def __init__(self, host):
self.host = host
# Select protocol based on OS
if os.name == 'nt':
socket_protocol = socket.IPPROTO_IP
else:
socket_protocol = socket.IPPROTO_ICMP
# Create the raw socket
self.socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
self.socket.bind((host, 0))
self.socket.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
# Enable promiscuous mode on Windows
if os.name == 'nt':
self.socket.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
def sniff(self):
"""
Start sniffing ICMP responses and check for the magic message.
"""
hosts_up = set([f'{str(self.host)} *'])
try:
while True:
# Read a packet
raw_buffer = self.socket.recvfrom(65535)[0]
# Parse the IP header from the first 20 bytes
ip_header = IP(raw_buffer[0:20])
# If it's an ICMP packet, we process it
if ip_header.protocol == "ICMP":
offset = ip_header.ihl * 4
buf = raw_buffer[offset:offset + 8]
# Parse ICMP header
icmp_header = ICMP(buf)
# Check for ICMP Type 3 and Code 3 (Destination Unreachable)
if icmp_header.type == 3 and icmp_header.code == 3:
src_ip = ipaddress.ip_address(ip_header.src_address)
# If the source IP is within our target subnet
if src_ip in ipaddress.IPv4Network(SUBNET):
# Check if the packet contains our magic message
if raw_buffer[len(raw_buffer) - len(MESSAGE):] == bytes(MESSAGE, 'utf8'):
tgt = str(ip_header.src_address)
# Avoid printing the host that sent the message
if tgt != self.host and tgt not in hosts_up:
hosts_up.add(tgt)
print(f'Host Up: {tgt}')
except KeyboardInterrupt:
# Handle Ctrl-C and print a summary of hosts up
if os.name == 'nt':
self.socket.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
print('\nUser interrupted.')
if hosts_up:
print(f'\nSummary: Hosts up on {SUBNET}')
for host in sorted(hosts_up):
print(f'{host}')
print('')
sys.exit()
if __name__ == '__main__':
# If a host is provided via command-line argument, use it; otherwise, default to 192.168.1.203
if len(sys.argv) == 2:
host = sys.argv[1]
else:
host = '192.168.0.71'
# Initialize the scanner
s = Scanner(host)
# Start the UDP sender in a separate thread
time.sleep(5) # Wait for 5 seconds before sending packets
t = threading.Thread(target=udp_sender)
t.start()
# Start sniffing for ICMP responses
s.sniff()
For each host that is up and responds with an ICMP Destination Unreachable message containing the magic message, the script will print:
By default, the script will target the subnet
192.168.0.0/24and use the IP192.168.0.71for the scanner machine.
Note
You can also pass the scanner's host IP via command-line arguments. For example:
However, if the target machine does not have the destination port open, it may send back an ICMP Destination Unreachable (Type 3, Code 3) message, indicating that the port is closed.
Key Points to Understand:
-
ICMP Destination Unreachable (Type 3, Code 3): This ICMP message is sent when a device receives a UDP packet on a closed port. The script is designed to capture these ICMP messages to identify active hosts.
-
The Magic Message ⚡️: The magic message (
'PYTHONRULES!') helps the script ensure that the ICMP responses are triggered by the UDP probes it sent, not from other random network traffic. -
Raw Socket Privileges: Since raw sockets are required for sniffing ICMP traffic, the script must be run with root/administrator privileges.
ICMP is a crucial part of network communication in the TCP/IP suite, primarily used for error reporting and diagnostics. It helps network devices understand when packets cannot be delivered or if network issues arise. Tools like ping and traceroute use ICMP to measure connectivity and route tracing.
In this script, ICMP helps detect live hosts by sending back messages when UDP packets are sent to closed ports. By checking these ICMP messages for the magic string, the script identifies hosts that are up and responsive on the network.
Conclusion
For a quick scan like the one we performed, it only took a few seconds to get the results. By cross-referencing these IP addresses with the DHCP table in a home router, we were able to verify that the results were accurate 🥷🏼
You can easily expand what you’ve learned in this chapter to decode TCP and UDP packets as well as to build additional tooling around the scanner. This type of scanners are also useful for building trojan. This would allow a deployed trojan to scan the local network looking for additional targets.
Now that you know the basics of how networks work on a high and low level, let’s explore a very mature Python library called Scapy !