NOTE: this tool is now outdated and is presented here for historical reasons.

FTester - Introduction and Tutorial

The Firewall Tester (ftester for friends), is a tool designed for testing
firewalls filtering policies, from version 0.6 it also includes an Intrusion
Detection System (IDS) testing feature. Basically ftester is made of a packet
generator tool (ftest) and a sniffer (ftestd), the first script injects custom
packets with a signature in the data part while the sniffer listens for such
marked packets, the comparison of the sniffer logs with the injector ones
permits the identification of firewall filtering rules. Unlike common firewall
testing tools or packet generators ftester is capable of generating network
traffic that will looks like real connections to the firewall or IDS system
tested, this feature allows us to test stateful inspection firewalls (like
netfilter or ipfilter) and IDS (like snort). Another advantage of this
architecture is that we can spoof crafted packets source address since the
sniffer knows which packets are generated by it's counterpart, some tricks
involving TTL permits the spoofing also when simulating real connections, this
is described as the 'connection spoofing mode'.

The ftester components are perl scripts so they can be executed on any platform
with a recent version of perl (at least 5.6.1 is recommended) and the three perl
modules Net::RawIP, Net::PcapUtils, NetPacket, they can be downloaded at
www.cpan.org or using the CPAN shell.

Installation is quite simple, just untar the archive ftester-X.X.tar.gz and
everything we need will be decompressed in the ftester-X.X directory, here
we'll find:

 - ftest (the client-side packet generator)
 - ftestd (the sniffer)
 - ftest.conf (ftest example configuration file)
 - freport (a script for comparing ftest and ftestd log files)

the rest is documentation (I recommend reading the man page ftester.8).

The invocation of ftest and ftestd without options will show an usage summary,
look at the documentation for a detailed explanation regarding these options and
the configuration file.

# ./ftest

Firewall Tester client v1.0
Copyright (C) 2001-2006 Andrea Barisani <[email protected]>

Configuration options:
  -f <conf_file>
  -c <source_ip>:<source_port>:<dest_ip>:<dest_port>:<flags>:<protocol>:<tos>
  -v <verbose>

Timing options:
  -d <delay, 0.25 = 250 ms>
  -s <sleep time, 1 = 1 s>

Evasion options:
  -e <evasion method>
  -t <ids_ttl>

Connection options:
  -r <reset connection>
  -F <end connection>
  -g <IP fragments number, es. 4|IP fragments size, es. 16b>
  -p <TCP segments number, es. 4|TCP segments size, es 6b>
  -k <cksum value, es. 60000>
  -m <marker>

# ./ftestd

Firewall Tester sniffer v1.0
Copyright (C) 2001-2006 Andrea Barisani <[email protected]>

Configuration options:
  -i <interface>
  -g <fragments reassembly>
  -m <marker>
  -v <verbose>

Connection options:
  -c <ttl1:ttl2> (ttl1 setting currently works only on Linux systems)

The definition of the packets we want to send for test if they can traverse the
firewall is mainly specified in a configuration file (usually ftest.conf), the
main syntax is:

Source Address:Source Port:Destination Address:Destination Port:Flags:Protocol:Type of Service

for TCP and UDP packets

Source Address:Source Port:Destination Address:Destination Port:Flags:ICMP:icmp_type:icmp_code

for ICMP packets

here's are a few examples:

# SYN packet to 10.1.7.1 port 80
192.168.0.10:1024:10.1.7.1:80:S:TCP:0

# PSH,ACK reply from 192.168.0.10
192.168.0.10:20:10.1.7.1:1022:AP:TCP:22

# UDP packet
192.168.0.10:53:10.1.7.1:53::UDP:0

# ICMP packet type 3 code 5
192.168.0.10::10.1.7.1:::ICMP:3:5

# ranges are allowed for source address, source port, destination port
# source address can also be specified in CIDR form
192.168.0.1-255:1024:10.1.7.1:22:S:TCP:0
192.168.0.1:1024:10.1.7.1:1-65535:S:TCP:0
192.168.0.1:1-1024:10.7.0.1:20-25:S:TCP:22
192.168.3.0/24:1-1024:10.7.0.1:20-25:S:TCP:0
192.168.0.0/22:1024:10.7.0.1:80:S:TCP:0

A stop signal is necessary for telling ftestd that our test is completed, this
can be accomplished with the following syntax:

stop_signal=192.168.0.1:666:10.1.7.1:666:S:TCP:

the stop_signal can be a TCP, UDP or ICMP packet, this packet will cause the
termination of the sniffer so every packet specified after the stop_signal
directive wont be seen by ftestd.

Let's first see it's basic usage in testing basic non-stateful firewall
filtering policies.  We'll use the ftest script on a machine placed outside the
firewall and ftestd on a machine placed inside the firewall on the DMZ we want
to test (of course if we want to test outbound filtering we can reverse ftest
and ftestd placement).

-----------                             ------------
| Host A  |----------<firewall>---------| Host B   |
| (ftest) |                             | (ftestd) |
-----------                             ------------

Host A IP Address: 192.168.0.10
Host A IP Address: 10.1.7.1

Step 1:

For testing which privileged services can be reached from the outside we'll use
the following simple configuration file:

- ftest.conf -
# checking privileged ports (<1025)
192.168.0.10:1025:10.1.7.1:1-1025:S:TCP:0
# checking proxy port
192.168.0.10:1025:10.1.7.1:3128:S:TCP:0
stop_signal=192.168.0.10:80:10.1.7.1:1025:AP:TCP:0

of course we must be sure that the stop signal will reach ftestd, in this case
we've verified that the firewalled host can surf the web so a reply from port 80
should be fine.

Step 2:

Let's start as root ftestd on Host B:

[root@hostb]# ./ftestd -i eth0 -v

Let's injects the packets as root on Host A:

[root@hosta]# ./ftest -f ftest.conf -v -d 0.01

We see an output displaying every packet sent and the ftest.log file is
created.

- ftest.log -
# ftest started on Tue Jun  4 22:37:11 CEST 2002
1 - 192.168.0.10:1025 > 10.1.7.1:1 S TCP 0
2 - 192.168.0.10:1025 > 10.1.7.1:2 S TCP 0
3 - 192.168.0.10:1025 > 10.1.7.1:3 S TCP 0
...
... (cutted for the sake of brevity)
...
1023 - 192.168.0.10:1025 > 10.1.7.1:1023 S TCP 0
1024 - 192.168.0.10:1025 > 10.1.7.1:1024 S TCP 0
1025 - 192.168.0.10:1025 > 10.1.7.1:1025 S TCP 0
1026 - 192.168.0.10:1025 > 10.1.7.1:3128 S TCP 0
1027 - 192.168.0.10:80 > 10.1.7.1:1025 PA TCP
# ftest stopped on Tue Jun  4 22:38:42 CEST 2002

On Host B ftestd shows the same messages and creates the ftestd.log file.

As we can see each packet is marked with an unique identifier (the first number
of the line) which corresponds to the packet IP ID field, this will help us for
comparing the different log files.

Step 3:

Finally we copy the two log files on the same host and we compare them using
freport:

[root@hostb]# ./freport ftest.log ftestd.log

Authorized packets:
-------------------

21 - 192.168.0.10:1025 > 10.1.7.1:21 S TCP 0
22 - 192.168.0.10:1025 > 10.1.7.1:22 S TCP 0
23 - 192.168.0.10:1025 > 10.1.7.1:23 S TCP 0
25 - 192.168.0.10:1025 > 10.1.7.1:25 S TCP 0
80 - 192.168.0.10:1025 > 10.1.7.1:80 S TCP 0
110 - 192.168.0.10:1025 > 10.1.7.1:110 S TCP 0
113 - 192.168.0.10:1025 > 10.1.7.1:113 S TCP 0
1027 - 192.168.0.10:80 > 10.1.7.1:1025 PA TCP 0

Modified packets (probably NAT):
--------------------------------

443 - 192.168.0.10:1025 > 10.1.7.1:443 S TCP 0
		>>>>>>>>
443 - 192.168.0.10:1025 > 10.1.7.5:443 S TCP 0

Filtered or dropped packets:
----------------------------

1 - 192.168.0.10:1025 > 10.1.7.1:1 S TCP 0
2 - 192.168.0.10:1025 > 10.1.7.1:2 S TCP 0
3 - 192.168.0.10:1025 > 10.1.7.1:3 S TCP 0
...
... (cutted for the sake of brevity)
...
1026 - 192.168.0.10:1025 > 10.1.7.1:3128 S TCP 0

As we can see ftp,telnet,ssh,smtp,http,pop3,auth are apparently reachable from
the outside, https is forwarded to host 10.1.7.5 and the rest, including the
proxy port is filtered, of course NAT regarding the destination address can be
detected only if we sniff on a non-switched environment otherwise only port
address translation or source address translation can be detected.

The sniffing of packet 1027 (the PSH,ACK from port 80) demonstrate that we are
dealing with a simple firewall that does not perform connection tracking,
however as of today stateful inspection firewalls are more likely to be found in
real environments. Stateful inspection firewalls usually performs connection
tracking and they are not suppose to pass unmatched packets that aren't part of
a previously initiated connection, for this reason if we need to test the
filtering policies regarding PSH,ACK and moreover URG,RST,FIN packets we have to
create a real connection with the proper SYN,SYN-ACK,ACK handshake before
sending them with the correct sequence numbers, the ftestd process will send the
correct replies.  However when simulating the connection if we are spoofing the
source address the stack of the destination host will reply to our spoofed
packets and the real host that we'are spoofing will consequently reset the
connection since it hasn't started it. So we have to do two things, hiding the
stack response to the spoofed host and to the firewall and make sure that ftestd
reply will traverse the firewall by not reaching the spoofed host. Hiding the
stack response could be done by setting a very low default TTL in
/proc/sys/net/ipv4/ip_default_ttl (Linux only). Hiding to the spoofed host
ftestd reply is done by setting its TTL to a low value equal to the hop delay
between ftestd and the firewall. The -s flag for ftest and the -c flag for
ftestd must be tuned for this mode, for example use ./ftestd -c 0:3 for
temporarily setting default stack ttl to 0 and ftestd reply ttl to 3, the
ip_default_ttl will be restored when a stop_signal is received.

The so called 'connection spoofing mode' can be activated with the 'connect='
prefix, let's use this mode:

- ftest.conf -
# simulating a ssh connection
connect=192.168.0.10:1025:10.1.7.1:22:AP:TCP:0
# checking unmatched RST packets
192.168.0.10:1025:10.1.7.1:23:UR:TCP:0
# checking matched RST packets
connect=192.168.0.10:1025:10.1.7.1:23:UR:TCP:0
stop_signal=192.168.0.10:1025:10.1.7.1:80:S:TCP:0

and again:

[root@hostb]# ./ftestd -i eth0 -c 0:3 -v

[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1

Sent Syn Probe => 192.168.0.10:1025 > 10.1.7.1:22 S TCP
Sleeping for 1 seconds
Sent Ack Reply => 192.168.0.10:1025 > 10.1.7.1:22 A TCP
3 - 192.168.0.10:1025 > 10.1.7.1:22 AP TCP 0
5 - 192.168.0.10:1025 > 10.1.7.1:23 UR TCP 0
Sent Syn Probe => 192.168.0.10:1025 > 10.1.7.1:23 S TCP
Sleeping for 1 seconds
Sent Ack Reply => 192.168.0.10:1025 > 10.1.7.1:23 A TCP
8 - 192.168.0.10:1025 > 10.1.7.1:23 UR TCP 0
Stop packet => 192.168.0.10:1025 > 10.1.7.1:80 S TCP

note that this time ftest is configured to wait 1 second for ftestd replies.

It's useful to see what traffic we are generating using tcpdump:

[root@hostb]# tcpdump -n -v -i eth0

# the first connection
15:16:04.969929 192.168.0.10.1025 > 10.1.7.1.22: S [tcp sum ok] 18679:18687(8) win 65535 (DF) (ttl 200, id 1, len 48)
15:16:04.973880 10.1.7.1.22 > 192.168.0.10.1025: S [tcp sum ok] 19703:19703(0) ack 18687 win 65535 (DF) [tos 0x10]  (ttl 200, id 1, len 40)
15:16:05.991877 192.168.0.10.1025 > 10.1.7.1.22: . [tcp sum ok] ack 1 win 65535 (DF) (ttl 200, id 2, len 40)
15:16:06.001927 192.168.0.10.1025 > 10.1.7.1.22: P [tcp sum ok] 1:16(15) ack 1 win 65535 (DF) (ttl 200, id 3, len 55)
15:16:06.012034 192.168.0.10.1025 > 10.1.7.1.22: P [tcp sum ok] 16:21(5) ack 1 win 65535 (DF) (ttl 200, id 4, len 45)
15:16:06.014798 10.1.7.1.22 > 192.168.0.10.1025: . [tcp sum ok] ack 21 win 65535 (DF) [tos 0x10]  (ttl 200, id 2, len 40)

# the unmatched RST
15:16:06.023391 192.168.0.10.1025 > 10.1.7.1.23: R [tcp sum ok] 0:15(15) win 65535 urg 0 [RST ftestertcpprobe] (DF) (ttl 200, id 5, len 55)

# the second connection
15:16:06.032223 192.168.0.10.1025 > 10.1.7.1.23: S [tcp sum ok] 63848:63856(8) win 65535 (DF) (ttl 200, id 6, len 48)
15:16:06.034946 10.1.7.1.23 > 192.168.0.10.1025: S [tcp sum ok] 64872:64872(0) ack 63856 win 65535 (DF) [tos 0x10]  (ttl 200, id 1, len 40)
15:16:07.051963 192.168.0.10.1025 > 10.1.7.1.23: . [tcp sum ok] ack 1 win 65535 (DF) (ttl 200, id 7, len 40)
# the correct RST
15:16:07.061863 192.168.0.10.1025 > 10.1.7.1.23: R [tcp sum ok] 63856:63871(15) win 65535 urg 0 [RST ftestertcpprobe] (DF) (ttl 200, id 8, len 55)
15:16:07.072021 192.168.0.10.1025 > 10.1.7.1.23: R [tcp sum ok] 63871:63876(5) win 65535 urg 0 [RST ackme] (DF) (ttl 200, id 9, len 45)
15:16:07.074616 10.1.7.1.23 > 192.168.0.10.1025: . [tcp sum ok] ack 21 win 65535 (DF) [tos 0x10]  (ttl 200, id 2, len 40)

# the stop_signal
15:16:07.083321 192.168.0.10.1025 > 10.1.7.1.80: S [tcp sum ok] 0:11(11) win 65535 (DF) (ttl 200, id 10, len 51)

again here's the logs comparison:

[root@hostb]# ./freport ftest.log ftestd.log

Authorized packets:
-------------------

3 - 192.168.0.10:1025 > 10.1.7.1:22 PA TCP 0
8 - 192.168.0.10:1025 > 10.1.7.1:23 UR TCP 0
10 - 192.168.0.10:1025 > 10.1.7.1:80 S TCP 0

Modified packets (probably NAT):
--------------------------------

		>>>>>>>>

Filtered or dropped packets:
----------------------------

5 - 192.168.0.10:1025 > 10.1.7.1:23 UR TCP 0

the first RST has been dropped this confirm that we'are probably dealing with a
stateful inspection firewall.

If more than one packet is specified with the 'connect=' prefix and the same
connection parameters (source address/port, destination address/port) the
-r/-F flags must be used to correctly close (with a RST or a FIN handshake) each
connections, otherwise, if the firewall is performing sequence numbers tracking,
connections other than the first will be blocked.

The IDS testing option permits the injection of arbitrary packets with custom
payload, using a payload that is supposed to trigger an IDS alert we can test
IDS visibility on the network and it's ability of sniffing fragmented/segmented
packets. A number of common IDS evasion techniques are also implemented for
activation during packets injection. The configuration file for this mode will
use the standard syntax with the 'ids=' and 'ids-conn=' prefixes, additionally
ftest can directly use a snort (http://www.snort.org) rule definition file
(check documentation for currently supported keywords). Here's a syntax example:

- ftest.conf -
ids=192.168.0.10:1025:10.1.7.1:25:S:TCP:0:VRFY
ids=192.168.0.10::10.1.7.1:::ICMP:3:5:+++ath
ids-conn=192.168.0.10:23:10.1.7.1:1025:PA:TCP:0:to su root
ids-conn=192.168.0.10:1025:10.1.7.1:80:PA:TCP:0:cmd.exe
ids-conn=192.168.0.10:1026:10.1.7.1:80:PA:TCP:0:ftp.exe
insert /etc/snort/exploit.rules 192.168.0.10 10.1.7.1 0
insert-conn /etc/snort/web-misc.rules 192.168.0.10 10.1.7.1 0

The '-conn' options works just like the 'connect' option, this is useful if the
IDS is performing stateful inspection, the ftestd presence is needed when using
this mode otherwise since we are only testing the IDS the sniffer is not
required.

The 'insert' option take 4 arguments, the definition file, the source address,
the destination address and the type of service.

Let's use this feature to test snort ability in eluding evasion techniques,
we'll use a common alert and we'll assume that snort is performing stateful
inspection discarding unmatched traffic.

Step 1:

The configuration file:

- ftest.conf -
ids-conn=192.168.0.1:1025:10.7.0.1:80:PA:TCP:0:cmd.exe

Step 2:

Let's start snort on our sensor:

[root@ids1] snort -A full -c /etc/snort/snort.conf -D -b -d -i eth0 -l /var/log \
            -s -z

Let's start as root ftestd on Host B:

[root@hostb]# ./ftestd -i eth0 -c 0:3 -v

Let's injects the packets as root on Host A:

[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1 -F

Sent Syn Probe => 192.168.0.10:1025 > 10.1.7.1:80 S TCP
Sleeping for 1 seconds
Sent Ack Reply => 192.168.0.10:1025 > 10.1.7.1:80 A TCP
6 - 192.168.0.10:1025 > 10.1.7.1:80 PA TCP 0 "cmd.exe"

Immediately snort alerts us with the following syslog message (notice that snort
also warn us about ftest signature in the SYN packet):

Jun  5 16:25:13 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection [Classification: Unknown Traffic] [Priority: 3]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:25:14 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80

We can repeat the process using some dirty tricks like fragmentation and evasion
techniques (see documentation for a detailed description):

[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1 -F -g 2
[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1 -F -e stream -p 3
[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1 -F -e stream -p 1b
[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1 -F -e desync1

and so on...

Since snort rocks it identifies correctly the triggering payload each time:

Jun  5 16:29:32 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:29:34 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:30:16 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:30:18 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:30:40 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:30:42 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:31:00 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
Jun  5 16:31:02 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80

The generated log will help in clarifying what has been sent:

- ftest.log -
# ftest started on Wed Jun  5 16:25:13 CEST 2002
IDS mode >> 3 - 192.168.0.10:1025 > 10.1.7.1:80 "cmd.exe" PA TCP 0
# ftest stopped on Wed Jun  5 16:25:15 CEST 2002
# ftest started on Wed Jun  5 16:29:32 CEST 2002
IDS mode >> 3 - 192.168.0.10:1025 > 10.1.7.1:80 "cmd.exe" PA TCP 0
# ftest stopped on Wed Jun  5 16:29:35 CEST 2002
# ftest started on Wed Jun  5 16:30:16 CEST 2002
IDS mode >> 3 - 192.168.0.10:1025 > 10.1.7.1:80 "cm" PA TCP 0
IDS mode >> 4 - 192.168.0.10:1025 > 10.1.7.1:80 "d." PA TCP 0
IDS mode >> 5 - 192.168.0.10:1025 > 10.1.7.1:80 "exe" PA TCP 0
# ftest stopped on Wed Jun  5 16:30:18 CEST 2002
# ftest started on Wed Jun  5 16:30:40 CEST 2002
IDS mode >> 3 - 192.168.0.10:1025 > 10.1.7.1:80 "c" PA TCP 0
IDS mode >> 4 - 192.168.0.10:1025 > 10.1.7.1:80 "m" PA TCP 0
IDS mode >> 5 - 192.168.0.10:1025 > 10.1.7.1:80 "d" PA TCP 0
IDS mode >> 6 - 192.168.0.10:1025 > 10.1.7.1:80 "." PA TCP 0
IDS mode >> 7 - 192.168.0.10:1025 > 10.1.7.1:80 "e" PA TCP 0
IDS mode >> 8 - 192.168.0.10:1025 > 10.1.7.1:80 "x" PA TCP 0
IDS mode >> 9 - 192.168.0.10:1025 > 10.1.7.1:80 "e" PA TCP 0
# ftest stopped on Wed Jun  5 16:30:42 CEST 2002
# ftest started on Wed Jun  5 16:31:00 CEST 2002
IDS mode >> 3 - 192.168.0.10:1025 > 10.1.7.1:80 "cmd" PA TCP 0
IDS mode >> 4 - 192.168.0.10:1025 > 10.1.7.1:80 "" S TCP 0 EVASION PACKET!
IDS mode >> 5 - 192.168.0.10:1025 > 10.1.7.1:80 ".exe" PA TCP 0
# ftest stopped on Wed Jun  5 16:31:02 CEST 2002