You have mail.IPFW Primer
Copyright © 2024 The FreeBSD Documentation Project
trademarks
FreeBSD is a registered trademark of the FreeBSD Foundation.
Git and the Git logo are either registered trademarks or trademarks of Software Freedom Conservancy, Inc., corporate home of the Git Project, in the United States and/or other countries.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this document, and the FreeBSD Project was aware of the trademark claim, the designations have been followed by the “™” or the “®” symbol.
Table of Contents
[ Split HTML / Single HTML ]
Abstract
ipfw(8) is a firewall application that comes standard with FreeBSD. This book provides an introduction to IPFW and its capabilities. These capabilities are demonstrated using QEMU virtual machines in real-world scenarios.
Content includes descriptions of the main features of ipfw(8) and lab examples on using IPFW as a firewall or a network gat eway.
Like IPFW itself, this book is a work in progress.
Acknowledgments
Thanks to the FreeBSD developers who added and maintain IPFW.
Also, thanks to those who took the time to read early drafts of this document and offered many valuable comments and criticisms.
Attributions
Artwork from the following sources was used in preparing this book:
Stylized nginx logo
Original image: NGINX Logo
Obtained from Wikimedia https://en.wikipedia.org/wiki/Nginx#/media/File:Nginx_logo.svg. Image is listed as in the Public Domain.
The image was converted to .PNG and also to .BMP formats for use in this book and accompanying code. Conversions were accomplished via ImageMagick’s convert program.
Stylized bind 9 logo
Original image: BIND 9’s New Logo
Obtained from https://www.isc.org/blogs/bind-9s-new-logo/
Logo used with permission from Internet Systems Consortium (ISC).
The image was converted to .PNG and also to .BMP formats for use in this book and accompanying code. Conversions were accomplished via ImageMagick’s convert program.
Stylized IPv6 logo
Original image: World IPv6 Launch Logo
Obtained from Wikipedia: https://commons.wikimedia.org/wiki/File:World_IPv6_launch_logo.svg Original CC 3.0 license: https://creativecommons.org/licenses/by/3.0/deed.en Updated CC 4.0 license: https://creativecommons.org/licenses/by/4.0/deed.en
The image was converted to .PNG and also to .BMP formats for use in this book and accompanying code. Conversions were accomplished via ImageMagick’s convert program.
Use of the above artwork does not imply endorsement of any opinions expressed in this book.
All other artwork is orginial artwork by the author.
Preface
Shell Prompts
This table shows the default system prompt and superuser prompt. The examples use these prompts to indicate which type of user is running the example.
| User | Prompt |
|---|---|
Normal user | % |
| # |
Typographic Conventions
This table describes the typographic conventions used in this book.
| Meaning | Examples |
|---|---|
The names of commands. | Use |
The names of files. | Edit .login. |
On-screen computer output. | |
What the user types, contrasted with on-screen computer output. | |
Manual page references. | Use su(1) to change user identity. |
User and group names. | Only |
Emphasis. | The user must do this. |
Text that the user is expected to replace with the actual text. | To search for a keyword in the manual pages, type |
Environment variables. |
|
Notes, Tips, Important Information, Warnings, and Examples
Notes, warnings, and examples appear within the text.
Notes are represented like this, and contain information to take note of, as it may affect what the user does. |
Tips are represented like this, and contain information helpful to the user, such as showing an easier way to do something. |
Important information is represented like this. Typically, these show extra steps the user may need to take. |
Warnings are represented like this, and contain information warning about possible damage if the instructions are not followed. This damage may be physical, to the hardware or the user, or it may be non-physical, such as the inadvertent deletion of important files. |
Example 1. A Sample Example
Examples are represented like this, and typically contain examples showing a walkthrough, or the results of a particular action.
Chapter 1. Introduction
This book is about one of the native firewalls included with FreeBSD, ipfw(8) - the Internet Protocol FireWall. The book makes frequent reference to the ipfw(8) manual page and the reader is advised to become familiar with the manual page alongside this book. There is also a section on ipfw in the FreeBSD Handbook Page on IPFW. The intent with this book is to provide examples and informative material beyond the manual page and handbook to increase understanding and usage of ipfw.
It is, of course, entirely possible to perform all the examples in this book with real hardware. QEMU provides a way to perform the examples without spending any money for hardware. In either case, some setup is required.
Throughout this book are many examples of using ipfw with virtual machines to simulate actual hardware. These examples were developed with QEMU version 9.0.2. Note that QEMU command syntax with some of the examples may have changed slightly by the time this book becomes available. Use the latest QEMU release where possible, and check the QEMU documentation if the examples in this book do not work correctly.
Also used are a number of scripts that allow easy if_bridge(4) and tap(4) setup, virtual machine setup, and data transfer from external VMs to or through a firewall VM. In the early examples, data transfer is accomplished with the netcat program, specifically the version distributed with the nmap package (www.nmap.org). This version, ncat(1), has the best coverage of features that are used throughout the book. A familiarity with the man page for ncat(1) is helpful, but not required.
All scripts used in this book are found in Appendix B and published under the BSD 3-clause license. The scripts are also available on the Github IPFW Primer page.
When copy/pasting examples, be aware that some desktop copy/paste functions add an extra space (or multiple spaces) to the end of a line, messing up the Unix continuation character convention ' … \' at the end of a line. You should ensure that the paste function does not introduce extra spaces at the end of the line.
The lab examples in this book involve passing data between interfaces on the host system. A running firewall on the host such as pf, ipfw, or ipfilter (also known as ipf) may interfere with data transfer, so ensure that any host system firewall is disabled. In addition, take any necessary steps to ensure that this does not compromise the security of the host. |
1.1. Quick Start
This section details the steps to quickly begin using two QEMU virtual machines, one named "firewall" and the other named "external1". These are the first two of several that will be used throughout this book. Additional detail, along with setup instructions for all virtual machines, is provided in Appendix A.
Figure 1. Setting Up the Initial Virtual Machines
The initial setup is that shown in Figure 1.
1.1.1. QEMU VM Installation Process
Follow the steps below to install and configure the QEMU virtual machines for this example.
Install the necessary packages - qemu(1), nmap(1), sudo(1) (or doas(1)). Sudo, (or doas) is necessary for running the virtual machines as these QEMU configurations open a separate console window through SDL. The examples in this book use sudo.
# pkg install qemu nmap sudo
Configure sudo as desired.
Create a directory layout for virtual machines and scripts, and download an install ISO for FreeBSD.
% mkdir -p ~/ipfw/VM ~/ipfw/SCRIPTS ~/ipfw/ISO % cd ~/ipfw/ISO % fetch https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/<latest version>/FreeBSD-<latest-version>-RELEASE-amd64-dvd1.iso
Create the bridge and tap devices for the virtual machines (VMs) to use.
# ifconfig tap0 create # ifconfig tap1 create # sysctl net.link.tap.up_on_open=1 net.link.tap.up_on_open: 0 -> 1 # sysctl net.link.tap.user_open=1 net.link.tap.user_open: 0 -> 1 # ifconfig bridge0 create # ifconfig bridge0 addm tap0 addm tap1 addm hostintf <--- replace hostintf with host network interface (em0, bge0, etc.) # ifconfig bridge0 up
A script for creating and managing bridge and tap devices will be introduced in the next section.
Create two new VM image files and install FreeBSD on one.
% cd ~/ipfw/VM % qemu-img create -f qcow2 -o preallocation=full firewall.qcow2 8G % qemu-img create -f qcow2 -o preallocation=full external1.qcow2 8G % cd ~/ipfw/ISO % ln -s FreeBSD-<latest-version>-RELEASE-amd64-dvd1.iso fbsd.iso Copy the below text into a file (say, firewall.sh) and run: % sudo /bin/sh firewall.sh ------- #!/bin/sh # firewall.sh /usr/local/bin/qemu-system-x86_64 -monitor stdio \ -cpu qemu64 \ -vga std \ -m 4096 \ -smp 4 \ -cdrom ../ISO/fbsd.iso \ -boot order=cd,menu=on \ -blockdev driver=file,aio=threads,node-name=imgright,filename=../VM/firewall.qcow2 \ -blockdev driver=qcow2,node-name=drive0,file=imgright \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -netdev tap,id=nd0,ifname=tap0,script=no,downscript=no,br=bridge0 \ -device e1000,netdev=nd0,mac=02:69:70:66:77:00 \ -name \"Firewall\" exit -------The FreeBSD installer should boot. Perform a standard installation of FreeBSD.
During the installation note the following:
Select to use UFS as the filesystem. ZFS does not perform well with small memory sizes.
In this Quick Start, use DHCP for networking. If desired, configure IPv6 if supported by the local LAN.
When adding the default user, ensure they are a member of the wheel group.
Once the installation completes, the virtual machine reboots into the newly installed FreeBSD image.
Login as root and update the system if desired.
Repeat the above step to create another QEMU script file, and perform another installation with these changes:
Copy the below text into a file (say, external1.sh) and run: % sudo /bin/sh external1.sh ------- #!/bin/sh # external1.sh /usr/local/bin/qemu-system-x86_64 -monitor stdio \ -cpu qemu64 \ -vga std \ -m 4096 \ -smp 4 \ -cdrom ../ISO/fbsd.iso \ -boot order=cd,menu=on \ -blockdev driver=file,aio=threads,node-name=imgleft,filename=../VM/external1.qcow2 \ -blockdev driver=qcow2,node-name=drive0,file=imgleft \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,br=bridge0 \ -device e1000,netdev=nd0,mac=02:20:65:78:74:31 \ -name \"External 1\" exit -------As above, login and update the system if desired.
On both virtual machines (and all later installed VMs) , install the nmap(1) package. This brings in the version of ncat(1) used by scripts on the firewall and external VMs.
# pkg install nmap
Finally, download IPFW_root_bin.tgz file to both VMs. This tar file has a number of scripts needed for the virtual machines.
Move the tarzip file into /root and:
# mv IPFW_root_bin.tgz /root # # cd /root # # tar xvzf IPFW_root_bin.tgz ... files are extracted into /root/bin # # chmod +x /root/bin/*.sh
(End installation procedure.)
For this Quick Start, it is Ok to use DHCP for both VMs.
In later examples there will be multiple external VMs using the 203.0.113.0/24 network, all set up the same way and attached via tap(4) interfaces to a if_bridge(4) on the FreeBSD host.
To ensure you have the first two VMs setup correctly, ping the firewall VM from the external1 VM and vice-versa. Communications should be successful. If not, check the above installation details and troubleshoot any network issues. It should be possible ping in both directions, and even ssh(1) from one VM to the other.
If you are having difficulty getting QEMU set up correctly, check the QEMU virtualization section in the FreeBSD Handbook. |
If the mouse is clicked in the QEMU console window, QEMU will “grab” the mouse. If this happens,type, Ctl+Alt+G to release the mouse. |
If suddenly, the QEMU console window is full screen, you have accidently typed Ctl+Alt+F. If this happens, retype Ctl+Alt+F to restore the desktop screen. |
1.1.2. Disabling Syslog Messages to the Console in the Virtual Machines
You may find it advantageous (even necessary) to stop syslog messages from being sent to the console (either the QEMU console, or the serial port).
To configure syslog to stop logging to the console, configure a file to receive console messages:
# touch /var/log/console.log # # chmod 0600 /var/log/console.log
Then, as root, modify the line in /etc/syslog.conf to read (instead of /dev/console:
*.err;kern.warning;auth.notice;mail.crit /var/log/console.log
And, if necessary:
# service syslogd restart
All messages previously bound for the console, will be directed to /var/log/console.log instead.
Before continuing, we have one more piece to add to each VM - a serial console. We will need a serial console to examine the state of each VM independent of the main console.
1.1.3. Adding the Serial Console Interface to the VMs
To add a serial console to each QEMU VM, start up the VM and edit the file /boot/loader.conf to add console=“comconsole” to allow use of the serial console. QEMU redirects the serial port I/O to a TCP port on the host system at VM startup, and the QEMU monitor waits to activate the guest VM until a telnet(1) connection occurs on that port on the host. Once the FreeBSD system starts booting and recognizes the console directive in /boot/loader.conf it starts up a serial console. The QEMU monitor detects this and directs the necessary character I/O on that serial port to the TCP port on the host.
It is important to note that the this serial redirect over TCP takes place outside the virtual machine. There is no interaction with any network on the virtual machine and thus it is not subject to any firewall rules. Think of it just like a "dumb terminal" sitting on an RS-232 serial port on a real machine.
Edits for the firewall VM are shown below:
#!/bin/sh ## firewall.sh echo echo "NOTE!!! telnet server running!" echo "To start QEMU, start another session and telnet to localhost 4450" echo /usr/local/bin/qemu-system-x86_64 -monitor stdio \ -serial telnet:localhost:4450,server=on,wait=on \ -cpu qemu64 \ -vga std \ -m 4096 \ -smp 4 \ -cdrom ../ISO/fbsd.iso \ -boot order=cd,menu=on \ -blockdev driver=file,aio=threads,node-name=imgright,filename=../VM/firewall.qcow2 \ -blockdev driver=qcow2,node-name=drive0,file=imgright \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -netdev tap,id=nd0,ifname=tap0,script=no,downscript=no,br=bridge0 \ -device e1000,netdev=nd0,mac=02:69:70:66:77:00 \ -name \"Firewall\" exit -------
Edits for the external1 VM:
------- #!/bin/sh ## external1.sh echo echo "NOTE!!! telnet server running!" echo "To start QEMU, start another session and telnet to localhost 4410" echo /usr/local/bin/qemu-system-x86_64 -monitor stdio \ -serial telnet:localhost:4410,server=on,wait=on \ -cpu qemu64 \ -vga std \ -m 4096 \ -smp 4 \ -cdrom ../ISO/fbsd.iso \ -boot order=cd,menu=on \ -blockdev driver=file,aio=threads,node-name=imgleft,filename=../VM/external1.qcow2 \ -blockdev driver=qcow2,node-name=drive0,file=imgleft \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,br=bridge0 \ -device e1000,netdev=nd0,mac=02:20:65:78:74:31 \ -name \"External 1\" exit -------
Manually configure the FreeBSD host, firewall VM, and external1 VM with the addressing as shown in the figure above. You should have full connectivity between your FreeBSD host, the firewall VM and the external1 VM.
Figure 2. External1 and Firewall VMs Startup with Serial Console
Figure Figure 2 shows both VMs starting up with (from the bottom up):
The QEMU monitor window running on the host.
The serial console (ttyu0) running on the VM serial console
The main console (ttyv0) running on the VM main console
Scripts used by the two VMs are discussed in the next section.
Chapter 2. IPFW Operation
If you have been successful so far, you now have the initial lab setup shown in Figure 3 except for IP addressing. All the examples in this book use "Special Use Addresses" for both IPv4 and IPv6. Address references used are described here for IPv4 Addresses (RFC 5737) and here for IPv6 Addresses (RFC 3849).
Configure both VMs to use the addresses shown. Once these addresses are in place on the VMs (and the alias address on the FreeBSD host), it is unlikely that the VMs will be able to access sites on the Internet. The 203.0.113.0/24 network is considered non-routable by IPS and major telecom carriers on the Internet. However, this does not really matter, since all the communications for this book are local to the VMs running on the FreeBSD host. Bottom line - if you want your VMs to access the Internet, leave them on DHCP, but for the examples in this book, you’ll want to use manual addressing as shown in the figure below.
Figure 3. ipfw external1 and firewall VMs With Special Use Addresses
Scripts used by the two VMs are discussed in the next section.
2.1. Firewall Server Scripts
To demonstrate the firewall capabilities of ipfw, the firewall VM runs certain scripts that open local UDP and TCP ports using ncat(1) for TCP operation, and nc(1) for UDP operation. While the commands perform similar functions, the nc version on the base system is the one that performs best for receiving UDP traffic as it supports the keep-open option (-k).
The firewall VM runs one of four basic scripts. Recall that these are localed in /root/bin:
tserv.sh (userv.sh): Script that opens one TCP (UDP) port and listens for incoming traffic.
tserv3.sh (userv3.sh): Script that opens 3 TCP (UDP) ports and listens for incoming traffic.
These scripts will listen for an incoming connection and print whatever is sent over during the connection. When the connection is closed, the script will continue to listen for the next connection.
These script provide the basic mechanism for receiving a TCP or UDP connection. Once we start the firewall and populate the ipfw ruleset, we will see the effect each rule has on the connection capabilities.
2.2. External VM Scripts
Likewise, there are other basic scripts the external1 (and later, external2 and external3) VMs use for initiating or establishing communications with the firewall VM. The TCP and UDP versions perform similarly:
tcon.sh (ucon.sh): Connect via TCP (or UDP). This script takes a single argument, a port number to use for the connection. The external VM host can change the port number at each prompt. If there is no script listening on the port on the firewall, the script will indicate a "connection refused" error.
tconr.sh (uconr.sh): Connection takes a random port number and a sleep value. The script randomly selects one of three ports for its connection in a loop, controlled by the sleep value. If there is a listener on the firewall active on the port, the connection succeeds - otherwise the connection is refused.
tcont.sh (ucont.sh): Connection takes a port number and sleep value. The communication uses the same port in a loop, controlled by the sleep value.
These are simple scripts, but they allow for independent activity by the external VMs, while the firewall VM admin creates and tests ipfw rulesets. Most of the examples in the first part of this book can be done with just these scripts, so it is a good idea to familiarize yourself with their operation. Later scripts will use hping3(8), and iperf3(1), versatile tools used in network analysis.
By default, the external VMs and firewall VM scripts work on ports 5656, 5657, and 5658.
The randomized communication scripts also utilize port 5659, but since no services are listening on that port on the firewall VM, the connection fails.
It is important to understanding the underlying network activity. If you are unfamiliar with the basics of Internet protocols, network traffic, traffic monitoring and so forth, there are a number of excellent books, white papers, and tutorials, many free over the Internet. Check Appendix C for a modest selection.
2.3. Loading IPFW
ipfw can be built into the FreeBSD kernel directly, or it can be loaded as a kernel module. We will use the ipfw.ko loadable kernel module for most of the examples in this book. You can load ipfw as root with the command:
# kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled
Notice the kernel display output - “ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled”. This gives a quick summary of this host’s ipfw's capabilities. The most important note is "default to deny" which indicates that, by default, the firewall has an immutable rule located at the end of the ruleset that denies all Internet Protocol (IP) traffic. This rule depends on how the kernel was configured when it was built. By default it is "default to deny". However, if you are working on a FreeBSD system where you do not know its provenance, use the ipfw list command to make sure:
# ipfw list 65535 deny ip from any to any #
Note that this does not mean it denies all network traffic, only traffic that is based on the Internet Protocol (RFC 791), and all of its derivatives (TCP, UDP, ICMP, etc). If you had the capability to send and receive non-IP based traffic you could possibly still send and receive it. You would need special rules to deny all traffic. We will see an example of that later in this book.
To begin the next section, start with the ipfw firewall unloaded:
# kldunload ipfw
Unloading (# kldunload ipfw) and loading (# kldload ipfw) the ipfw kernel module is a handy way of completely reinitializing ipfw. This removes all rules, sets, queues, pipes, and other ipfw operations. See kldload(8), kldunload(8), and kldstat(8) for details. |
2.4. Initial Firewall Setup
In this section we will introduce the operation of the scripts described above and demonstrate simple traffic filtering.
In the first example, the firewall host runs tserv.sh which opens TCP port 5656.
Figure 4. Simple Transmit with No ipfw In Place
The external1 VM runs tcon.sh, which repetitively opens a TCP connection and sends data to the firewall VM.
Since there is no firewall in place, all TCP connections succeed.
On the host machine, it is possible to run tcpdump(1) on the bridge0 device to see the traffic in real time. Shown below is one successful transfer, i.e there are no firewall rules preventing the connection.
It follows the basic TCP connection sequence – 3 way handshake setup, send data, and close the connection:
Figure 5. tcpdump(1) of Bridge Traffic During Transfer
We now load the ipfw firewall and retry the communication.
# kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled # # ipfw list 65535 deny ip from any to any #
Figure 6. Simple Transmit with Default Rule In Place
No communications were successful - the connections time out because the ipfw firewall has denied traffic with the default deny rule as described above.
The external1 VM host sends SYN packets to start the connection, but they never reach the firewall VM’s TCP service on port 5656.
The TCP 3-way handshake is never completed.
We will use these same techniques to show how communications and data transfer operate. We now turn our attention to firewall rule creation in the next section.
Chapter 3. IPFW Rules
The manual page for ipfw(8) lists the entire command syntax including those for general ruleset construction. We will look first at basic traffic rules and study the more complex capabilities as we progress. Each section will build on the previous sections so that you will be better positioned to understand the advanced material later in the book.
Almost all the examples in this section can be run on the architecture used in the previous chapter, specifically that of Chapter 2, Figure 1. The Quick Start section in the Introduction can provide guidance on setting this back up. Ensure you have the addressing set as shown in the above Chapter 2, Figure 1.
ipfw rules take the general format of:
# ipfw command [rule_number] [set set_number] [prob match_probability] action [log [logamount number]] [altq queue] [{tag | untag} number] body
Bolded keywords indicate literal option text that is added to a rule. Italicized keywords indicate a block of additional content that is rule dependent.
The rule body has its own syntax format:
[ proto from src to dst ] [ options ]
Here is an example with basic syntax for an entire rule:
Figure 7. Description of ipfw Rule Syntax
The FreeBSD ipfw shell command
The ipfw rule command (add)
The optional rule number (1000)
An optional set keyword and value (set 2)
An optional probability keyword and value (prob 0.5)
The rule action keyword (deny)
An optional log keyword (log)
An optional logamount keyword and value (logamount 50000)
An optional altq keyword and value (altq red)
An optional tag keyword and value (tag 27)
The body of the rule starting with a protocol keyword (tcp)
A source direction keyword (from)
A source address (any internet address)
A destination keyword (to)
A destination address (any interface on the local system)
A destination port number (8081)
A comment
The example in Figure 4 would not actually load - there is no altq(9) queue named red set up yet - but it does show the overall format of how rules are constructed.
This example also shows the use of the Unix line continuation convention using a single backslash at the end of the line (with no following spaces) to continue to the next line.
The essence of ipfw rule processing is found when patterns that are defined in the rule body are matched on incoming or outgoing packets, one at a time and the action keywords are processed in turn.
In other words, ipfw directs traffic flow by first matching each incoming or outgoing packet against patterns supplied within the body of the rule. The patterns include protocols (tcp, udp, igmp, eigrp, etc.), source and destination addresses and ports, and options that apply to the context of the traffic.
3.1. Practical Ruleset Development
In this section, we will concentrate on the basic commands and actions.
The basic command keywords are these:
enable/disable - commands to disable or enable ipfw rule processing. The kernel module remains loaded - the effect is to suspend or resume rule processing. This is important to understand early as these commands function like an 'on/off' switch to firewall operation.
add - adds a rule.
delete - deletes a rule. The rule number must be specified - for example, ipfw delete 1000.
list - lists the contents of the current ruleset. Even if there have been no rules added, the list command should always list out the default rule, by default, 65535 deny ip from any to any.
show - similar to the list command, show includes counters for each rule matched.
flush - delete all the rules in the ruleset except for rules in set 31. (Sets are described later in this Chapter.) Since this is a command with enormous impact, a Yes/No prompt is issued before continuing.
The basic traffic flow action keywords are these:
allow | accept | pass | permit - direct ipfw to allow a packet through this rule should the packet match the rule body.
count - increment a counter applied to a rule. No other processing is applied to the packet.
deny | drop - do not allow a packet to pass through this rule should the packet match the rule body.
check-state [:flowname | :any] - check if a dynamic rule already exists.
reset - resets Network Address Translation tables.
The ipfw(8) man page has the complete list of action keywords, and describes each in detail.
We will also examine these keywords in the rules section:
prob - assign a probability (a value between 0 and 1) to the rule action.
set - use a collection of rules.
tag and untag - apply an internal tag to a packet affected by the rule.
log and logamout - log keywords.
reset - send a TCP reset on connection
tee - cause packets to flow in multiple ways
unreach - specify an action if a packet’s destination is unreachable
setdscp - set DiffServe parameters for outbound packets
skipto - jump around a ruleset
divert - pull packets into userspace for programmatic purposes
limit - limit the number of active connections
call and return - another way to jump around a ruleset
lookup tables and the lookup - log keywords.
Most action keywords, such as allow or deny, determine traffic flow.
It is important to become familiar with these actions as they will be used in almost every rule.
In addition, carefully note what ipfw does after it matches a packet and applies an action - it either terminates its search, or it goes on to the next rule.
Other action keywords perform an activity that does not have any impact on traffic flow. For example, the count action simply updates counters that apply to a rule. It has no effect on traffic flow and ipfw continues processing with the next rule.
Recall that ipfw is a command line program that uses all the words on the command line as parameters. In developing rules, remember that certain constructs such as braces ({,}), brackets ([,]) and even parentheses themselves are all recognized by the shell and must be escaped with a backslash '\'. The ipfw(8) man page has additional caveats on rule syntax.
We start with the smallest possible ruleset that permits the external1 VM to make a TCP connection to the a service running on the firewall VM.
Start up the external1 VM, and the firewall VM. Load the ipfw.ko kernel module and start the tserv.sh service on the firewall VM to listen for incoming connections. Then create the following ruleset on the firewall VM:
# ipfw add 100 check-state 00100 check-state :default # ipfw add 1000 allow tcp from any to me 5656 in via em0 setup keep-state 01000 allow tcp from any to me 5656 in via em0 setup keep-state :default
You can test this ruleset immediately by again running the tserv.sh on the firewall VM and the tcon.sh script on the external1 VM as described in the previous chapter. The connection should succeed.
Rule 100 contains the check-state option. It checks to see of a connection is already established and a dynamic rule is in place. If so, any additional packets matching the dynamic rule would be passed.
Rule 1000 contains the add command. This command inserts the requested rule into the ipfw ruleset where it can process packets against the specified actions in the rule body.
It uses the setup and keep-state options to create a dynamic rule for the connection.
In a stateful firewall like ipfw, once a connection from an external host to an internal host is established, the firewall creates a dynamic rule permitting continued traffic along this path until the connection is reset.
Note that without the check-state keyword, no check for a dynamic rule is performed and without the keep-state, no creation of a dynamic rule is performed.
Consider this ruleset with just a single rule:
# ipfw add 1000 allow tcp from any to me 5656
This rule looks like it should work, but it does not. A TCP packet entering ipfw has no pre-existing dynamic rule. Further, the rule does not create one. And, because there is no corresponding rule for outbound traffic, no TCP 3-way handshake is ever completed. Note that a SYN packet is received by the firewall, but not by the destination service.
If we add the rule:
# ipfw add 2000 allow tcp from me to any
the TCP 3-way handshake is allowed to complete and the data is sent from the external1 VM host to the tserv.sh process running on the firewall VM.
While this method works, it uses two rules instead of one. In this case, the better solution is to use the keep-state and check-state options early in the ruleset as shown in the original example in this section.
3.2. Dynamic Rules
So, what exactly are "dynamic rules"? The scripts we are using close the TCP connection each time, so the dynamic rules are short lived, and cannot be easily examined.
To see dynamic rules in action, manually set up a Netcat listener on the firewall host and send data with a Netcat sender on the external VM host:
On the firewall VM, start up the listener service manually:
# ncat -l 203.0.113.50 5656
Then, on the External1 VM, use ncat to connect to the service on the firewall and type a message:
# ncat 203.0.113.50 5656 hello there
The message should appear on the console of the firewall VM. If it does not, ensure that the original rule from the previous section is active.
Figure 8. Manually Creating Traffic to Examine Dynamic Rules
Figure 5 shows the connection is open between the external1 and firewall VMs.
While the connection is still open, run the following command on the firewall VM serial console:
Figure 9. Viewing Dynamic Rules
You should see output similar to that in Figure 6.
The -d option displays dynamic rules in addition to regular rules.
The -D option displays just dynamic rules.
3.2.1. Notes on Rule Numbering
Each rule is assigned a rule number, even if you do not specify one. The details for rule number handing are found in the ipfw(8) man page. Note that rules are assigned numbers in increments specified by the sysctl net.inet.ip.fw.autoinc_step
# sysctl net.inet.ip.fw.autoinc_step net.inet.ip.fw.autoinc_step: 100
We start with a simple check-state rule and see that ipfw has assigned a number associated with the increment sysctl shown above:
# ipfw add check-state 00000 check-state :default # ipfw list 00100 check-state :default 65535 deny ip from any to any00000 check-state :default
ipfw has automatically assigned the rule number 100. While it can be convenient to have ipfw add a rule number automatically, you should always assign rule numbers yourself. This ensures that you consciously put a rule in a specific place within the ruleset. With large rulesets this is critical. A rule automatically assigned by ipfw can be placed where it can have an unexpected effect.
Consider this ruleset:
# ipfw list 00300 deny ip from any to 200.200.200.200 00400 deny ip from any to 200.200.200.201 00500 deny ip from any to 200.200.200.202 00600 deny ip from any to 200.200.200.203 00700 deny ip from any to 200.200.200.204 00800 deny ip from any to 200.200.200.205 00800 deny ip from any to 200.200.200.206 65535 deny ip from any to any
Having forgotten to add the check-state rule you quickly add it:
# *ipfw add check-state* 00000 check-state :default
But the result may not be what you intended:
# ipfw list 00300 deny ip from any to 200.200.200.200 00400 deny ip from any to 200.200.200.200 00500 deny ip from any to 200.200.200.200 00600 deny ip from any to 200.200.200.200 00700 deny ip from any to 200.200.200.200 00800 deny ip from any to 200.200.200.200 00800 deny ip from any to 200.200.200.200 00900 check-state :default 65535 deny ip from any to any
ipfw allows rules with the same rule number to be added to the ruleset, and it will keep track of the rules in the order they were entered. This is easy to forget when manually entering rules from the command line and using command line editing to change something simple like the last byte of an IP address.
It is important to remember that all such rules are affected by commands that operate on one or more lines, such as the delete command:
# ipfw add 100 check-state 00100 check-state :default # ipfw add 1000 allow tcp from 203.0.113.10 to me 5656 keep-state 01000 allow tcp from 203.0.113.10 to me 5656 keep-state :default # ipfw add 1000 allow tcp from 203.0.113.20 to me 5656 keep-state 01000 allow tcp from 203.0.113.20 to me 5656 keep-state :default # ipfw add 1000 allow tcp from 203.0.113.30 to me 5656 keep-state 01000 allow tcp from 203.0.113.30 to me 5656 keep-state :default # ipfw add 1000 allow tcp from 203.0.113.40 to me 5656 keep-state 01000 allow tcp from 203.0.113.40 to me 5656 keep-state :default # # ipfw list 00100 check-state :default 01000 allow tcp from 203.0.113.10 to me 5656 keep-state :default 01000 allow tcp from 203.0.113.20 to me 5656 keep-state :default 01000 allow tcp from 203.0.113.30 to me 5656 keep-state :default 01000 allow tcp from 203.0.113.40 to me 5656 keep-state :default 65535 deny ip from any to any # # ipfw delete 1000 # # ipfw list 00100 check-state :default 65535 deny ip from any to any
The delete command can also process both ranges and lists of rules.
Consider the following ruleset:
# ipfw list 00100 check-state :default 01000 allow tcp from 203.0.113.10 to me 5656 keep-state :default 01100 allow tcp from 203.0.113.11 to me 5656 keep-state :default 01200 allow tcp from 203.0.113.12 to me 5656 keep-state :default 01300 allow tcp from 203.0.113.13 to me 5656 keep-state :default 02000 allow tcp from 203.0.113.20 to me 5656 keep-state :default 02100 allow tcp from 203.0.113.21 to me 5656 keep-state :default 02200 allow tcp from 203.0.113.22 to me 5656 keep-state :default 02300 allow tcp from 203.0.113.23 to me 5656 keep-state :default 03000 allow tcp from 203.0.113.30 to me 5656 keep-state :default 03100 allow tcp from 203.0.113.31 to me 5656 keep-state :default 03200 allow tcp from 203.0.113.32 to me 5656 keep-state :default 03300 allow tcp from 203.0.113.33 to me 5656 keep-state :default 04000 allow tcp from 203.0.113.40 to me 5656 keep-state :default 04100 allow tcp from 203.0.113.41 to me 5656 keep-state :default 04200 allow tcp from 203.0.113.42 to me 5656 keep-state :default 04300 allow tcp from 203.0.113.43 to me 5656 keep-state :default 04400 allow tcp from 203.0.113.44 to me 5656 keep-state :default 04500 allow tcp from 203.0.113.45 to me 5656 keep-state :default 04600 allow tcp from 203.0.113.46 to me 5656 keep-state :default 04700 allow tcp from 203.0.113.47 to me 5656 keep-state :default 04800 allow tcp from 203.0.113.48 to me 5656 keep-state :default 04900 allow tcp from 203.0.113.49 to me 5656 keep-state :default 65535 deny ip from any to any
A range is specified by two number separated by a dash: for example 5000-7350; whereas a list is a space-separated collection of numbers on the command line.
We wish to delete rules from 1000 to 2999 and all odd-numbered rules between 4000 and 5000:
# ipfw delete 1000-2999 4100 4300 4500 4700 4900 # # ipfw list 00100 check-state :default 03000 allow tcp from 203.0.113.30 to me 5656 keep-state :default 03100 allow tcp from 203.0.113.31 to me 5656 keep-state :default 03200 allow tcp from 203.0.113.32 to me 5656 keep-state :default 03300 allow tcp from 203.0.113.33 to me 5656 keep-state :default 04000 allow tcp from 203.0.113.40 to me 5656 keep-state :default 04200 allow tcp from 203.0.113.42 to me 5656 keep-state :default 04400 allow tcp from 203.0.113.44 to me 5656 keep-state :default 04600 allow tcp from 203.0.113.46 to me 5656 keep-state :default 04800 allow tcp from 203.0.113.48 to me 5656 keep-state :default 65535 deny ip from any to any
Note that the delete command will operate on comma separated values, but the delete command will only remove the first value in a comma separated list which is usually not what you intend. The command does not throw an error, but it does not delete all the lines requested.
# ipfw delete 3100,3200,3300 # echo $? 0 <--- No error found with the previous command. # # ipfw list 00100 check-state :default 03000 allow tcp from 203.0.113.30 to me 5656 keep-state :default 03200 allow tcp from 203.0.113.32 to me 5656 keep-state :default 03300 allow tcp from 203.0.113.33 to me 5656 keep-state :default 04000 allow tcp from 203.0.113.40 to me 5656 keep-state :default 04200 allow tcp from 203.0.113.42 to me 5656 keep-state :default 04400 allow tcp from 203.0.113.44 to me 5656 keep-state :default 04600 allow tcp from 203.0.113.46 to me 5656 keep-state :default 04800 allow tcp from 203.0.113.48 to me 5656 keep-state :default 65535 deny ip from any to any
The show command is similar to the list command but it also includes packet count and byte count for each rule. Consider the following ruleset on the firewall VM:
# ipfw list 00100 check-state :default 01000 allow udp from 203.0.113.10 to me 5656 02000 allow udp from 203.0.113.10 to me 5657 03000 allow udp from 203.0.113.10 to me 5658 04000 allow udp from 203.0.113.10 to me 5659 65535 deny ip from any to any
On the external1 VM, run the uconr.sh script to send packets to ports 5656, 5657, and 5658, randomly:
# sh uconr.sh 5656 1 PORT1 = [5656] SLEEPVAL = [1] UDP packet from [203.0.113.10],[5656],[1] UDP packet from [203.0.113.10],[5656],[2] UDP packet from [203.0.113.10],[5658],[3] UDP packet from [203.0.113.10],[5657],[4] UDP packet from [203.0.113.10],[5659],[5] UDP packet from [203.0.113.10],[5659],[6] UDP packet from [203.0.113.10],[5659],[7] UDP packet from [203.0.113.10],[5656],[8] UDP packet from [203.0.113.10],[5658],[9] UDP packet from [203.0.113.10],[5659],[10] UDP packet from [203.0.113.10],[5658],[11] UDP packet from [203.0.113.10],[5656],[12] UDP packet from [203.0.113.10],[5656],[13] UDP packet from [203.0.113.10],[5656],[14] UDP packet from [203.0.113.10],[5659],[15] UDP packet from [203.0.113.10],[5656],[16] UDP packet from [203.0.113.10],[5657],[17] UDP packet from [203.0.113.10],[5659],[18] UDP packet from [203.0.113.10],[5659],[19] UDP packet from [203.0.113.10],[5657],[20] UDP packet from [203.0.113.10],[5659],[21] ^C
Running the ipfw show command outputs:
# ipfw show 00100 0 0 check-state :default 01000 7 494 allow udp from 203.0.113.10 to me 5656 02000 3 212 allow udp from 203.0.113.10 to me 5657 03000 3 211 allow udp from 203.0.113.10 to me 5658 04000 8 565 allow udp from 203.0.113.10 to me 5659 65535 11 897 deny ip from any to any
The output shows the number of packets and the number of bytes processed by each rule, including the default rule which may have processed many more packets.
Note that the -D command parameter will show counts for dynamic rules similar to the above.
# ipfw -D show Dynamic rules (2 288): 04000 2 100 (1s) STATE tcp 203.0.113.10 31662 <-> 203.0.113.50 5659 :default 03000 8 482 (1s) STATE tcp 203.0.113.10 45732 <-> 203.0.113.50 5658 :default
This is a useful tool for debugging. Paired with the zero command which can clear counters with precise rule selection, it can show what rules are still processing a rule match.
The zero command takes a space separated list of rules (similar to the delete command) to clear counters. However, unlike the delete command, ranges (e.g 2000-3000) are not allowed.
# ipfw zero 2000 3000 # # ipfw show 00100 0 0 check-state :default 01000 7 494 allow udp from 203.0.113.10 to me 5656 02000 0 0 allow udp from 203.0.113.10 to me 5657 03000 0 0 allow udp from 203.0.113.10 to me 5658 04000 8 565 allow udp from 203.0.113.10 to me 5659 65535 11 897 deny ip from any to any
Clearing all rule match counters can be done with ipfw zero with no parameters.
Counters are also a feature of rules that specify the log keyword. We will see an example of this when we explore the log and logamount keywords later.
3.3. Keywords
3.3.1. Protocols
Protocols are those defined by IANA - the Internet Assigned Numbers Authority (https://www.iana.org) and are included in Unix systems in /etc/protocols. From this file, we can learn what numbers are assigned to common (and some very obscure) protocols - ip (0), tcp (6), udp (17), icmp(1), and many others.
Source and destination protocols can be the conventional IP or IPv6 addresses. However, the ipfw(8) manual page has this more detailed explanation:
"The first part (proto from src to dst) is for backward compatibility with earlier versions of FreeBSD. In modern FreeBSD any match pattern (including MAC headers, IP protocols, addresses and ports) can be specified in the options section."
The ipfw keywords for common protocols include:
ip4 | ipv4 Matches IPv4 packets.
ip6 | ipv6 Matches IPv6 packets.
ip | all Matches any IP packet.
The logical operator "or" can be use to combine multiple protocols where any one of them applies. The "{" and "}" braces can be used to group "or" conditions (known as "or-blocks"). Only one level of braces can be used. Braces must be escaped with a backslash '\' to prevent them from being interpreted directly by the command line shell:
# ipfw add 1100 deny \{ tcp or udp or eigrp or chaos \} from 1.2.3.4 to 5.6.7.8
01100 deny { tcp or udp or eigrp or chaos } from 1.2.3.4 to 5.6.7.8Consider this ruleset in a shell script:
#!/bin/sh
ipfw add 5000 deny \{ icmp or ip or igmp or ggp or ipencap or st2 or tcp or cbt or egp or igp or bbn-rcc or nvp or pup or argus or emcon or xnet or chaos or udp or mux or dcn or hmp or prm or xns-idp or trunk-1 or trunk-2 or leaf-1 or leaf-2 or rdp or irtp or iso-tp4 or netblt or mfe-nsp or merit-inp or dccp or 3pc or idpr or xtp or ddp or idpr-cmtp or tp++ or il or ipv6 or sdrp or ipv6-route or ipv6-frag or idrp or rsvp or gre or dsr or bna or esp or ah or i-nlsp or swipe or narp or mobile or tlsp or skip or ipv6-icmp or ipv6-nonxt or ipv6-opts or cftp or sat-expak or kryptolan or rvd or ippc or sat-mon or visa or ipcv or cpnx or cphb or wsn or pvp or br-sat-mon or sun-nd or wb-mon or wb-expak or iso-ip or vmtp or secure-vmtp or vines or ttp or nsfnet-igp or dgp or tcf or eigrp or ospf or sprite-rpc or larp or mtp or ax.25 or ipip or micp or scc-sp or etherip or encap or gmtp or ifmp or pnni or pim or aris or scps or qnx or a/n or ipcomp or snp or compaq-peer or ipx-in-ip or carp or pgm or l2tp or ddx or iatp or stp or srp or uti or smp or sm or ptp or isis or fire or crtp or crudp or sscopmce or iplt or sps or pipe or sctp or fc or rsvp-e2e-ignore or mobility-header or udplite or mpls-in-ip or manet or hip or shim6 or wesp or rohc or pfsync or divert \} from any to me
exit
Note that the above file is shown as one very long line and does not use the Unix line continuation convention.
This command will deny all traffic using all protocols defined in /etc/protocols. The above command will complete successfully. However, due to a bug in the "or-block" parser, you cannot list the "ip" protocol first. If you swap the first two protocols - icmp and ip - the command throws an error.
For example,
# ipfw add 1000 deny \{ igmp or ip or eigrp \} from any to meworks ok but
# ipfw add 1000 deny \{ ip or igmp or eigrp \} from any to me
ipfw: invalid OR blockfails.
The use of the logical "and" operator in a protocol block is an error. A packet can be in only one protocol at a time. However, the use of the logical "not" operator is permitted in front of a protocol identifier:
# ipfw add 1000 deny \{ icmp or not igmp \} from any to meCareful consideration of all logical conditions is essential to correct operation of a ruleset.
In later versions of FreeBSD, the use of the protocol "or-block" is noted as deprecated in ipfw(8) but the operation may still complete successfully until the feature is removed completely. |
3.3.2. Addresses
Source and destination addresses can be any of the following:
any - matches any IP address. We have already seen many examles of this keyword.
me - matches any IP address configured on an interface in the system.
Note that the interface does not have to have an IP or IPv6 address, nor does it have to be up or even exist at the time the rule is entered. Thus, be aware that a rule with keyword me may affect traffic on interfaces that are configured at a later time. Consider the following system interface list:
# ifconfig -a
em0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=481209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP>
ether 02:49:50:46:57:41
inet 203.0.113.50 netmask 0xffffff00 broadcast 203.0.113.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: loThe following attempt to add a rule with a non-existent interface succeeds even though there is no wlan0 interface defined:
# ipfw add 1000 deny tcp from 1.2.4.4 to me via wlan0 01000 deny tcp from 1.2.4.4 to me via wlan0
me6 - matches any IPv6 address similarly to the above me keyword.
table(name[,value]) - matches any IPv4 or IPv6 address for an entry in the named table.
Tables are discussed below in Lookup Tables.
IPv4 and IPv6 addresses follow the usual conventions regarding address and mask length. Addresses can also be grouped into a list, similar to the capability discussed for protocols above.
# ipfw add 2000 allow tcp from \{ 2.3.4.5/32, 3.4.5.0/24, 10.0.0.0/8 \} to me
# ipfw add 2000 allow tcp from \{ 2607:fcc0:0:35::dd/64, 2608:abcd:0:2300::9eac/64 \} to meHowever, you cannot mix IPv4 and IPv6 addresses in the same list. Use two different rules instead.
For sparse collections of addresses, consider the alternate form allowed by
addr-set: addr[/masklen]{list}
such as:
# ipfw add 1000 allow tcp from 1.2.3.0/24\{128,9,35-45,7\} to meIn this form, lists and increasing ranges are allowed. ipfw will consolidate overlapping ranges, and will reorder the list in the display to show increasing addresses from left to right. Note that spaces are only allowed after commas between list elements, nowhere else.
This form does not work due to incorrect placement of spaces:
# ipfw add 1000 allow tcp from 1.2.4.0/24\{ 128,9,25-45,7-25 \} to me
ipfw: missing ``to''This form works by correcting where spaces occur on the command line.
# ipfw add 1000 allow tcp from 1.2.4.0/24\{128,9,25-45,7-25\} to me
01000 allow tcp from 1.2.4.0/24{7-45,128} to meAs noted, ipfw will simplify, reorder, and display the list:
# ipfw list
01000 allow tcp from 1.2.4.0/24{7-45,128} to me
65535 deny ip from any to anyNote that ranges must be defined as increasing. Also, as noted in ipfw(8), there is no support for sets of IPv6 addresses.
3.3.3. Ports
Ports may be specified by number or by service name. Service names, also under the provenance of IANA, are found in /etc/services on Unix systems.
Ports can be specified as individual items, lists, or ranges. Typically ports are used to determine a destination service and so apply to the destination address (although source ports are also permitted):
# ipfw add 1000 allow tcp from 1.2.3.4 to me daytime 01000 allow tcp from 1.2.3.4 to me 13 # ipfw add 2000 allow tcp from 2.3.4.5 to me ssh, telnet, smtp 02000 allow tcp from 2.3.4.5 to me 22,23,25 # ipfw add 3000 allow tcp from 3.4.5.6 to me auditd-domain 03000 allow tcp from 3.4.5.6 to me 48-53
General Notes on Port Ranges
A range such as that shown above (
auditd-domain) may accidentally include ports you do not want. In this case, portstacacs (49),re-mail-ck (50),la-maint (51), andxns-time (52)would be included. Even if this is really what you want, you should always check actual port numbers when using named ranges for ports.Some service names include the dash character '-' as part of the name, as in the point above. In these cases you will have to escape the dash with a double backslash, one for the shell and one for ipfw:
# ipfw add 4000 allow tcp from 4.5.6.7 to me ftp, ftp\\-data 04000 allow tcp from 4.5.6.7 to me 21,20
Some applications require a range of source and destination ports in both directions. This is easy to accomplish with ranges and a
keep-staterule:# ipfw add 1000 allow tcp from 203.0.113.10 5200-5205 to me 5656-5658 keep-state 01000 allow tcp from 203.0.113.10 5200-5205 to me 5656-5658 keep-state :default # # ipfw list 01000 allow tcp from 203.0.113.10 5200-5205 to me 5656-5658 keep-state :default 65535 deny ip from any to any
To test common ports in both directions we have to manually connect using Netcat so we can set up the source and destination ports as we need them:
# # ipfw list 01000 allow tcp from 203.0.113.10 5200-5205 to me 5656-5658 keep-state :default 65535 deny ip from any to any
Note: the transmission and reception lines have been aligned on each side.
Figure 10. Manually Testing Common Ports in Both Directions
Note that no connection was achieved when the destination port was out of bounds (5659) and when the source port was out of bounds (5206).
3.3.4. Prob
The prob keyword is used to assign a chance, that is a probability (a floating point value between 0 and 1), that an incoming packet will be matched. If the chance is successful, the corresponding rule performs the required action. If the chance is not successful, the rule is not matched and rule processing continues to the next rule.
To test this keyword, we use "sh ucont.sh 5656 1" script on the external1 VM’s side to repeatedly send a UDP packet to the firewall VM, who is listening on UDP port 5656. Using the prob keyword, we set a probability of .5 (a 50% chance) that the packet will be matched. The action is to let the packet pass to the service, which just prints the contents of the packet.
As you can see below, there were 24 out of 50 packets received, very close to 50% for such a small sample:
# # ipfw add 3000 prob 0.5 allow udp from any to me 5656 03000 prob 0.500000 allow udp from any to me 5656 # # ipfw list 03000 prob 0.500000 allow udp from any to me 5656 65535 deny ip from any to any # # sh userv.sh 5656 PORT1 = [5656] Starting UDP listener on [203.0.113.50],[5656] UDP packet from [203.0.113.10],[5656],[5] UDP packet from [203.0.113.10],[5656],[6] UDP packet from [203.0.113.10],[5656],[7] UDP packet from [203.0.113.10],[5656],[10] UDP packet from [203.0.113.10],[5656],[11] UDP packet from [203.0.113.10],[5656],[13] UDP packet from [203.0.113.10],[5656],[14] UDP packet from [203.0.113.10],[5656],[16] UDP packet from [203.0.113.10],[5656],[19] UDP packet from [203.0.113.10],[5656],[20] UDP packet from [203.0.113.10],[5656],[22] UDP packet from [203.0.113.10],[5656],[28] UDP packet from [203.0.113.10],[5656],[29] UDP packet from [203.0.113.10],[5656],[32] UDP packet from [203.0.113.10],[5656],[33] UDP packet from [203.0.113.10],[5656],[34] UDP packet from [203.0.113.10],[5656],[35] UDP packet from [203.0.113.10],[5656],[37] UDP packet from [203.0.113.10],[5656],[38] UDP packet from [203.0.113.10],[5656],[39] UDP packet from [203.0.113.10],[5656],[41] UDP packet from [203.0.113.10],[5656],[43] UDP packet from [203.0.113.10],[5656],[45] UDP packet from [203.0.113.10],[5656],[49] ^C#
3.3.5. Sets
Firewall rules can be grouped into different sets, and you can switch between sets atomically. Why would you want this feature? Consider a datacenter with two sets of identical servers on separate networks. You want to take down one set for maintenance, but first you want to transfer traffic to the other set of servers. Bingo - you have a solution.
Sets are useful, but they do come with some caveats which we will describe throughout this section.
The default set is set 0. Let’s create rules in set 0 and 1 with slight differences between the two. Note that this example also shows the use of the ipfw comment feature - allowing comments on a per-rule basis.
# ipfw add 1000 set 0 check-state 01000 check-state :default # # ipfw add 1100 set 0 allow tcp from any to me 5656 setup keep-state // 5656 only 01100 allow tcp from any to me 5656 setup keep-state :default // 5656 only # # ipfw add 2000 set 1 check-state 02000 check-state :default # # ipfw add 2100 set 1 allow tcp from any to me 5657 setup keep-state // 5657 only 02100 allow tcp from any to me 5657 setup keep-state :default // 5657 only # # ipfw set show enable 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
The above ipfw set show command lists all enabled sets, here showing both 0 and 1 as enabled.
From the external1 VM, commence communications using the tcpcon.sh script as shown below. The first two communications ([1], [2]) are to port 5656, the next two communications are to port 5657 ([3],[4]). The firewall host shows all four communications received:
Figure 11. Use of Sets
Right before communication 5, the firewall host admin disabled set 0, (# ipfw set disable 0) effectively blocking access to port 5656. Disabling set 0, effectively removes the rules for port 5656 and so the communications [5] and [6] fail. The external1 VM goes back to port 5657 for communications [7] and [8], which are successful.
Notice that if you list the firewall ruleset with just ipfw list, only the sets that are enabled actually show up.
If you want to make sure which sets are enabled / disabled, use the -S flag on the ipfw command as shown below.
# ipfw set enable 0 # # ipfw set show enable 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 # # ipfw -S list 01000 set 0 check-state :default 01100 set 0 allow tcp from any to me 5656 setup keep-state :default // 5656 only 02000 set 1 check-state :default 02100 set 1 allow tcp from any to me 5657 setup keep-state :default // 5657 only 65535 set 31 deny ip from any to any # # ipfw set disable 0 # # ipfw set show disable 0 enable 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 # # ipfw -S list # DISABLED 01000 set 0 check-state :default # DISABLED 01100 set 0 allow tcp from any to me 5656 setup keep-state :default // 5656 only 02000 set 1 check-state :default 02100 set 1 allow tcp from any to me 5657 setup keep-state :default // 5657 only 65535 set 31 deny ip from any to any #
To atomically change from one set to the other use the ipfw set swap command:
# ipfw set swap 0 1
What actually happens is that the rules in the sets get swapped; that is, all the rules in set 0 get put in set 1 and all the rules in set 1 get put in set 0.
# # ipfw -S list # DISABLED 01000 set 0 check-state :default # DISABLED 01100 set 0 allow tcp from any to me 5656 setup keep-state :default // 5656 only 02000 set 1 check-state :default 02100 set 1 allow tcp from any to me 5657 setup keep-state :default // 5657 only 65535 set 31 deny ip from any to any # # ipfw set swap 0 1 # # ipfw -S list 01000 set 1 check-state :default 01100 set 1 allow tcp from any to me 5656 setup keep-state :default // 5656 only # DISABLED 02000 set 0 check-state :default # DISABLED 02100 set 0 allow tcp from any to me 5657 setup keep-state :default // 5657 only 65535 set 31 deny ip from any to any
Note carefully that if you swap sets where one of the sets is disabled, the set number is still disabled after the swap, even though the rules are now different. This can lead to unexpected consequences such as the following:
# ipfw -S list 01000 set 0 check-state :default 01100 set 0 allow tcp from any to me 5656 setup keep-state :default // 5656 only 02000 set 1 check-state :default 02100 set 1 allow tcp from any to me 5660 setup keep-state :default // 5660 only 65535 set 31 deny ip from any to any # # ipfw set disable 0 # # ipfw -S list # DISABLED 01000 set 0 check-state :default # DISABLED 01100 set 0 allow tcp from any to me 5656 setup keep-state :default // 5656 only 02000 set 1 check-state :default 02100 set 1 allow tcp from any to me 5660 setup keep-state :default // 5660 only 65535 set 31 deny ip from any to any # # ipfw set swap 0 1 # # ipfw -S list 01000 set 1 check-state :default 01100 set 1 allow tcp from any to me 5656 setup keep-state :default // 5656 only # DISABLED 02000 set 0 check-state :default # DISABLED 02100 set 0 allow tcp from any to me 5660 setup keep-state :default // 5660 only 65535 set 31 deny ip from any to any
Here set 0 is disabled, and then swapped. After the swap, set 0 is still disabled, though the rules have changed.
Note that all sets are initially enabled. When you disable a set, say set 3, all other sets are still active, even if no rule references them. You can think of sets as the pieces on the back row of a chess board. If you remove a knight or a bishop, that one piece is not able to play, but all the others are able to play.
set 31 cannot be deleted or changed. It can however partially participate in a swap.
# ipfw set swap 1 31
This swap will complete successfully (return code 0), but the effect is not the same as the swaps above. The default rule in set 31 is not swapped, but the set number for the other rules (rules in set 1) are set to 31:
# ipfw -S list 01000 set 1 check-state :default 01100 set 1 allow tcp from 1.2.3.4 to me 8080 01200 set 1 allow tcp from 1.2.3.4 to me 8081 65535 set 31 deny ip from any to any # # ipfw set swap 1 31 # # ipfw -S list 01000 set 31 check-state :default 01100 set 31 allow tcp from 1.2.3.4 to me 8080 01200 set 31 allow tcp from 1.2.3.4 to me 8081 65535 set 31 deny ip from any to any
As noted in the ipfw(8) manual page, rules in set 31 cannot be flushed. We now have 4 rules in set 31:
# ipfw -f flush Flushed all rules. # # ipfw -S list 01000 set 31 check-state :default 01100 set 31 allow tcp from 1.2.3.4 to me 8080 01200 set 31 allow tcp from 1.2.3.4 to me 8081 65535 set 31 deny ip from any to any
We cannot use ipfw flush to clean out non-default rules, but we can use ipfw delete set 31 to clean out all but the default rule, which cannot be changed:
# ipfw delete set 31 # # ipfw -S list 65535 set 31 deny ip from any to any
Further, you should note that any set you disable remains disabled after a flush. If you disable a set and then flush the ruleset, any rules added back into that set will still be disabled. This includes set 0, the default set.
Consider the following:
# ipfw list 01000 check-state :default 01100 allow tcp from any to me 1111 02000 allow tcp from any to me 2222 03000 allow tcp from any to me 3333 65535 deny ip from any to any # # ipfw set disable 0 # # ipfw -S list # DISABLED 01000 set 0 check-state :default # DISABLED 01100 set 0 allow tcp from any to me 1111 # DISABLED 02000 set 0 allow tcp from any to me 2222 # DISABLED 03000 set 0 allow tcp from any to me 3333 65535 set 31 deny ip from any to any # # ipfw -f flush Flushed all rules. # # ipfw add 100 check-state 00100 check-state :default # ipfw add 200 allow tcp from any to me 5555 00200 allow tcp from any to me 5555 # ipfw add 300 allow tcp from any to me 6666 00300 allow tcp from any to me 6666 # ipfw add 400 allow tcp from any to me 7777 00400 allow tcp from any to me 7777 # # ipfw -S list # DISABLED 00100 set 0 check-state :default # DISABLED 00200 set 0 allow tcp from any to me 5555 # DISABLED 00300 set 0 allow tcp from any to me 6666 # DISABLED 00400 set 0 allow tcp from any to me 7777 65535 set 31 deny ip from any to any #
Because set 0 was disabled before the flush, the flush has no effect on the enable/disable state of that set.
Note that you can even disable sets that do not yet exist:
# kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled # # ipfw -S list 65535 set 31 deny ip from any to any # # ipfw set show enable 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 # # ipfw set disable 4 5 6 7 8 9 # # ipfw set show disable 4 5 6 7 8 9 enable 0 1 2 3 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 #
Using sets can be very helpful, as long as you understand their properties and limitations.
3.3.6. Tags
Tags allow for marking incoming packets in such a way that later rules can be applied based on the tag.
For a simple example, consider tagging incoming packets from different networks. Later rules determine if the tagged packets are allowed or denied:
# ipfw add 100 check-state 00100 check-state :default # # ipfw add 1000 count tag 10 tcp from 172.16.200.0/24 to me 5656 01000 count tag 10 tcp from 172.16.200.0/24 to me 5656 # # ipfw add 1100 count tag 20 tcp from 172.16.225.0/24 to me 5656 01100 count tag 20 tcp from 172.16.225.0/24 to me 5656 # # ipfw add 1200 count tag 30 tcp from 203.0.113.0/24 to me 5656 01200 count tag 30 tcp from 203.0.113.0/24 to me 5656 # # ipfw add 3000 allow tcp from any to me tagged 30 setup keep-state 03000 allow tcp from any to me tagged 30 setup keep-state :default # # ipfw add 4000 deny tcp from any to me tagged 10,20 04000 deny tcp from any to me tagged 10,20 #
We can test this with manual use of Netcat and using the ncat(1) option to set its own source address.
To do this we first setup two alias address on the em0 interface:
# ifconfig em0 172.16.200.10/24 alias
# ifconfig em0 172.16.225.10/24 alias
#
# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
ether 02:49:50:46:57:10
inet 203.0.113.10 netmask 0xffffff00 broadcast 203.0.113.255
inet 172.16.200.10 netmask 0xffffff00 broadcast 172.16.200.255
inet 172.16.225.10 netmask 0xffffff00 broadcast 172.16.225.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
#
Figure 12. Use of Tags
Because of the rule tagging in this ruleset, only traffic tagged with value "30" is allowed to pass.
Tags, combined with lookup tables allow for powerful policy based network access.
3.3.7. Logging
ipfw supports two methods of logging:
3.3.7.1. Method 1 – using ipfw0, the ipfw pseudointerface
# kldload ipfw # ifconfig ipfw0 create
Note that you must have the ipfw.ko kernel module loaded before you can create the ipfw0 interface. Also, note that if you unload ipfw.ko, the interface is destroyed and is no longer available.
Why use the ipfw0 interface?
It is possible to read logs in real time with programs such as tcpdump(1), wireshark(1), or other network monitoring programs.
An example is given further below.
3.3.7.2. Method 2 – use syslogd(8)
Setting the sysctl variable net.inet.ip.fw.verbose = 1 will instruct the firewall to log packets to syslogd(8) even when the ipfw0 interface exists.
Syslogd must be configured via /etc/syslog.conf.
ipfw packets will be logged with a LOG_SECURITY facility.
The logging limit is configurable via net.inet.ip.fw.verbose_limit, which is set to 0 (unlimited) by default.
Why use syslog?
Of the two methods, it is the only one that processes count actions.
To test logging, create a rule with the log keyword:
# ipfw add 1000 allow log tcp from any to me 5656 setup keep-state
Counters are also a feature of rules that specify the log keyword. If we replace the above ruleset and add the log keyword, matches for all rules will be included in the log entries with associated counts.
With the ruleset changes to add the log keyword, our ruleset looks like this:
# ipfw list 00100 check-state :default 01000 allow log tcp from 203.0.113.10 to me 5656 keep-state :default 02000 allow log tcp from 203.0.113.10 to me 5657 keep-state :default 03000 allow log tcp from 203.0.113.10 to me 5658 keep-state :default 04000 allow log tcp from 203.0.113.10 to me 5659 keep-state :default 65535 deny ip from any to any
Using Method 1
Using Method 1, we capture/view logs with:
# tcpdump -i ipfw0 -X
Note: You can redirect binary data to a 'savefile' with the -w file option.
If you are just reading tcpdump output, experiment with the -v, -vv, and -vvv options, which gives increasingly more verbose output.
The example below examines traffic with the ucont.sh on the external1 VM and the userv3.sh script on the firewall VM.
Figure 13. UDP Traffic From External1 to Firewall
Logged traffic from the above communication appears on the log device ipfw0:
#
# tcpdump -i ipfw0 -X -vvv
tcpdump: WARNING: ipfw0: That device doesn't support promiscuous mode
(BIOCPROMISC: Invalid argument)
tcpdump: listening on ipfw0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:50:44.259127 IP (tos 0x0, ttl 64, id 61929, offset 0, flags [none], proto UDP (17), length 70)
203.0.113.10.27337 > 203.0.113.50.5656: [udp sum ok] UDP, length 42
0x0000: 4500 0046 f1e9 0000 4011 1c61 ac10 0a0a E..F....@..a....
0x0010: ac10 0a32 6ac9 1618 0032 00ff 5544 5020 ...2j....2..UDP.
0x0020: 7061 636b 6574 2066 726f 6d20 5b31 3732 packet.from.[172
0x0030: 2e31 362e 3130 2e31 305d 2c5b 3536 3536 .16.10.10],[5656
0x0040: 5d2c 5b31 5d0a ],[1].
20:50:44.581025 IP (tos 0x0, ttl 64, id 61930, offset 0, flags [none], proto UDP (17), length 70)
203.0.113.10.41914 > 203.0.113.50.5656: [udp sum ok] UDP, length 42
0x0000: 4500 0046 f1ea 0000 4011 1c60 ac10 0a0a E..F....@..`....
0x0010: ac10 0a32 a3ba 1618 0032 c80c 5544 5020 ...2.....2..UDP.
0x0020: 7061 636b 6574 2066 726f 6d20 5b31 3732 packet.from.[172
0x0030: 2e31 362e 3130 2e31 305d 2c5b 3536 3536 .16.10.10],[5656
0x0040: 5d2c 5b32 5d0a ],[2].
20:50:45.960845 IP (tos 0x0, ttl 64, id 61931, offset 0, flags [none], proto UDP (17), length 70)
203.0.113.10.33126 > 203.0.113.50.5656: [udp sum ok] UDP, length 42
0x0000: 4500 0046 f1eb 0000 4011 1c5f ac10 0a0a E..F....@.._....
0x0010: ac10 0a32 8166 1618 0032 ea5f 5544 5020 ...2.f...2._UDP.
0x0020: 7061 636b 6574 2066 726f 6d20 5b31 3732 packet.from.[172
0x0030: 2e31 362e 3130 2e31 305d 2c5b 3536 3536 .16.10.10],[5656
0x0040: 5d2c 5b33 5d0a ],[3].
20:50:47.303505 IP (tos 0x0, ttl 64, id 61932, offset 0, flags [none], proto UDP (17), length 70)
203.0.113.10.39687 > 203.0.113.50.5656: [udp sum ok] UDP, length 42
0x0000: 4500 0046 f1ec 0000 4011 1c5e ac10 0a0a E..F....@..^....
0x0010: ac10 0a32 9b07 1618 0032 d0bd 5544 5020 ...2.....2..UDP.
0x0020: 7061 636b 6574 2066 726f 6d20 5b31 3732 packet.from.[172
0x0030: 2e31 362e 3130 2e31 305d 2c5b 3536 3536 .16.10.10],[5656
0x0040: 5d2c 5b34 5d0a ],[4].
^CUsing Method 2
Using Method 2, first examine /etc/syslog.conf to see if there is already a facility and level for security listed. In modern versions of FreeBSD it is common to see:
security.* /var/log/security
ipfw creates logs with the LOG_SECURITY facility and sends output to the file /var/log/security in this case.
If this entry exists on your system, you are all set, otherwise read through the below and set up your own entry for an ipfw logfile.
To log to syslogd(8), add the following line to the end of /etc/syslog.conf:
security.info /tmp/ipfw.info
(FreeBSD syslog.conf allows tabs or spaces to be used.)
Create the logfile with
# touch /tmp/ipfw.info
then send a HANGUP signal to the syslogd daemon:
# kill -HUP <pid of syslogd>
You can use tail -f to see logs in real time.
Note that ipfw only logs matched rules with this method.
Note that ipfw logs to the .info level, and to the .debug level.
# tail -f /var/log/security Apr 3 14:50:12 firewall newsyslog[401]: logfile first created Apr 9 21:05:13 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:59203 203.0.113.50:5656 in via em0 Apr 9 21:05:15 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:12401 203.0.113.50:5656 in via em0 Apr 9 21:05:16 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:45319 203.0.113.50:5656 in via em0 Apr 9 21:05:26 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:26190 203.0.113.50:5656 in via em0 Apr 9 21:05:29 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:55268 203.0.113.50:5656 in via em0 ^C
The log entry includes the date, time, host, service, and rule number (1000 above) to make it easy to track which rule is being matched.
General Notes on logging
Consider the following ruleset:
# ipfw list 00100 check-state :default 01000 allow tcp from 203.0.113.100 to me setup keep-state :default 02000 allow icmp from 203.0.113.100 to me 02100 allow icmp from me to 203.0.113.100 03000 allow log udp from 203.0.113.10 to me 5656 04000 allow log logamount 20 udp from 203.0.113.10 to me 5657 05000 allow log logamount 20 udp from 203.0.113.10 to me 5658 65535 deny ip from any to any
Using Method 2 (syslog), on a quiet system, you may notice that the entries do not appear right away if you are reading the security log file in real time (for example, tail -f /var/log/security).
This is because syslog will buffer identical lines and output a notification only occasionally as in the below example:
Mar 28 22:30:01 firewall kernel: ipfw: 3000 Accept UDP 203.0.113.10:27519 203.0.113.50:5656 in via em0 Mar 28 22:30:03 firewall syslogd: last message repeated 4 times Mar 28 22:32:31 firewall syslogd: last message repeated 31 times
Also, Method 1 (using the ipfw0 interface) and Method 2 (syslog) are mutually exclusive.
You cannot have both active at the same time.
If net.inet.ip.fw.verbose=0, the output will be to the ipfw0 interface; if the value is 1, the log output will be to syslog.
logamount values in rules only apply to Method 2 - syslog.
They have no effect on limiting the number of packets sent out the ipfw0 interface.
In Method 2 - syslog, when the log limit is reached, ipfw will send a notification similar to the following into the designated security logging file (default: /var/log/security):
Mar 28 23:00:10 firewall kernel: ipfw: 5000 Accept UDP 203.0.113.10:63367 203.0.113.50:5658 in via em0 Mar 28 23:00:11 firewall kernel: ipfw: 5000 Accept UDP 203.0.113.10:30909 203.0.113.50:5658 in via em0 Mar 28 23:00:11 firewall kernel: ipfw: limit 20 reached on entry 5000
And, at the same time, it also conveniently sends the same message to /var/log/messages, the standard FreeBSD logfile:
Mar 28 23:00:11 firewall kernel: ipfw: limit 20 reached on entry 5000
After that notification is sent, no more syslog entries will be sent from the matching rule until the log counters are reset with:
# ipfw resetlog <rule number>
When the resetlog command is entered, ipfw will send a reset notification to syslog:
Mar 28 23:04:51 firewall kernel: ipfw: logging count reset.
Unfortunately, as of FreeBSD version 14.1, it does not tell you which rule the count was reset for. Presumably, you should know which rule since you just entered the command, at least if you are the only person configuring the ruleset. In any case, you may have to keep track of that manually when working with many rules that include the log keyword.
If you issue an ipfw resetlog command without specifying a rule number, all counters in all rules are reset and ipfw sends the following notification:
Mar 28 23:08:52 firewall kernel: ipfw: All logging counts reset.
Finally, note that the sysctl variable net.inet.ip.fw.verbose_limit provides a "default limit" if one is not specified with the logamount keyword in the ruleset.
Consider this scenario:
# sysctl net.inet.ip.fw.verbose_limit net.inet.ip.fw.verbose_limit: 5 <---- The limit is preset to 5 # ipfw list 65535 deny ip from any to any
As new rules are added, ipfw will apply any logamount value it finds in the body of a rule.
If a rule being entered does not have a logamount entry, the value defaults to the current net.inet.ip.fw.verbose_limit amount.
# ipfw add 100 check-state 00100 check-state :default # # ipfw add 3000 allow log udp from 203.0.113.10 to me 5656 03000 allow log logamount 5 udp from 203.0.113.10 to me 5656 # # ipfw add 4000 allow log logamount 20 udp from 203.0.113.10 to me 5657 04000 allow log logamount 20 udp from 203.0.113.10 to me 5657 # # ipfw list 00100 check-state :default 03000 allow log logamount 5 udp from 203.0.113.10 to me 5656 04000 allow log logamount 20 udp from 203.0.113.10 to me 5657 65535 deny ip from any to any
If the sysctl for net.inet.ip.fw.verbose_limit is changed after the rule is entered, it has no effect:
# sysctl net.inet.ip.fw.verbose_limit=3 net.inet.ip.fw.verbose_limit: 5 -> 3
and later in /var/log/messages
Mar 29 11:07:02 firewall kernel: ipfw: limit 5 reached on entry 3000 ... Mar 29 11:07:24 firewall kernel: ipfw: limit 20 reached on entry 4000
3.3.8. Reset
The reset keyword sends an immediate TCP reset on a rule match containing that keyword. This immediately shuts down any TCP connection from the source matching the rule.
# ipfw list 00100 check-state :default 01000 allow log tcp from 203.0.113.10 to me 5656 setup keep-state :default 02000 reset log tcp from 203.0.113.10 to me 5657 03000 reset log udp from 203.0.113.10 to me 5658 65535 deny ip from any to any
The syslog view of a TCP reset rule match looks like this:
Apr 9 21:44:49 firewall kernel: ipfw: 1000 Accept TCP 203.0.113.50:5656 203.0.113.10:28218 out via em0 Apr 9 21:44:49 firewall syslogd: last message repeated 1 times Apr 9 21:44:49 firewall kernel: ipfw: 1000 Accept TCP 203.0.113.10:28218 203.0.113.50:5656 in via em0 Apr 9 21:45:01 firewall kernel: ipfw: 2000 Reset TCP 203.0.113.10:12998 203.0.113.50:5657 in via em0 Apr 9 21:45:07 firewall kernel: ipfw: 2000 Reset TCP 203.0.113.10:13782 203.0.113.50:5657 in via em0
When sysctl net.inet.ip.fw.verbose=0, there is no discernable output on ipfw0 for a reset action.
A TCP SYN packet arrives and that is all that is displayed.
To actually witness the reset, run tcpdump(8) on the specific interface:
# tcpdump -i em0 -X -vvv
tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:53:39.376825 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.10.32945 > 203.0.113.50.5657: Flags [S], cksum 0x68fa (correct), seq 1926269947, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 648984165 ecr 0], length 0
0x0000: 4500 003c 0000 4000 4006 ce5f ac10 0a0a E..<..@.@.._....
0x0010: ac10 0a32 80b1 1619 72d0 8bfb 0000 0000 ...2....r.......
0x0020: a002 ffff 68fa 0000 0204 05b4 0103 0306 ....h...........
0x0030: 0402 080a 26ae b665 0000 0000 ....&..e....
21:53:39.377143 IP (tos 0x10, ttl 64, id 1965, offset 0, flags [none], proto TCP (6), length 40)
203.0.113.50.5657 > 203.0.113.10.32945: Flags [R.], cksum 0xaddc (correct), seq 0, ack 1926269948, win 0, length 0
0x0000: 4510 0028 07ad 0000 4006 06b7 ac10 0a32 E..(....@......2
0x0010: ac10 0a0a 1619 80b1 0000 0000 72d0 8bfc ............r...
0x0020: 5014 0000 addc 0000 P.......
^CA UDP rule containing the reset keyword just drops the packet. Nothing is sent back to the source address. If the log keyword is also used on the rule, a log entry is generated for syslog (if enabled):
Apr 9 21:58:49 firewall kernel: ipfw: 3000 Reset UDP 203.0.113.10:56503 203.0.113.50:5658 in via em0
3.3.9. Tee
The tee rule requires a divert(4) socket set up beforehand. Refer to the divert rule covered below for setting up the socket. Once the socket is set up, the tee keyword works like divert except that it is not interested in any packet return. It is simply copying the packet to the socket. Processing continues with the next rule.
In essence, tee allows the packet to be sent to userspace for any purpose desired - monitoring, copying, counting - whatever.
ipfw add 1000 tee 700 ip from any to me
3.3.10. Unreach
The unreach keyword directs ipfw to respond back to the source when packets arrive with a destination port that is not opened by any service. ipfw sends an ICMP reply with the code set to the keyword parameter. This works for any IP protocol.
Because ipfw sends an ICMP packet back to the source, the ruleset must allow outbound ICMP.
Consider the following ruleset:
# ipfw -a list 00100 0 0 allow icmp from me to any 01000 0 0 unreach 100 log udp from any to me 5656 02000 0 0 unreach 200 log tcp from any to me 5657 03000 0 0 unreach 250 log ip from any to me 5658
The counters are zero when the external1 VM sends its first packet, a UDP packet destined for port 5656.
ipfw matches this with rule 1000 and sends an ICMP unreachable notice with code 100 (arbitrary value, but see list in ipfw(8)). The offending packet is encapsulated in the data portion of the ICMP reply.
# tcpdump -i bridge0 -X -vvv
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:55:59.666002 IP (tos 0x0, ttl 64, id 25431, offset 0, flags [none], proto UDP (17), length 70)
203.0.113.10.36146 > 203.0.113.50.5656: [udp sum ok] UDP, length 42
0x0000: 4500 0046 6357 0000 4011 aaf3 ac10 0a0a E..FcW..@.......
0x0010: ac10 0a32 8d32 1618 0032 de95 5544 5020 ...2.2...2..UDP.
0x0020: 6174 7461 636b 2066 726f 6d20 5b31 3732 communication.from.[172
0x0030: 2e31 362e 3130 2e31 305d 2c5b 3536 3536 .16.10.10],[5656
0x0040: 5d2c 5b31 5d0a ],[1].The results for a TCP unreachable are almost the same. The ICMP packet encapsulates the SYN packet in the data portion of the reply.
Here is a view of an ICMP Reply to unreachable TCP port 5657:
# tcpdump -i bridge0 -X -vvv
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:24:03.663104 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.10.58575 > 203.0.113.50.5657: Flags [S], cksum 0x092c (correct), seq 1062429515, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3566166113 ecr 0], length 0
0x0000: 4500 003c 0000 4000 4006 ce5f ac10 0a0a E..<..@.@..-....
0x0010: ac10 0a32 e4cf 1619 3f53 634b 0000 0000 ...2....?ScK....
0x0020: a002 ffff 092c 0000 0204 05b4 0103 0306 .....,..........
0x0030: 0402 080a d48f 6061 0000 0000 ......`a....
15:24:03.664168 IP (tos 0x0, ttl 64, id 37717, offset 0, flags [none], proto ICMP (1), length 88)
203.0.113.50 > 203.0.113.10: ICMP # 200 203.0.113.50 unreachable, length 68
IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.10.58575 > 203.0.113.50.5657: Flags [S], cksum 0x092c (correct), seq 1062429515, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3566166113 ecr 0], length 0
0x0000: 4500 0058 9355 0000 4001 7af3 ac10 0a32 E..X.U..@.z....2
0x0010: ac10 0a0a 03c8 68c3 0000 0000 4500 003c ......h.....E..<
0x0020: 0000 4000 4006 ce5f ac10 0a0a ac10 0a32 ..@.@.._.......2
0x0030: e4cf 1619 3f53 634b 0000 0000 a002 ffff ....?ScK........
0x0040: 092c 0000 0204 05b4 0103 0306 0402 080a .,..............
0x0050: d48f 6061 0000 0000 ..`a....3.3.11. Setdscp
The setdcsp action directs ipfw to set an IP header option on outbound packets. The action has no effect on inbound packets. The header option, formerly known as the "Type of Service" (ToS) option, now defines several classes of differentiated services (DiffServ) per several RFCs - RFC 2474, RFC 3168, and RFC 3260.
These service classes such as "Network Control", "Telephony", "Multimedia Conferencing", "Broadcast Video", "Low-latency Data", etc. all require their packets receive special handling in the network. The handling is achieved by inserting "code points" - numerical values in the packet header - that define each class.
Firewalls, routers, switches, and other network devices interpret these values and, in theory, service the packets according to their class. See this Wikipedia article on Differentiated Services: https://en.wikipedia.org/wiki/Differentiated_services.
In practice, support for service classes vary among network operators. Check the man page for a list of code points settable by ipfw.
The example below sets the DSCP value to "af31", a codepoint in the "Multimedia Streaming" class.
# ipfw add 2000 setdscp af31 udp from me to any 5656
Note that rule processing continues to the next rule.
Note also that the DSCP value takes up only a partial byte in the IP header, sharing it with two bits of ECN (Explicit Congestion Notification). The binary value for "af31" is 011010nn, where 'nn' are the two bits for ECN. If no ECN, the value resolves to 0x68 (104 decimal).
An outbound traffic example, generated by ncat -u 203.0.113.10 5656 from the firewall VM is shown as received by external1 VM:
# tcpdump -i em0 -X -vvv
tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:14:22.173252 IP (tos 0x68, ttl 64, id 30816, offset 0, flags [none], proto UDP (17), length 57)
203.0.113.50.17767 > 203.0.113.10.5656: [udp sum ok] UDP, length 29
0x0000: 4568 0039 7860 0000 4011 958f ac10 0a32 Eh.9x`..@......2
0x0010: ac10 0a0a 4567 1618 0025 0f5b 4772 6565 ....Eg...%.[Gree
0x0020: 7469 6e67 7320 6672 6f6d 2074 6865 2066 tings.from.the.f
0x0030: 6972 6577 616c 6c2e 0a irewall..Diffserve codepoints can be set on any IP based protocol or restricted to selected protocols and/or ports through suitable rules.
3.3.12. Skipto
The skipto action directs the firewall engine to pass over any rules less than the skipto parameter number. If an early rule can match a packet characteristic such as an address, port, TCP or UDP header option or similar, a skipto rule can jump to a potentially much later section of the firewall ruleset to handle to packet.
Consider the following (contrived) ruleset:
# ipfw add 100 check-state # ipfw add 1000 allow tcp from me to any established keep-state # ipfw add 2000 allow tcp from 203.0.113.10 to me 4500 setup keep-state # ipfw add 3000 allow tcp from 203.0.113.10 to me 4502 setup keep-state # ipfw add 4000 allow tcp from 203.0.113.10 to me 4504 setup keep-state # ipfw add 5000 allow tcp from 203.0.113.10 to me 4506 setup keep-state # ipfw add 6000 allow tcp from 203.0.113.10 to me 4508 setup keep-state # ipfw add 7000 allow tcp from 203.0.113.10 to me 4510 setup keep-state # ipfw add 8000 allow tcp from 203.0.113.10 to me 4512 setup keep-state # ipfw add 9000 allow tcp from 203.0.113.10 to me 4512 setup keep-state # ipfw add 10000 allow tcp from 203.0.113.10 to me 5656 setup keep-state
With the external1 VM using our standard "tcon.sh 5656" TCP connection script, the firewall VM has to traverse the entire firewall ruleset, checking each rule in turn for a match. When testing this ruleset, ensure that the firewall VM is running the appropriate service script such as tserv3.sh.
By placing a skipto action rule after the check-state action, we can have ipfw jump directly to the rule we care about:
# ipfw add 100 check-state # *ipfw add 500 skipto 10000 tcp from 203.0.113.10 to me 5656 # ipfw add 1000 allow tcp from me to any established keep-state # ipfw add 2000 allow tcp from 203.0.113.10 to me 4500 setup keep-state # ipfw add 3000 allow tcp from 203.0.113.10 to me 4502 setup keep-state # ipfw add 4000 allow tcp from 203.0.113.10 to me 4504 setup keep-state # ipfw add 5000 allow tcp from 203.0.113.10 to me 4506 setup keep-state # ipfw add 6000 allow tcp from 203.0.113.10 to me 4508 setup keep-state # ipfw add 7000 allow tcp from 203.0.113.10 to me 4510 setup keep-state # ipfw add 8000 allow tcp from 203.0.113.10 to me 4512 setup keep-state # ipfw add 9000 allow tcp from 203.0.113.10 to me 4512 setup keep-state # ipfw add 10000 allow tcp from 203.0.113.10 to me 5656 setup keep-state
You can determine if your skipto action is working by using the -a command line parameter:
# ipfw -a list 00100 0 0 check-state :default 00500 1 60 skipto 10000 tcp from 203.0.113.10 to me 5656 01000 0 0 allow tcp from me to any established keep-state :default 02000 0 0 allow tcp from 203.0.113.10 to me 4500 setup keep-state :default 03000 0 0 allow tcp from 203.0.113.10 to me 4502 setup keep-state :default 04000 0 0 allow tcp from 203.0.113.10 to me 4504 setup keep-state :default 05000 0 0 allow tcp from 203.0.113.10 to me 4506 setup keep-state :default 06000 0 0 allow tcp from 203.0.113.10 to me 4508 setup keep-state :default 07000 0 0 allow tcp from 203.0.113.10 to me 4510 setup keep-state :default 08000 0 0 allow tcp from 203.0.113.10 to me 4512 setup keep-state :default 09000 0 0 allow tcp from 203.0.113.10 to me 4514 setup keep-state :default 10000 8 474 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 65535 0 0 deny ip from any to any
General notes on skipto:
The action does not allow negative numbers or zero as a parameter.
skipto does allow numbers greater than 65536. What happens is the system accepts the input, but the result is the integer remainder of the number modulo 65536. You should ensure the value you enter is the value displayed by the command output. Numbers greater than 65534 should not be used.
You can use the skipto action to skip between sets. However, if the set containing the skipto target is disabled, processing continues with the next rule in any set that is enabled.
For example, if you have three sets - 0, 1, and 2, with a disabled set 1 containing the destination of the skipto action, processing will continue with the next rule. See the below ruleset and counters:
# ipfw -Sa list 00100 0 0 set 0 check-state :default 00101 1 60 set 0 skipto 2000 tcp from 203.0.113.10 to me 00120 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00150 0 0 set 0 allow tcp from me to any established keep-state :default 01000 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default # DISABLED 01200 0 0 set 1 allow tcp from 203.0.113.10 to me 6500 setup keep-state :default # DISABLED 01800 0 0 set 1 allow tcp from 203.0.113.10 to me 6512 setup keep-state :default # DISABLED 01900 0 0 set 1 allow tcp from 203.0.113.10 to me 6514 setup keep-state :default # DISABLED 02000 0 0 set 1 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 02700 0 0 set 2 allow tcp from 203.0.113.10 to me 7510 setup keep-state :default 02800 0 0 set 2 allow tcp from 203.0.113.10 to me 7512 setup keep-state :default 02900 0 0 set 2 allow tcp from 203.0.113.10 to me 7514 setup keep-state :default 03000 8 474 set 2 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 65535 0 0 set 31 deny ip from any to any
If you skipto a rule number that has multiple rules, the first matching rule at or after that number is executed:
# ipfw -Sa list 00100 0 0 set 0 check-state :default 00101 1 60 set 0 skipto 2000 tcp from 203.0.113.10 to me 00120 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00150 0 0 set 0 allow tcp from me to any established keep-state :default 01600 0 0 set 1 allow tcp from 203.0.113.10 to me 6508 setup keep-state :default 01700 0 0 set 1 allow tcp from 203.0.113.10 to me 6510 setup keep-state :default 02000 0 0 set 1 allow tcp from 203.0.113.10 to me 6512 setup keep-state :default 02000 0 0 set 1 allow tcp from 203.0.113.10 to me 6514 setup keep-state :default 02000 0 0 set 2 allow tcp from 203.0.113.10 to me 7500 setup keep-state :default 02000 0 0 set 2 allow tcp from 203.0.113.10 to me 7502 setup keep-state :default 02000 0 0 set 2 allow tcp from 203.0.113.10 to me 7504 setup keep-state :default 02500 0 0 set 2 allow tcp from 203.0.113.10 to me 7506 setup keep-state :default 02600 0 0 set 2 allow tcp from 203.0.113.10 to me 7508 setup keep-state :default 02700 0 0 set 2 allow tcp from 203.0.113.10 to me 7510 setup keep-state :default 02800 0 0 set 2 allow tcp from 203.0.113.10 to me 7512 setup keep-state :default 02900 0 0 set 2 allow tcp from 203.0.113.10 to me 7514 setup keep-state :default 03000 10 567 set 2 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 65535 0 0 set 31 deny ip from any to any
You can enter a rule with a skipto rule number that is lower than the current rule number, attempting to go backward in the ruleset, but this has no effect, and processing continues with the next rule:
# ipfw -Sa list 00100 0 0 set 0 check-state :default 00101 1 60 set 0 skipto 1000 tcp from 203.0.113.10 to me 00120 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00150 0 0 set 0 allow tcp from me to any established keep-state :default 00200 0 0 set 0 allow tcp from 203.0.113.10 to me 4500 setup keep-state :default 00300 0 0 set 0 allow tcp from 203.0.113.10 to me 4502 setup keep-state :default 00400 0 0 set 0 allow tcp from 203.0.113.10 to me 4504 setup keep-state :default 00500 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00600 0 0 set 0 allow tcp from 203.0.113.10 to me 4508 setup keep-state :default 00700 0 0 set 0 allow tcp from 203.0.113.10 to me 4512 setup keep-state :default 00800 0 0 set 0 allow tcp from 203.0.113.10 to me 4514 setup keep-state :default 01000 1 60 set 0 skipto 500 tcp from 203.0.113.10 to me 01100 8 475 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 65535 0 0 set 31 deny ip from any to any
You can (but probably should not) skipto a skipto rule:
00100 0 0 set 0 check-state :default 00101 1 60 set 0 skipto 1000 tcp from 203.0.113.10 to me 00120 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00150 0 0 set 0 allow tcp from me to any established keep-state :default 00200 0 0 set 0 allow tcp from 203.0.113.10 to me 4500 setup keep-state :default 00300 0 0 set 0 allow tcp from 203.0.113.10 to me 4502 setup keep-state :default 00400 0 0 set 0 allow tcp from 203.0.113.10 to me 4504 setup keep-state :default 00500 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 00600 0 0 set 0 allow tcp from 203.0.113.10 to me 4508 setup keep-state :default 00700 0 0 set 0 allow tcp from 203.0.113.10 to me 4510 setup keep-state :default 00800 0 0 set 0 allow tcp from 203.0.113.10 to me 4512 setup keep-state :default 00900 0 0 set 0 allow tcp from 203.0.113.10 to me 4514 setup keep-state :default 01000 1 60 set 0 skipto 1500 tcp from 203.0.113.10 to me 01000 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 01500 1 60 set 0 skipto 2000 tcp from 203.0.113.10 to me 01600 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 02000 1 60 set 0 skipto 2500 tcp from 203.0.113.10 to me 02100 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 02500 1 60 set 0 skipto 3000 tcp from 203.0.113.10 to me 02600 0 0 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 03000 8 475 set 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 65535 0 0 set 31 deny ip from any to any
See the section on Lists for additional caveats.
3.3.13. Divert
The divert mechanism in ipfw allows you to pull packets into user space for programmatic purposes. The divert rule snatches the packet and presents it to a divert(4) socket, a special socket type that can be created from an external program. See Appendix B for the sample program used for this book.
Divert sockets can be used as the basis for many specialized applications such as packet examination, in-flight packet modification, experimental routing techniques, etc. The program shown here simply reads from the socket and dumps the contents of the packet in hex and ASCII. It then writes the packet back into the divert socket.
To work with the divert keyword, the divert(4) packet diversion mechanism has to be compiled into the kernel or loaded at runtime:
# kldload ipdivert
This loads the divert kernel module which provides divert(4) functionality.
Once this is done, an application can open a divert(4) socket and process packets.
argv[1] = 700 after strtonum. num = [700] Opening divert on port 700 ... (see below)
In another window, run netstat(1) to see the divert socket:
# netstat -an | more Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 *.22 *.* LISTEN tcp6 0 0 *.22 *.* LISTEN udp4 0 0 *.514 *.* udp6 0 0 *.514 *.* div4 0 0 *.700 *.* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr fffff80003b83000 stream 0 0 fffff80003cac5a0 0 0 0 /var/run/devd.pipe fffff80003baf800 dgram 0 0 0 fffff80003bafc00 0 fff ...
To examine the divert operation, we first create a suitable rule:
# ipfw add 700 divert 700 ip from any to any
The syntax is a bit odd in this case. The divert keyword takes a number argument that functions as the divert object. This is similar syntax to pipes, queues, and NAT (network address translation) rules which we will see later.
A common convention, though not required, is to make the divert port the same number as the rule number in the ruleset. Whatever the rule number, when the packet is diverted and processed, and then returned to ipfw, the firewall picks up the packet at the divert rule number, plus one - that is, the next rule.
# ipfw list 00700 divert 700 ip from any to any 01000 allow udp from 203.0.113.10 to me 01100 allow udp from me to 203.0.113.10 65535 deny ip from any to any
Let’s look at the ruleset above. The packet is diverted to a divert socket, port 700 at the first rule. When it is returned from the mydivert program, it renters the ruleset at rule 1000. The ruleset allows UDP packets from and to the external1 VM.
If we set up the firewall VM host to run userv3.sh and the external1 VM host to run ucon.sh 5656, we get the following expected output from the divert program:
# ./divert 700 Opening divert on port 700 203.0.113.10:51417 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 0c 00 00 40 11 65 3e ac 10 0a 0a |E..F....@.e>....| |0010 ac 10 0a 32 c8 d9 16 18 00 32 a2 eb 55 44 50 20 |...2.....2..UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 34 5d 0a |],[4]. |
And we also see the output from userv3.sh on the firewall console:
# sh userv3.sh Starting UDP listeners on [5656],[5657],[5658] UDP communication from [203.0.113.10],[5656],[1]
However, if we shut down the userv3.sh services on the firewall VM, the incoming packets find no open port and are rejected by the firewall VM host. However, they still go through the divert socket:
# ./mydivert 700 argc = 2 argv[1] = 700 after strtonum. num = [700] Opening divert on port 700 203.0.113.10:26058 -> 203.0.113.50:5656 |0000 45 00 00 47 a9 12 00 00 40 11 65 37 ac 10 0a 0a |E..G....@.e7....| |0010 ac 10 0a 32 65 ca 16 18 00 33 28 a9 55 44 50 20 |...2e....3(.UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 31 30 5d 0a |],[10]. | 203.0.113.50:771 -> 203.0.113.10:27038 |0000 45 00 00 63 3a 61 00 00 40 01 65 37 ac 10 0a 32 |E..c:a..@.e7...2| |0010 ac 10 0a 0a 03 03 69 9e 00 00 00 00 45 00 00 47 |......i.....E..G| |0020 a9 12 00 00 40 11 65 37 ac 10 0a 0a ac 10 0a 32 |....@.e7.......2| |0030 65 ca 16 18 00 33 28 a9 55 44 50 20 70 61 63 6b |e....3(.UDP pack| |0040 65 74 20 66 72 6f 6d 20 5b 31 37 32 2e 31 36 2e |et from [172.16.| |0050 31 30 2e 31 30 5d 2c 5b 35 36 35 36 5d 2c 5b 31 |10.10],[5656],[1| |0060 30 5d 0a |0]. | mydivert: sendto: Permission denied
The last output line, "Permission denied", is because the kernel, faced with a packet and no port to send it to, instead sends an ICMP port unreachable response. The kernel sends the ICMP packet back out the divert port, but there is no ipfw rule for it to re-enter the firewall - thus "Permission denied". The packet is dropped.
To fix, we add a rule for ICMP traffic in either direction:
# ipfw list 00700 divert 700 ip from any to any 00800 allow icmp from any to any 01000 allow udp from 203.0.113.10 to me 01100 allow udp from me to 203.0.113.10 65535 deny ip from any to any
The divert operation now works as expected and the packet re-enters the firewall after rule 700. The next rule (800) permits ICMP in either direction and the packet is sent back to the source host. In the listings below, the ucon.sh script was run for 5 cycles, and after cycle #3, the firewall userv3.sh script was shut down.
The remaining two cycles result in an ICMP message being returned back to the external1 VM:
# ./mydivert 700 Opening divert on port 700 203.0.113.10:36083 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 20 00 00 40 11 65 2a ac 10 0a 0a |E..F. ..@.e*....| |0010 ac 10 0a 32 8c f3 16 18 00 32 de d4 55 44 50 20 |...2.....2..UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 31 5d 0a |],[1]. | 203.0.113.10:25662 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 21 00 00 40 11 65 29 ac 10 0a 0a |E..F.!..@.e)....| |0010 ac 10 0a 32 64 3e 16 18 00 32 07 89 55 44 50 20 |...2d>...2..UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 32 5d 0a |],[2]. | 203.0.113.10:40345 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 22 00 00 40 11 65 28 ac 10 0a 0a |E..F."..@.e(....| |0010 ac 10 0a 32 9d 99 16 18 00 32 ce 2c 55 44 50 20 |...2.....2.,UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 33 5d 0a |],[3]. | 203.0.113.10:53482 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 23 00 00 40 11 65 27 ac 10 0a 0a |E..F.x..@.e'....| |0010 ac 10 0a 32 d0 ea 16 18 00 32 9a da 55 44 50 20 |...2.....2..UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 34 5d 0a |],[4]. | 203.0.113.50:771 -> 203.0.113.10:27037 (ICMP packet) |0000 45 00 00 62 3a 68 00 00 40 01 65 27 ac 10 0a 32 |E..b:h..@.e'...2| |0010 ac 10 0a 0a 03 03 69 9d 00 00 00 00 45 00 00 46 |......i.....E..F| |0020 a9 23 00 00 40 11 65 27 ac 10 0a 0a ac 10 0a 32 |.x..@.e'.......2| |0030 d0 ea 16 18 00 32 9a da 55 44 50 20 70 61 63 6b |.....2..UDP pack| |0040 65 74 20 66 72 6f 6d 20 5b 31 37 32 2e 31 36 2e |et from [172.16.| |0050 31 30 2e 31 30 5d 2c 5b 35 36 35 36 5d 2c 5b 34 |10.10],[5656],[4| |0060 5d 0a |]. | 203.0.113.10:35359 -> 203.0.113.50:5656 |0000 45 00 00 46 a9 24 00 00 40 11 65 26 ac 10 0a 0a |E..F.$..@.e&....| |0010 ac 10 0a 32 8a 1f 16 18 00 32 e1 a4 55 44 50 20 |...2.....2..UDP | |0020 70 61 63 6b 65 74 20 66 72 6f 6d 20 5b 31 37 32 |packet from [172| |0030 2e 31 36 2e 31 30 2e 31 30 5d 2c 5b 35 36 35 36 |.16.10.10],[5656| |0040 5d 2c 5b 35 5d 0a |],[5]. | 203.0.113.50:771 -> 203.0.113.10:27037 (ICMP packet) |0000 45 00 00 62 3a 69 00 00 40 01 65 26 ac 10 0a 32 |E..b:i..@.e&...2| |0010 ac 10 0a 0a 03 03 69 9d 00 00 00 00 45 00 00 46 |......i.....E..F| |0020 a9 24 00 00 40 11 65 26 ac 10 0a 0a ac 10 0a 32 |.$..@.e&.......2| |0030 8a 1f 16 18 00 32 e1 a4 55 44 50 20 70 61 63 6b |.....2..UDP pack| |0040 65 74 20 66 72 6f 6d 20 5b 31 37 32 2e 31 36 2e |et from [172.16.| |0050 31 30 2e 31 30 5d 2c 5b 35 36 35 36 5d 2c 5b 35 |10.10],[5656],[5| |0060 5d 0a |]. | ^C
Note the two icmp packets logged by rule 800:
# ipfw show 00700 19 2466 divert 700 ip from any to any 00800 2 196 allow icmp from any to any 01000 5 350 allow udp from 203.0.113.10 to me 01100 0 0 allow udp from me to 203.0.113.10 65535 477 114991 deny ip from any to any
General notes on the divert action:
The ipdivert.ko kernel module must be loaded or compiled into the kernel to create a divert rule, and thus to use divert(4) sockets.
You cannot create a rule with a divert port of 0 or 65535. The port number must be between 1 and 65534 (inclusive). *If you create a rule with a divert port on rule 65534, the returning packet will restart firewall rule processing at the default rule, 65535, which cannot be changed.
You can create a divert rule for any protocol in /etc/protocols.
You can use the same divert port for multiple rules.
After returning from a divert rule, if the next rule is in another set, processing will continue with that rule unless the set is disabled. If disabled it will skip to the next rule in any set that is not disabled.
General notes on creating the divert socket:
Only root can create a divert socket.
Opening a divert socket on port 0 or port 65536 results in a random divert port number.
Opening a divert socket on port 65535 is permitted, but not advised.
Opening a divert port greater than 65536 or less than 0 results in a positive port number modulo 65536.
As with other sockets, you cannot open two divert sockets on the same port number. However, you can open a divert socket on a port already in use for any protocols based on IPv4 or IPv6.
3.3.14. Other Protocols
You can use any protocol in /etc/protocols in a rule.
# ipfw add 1000 allow ospf from any to me # ipfw add 2000 allow chaos from any to me etc.
3.3.15. Limit
ipfw can restrict the number of active connections with the limit option. This option allows for specifying a parameter in the rule that is regarded as a flow element, e.g. one of src-addr, src-port, dst-addr, or dst-port. In addition, the limit keyword takes a value, N, that is the maximum number of connections desired.
Concurrent connections via TCP, UDP, ICMP, or any protocol can be limited in this way.
ipfw creates a dynamic rule for each connection allowed by the rule. When the maximum number of connections is reached, additional packets are considered no longer matched and are dropped by the rule after being counted, and the search terminates.
To test, we write a simple script on the external1 VM to write UDP packets multiple times to the firewall VM:
# export NUM=1
# for i in 5656 5657 5658 5656 5657 5658 5656 5657 5658 5656
> do
> echo "hello [$NUM] to port [$i]" | ncat -u 203.0.113.50 $i
> NUM=expr $NUM + 1
> doneOn the firewall VM, ipfw starts creating dynamic rules as soon as the first matching packet is received.
Additional dynamic rules, up to the limit number are created.
Each UDP based dynamic rule has a default 10 second lifetime, controlled by the sysctl node net.inet.ip.fw.dyn_udp_lifetime.
As they expire under the limit value, space for additional connections is created.
The number of open dynamic rules at any point in time can be viewed with the sysctl node net.inet.ip.fw.dyn_count.
# ipfw -SaD list 00500 0 0 check-state :default 01000 10 531 allow udp from 203.0.113.10 to me limit src-addr 3 :default 65535 0 0 deny ip from any to any ## Dynamic rules (4 560): 01000 1 53 (8s) LIMIT udp 203.0.113.10 23755 <-> 203.0.113.50 5657 :default 01000 1 53 (8s) LIMIT udp 203.0.113.10 30144 <-> 203.0.113.50 5656 :default 01000 0 0 (4s) PARENT 3 udp 203.0.113.10 0 <-> 0.0.0.0 0 :default 01000 1 53 (8s) LIMIT udp 203.0.113.10 22722 <-> 203.0.113.50 5658 :default
# sh userv3.sh Starting UDP listeners on [5656],[5657],[5658] hello [1] to port [5656] hello [2] to port [5657] hello [3] to port [5658] ^C
3.3.16. Call and Return
The call and return actions allow you to change ruleset processing by jumping to a rule number elsewhere in the ruleset. If the rules at that location contain a return action, processing will jump back to the statement immediately after the original call statement. In practice, this acts like a program function call, or as the man page notes, like an assembly language subroutine.
Creating a ruleset with call and return actions:
# # ipfw add 500 check-state 00500 check-state :default # # ipfw add 1000 call 20000 udp from 203.0.113.10 to me 5656 01000 call 20000 udp from 203.0.113.10 to me 5656 # # ipfw add 1100 count udp from 203.0.113.10 to me 01100 count udp from 203.0.113.10 to me # # ipfw add 1200 allow udp from 172.168.10.10 to me 5656 01200 allow udp from 203.0.113.10 to me 5656 # # ipfw add 20000 count udp from 203.0.113.10 to me 20000 count udp from 203.0.113.10 to me # # ipfw add 21000 return via any 21000 return # # ipfw -a list 00500 0 0 check-state :default 01000 0 0 call 20000 udp from 203.0.113.10 to me 5656 01100 0 0 count udp from 203.0.113.10 to me 01200 0 0 allow udp from 203.0.113.10 to me 5656 20000 0 0 count udp from 203.0.113.10 to me 21000 0 0 return 65535 0 0 deny ip from any to any
As noted in the man page, you have to put some extra syntactic sugar on the return statement:
# ipfw add 21000 return via any
In this example we have used several count statements to try to trace ruleset processing.
To test, we have the firewall VM host startup 3 listeners with userv3.sh. After the external1 VM uses ucon.sh 5656 to send a udp packet, the count list looks like this:
# ipfw -a list 00500 0 0 check-state :default 01000 1 70 call 20000 udp from 203.0.113.10 to me 5656 01100 1 70 count udp from 203.0.113.10 to me 01200 1 70 allow udp from 203.0.113.10 to me 5656 20000 1 70 count udp from 203.0.113.10 to me 21000 1 70 return 65535 0 0 deny ip from any to any
The packet was matched at rule 1000 where it encountered a call action to jump to rule 20000 where it was then matched at 20000. The next rule was a matched return action at rule 21000. Returning to the rule after the call action, it matched a count action at 1100, then matched an allow action at 1200 where it was sent through to the application layer and was received by userv3.sh.
If we remove the return action at rule 21000, the counts look much different:
# ipfw -a list 00500 0 0 check-state :default 01000 1 70 call 20000 udp from 203.0.113.10 to me 5656 01100 0 0 count udp from 203.0.113.10 to me 01200 0 0 allow udp from 203.0.113.10 to me 5656 20000 1 70 count udp from 203.0.113.10 to me 65535 1 70 deny ip from any to any
Without a return at rule 21000, the only rule left is the default deny rule, and there is nothing received by userv3.sh.
Because you can jump both forward and backward with the call action, it is possible to create an endless loop. |
In this example we create an endless loop with in incorrect call rule:
# # ipfw -a list 01000 0 0 count udp from 203.0.113.10 to me 05000 0 0 check-state :default 06000 0 0 call 1000 udp from 203.0.113.10 to me 5656 07000 0 0 count udp from 203.0.113.10 to me 65535 0 0 deny ip from any to any
In this example we are missing a return action between rule 1000, the target of the call action at rule 6000. This creates a loop. ipfw eventually figures out that a loop exists, and breaks out at the next call action with the diagnostic:
# ipfw: call stack error, go to next rule
but it does not currently tell you where the missing return action is or which rule it went to next.
You can pick up a clue by watching the rule counts. Below is the rule count for this errant ruleset after just one packet was received from the external1 VM:
# ipfw -a list 01000 17 1207 count udp from 203.0.113.10 to me 05000 0 0 check-state :default 06000 16 1136 call 1000 udp from 203.0.113.10 to me 5656 07000 1 71 count udp from 203.0.113.10 to me 65535 1 71 deny ip from any to any
This shows that ipfw went around this call loop 16 times before throwing an error.
For TCP connections, call and return operate almost the same. Below is a ruleset with TCP instead of UDP for the desired protocol and including the required setup and keep-state keywords:
# ipfw -a list 00500 0 0 check-state :default 01000 0 0 call 20000 tcp from 203.0.113.10 to me 5656 01100 0 0 count tcp from 203.0.113.10 to me 02000 0 0 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 20000 0 0 count tcp from 203.0.113.10 to me 21000 0 0 return 65535 0 0 deny ip from any to any
After the connection is successfully made from external1, the observed counts are:
# ipfw -a list 00500 0 0 check-state :default 01000 1 60 call 20000 tcp from 203.0.113.10 to me 5656 01100 1 60 count tcp from 203.0.113.10 to me 02000 8 479 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 20000 1 60 count tcp from 203.0.113.10 to me 21000 1 60 return 65535 0 0 deny ip from any to any
The difference in rule counts is due to dynamic rules created by the setup and keep-state keywords on rule 2000.
If we manually run the connection we can observe the counts with the -d command line parameter.
The numbers below are from just the initial 3-way handshake:
# ipfw -ad list 00500 0 0 check-state :default 01000 1 60 call 20000 tcp from 203.0.113.10 to me 5656 01100 1 60 count tcp from 203.0.113.10 to me 02000 5 276 allow tcp from 203.0.113.10 to me 5656 setup keep-state :default 20000 1 60 count tcp from 203.0.113.10 to me 21000 1 60 return 65535 0 0 deny ip from any to any ## Dynamic rules (1 152): 02000 5 276 (296s) STATE tcp 203.0.113.10 19179 <-> 203.0.113.50 5656 :default
The above numbers indicate that the 3-way handshake occurred during the dynamic rule setup.
General notes on call and return:
You can call out to an address either before or after the current rule.
You must have a return for every call.
You can nest call / return pairs up to 16 levels deep. If you add another call rule, ipfw will throw the "call stack error, go to next rule" error and continue with the next rule.
Similar to skip-to, you cannot call to rule 0, 65535, or 65536. If you call to a value greater then 65536, the rule is accepted and the call target rule number is the requested value modulo 65536.
Similar also to sets, if a call is made to a target rule in a set that is disabled, the call will land on the next non-disabled rule in any set and processing continues from there.
If a return is encountered when no call has been made, the return rule is ignored and processing continues with the next rule.
3.3.17. Using uid and gid in rules
An interesting capability of ipfw is its ability to match packets on Unix uid and gid values. Network packets themselves have no inherent ownership, so where does this capability come from? Answer - it comes from the applications that are the source and destination of those packets.
Let’s look at a simple example. First, let’s examine the actual syntax that allows this matching capability.
Locate or add a user (here 'quarven') if needed: # grep quarven /etc/passwd quarven:*:1002:1002:Quarven:/home/quarven:/bin/sh Now use 'quarven' in an ipfw rule: # ipfw add 700 allow udp from 203.0.113.10 to me uid quarven 00700 allow udp from 203.0.113.10 to me uid quarven # # ipfw show 00700 0 0 allow udp from 203.0.113.10 to me uid quarven 65535 0 0 deny ip from any to any
Next, login as user 'quarven' and run the script /root/bin/userv.sh 5656. Then switch to the external1 VM and run /root/bin/ucon.sh 5656 for just one cycle.
The results are shown below.
quarven@firewall:~ $ /bin/sh /root/bin/userv.sh 5656 PORT1 = [5656] Starting UDP listener on [203.0.113.50],[5656] UDP packet from [203.0.113.10],[5656],[1] ^Cquarven@firewall:~ $
This works because the instance of userv.sh is run under the uid quarven as shown below
Show the user information for the userv.sh instance: USER PID PPID PGID SID JOBC STAT TT TIME COMMAND quarven 51689 27186 51689 27186 1 I v0 0:00.08 -sh (sh) quarven 54015 51689 54015 27186 1 I+ v0 0:00.02 /bin/sh /root/bin/userv.sh 5656 quarven 56230 54015 54015 27186 1 I+ v0 0:00.02 nc -l -k -u 203.0.113.50 5656
ipfw has matched an incoming network packet to a program owned by a userid. If the rule is changed to another userid, even root, the match will not succeed and the packet will be picked up by the default rule.
It is sometimes necessary to shut down all traffic from a certain user. This capability is ideal for that purpose:
# ipfw add 200 deny ip from any to any uid quarven
Notes on using uid and gid:
The gid keyword works in an identical fashion to uid described above.
If using a name as the uid or gid, the name must exist in the indicated system file.
Ranges and lists are not allowed. You can’t use "… uid tom,dick,harry" or "… uid 1000-1002".
Outbound traffic works in a similar way, just reversing the source and destination.
Denied traffic will generally have an indication of "<application>: sendto: Permission denied"
ICMP traffic cannot be reliably filtered using uid/gid. This is a known limitation.
As noted in ipfw(8) some contexts, such as initial incoming SYN packets may have no uid/gid associated with them.
Programs using setuid(2) system calls may not behave as expected, though it may be possible to set the uid/gid to the effective id if it can be determined.
Using uid/gid keywords for matching is resource intensive and should be used sparingly if at all.
3.4. Lookup Tables
Lookup tables are a versatile feature of many firewall systems, including ipfw. A lookup table is a virtual container that holds tuples of elements, one of which is a key that functions as a fast lookup feature. Using a key provided by a rule, ipfw can quickly determine if the element is in the table. If it is, that portion of the rule is matched.
Lookup tables are a powerful feature of ipfw and useful in many situations.
ipfw provides four types of lookup tables:
Address tables (addr) - These tables hold addresses that ipfw can rapidly find with an address as a key. If the address is matched the lookup is considered matched and used by the associated rule. This table type takes an additional keyword 'valtype' that can be used to specify IPv4 addresses or IPv6 addresses.
Interface tables (iface) - These tables hold interface names. Each entry is the name of an interface. Note that wildcards such as
em*are not supported.Number tables (number) - These tables are used for protocols, ports, uids/gids, or jail ids. Entries are 32 bit unsigned integers. Ranges (e.g. 1234-5678) are not supported.
Flow tables (flow) - These tables contain flow type suboptions that are used in looking up existing traffic flows.
3.4.1. Creating Lookup Tables
All tables must be created before they can be referenced by a rule. Note that the commands to manage tables do not have line numbers - they are independent shell commands that exist outside the ruleset.
# ipfw table foo create
By default, the ipfw table create command creates tables of type addr. Table names share the same namespace and so must be unique even among tables of different types.
To specify other types, add the type keyword:
# ipfw table bar create type iface # # ipfw table bar create type flow # # ipfw table bop create type number
Previously, when creating a table of type number, a bug existed that required an algorithm option such as |
To see all tables, use the list subcommand to show the table and any contents. Shown below, all four table types are created and one entry is added to each table:
# ipfw table all list # # ipfw table foo create type addr # ipfw table foo add 192.168.1.100 added: 192.168.1.100/32 0 # # ipfw table bar create type iface # ipfw table bar add em0 added: em0 0 # # ipfw table baz create type number # ipfw table baz add 9999 added: 9999 0 # # ipfw table bop create type flow # ipfw table bop add 203.0.113.10,192.168.1.100 ignored: 0 ipfw: Adding record failed: Invalid argument # # ipfw table bop destroy # # ipfw table bop create type flow:src-ip,dst-ip # ipfw table bop add 203.0.113.10,192.168.1.100 added: 203.0.113.10,192.168.1.100 0 # # ipfw table all list --- table(bar), set(0) --- em0 0 --- table(baz), set(0) --- 9999 0 --- table(bop), set(0) --- 203.0.113.10,192.168.1.100 0 --- table(foo), set(0) --- 192.168.1.100/32 0 #
Note the error on the flow table above. Flow tables take an explicit flow specification (discussed below) when they are created. If you try to add an entry to a flow table that does not match the flow specification, ipfw throws an error.
Note that the output list of tables is ordered by table name, not table type.
Tables can be removed one at a time with the destroy subcommand, provided the table is not used in any rule:
# ipfw table bar destroy
or removed all at once by specifying the special name all:
# ipfw table all destroy
Note that there is no confirmation with this command as there is with the flush command, so make sure that is really what you want to do. |
3.4.2. Using Tables in Rules
First, on the firewall VM, create a table called goodhosts, and populate it with the addresses of hosts allowed to accept traffic:
# ipfw table goodhosts create type addr # ipfw table goodhosts add 203.0.113.10/32 added: 203.0.113.10/32
Then create a rule that uses the table.
Note the single quotes around the table(goodhosts) entry to placate the shell:
# ipfw add 2000 allow udp from 'table(goodhosts)' to me 02000 allow udp from table(goodhosts) to me
To test, start up userv3.sh on the firewall VM:
# sh userv3.sh Starting UDP listeners on [5656],[5657],[5658]
And on the external1 VM, start up ucon.sh:
# sh ucon.sh 5656 UDP communicationing [203.0.113.50],[5656],[1] ncat [2] ready. Enter a valid PORTNUM: 5657 UDP communicationing [203.0.113.50],[5657],[2] ncat [3] ready. Enter a valid PORTNUM: 5658 UDP communicationing [203.0.113.50],[5658],[3] ncat [4] ready. Enter a valid PORTNUM: ^C
If you remove the entry for 203.0.113.10/32 from the goodhosts table, the communications fail to reach the userv3.sh services listening on the firewall VM:
# ipfw table goodhosts delete 203.0.113.10 deleted: 203.0.113.10/32 0
Retrying the ucon.sh communications above will not be successful.
The next example uses the lookup keyword on table goodports.
Start from scratch by unloading and reloading ipfw from the kernel.
This ensures we don’t leave any cruft lying around:
# kldunload ipfw # kldload ipfw
Next, create a table of type number and populate it with ports 5656 and 5657, but not 5658:
# ipfw table goodports create type number # # ipfw table goodports add 5656 added: 5656 0 # # ipfw table goodports add 5657 added: 5657 0 # # ipfw table goodports list 5656 0 5657 0 #
The lookup keyword searches for its parameter in a specified table. If the entry is found, the match succeeds and tablearg is set to the value returned from the table. Create a rule that uses this table with the lookup keyword:
# ipfw add 500 allow udp from 203.0.113.10 to me lookup dst-port goodports 00500 allow udp from 203.0.113.10 to me dst-ip lookup dst-port goodports
On the external1 VM, start up the ucon.sh script, and enter ports 5656, 5657, 5658, 5659, and 5656:
# sh ucon.sh 5656 UDP communicationing [203.0.113.50],[5656],[1] ncat [2] ready. Enter a valid PORTNUM: 5657 UDP communicationing [203.0.113.50],[5657],[2] ncat [3] ready. Enter a valid PORTNUM: 5658 UDP communicationing [203.0.113.50],[5658],[3] ncat [4] ready. Enter a valid PORTNUM: 5659 UDP communicationing [203.0.113.50],[5659],[4] ncat [5] ready. Enter a valid PORTNUM: 5656 UDP communicationing [203.0.113.50],[5656],[5] ncat [6] ready. Enter a valid PORTNUM: ^C
The results on the firewall host are:
# sh userv3.sh Starting UDP listeners on [5656],[5657],[5658] UDP communication from [203.0.113.10],[5656],[1] UDP communication from [203.0.113.10],[5657],[2] UDP communication from [203.0.113.10],[5656],[5] ^C
The two ports 5658 and 5659 were not in the table and thus not matched by rule 500.
It would be tempting to combine the previous two examples into something like:
# ipfw add 1000 allow udp from 'table(goodhosts)' to me dst-port 'table(goodports)' ipfw: invalid destination port table(goodports)
but ipfw does not allow the use of more than one table in a rule.
However, instead of a second table keyword, you could use the lookup keyword for the port:
# ipfw add 500 allow udp from 'table(goodhosts)' to me lookup dst-port goodports 00500 allow udp from table(goodhosts) to me dst-ip lookup dst-port goodports
While does work, the better solution is to use the a flow table:
# ipfw table goodflow create type flow:src-ip,dst-port # # ipfw table goodflow add 203.0.113.10,5656 added: 203.0.113.10,5656 0 # # ipfw table goodflow add 203.0.113.10,5657 added: 203.0.113.10,5657 0 # # ipfw add 500 allow udp from any to me flow 'table(goodflow)' 00500 allow udp from any to me flow table(goodflow)
This gives you much more granular control of exactly what host and what port you want to match together.
Using the valtype keyword for addr tables you can have separate tables for IPv4 and IPv6:
# Create the translation tables. # ipfw table T46 create type addr valtype ipv6 # ipfw table T64 create type addr valtype ipv4
3.4.2.1. Understanding the word tablearg
A tablearg is a computed value that is the result of a table lookup.
In the manual page ipfw(8), the word tablearg is used in two ways:
The term tablearg is used to show where in the rule the computed lookup will be applied. In this usage tablearg is just a placeholder.
In a rule, the actual word tablearg becomes the computed value taken from the last table lookup. This is typically described as the tablearg keyword in the documentation.
We have already seen the first usage when using the lookup keyword example above.
Essentially
# ipfw add 500 allow udp from 203.0.113.10 to me lookup dst-port goodports
becomes
# ipfw add 500 allow udp from 203.0.113.10 to me 5656
Below is an example of the second usage using the skipto action and the actual word tablearg in a rule.
Restart ipfw:
# kldunload ipfw # kldload ipfw
Next, create a number table holding our favorite ports 5656, 5657, 5658.
# ipfw table create goodports # ipfw table goodports add 5656 # ipfw table goodports add 5657 # ipfw table goodports add 5658
Then add rules:
# ipfw add 400 skipto tablearg udp from 203.0.113.10 to me lookup dst-port goodports # ipfw add 1000 allow icmp from me to any # ipfw add 4000 count ip from any to any # ipfw add 5656 allow udp from 203.0.113.10 to me # ipfw add 6000 count ip from any to any
This ruleset uses the skipto action to jump immediately to the rule number if the number is found in the table goodports.
In effect, the goodports table acts as a dispatch table for rule processing.
If a packet comes in with port 5656, 5657, or 5658, the skipto action is applied and processing begins with the rule number found in the table.
To demonstrate, start up the userv3.sh scrip on the firewall, and run the ucon.sh script on exteranal1.
Below is a transcript showing what happened.
# ipfw table goodports list 5656 0 5657 0 5658 0 # # ipfw show 00400 0 0 skipto tablearg udp from 203.0.113.10 to me dst-ip lookup dst-port goodports 01000 0 0 allow icmp from me to any 04000 0 0 count ip from any to any 05656 0 0 allow udp from 203.0.113.10 to me 06000 0 0 count ip from any to any 65535 0 0 deny ip from any to any #
Now, start sh userv3.sh on the firewall console, listening for UDP packets on the three ports and run sh ucon.sh 5656 on the external1 VM. This will send one packet.
Show counters:
# ipfw show 00400 1 71 skipto tablearg udp from 203.0.113.10 to me dst-ip lookup dst-port goodports 01000 0 0 allow icmp from me to any 04000 1 71 count ip from any to any 05656 1 71 allow udp from 203.0.113.10 to me 06000 0 0 count ip from any to any 65535 0 0 deny ip from any to any
The counters show the skipto rule was matched, and ipfw walked the ruleset up to rule 5656.
There was a match on 5656 the destination rule, the userv3.sh received a UDP packet.
Next, send a UDP packet with an outlier port, 9000.
The counters now reveal:
# ipfw show 00400 1 71 skipto tablearg udp from 203.0.113.10 to me dst-ip lookup dst-port goodports 01000 1 99 allow icmp from me to any 04000 2 142 count ip from any to any 05656 2 142 allow udp from 203.0.113.10 to me 06000 0 0 count ip from any to any 65535 0 0 deny ip from any to any
The packet was not matched at rule 400.
ipfw then walked the ruleset and found a matching rule at 4000 and next at 5656 where the packet was allowed through.
However, rule 1000 shows that since there was no service listening on port 9000, the firewall sent an ICMP destination port unreachable packet back to the originator (external1).
From the ipfw(8) man page - The tablearg argument can be used with the following actions: nat, pipe, queue, divert, tee, netgraph, ngtee, fwd, skipto, setfib; wth action parameters: tag, untag; and with rule options: limit, tagged. |
3.4.2.2. More on flow tables
A flow table maintains a list of flows. A flow is a designation given to traffic between two endpoints. The designation can be any of:
src-ip
src-port
proto
dst-ip
dst-port
or any combination that makes sense in source → destination order. Examples include:
# ipfw table zoo create type flow:src-ip // Creates a flow based on soure IP address # ipfw table zar create type flow:proto // Creates a flow based on just a protocol. * # *ipfw table zaz create type flow:dst-ip // Creates a flow based on just the desitination IP # ipfw table zop create type flow:dst-port // Creates a flow based on the destination port
or with extra specificity:
# ipfw table zip create type flow:src-ip,proto,dst-ip,dst-port // Flow based on all four # ipfw table zim create type flow:src-ip,proto // Just source IP and protocol # ipfw table zam create type flow:src-ip,proto,dst-ip // Source IP, dest. IP and protocol # ipfw table zap create type flow:src-ip,dst-ip // Just IP address endpoints
Once the table is created for a flow, entries can be placed in the table provided they match the table flow specification.
Match these additions to the tables created above:
# ipfw table zim add 192.168.200.30,tcp
would succeed, but
# ipfw table zip add 192.168.30.20,172.20.15.20
would fail because the flow specification for table zip is "src-ip,proto,dst-ip,dst-port" and neither the protocol nor the destination port was given.
A correct flow specification for table zip would be something like:
# ipfw table zip add 192.168.30.20,tcp,172.20.15.20,80
flow tables allow for precise definition of traffic between a source and a destination. Once in the table, rules can be applied to commands allow, deny, divert, queue, etc. for modifying traffic flow.
General notes on all tables:
Table names can be numeric or alphanumeric and can include only hyphen (-), underscore (_), and period (.) as special characters.
Maximum table name length is 63 characters.
Tables names must be unique within a set. Tables can have the same name across different sets, however any rules for tables in sets other than set 0, must include the set number. The sysctl variable net.inet.ip.fw.tables_sets controls this behavior.
The maximum number of tables across all sets is 65535. Practically however, the number is controlled by the sysctl variable 'net.inet.ip.fw.tables_max'. The default is 128.
If a table is in use in a rule, it cannot be destroyed. The rule must be removed first, then the table can be destroyed. However, a table can be flushed at anytime.
All table types survive a flush action, and table contents are not affected.
Make table names as descriptive as possible to avoid confusion when used in rules. The names we are using here (foo, bar, zip, zap, etc.) are just examples.
As noted in the man page, if two tables are used in a rule, the result of the second (destination) is used. Therefore avoid using two tables in a rule, or try using the lookup keyword instead.
Notes on specific table types:
Address tables (addr)
Address tables (addr) support IPv4 and IPv6 address types, and varying mask lengths appropriate for each address type. The default mask length for IPv4 is 32 bits (/32) and the default for IPv6 is 128 bits (/128).
Table lookups will return the most specific entry, so 203.0.113.20/32 is preferred over 203.0.113.0/24.
Interface tables (iface)
Interface tables store interface names as alphanumeric text. The text does not actually have to match a current valid interface.
Special characters in interface names can include any from the set
[-+_?,.!~@#%^&*()=;:/,<>{}|]Note however, that the shell may recognize some of these characters when adding and during lookup, thus interfering with the table operation, and so special characters in interface names should be avoided.
The maximum key length is 15 characters.
There is no support for interface ranges i.e.
em0-4, even though an interface name "em0-4" can be entered.
Number tables (number)
Number tables support unsigned 32-bit integer types.
Entries can be positive or negative. Negative entries perform unsigned ones complement arithmetic, and positive numbers roll over from 4294967295 to 0. So don’t do that.
Any shell element that evaluates to a number can be used: shell variables that resolve to a number (
$MAILCHECK,$PPID), backtick operations such asexpr 5 + $UID,id -u,date "+%s", and special variables like$RANDOMin bash.As with other table types, ranges are not supported.
Flow tables (flow)
Flow tables describe network traffic based on the desired attributes. The best matches include as much detail as possible:
src-ip,proto,dst-ip,port. Including less than that may make it difficult to add an element in the table:ipfw table foo create flow:dst-port // Table based on just the destination port
ipfw table foo add telnet // Fails to add!
ipfw table bar create flow:dst-ip,dst-port
ipfw table bar add 203.0.113.10,5656
3.5. Stream Control Transport Protocol (SCTP)
A readable introduction to SCTP is found in Wikipedia - https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#RFCs and ever more detail is found in the accompanying RFCs.
3.5.1. SCTP Versions
As of FreeBSD 14.1, there are three different versions of SCTP - a "native" version, a "separate portable" version, and a "userland" version.
FreeBSD ships with the "native" version of the protocol, described in sctp(4). (This is actually the reference implementation of SCTP initially developed on FreeBSD 7.) This version requires loading the kernel module sctp.ko before use. Native SCTP uses the SCTP library on FreeBSD which provides access to the various functions found in <netinet/sctp.h>. You can use the features described in sctp(4) to develop SCTP applications based on the native protocol.
Then, there are a couple of notable packages regarding SCTP:
libusrsctp “Portable SCTP userland stack” – This is a non-kernel implementation of the protocol using “usrsctp_*” functions. It also provides UDP encapsulation as shown in the Wireshark image below:
sctplib “User-space implementation of the SCTP protocol RFC 4960” – This package is a re-implementation of the native SCTP code that can function as a replacement for the default installed code. It does not require the sctp.ko kernel module.
As noted in the BUGS section of sctp(4), the sctp.ko kernel module cannot be unloaded. You will have to restart FreeBSD to remove the module from the kernel. |
Interestingly, the reference implementation for SCTP that was developed on FreeBSD 7 and has been archived on Github here. ipfw played a role in its development and the vestiges of that development remain in the ipfw code base and in the collection of sysctls that support it.
We will study the native SCTP protocol usage and the encapsulated usage with ipfw.
3.5.2. SCTP Protocol Operation
The image below shows a Wireshark view of a typical native SCTP association (connection):
Figure 14. Wireshark View of Native SCTP
Notice how there are two different interfaces involved with this transfer - 127.0.0.1, and 192.168.1.78. In fact, SCTP supports association setup and data exchange with multiple interfaces to a remote node. The RFC’s explain this capability in detail.
Also in this image, you can see the basic 4-way handshake - INIT, INIT_ACK, COOKIE_ECHO, and COOKIE_ACK, the HEARTBEAT, DATA, SACK (Stream Acknowledgement) messages, and the shutdown sequence of SHUTDOWN, SHUTDOWN_ACK, and SHUTDOWN_COMPLETE.
Below is a view of netstat -an showing the display of a separate SCTP section containing all current associations:
% netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 *.22 *.* LISTEN
tcp6 0 0 *.22 *.* LISTEN
tcp4 0 0 127.0.0.1.631 *.* LISTEN
tcp6 0 0 ::1.631 *.* LISTEN
udp4 0 0 127.0.0.1.123 *.*
udp6 0 0 fe80::1%lo0.123 *.*
udp6 0 0 ::1.123 *.*
udp4 0 0 192.168.1.78.123 *.*
udp6 0 0 2600:1700:3901:4.123 *.*
udp6 0 0 fe80::3e97:eff:f.123 *.*
udp4 0 0 *.123 *.*
udp6 0 0 *.123 *.*
Active SCTP associations (including servers)
Proto Type Local Address Foreign Address (state)
sctp4 1to1 127.0.0.1.5000 127.0.0.1.42227 ESTABLISHED
192.168.1.78.5000 192.168.1.78.42227
sctp4 1to1 127.0.0.1.42227 127.0.0.1.5000 ESTABLISHED
192.168.1.78.42227 192.168.1.78.5000
sctp4 1to1 127.0.0.1.5000 LISTEN
192.168.1.78.5000
Active UNIX domain sockets
Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr
fffff8015575c000 stream 0 0 0 fffff80155758c00 0 0
. . .Internally, the SCTP data packets look different from TCP and/or UDP packets. However, like those two, SCTP has a protocol number, source address, destination address, and port numbers, so using ipfw with SCTP will be fairly straightforward.
Here’s a look at a typical packet:
Figure 15. Internal View of the SCTP INIT Data Packet
To study SCTP’s behavior with ipfw we will use the client/server echo program, shown above. We will also look at a streaming application based on the chargen small server of yore.
3.5.3. Using the TSCTP Testing Tool on FreeBSD
A handy testing tool is the FreeBSD package tsctp.
Install the package on the external1 and internal VMs.
(You will have to configure the VMs to access the Internet to install the package.
Refer to the Quick Start section for details.)
# pkg install tsctp
This tool uses the native SCTP protocol version. To use, we first load the sctp.ko kernel module:
# *kldload stcp.ko
Running the program is discussed below.
To get started, initialize the bridge and tap devices to the same architecture as "Simple NAT", shown in the Figure below.
Figure 16. Setting Up to Test Native SCTP
% sudo /bin/sh mkbr.sh reset bridge0 tap1 tap4 bridge1 tap0 tap5 Then start the VMs we need: % /bin/sh runvm.sh firewall external1 internal
Then follow these steps:
Apply the addressing shown in the above figure and ensure connectivity with adjacent interfaces, and with the opposite side interfaces (10.10.10.20 from the external1 VM and 203.0.113.10 from the internal VM).
The default route for the external1 VM should point to 203.0.113.50, and for the internal VM it should point to 10.10.10.50.
Load the sctp.ko kernel module on the external1 and the internal VMs. This enables communication via native SCTP on these VMs.
On the internal VM, run in server mode: tsctp -L 127.0.0.1 -L 10.10.10.20 -p 1234 and on the external1 VM, run in client mode: tsctp -L 203.0.113.10 -p 1234 -n 10 -l 1000 192.168.1.128
Use tscp --help to get a list of the options and meanings.
The client program sends 10 messages (-n 10) to the server.
Simple statistics are shown once the program terminates.
With the above procedure, we have established SCTP communications across the firewall. The firewall VM does not need to load the sctp.ko module. To the firewall VM, this is simply normal IP traffic.
Restart the client with -n 0, for unlimited messages.
While normal SCTP traffic is established from client to server, load ipfw on the firewall VM.
Traffic is immediately halted. Eventually, the client will recognize it has been disconnected. This may take a couple of minutes. The disconnection is shown in the image below:
Figure 17. IPFW Firewall Disrupts SCTP Traffic
Creating suitable rules for SCTP traffic is quite similar to other rules we have performed before. You can use the "two rule" version:
Figure 18. IPFW Traffic Flow With Two Rules
One would think the "one-rule" version using setup and keep-state keywords would work, but it does not. You have to use the "two-rule" version:
root@firewall:~ # ipfw show 00100 0 0 check-state :default 01000 40 4152 allow sctp from 203.0.113.10 to 10.10.10.20 02000 48 4424 allow sctp from 10.10.10.20 to 203.0.113.10 65535 0 0 deny ip from any to any root@firewall:~ #
Also, keep in mind that SCTP is typically used where there are multiple interfaces per association. This example has used only one, but the principles are the same.
3.5.4. Downloading and Building usrsctp Programs
To test SCTP encapsulation with UDP, we must download and build the usrsctp kit. Follow this procedure to download and build the programs for the usrsctp kit. On the external1 VM and the internal VM, the procedure is the same.
For this procedure, you will have to reconfigure the external1 and internal VMs to access the Internet as described in Quick Start.
# pkg install git # pkg install cmake # mkdir /root/src # cd /root/src # git clone https://github.com/sctplab/usrsctp.git # mkdir tmp # cd tmp # cmake ../usrsctp # cmake --build . Once finished, the test programs are located in /root/src/tmp/programs.
Now reconfigure external1 and internal VMs to the architecture in Figure 10.
3.5.5. Encapsulated Echo Server and Client with IPFW
Figure 13 shows encapsulated usage of SCTP with the chargen_server_upcall program running on the internal VM and the client_upcall program running on the external1 VM. The output is similar to that of the chargen small server program.
Figure 19. SCTP Traffic Encapsulated in UDP Datagrams
All the data exchanged between the two systems was encapsulated in UDP datagrams as shown in Figure 14.
Figure 20. Wireshark View of UDP Encapsulation of SCTP
Any ipfw rules for this traffic only have to be concerned with UDP, not SCTP.
Chapter 4. IPFW Dummynet and Traffic Shaping
FreeBSD’s dummynet is not a network for dummies. It is a sophisticated network traffic shaping tool for bandwidth usage and scheduling algorithms. In this use of ipfw, the focus is not on ruleset development, although rules are still used to select traffic to pass to dummynet objects. Instead, the focus is on setting up a system to shape traffic flows.
Imagine if you had the ability to model traffic flow across the wild Internet. dummynet allows you to model scheduling, queuing, and similar tasks similar to the real-world Internet.
dummynet works with three main types of objects - a pipe, a queue, and a sched (short for scheduler) which also happen to be the three keywords to use with dummynet.
A pipe (not to be confused with a Unix pipe(2)) is a model of a network link with a configurable bandwidth, and propagation delay.
A queue is an abstraction used to implement packet scheduleing using one of several different scheduling algorithms. Packets sent to a queue are first grouped into flows according to a mask on a 5-tuple (protocol, source address, source port, destination address, destination port) specification. Flows are then passed to the scheduler associated with the queue, and each flow uses scheduling parameters (weight, bandwidth, etc.) as configured in the queue itself. A sched (scheduler) in turn is connected to a pipe (an emulated link) and arbitrates the link’s bandwidth among backlogged flows according to weights and to the features of the scheduling algorighm in use.
Network performance testing is a complex subject that can encompass many variables across many different testing strategies. For our purposes, we want to understand the basics behind dummynet, so we will not be diving into the deepest levels of network performance testing - only enough to understand how to use dummynet. Also, for these tests, we will restrict our methodologies to using IP and TCP exclusively.
4.1. Measuring Default Throughput
The idea behind dummynet is that it lets you model and/or shape network speeds, available bandwidth, and scheduling algorithms. But first you have to know what your current transfer speeds are for the current environment (QEMU virtual machines over a FreeBSD bridge). To find out, we take a short detour to learn iperf3, the network bandwidth testing tool to perform simple transfer and bitrate calculations.
With iperf3, you can determine the effective throughput of data transfer for your system. Often called "goodput", this is the basic speed the user will see for transferring data across the network - the value that is unencumbered by protocol type and overhead.
To use iperf3, ensure that the software is installed on both the firewall VM system, and the external1 VM (and external2 and external3), and that ipfw on the firewall host is disabled (# kldunload ipfw).
The basic operation of iperf3 is as a client-server architecture, so on the external1 VM system, start the iperf3 software in server mode:
# iperf3 -s <--- run iperf3 in server mode ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- . . .
Then, on the firewall VM, run the client:
# iperf3 -c 203.0.113.10 <--- connect to external1 server and send test data Connecting to host 203.0.113.10, port 5201 [ 5] local 203.0.113.50 port 19359 connected to 203.0.113.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.03 sec 12.5 MBytes 102 Mbits/sec 0 1.07 MBytes [ 5] 1.03-2.09 sec 13.8 MBytes 108 Mbits/sec 0 1.07 MBytes [ 5] 2.09-3.07 sec 12.5 MBytes 107 Mbits/sec 0 1.07 MBytes [ 5] 3.07-4.09 sec 12.5 MBytes 103 Mbits/sec 0 1.07 MBytes [ 5] 4.09-5.08 sec 12.5 MBytes 106 Mbits/sec 0 1.07 MBytes [ 5] 5.08-6.09 sec 12.5 MBytes 105 Mbits/sec 0 1.07 MBytes [ 5] 6.09-7.07 sec 12.5 MBytes 107 Mbits/sec 0 1.07 MBytes [ 5] 7.07-8.05 sec 12.5 MBytes 107 Mbits/sec 0 1.07 MBytes [ 5] 8.05-9.04 sec 12.5 MBytes 106 Mbits/sec 0 1.07 MBytes [ 5] 9.04-10.02 sec 12.5 MBytes 107 Mbits/sec 0 1.07 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.02 sec 126 MBytes 106 Mbits/sec 0 sender [ 5] 0.00-10.02 sec 126 MBytes 106 Mbits/sec receiver iperf Done. #
A key test for measuring throughput is to send a file of data and measure the transfer speed. To create the file, we can use the jot(1) program:
# jot -r -s "" 10000000 > A.bin
This command creates a 10MB file of random ASCII digits. (Note that this takes roughly 30 seconds to a minute to create the file on a QEMU virtual machine.)
To transfer the file to the server on the firewall VM we can use this command:
# iperf3 -F A.bin -c 203.0.113.10
Connecting to host 203.0.113.10, port 5201
[ 5] local 203.0.113.50 port 51657 connected to 203.0.113.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.04 sec 12.5 MBytes 101 Mbits/sec 0 490 KBytes
[ 5] 1.04-1.52 sec 5.81 MBytes 101 Mbits/sec 0 490 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-1.52 sec 18.3 MBytes 101 Mbits/sec 0 sender
Sent 18.3 MByte / 18.3 MByte (100%) of A.bin
[ 5] 0.00-1.52 sec 18.3 MBytes 101 Mbits/sec receiver
iperf Done.
#Running this command several times shows that a consistent averate bitrate for throughput on this system is about 101Mbits/second - or about 18.3 MBytes/second.
We now have a baseline TCP-based "goodput" value for testing dummynet traffic shaping commands.
4.2. IPFW Commands for Dummynet
To use dummynet, load the kernel module dummynet.ko:
# kldload dummynet load_dn_sched dn_sched FIFO loaded load_dn_sched dn_sched QFQ loaded load_dn_sched dn_sched RR loaded load_dn_sched dn_sched WF2Q+ loaded load_dn_sched dn_sched PRIO loaded load_dn_sched dn_sched FQ_CODEL loaded load_dn_sched dn_sched FQ_PIE loaded load_dn_aqm dn_aqm CODEL loaded load_dn_aqm dn_aqm PIE loaded #
dummynet announces the schedulers it is configured to use.
4.2.1. Simple Pipe Configuration
Recall that dummynet uses pipes, queues, and sched (schedulers) to shape traffic.
To see dummynet in action, create a pipe with limited bandwidth, and assign it to a rule matching traffic to the external1 VM:
# ipfw pipe 1 config bw 300Kbit/s # ipfw pipe 1 show 00001: 300.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active #
The above output shows the pipe configration limiting bandwidth (bw) to 300Kbits/sec.
Now add ipfw rules to send traffic between the firewall VM and the external1 VM through the pipe:
# ipfw add 100 check-state 00100 check-state :default # # ipfw add 1000 pipe 1 ip from any to any 01000 pipe 1 ip from any to any # # ipfw list 00100 check-state :default 01000 pipe 1 ip from any to any 65535 deny ip from any to any #
By adding the matching phrase "ip from any to any" and assigning it to pipe 1, we have configured the firewall to send all ip-based traffic through the pipe, now configured as a 300K bps link.
If we re-run the basic file transfer command for iperf3 the same as we did earlier, we can see the difference take shape:
# iperf3 -F A.bin -c 203.0.113.10
Connecting to host 203.0.113.10, port 5201
[ 5] local 203.0.113.50 port 22303 connected to 203.0.113.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 69.1 KBytes 564 Kbits/sec 0 15.6 KBytes
[ 5] 1.00-2.01 sec 36.3 KBytes 294 Kbits/sec 0 18.4 KBytes
[ 5] 2.01-3.01 sec 33.9 KBytes 278 Kbits/sec 0 21.3 KBytes
[ 5] 3.01-4.00 sec 47.6 KBytes 394 Kbits/sec 0 24.1 KBytes
[ 5] 4.00-5.01 sec 26.9 KBytes 218 Kbits/sec 0 25.5 KBytes
[ 5] 5.01-6.00 sec 37.7 KBytes 312 Kbits/sec 0 27.0 KBytes
[ 5] 6.00-7.00 sec 43.8 KBytes 360 Kbits/sec 0 28.4 KBytes
[ 5] 7.00-8.01 sec 34.9 KBytes 282 Kbits/sec 0 29.8 KBytes
[ 5] 8.01-9.00 sec 29.7 KBytes 246 Kbits/sec 0 31.2 KBytes
[ 5] 9.00-10.00 sec 46.2 KBytes 378 Kbits/sec 0 32.7 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 406 KBytes 332 Kbits/sec 0 sender
Sent 406 KByte / 18.3 MByte (2%) of A.bin
[ 5] 0.00-10.55 sec 358 KBytes 278 Kbits/sec receiver
iperf Done.
#Here, we can see that during iperf3's 10-second run, the ipfw dummynet configuration limited the transfer speed to an average of about 332 Kbits/sec, and only about 2% of the entire 10MB file was transferred.
To see how we can use dummynet to configure different link speeds, we will set up a second pipe:
# ipfw pipe 2 config bw 3Mbit/s # ipfw pipe show 00001: 300.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active 00002: 3.000 Mbit/s 0 ms burst 0 q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail sched 65538 type FIFO flags 0x0 0 buckets 0 active #
This pipe is set up to be 10 times faster (3Mb/sec instead of 300Kb/sec) than pipe 1. We then reconfigure the ipfw rules to send to the external2 VM (also running iperf3 -s) through pipe 2:
# ipfw list 00100 check-state :default 01000 pipe 1 ip from any to any 65535 deny ip from any to any # # ipfw delete 1000 # # ipfw add 1000 pipe 1 ip from me to 203.0.113.10 // external1 01000 pipe 1 ip from me to 203.0.113.10 # # ipfw add 1100 pipe 1 ip from 203.0.113.10 to me // external1 01100 pipe 1 ip from 203.0.113.10 to me # # ipfw add 2000 pipe 2 ip from me to 203.0.113.20 // external2 02000 pipe 2 ip from me to 203.0.113.20 # # ipfw add 2100 pipe 2 ip from 203.0.113.20 to me // external2 02100 pipe 2 ip from 203.0.113.20 to me # # ipfw list 00100 check-state :default 01000 pipe 1 ip from me to 203.0.113.10 01100 pipe 1 ip from 203.0.113.10 to me 02000 pipe 2 ip from me to 203.0.113.20 02100 pipe 2 ip from 203.0.113.20 to me 65535 deny ip from any to any #
As expected, pipe 2 is approximately 10 times faster than pipe 1:
# iperf3 -F A.bin -c 203.0.113.20
Connecting to host 203.0.113.20, port 5201
[ 5] local 203.0.113.50 port 21569 connected to 203.0.113.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 417 KBytes 3.41 Mbits/sec 0 34.1 KBytes
[ 5] 1.00-2.00 sec 325 KBytes 2.66 Mbits/sec 0 45.5 KBytes
[ 5] 2.00-3.00 sec 373 KBytes 3.06 Mbits/sec 0 55.5 KBytes
[ 5] 3.00-4.00 sec 334 KBytes 2.73 Mbits/sec 0 64.0 KBytes
[ 5] 4.00-5.00 sec 348 KBytes 2.85 Mbits/sec 0 64.0 KBytes
[ 5] 5.00-6.00 sec 337 KBytes 2.76 Mbits/sec 0 64.0 KBytes
[ 5] 6.00-7.00 sec 339 KBytes 2.78 Mbits/sec 0 64.0 KBytes
[ 5] 7.00-8.00 sec 348 KBytes 2.85 Mbits/sec 0 64.0 KBytes
[ 5] 8.00-9.00 sec 351 KBytes 2.87 Mbits/sec 0 64.0 KBytes
[ 5] 9.00-10.00 sec 351 KBytes 2.88 Mbits/sec 0 64.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 3.44 MBytes 2.89 Mbits/sec 0 sender
Sent 3.44 MByte / 18.3 MByte (18%) of A.bin
[ 5] 0.00-10.12 sec 3.42 MBytes 2.83 Mbits/sec receiver
iperf Done.
#Note that the pipe configuration can be changed without changing the ruleset. Below, the pipe 1 bandwidth is changed to the equivalent of a telecommunications T1 line in days of yore:
# ipfw pipe 1 config bw 1544Kbit/s # ipfw pipe show 00001: 1.544 Mbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active 00002: 3.000 Mbit/s 0 ms burst 0 q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail sched 65538 type FIFO flags 0x0 0 buckets 0 active #
Resending the 10MB file across the T1 configured line shows these results:
# iperf3 -F A.bin -c 203.0.113.10
Connecting to host 203.0.113.10, port 5201
[ 5] local 203.0.113.50 port 16696 connected to 203.0.113.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 222 KBytes 1.82 Mbits/sec 0 65.0 KBytes
[ 5] 1.00-2.00 sec 181 KBytes 1.48 Mbits/sec 0 65.0 KBytes
[ 5] 2.00-3.00 sec 181 KBytes 1.48 Mbits/sec 0 65.0 KBytes
[ 5] 3.00-4.00 sec 184 KBytes 1.51 Mbits/sec 0 65.0 KBytes
[ 5] 4.00-5.00 sec 181 KBytes 1.48 Mbits/sec 0 65.0 KBytes
[ 5] 5.00-6.00 sec 181 KBytes 1.48 Mbits/sec 0 65.0 KBytes
[ 5] 6.00-7.00 sec 178 KBytes 1.46 Mbits/sec 0 65.0 KBytes
[ 5] 7.00-8.00 sec 178 KBytes 1.46 Mbits/sec 0 65.0 KBytes
[ 5] 8.00-9.00 sec 181 KBytes 1.48 Mbits/sec 0 65.0 KBytes
[ 5] 9.00-10.00 sec 175 KBytes 1.44 Mbits/sec 0 65.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.80 MBytes 1.51 Mbits/sec 0 sender
Sent 1.80 MByte / 18.3 MByte (9%) of A.bin
[ 5] 0.00-10.18 sec 1.78 MBytes 1.46 Mbits/sec receiver
iperf Done.
#About half of the 3Mbits/sec speed of pipe 2, again as expected.
So far, we have only been working with the pipe object. By definition, a pipe has just one queue, and it is subject to "First In First Out" (FIFO) operation. All traffic that flows through this pipe shares the same characteristics.
However, creating a pipe also does something else. It creates a default sched (scheduler) that governs the pipe:
Start with no pipes or schedulers # # ipfw pipe list # # ipfw sched list # Create a simple pipe. # ipfw pipe 1 config bw 100KBit/s # # ipfw pipe list 00001: 100.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active # Observe the default scheduler for this pipe # ipfw sched list 00001: 100.000 Kbit/s 0 ms burst 0 sched 1 type WF2Q+ flags 0x0 0 buckets 0 active #
The default scheduler for a new pipe is of type WF2Q+, a version of the Weighted Fair Queueing algorithm for packet transfer.
We now have a single pipe of type FIFO operation that is managed by a WF2Q+ scheduling algorithm.
The ipfw(8) man page makes note of serveral other scheduling algorithms. These can be selected by using the "type" keyword on the pipe command. The type keyword selects the type of scheduler applied to the pipe - not the type of the pipe itself (the pipe remains FIFO):
# ipfw pipe list # # ipfw sched list # Create a pipe and assign a scheduler of type Round Robin (Deficit Round Robin) # ipfw pipe 1 config bw 100KBit/s type rr # # ipfw pipe list 00001: 100.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active # View the new sheduler of type RR (Deficit Round Robin) # ipfw sched list 00001: 100.000 Kbit/s 0 ms burst 0 sched 1 type RR flags 0x0 0 buckets 0 active #
*pipes and *sched*s (schedulers) are tighly bound. In fact, there is no command to delete a scheduler. The scheduler is deleted when the pipe is deleted.
Note however that the scheduler can be configured independently if desired. Below we change the scheduler type from the above type RR to QFQ, a variant of WF2Q+:
# # ipfw sched 1 config type qfq Bump qfq weight to 1 (was 0) Bump qfq maxlen to 1500 (was 0) # # ipfw sched list 00001: 100.000 Kbit/s 0 ms burst 0 sched 1 type QFQ flags 0x0 0 buckets 0 active #
There are other keywords that can be added to a pipe specification: delay, burst, profile, weight, buckets, mask, noerror, plr, queue, red or gred, codel, and pie. These are described in the ipfw(8) man page.
A contrived example might be:
Start fresh
# ipfw pipe 1 delete
#
# ipfw pipe 1 config bw 100kbit/s delay 20 burst 2000 weight 40 buckets 256 mask src-ip 0x000000ff noerror plr 0.01 queue 75 red .3/25/30/.5 type qfq
#
# ipfw pipe list
00001: 100.000 Kbit/s 20 ms burst 2000
q131073 75 sl.plr 0.010000 0 flows (1 buckets) sched 65537 weight 40 lmax 0 pri 0
RED w_q 0.299988 min_th 25 max_th 30 max_p 0.500000
sched 65537 type FIFO flags 0x1 256 buckets 0 active
mask: 0x00 0x000000ff/0x0000 -> 0x00000000/0x0000
#
# ipfw sched list
00001: 100.000 Kbit/s 20 ms burst 2000
sched 1 type QFQ flags 0x1 256 buckets 0 active
mask: 0x00 0x000000ff/0x0000 -> 0x00000000/0x0000
#Setting up two separate pipes to send data to the same destination is overkill. It’s like setting up two separate network links between the two points. While that may be desireable for reduncancy or high-availability, it makes no difference for bandwidth allocation. (Yes, link aggregation is possible, but we are not considering that case here.)
What is usually needed is a way to separate traffic into different "lanes" and assign different "speed limits" to each lane. That is exactly what queues are for.
4.2.2. Simple Pipe and Queue Configuration
Before we go further, it’s useful to disambiguate the two meanings of the word "queue".
In a pipe definition, by default, the pipe is assigned a queue where incoming packets are held before processing and transit. The size of this "pipe queue" is by default 50 packets, but can be changed with the queue keyword on the pipe definition:
# ipfw pipe 1 config bw 200Kbit/s # # ipfw pipe list 00001: 200.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active # # ipfw pipe 2 config bw 200Kbit/s queue 75 # # ipfw pipe list 00001: 200.000 Kbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active 00002: 200.000 Kbit/s 0 ms burst 0 q131074 75 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail sched 65538 type FIFO flags 0x0 0 buckets 0 active #
In contrast, dummynet has the concept of flow queues which are virtual groupings of packets assigned to a flow according to a mask in their own definition with an ipfw queue statements.
Configuring a queue is almost as simple as configuring a pipe.
Start with a clean slate (all objects and rules deleted):
# kldunload dummynet # kldunload ipfw # kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled # kldload dummynet load_dn_sched dn_sched FIFO loaded load_dn_sched dn_sched QFQ loaded load_dn_sched dn_sched RR loaded load_dn_sched dn_sched WF2Q+ loaded load_dn_sched dn_sched PRIO loaded load_dn_sched dn_sched FQ_CODEL loaded load_dn_sched dn_sched FQ_PIE loaded load_dn_aqm dn_aqm CODEL loaded load_dn_aqm dn_aqm PIE loaded # # ipfw queue 1 config pipe 1 # # ipfw queue show q00001 50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
Here we see that one queue of size 50 packets was created and assigned to pipe 1. Since we did not assign a weight, the default weight is 0 (zero), which is the least weight possible. The queue currently has 0 flows, meaning that this queue has no traffice flowing through it
Notice however, that we created the queue before we created the pipe. That is why the weight is zero. We have actually done this configuration out of order. To maintain your sanity, (and those reading the configuration after you), it’s best to configure the objects in the following order:
pipes (also creates a scheduler, which can be assigned a specific scheduler type)
queues - create queues and assign weights, source and destination masks, delay, and other characteristics to the queue
Assign rules to match traffic using standard 5-tuples or as needed
dummynet also has the ability to separate out different flows within the same pipe to perform different scheduling algorithms.
When transferring a file to the external1 VM and attempting to type interactively on the external1 VM at the same time, the ability to type at speed is dramatically reduced. The file transfer packets, being much larger than interactive typing packets are hogging all the bandwidth. This effect is a well known limitation to anyone who edits documents on a remote site. Since packets are created much faster by a file transfer program than you can type, the outbound queue is almost always full of large packets, leaving your keystrokes to be separated by large amounts of file transfer data in the queue.
You should try this out on the firewall VM by resetting the pipe 1 bandwidth to 300Kbit/sec, and in one session, run iperf3 as iperf3 -c 203.0.113.10 -t 60. Then in another session, add rules for ssh traffic and ssh to external1 VM and try to enter text into a scratch file. The typing delay is almost unbearable. |
To control traffic flow between the firewall VM and any external VM host, you will need to set up individual queues to separate traffic within a pipe. queues can be either static - you define them yourself with the ipfw queue config … - or they can be dynamic. Dynamic queues are created when using the mask keyword. Masks for queues are callled flow masks. The mask determines if a packet that arrives at the firewall is selected to be entered into a queue. Consider the following example:
# ipfw pipe 1 config bw 200Kbit/s mask src-ip 0x000000ff
Each /24 host transferring data through pipe 1 (based on suitable rules) will have its own dynamic queue, all sharing the bandwidth in the pipe equally.
If instead, we wish to create separate individual queues with different characteristics such as different weights or delay, we can create static queues and then assign them to individual pipes as desired:
#
# ipfw pipe 1 config bw 300kbit/s
#
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x0 0 buckets 0 active
#
# ipfw queue 1 config pipe 1 weight 10 mask dst-ip 0xffffffff dst-port 5201
Bump flowset buckets to 64 (was 0)
#
# ipfw queue 2 config pipe 1 weight 10 mask dst-ip 0xffffffff dst-port 5202
Bump flowset buckets to 64 (was 0)
#
# ipfw queue show
q00001 50 sl. 0 flows (64 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x1451
q00002 50 sl. 0 flows (64 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x1452
#
# ipfw list
00010 allow icmp from any to any
00100 check-state :default
01000 queue 1 tcp from me to 203.0.113.10 5201 setup keep-state :default
01100 queue 2 tcp from me to 203.0.113.20 5202 setup keep-state :default
65535 deny ip from any to any
#Running
# iperf3 -c 203.0.113.10 -p 5201 -t 180 -O 30
produces the output below.
The output is the result of using the "omit" flag (-O) on the sender to ignore the first 30 seconds of output. This removes the "slow start" portion of the TCP test, and focuses instead on the "steady state" that occurs after slow start gets up to speed.
Figure 21. Testing Separate Static Queues and Pipes
This example shows the steady-state results of transmitting data through one queue - queue 1. Throughput was consistently about 277Kbits/sec. During the transmission, a view of the queue status was:
# ipfw queue show
q00001 50 sl. 2 flows (64 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x1451
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
136 ip 0.0.0.0/0 203.0.113.10/5201 2293 3425216 42 63000 0
50 ip 0.0.0.0/0 203.0.113.50/1040 752 39104 1 52 0
q00002 50 sl. 0 flows (64 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x1452
#The queue mask, set to show the full destination address and destination port is highlighted.
Note that the port number is displayed in hexadecimal. A decimal/hexadecimal calculator may save you some confusion if you are looking at a lot of queue displays. |
The next example shows the result of starting transmission through the second queue about halfway through the first transmission. Notice how the queue is adjusted to accomodate the presence of a second queue of equal weight:
Figure 22. Testing Two Static Queues and Pipes
Since the queues were equally weighted, the result was that the transmission rate for both ended up at about 139Kbits/sec or roughly half of the previous transmission.
Queue characteristics can be changed at any time, even during an active flow. Consider the case below where, during simultaneous transmission through queues of equal weight, the queue weights were modifed as follows:
queue 1: original weight 10 modified weight 10
queue 2: original weight 10 modified weight 50
This change can be effected by the command:
# ipfw queue 2 config weight 50
Figure 23. Testing Two Static Queues and Pipes Changed In-flight
The transmission rate for queue 1 dropped from and average of 139 Kbits/sec to an average of 46.3 Kbits/sec; while queue 2, after restarting the transmission with the new queue weight, expanded from an average of 139Kbits/sec to an average of 232 Kbits/sec. As expected, 232 Kbits is about five times the transmission rate of 46.3 Kbits/sec.
Note however, that the above command had a side effect:
# ipfw queue show
q00001 50 sl. 0 flows (64 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x1451
q00002 50 sl. 0 flows (1 buckets) sched 1 weight 50 lmax 0 pri 0 droptail
#The flow mask for queue 2 has been deleted. In fact, all settings not explicitly reset will revert to their default settings. Here is a complicated queue setup:
# ipfw queue 1 config pipe 1 weight 40 buckets 256 mask src-ip 0x000000ff dst-ip 0x0000ffff noerror plr 0.01 queue 75 red .3/25/30/.5
#
# ipfw queue show
q00001 75 sl.plr 0.010000 0 flows (256 buckets) sched 1 weight 40 lmax 0 pri 0
RED w_q 0.299988 min_th 25 max_th 30 max_p 0.500000
mask: 0x00 0x000000ff/0x0000 -> 0x0000ffff/0x0000
#And if, similar to the previous example, we only change the weight:
# ipfw queue 1 config weight 20 # # ipfw queue show q00001 50 sl. 0 flows (1 buckets) sched 1 weight 20 lmax 0 pri 0 droptail #
All the other parameters of the queue are reset to their defaults. Therefore, it is best to retain the original commands used to construct queues, pipes, and schedulers, so that if you are only changing one parameter, all other parameters can be replicated on the command line. Otherwise you will have to reconstruct the parameters from the output of ipfw queue show which can be quite tedious.
4.2.3. Dynamic Pipes
Here, we note the simplest setup for pipes creates dynamic pipes when needed:
# ipfw pipe 1 config bw 300kbit/s weight 10 mask src-ip 0x0000ffff dst-ip 0xffffffff
Bump sched buckets to 64 (was 0)
#
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 10 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 0 active
mask: 0x00 0x0000ffff/0x0000 -> 0xffffffff/0x0000
#
# ipfw list
00050 allow icmp from any to any
00100 check-state :default
65535 deny ip from any to any
#
# ipfw add 1000 pipe 1 tcp from me to 203.0.113.0/24 5201-5203 setup keep-state
01000 pipe 1 tcp from me to 203.0.113.0/24 5201-5203 setup keep-state :default
#
# ipfw list
01000 pipe 1 tcp from me to 203.0.113.0/24 5201-5203 setup keep-state :default
65535 deny ip from any to any
#Sending some data with this configuration:
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 10 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 4 active
mask: 0x00 0x0000ffff/0x0000 -> 0xffffffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
6 ip 0.0.10.10/0 203.0.113.50/0 236 12272 0 0 0
78 ip 0.0.10.50/0 203.0.113.10/0 1493 2225216 43 64500 0
80 ip 0.0.10.50/0 203.0.113.20/0 1355 2018216 42 63000 0
58 ip 0.0.10.20/0 203.0.113.50/0 366 19032 0 0 0
#
# ipfw list
00050 allow icmp from any to any
00100 check-state :default
01000 pipe 1 tcp from me to 203.0.113.0/24 5201-5203 setup keep-state :default
65535 deny ip from any to any
#All three transmissions running together, single pipe:
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 10 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 6 active
mask: 0x00 0x0000ffff/0x0000 -> 0xffffffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
6 ip 0.0.10.10/0 203.0.113.50/0 588 30576 0 0 0
78 ip 0.0.10.50/0 203.0.113.10/0 1508 2247716 43 64500 0
80 ip 0.0.10.50/0 203.0.113.20/0 1357 2021216 43 64500 0
90 ip 0.0.10.50/0 203.0.113.30/0 1322 1981552 41 61500 0
46 ip 0.0.10.30/0 203.0.113.50/0 34 1768 0 0 0
58 ip 0.0.10.20/0 203.0.113.50/0 702 36504 0 0 0Because of the ipfw rule:
01000 pipe 1 tcp from me to 203.0.113.0/24 5201-5203 setup keep-state :default
All are getting 290 Kbit/sec from iperf3 and they are all sharing the pipe equally.
If we change iperf3 to send to different ports for each system (5201, 5202, 5203) to external1, external2, and external3 VMs respectively there is no change. It is only with queues, where you can set the individual flow rate, that you can effect change.
Below are examples of different masks and their effect on traffic flow:
* dst-ip 0x0000ffff
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 4 active
mask: 0x00 0x00000000/0x0000 -> 0x0000ffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
10 ip 0.0.0.0/0 0.0.10.10/0 1183 1760218 43 64500 0
20 ip 0.0.0.0/0 0.0.10.20/0 974 1446718 42 63000 0
30 ip 0.0.0.0/0 0.0.10.30/0 688 1017718 35 52500 0
50 ip 0.0.0.0/0 0.0.10.50/0 1717 89284 0 0 0
* dst-ip 0xffffffff
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 4 active
mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
18 ip 0.0.0.0/0 203.0.113.50/0 402 20888 0 0 0
42 ip 0.0.0.0/0 203.0.113.10/0 144 204722 0 0 0
52 ip 0.0.0.0/0 203.0.113.20/0 359 525971 0 0 0
62 ip 0.0.0.0/0 203.0.113.30/0 562 843000 37 55500 0
* src-ip 0x0000ffff
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 4 active
mask: 0x00 0x0000ffff/0x0000 -> 0x00000000/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
20 ip 0.0.10.10/0 0.0.0.0/0 361 19348 0 0 0
100 ip 0.0.10.50/0 0.0.0.0/0 2102 3079974 36 54000 27
40 ip 0.0.10.20/0 0.0.0.0/0 193 10416 0 0 0
60 ip 0.0.10.30/0 0.0.0.0/0 47 2612 0 0 0
* mask src-ip 0x0000ffff dst-ip 0x0000ffff <-only one keyword mask needs to be specified
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 6 active
mask: 0x00 0x0000ffff/0x0000 -> 0x0000ffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
14 ip 0.0.10.30/0 0.0.10.50/0 253 13156 0 0 0
26 ip 0.0.10.20/0 0.0.10.50/0 61 3172 0 0 0
38 ip 0.0.10.10/0 0.0.10.50/0 771 40094 0 0 0
110 ip 0.0.10.50/0 0.0.10.10/0 853 1265218 40 60000 0
112 ip 0.0.10.50/0 0.0.10.20/0 723 1083052 37 55500 0
122 ip 0.0.10.50/0 0.0.10.30/0 644 951718 34 51000 0
* mask src-ip 0x0000ffff dst-ip 0x0000ffff dst-port 5201
# ipfw pipe show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 6 active
mask: 0x00 0x0000ffff/0x0000 -> 0x0000ffff/0x1451
BKT Prot Source IP/port Dest. IP/port ot_pkt/bytes Pkt/Byte Drp
204 ip 0.0.10.50/0 0.0.10.10/5201 2132 3183718 43 64500 0
14 ip 0.0.10.30/0 0.0.10.50/4096 823 42796 0 0 0
210 ip 0.0.10.50/0 0.0.10.20/5201 2001 2987218 43 64500 0
152 ip 0.0.10.20/0 0.0.10.50/4161 663 34476 0 0 0
216 ip 0.0.10.50/0 0.0.10.30/5201 1981 2957218 43 64500 0
164 ip 0.0.10.10/0 0.0.10.50/65 471 24492 0 0 0
* mask src-ip 0xffffffff dst-ip 0xffffffff
# ipfw pipe 1 show
00001: 300.000 Kbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x1 64 buckets 6 active
mask: 0x00 0xffffffff/0x0000 -> 0xffffffff/0x0000
BKT Prot Source IP/port Dest. IP/port Tot_pkt/bytes Pkt/Byte Drp
64 ip 203.0.113.50/0 203.0.113.20/0 1215 1808218 43 64500 0
74 ip 203.0.113.50/0 203.0.113.30/0 1023 1533052 43 64500 0
22 ip 203.0.113.10/0 203.0.113.50/0 746 38792 0 0 0
94 ip 203.0.113.50/0 203.0.113.10/0 1863 2780218 42 63000 0
42 ip 203.0.113.20/0 203.0.113.50/0 481 25012 0 0 0
62 ip 203.0.113.30/0 203.0.113.50/0 159 8268 0 0 04.2.4. Other Pipe and Queue Commands
To delete pipes and queues use the following syntax:
For queues, specify the queue number on the command line: # ipfw queue delete 1 For pipes, specify the pipe number on the command line: # ipfw pipe delete 1
Note however that:
# ipfw delete pipe 1 <----- does not throw error, and does not delete the pipe.
The same is true for the corresponding queue keyword. You should take care to use the proper syntax.
You can delete a pipe with a pipe statement still in the ruleset. ipfw will not throw an error - but any data transfer matching a pipe statement will not work. |
scheds (schedulers) and pipes are tightly bound. To delete a scheduler, you must first delete the pipe. You can then re-create the pipe if needed. The scheduler for the new pipe is reset to the default scheduler.
To change the scheduler type: # ipfw sched config 1 type wfq2 # or rr or any other sched type
4.3. Adding Additional Virtual Machines
Up to this point, we have been using only two virtual machines for exploring ipfw. The later material in this book requires the use of several additional virtual machines.
In the NAT chapter, we will use several more VMs for:
Figure 24. Setting Up Simple NAT
Figure 25. Setting up Load Sharing NAT
Figure 26. Setting Up NAT64 and DNS64
Figure 27. Setting Up 464XLAT
4.3.1. Setting Up The Entire IPFW Lab
A suggested host machine file directory layout for these machines is shown below. All scripts use relative path names, so the directory can be located anywhere.
~/ipfw
/SCRIPTS
_CreateAllVMs.sh (script to create QEMU disks images)
mkbr.sh (script to create bridge and tap devices)
vm_envs.sh (script to manage all parameters)
dnshost.sh (script for running a BIND 9 DNS server)
external1.sh (scripts for running 'external VM host' VMs)
external2.sh "
external3.sh "
firewall.sh (script for running a firewall VM)
firewall2.sh (script for running a firewall VM)
internal.sh (script for running an internal VM)
v6only.sh (script for running an IPv6 only VM)
/ISO
fbsd.iso (link to latest FreeBSD install iso)
/VM
dnshost.qcow2 (QEMU disk image for a BIND 9 DNS server)
external1.qcow2 (QEMU disk image for 'external' hosts)
external2.qcow2 "
external3.qcow2 "
firewall.qcow2 (QEMU disk image for the 'firewall')
firewall2.qcow2 (QEMU disk image for the 'firewall2' host)
internal.qcow2 (QEMU disk image for an internal host)
v6only.qcow2 (QEMU disk image for an IPv6 only host)
/BMP
dns_splash_640x480.bmp (QEMU splash image)
external1_splash_640x480.bmp "
external2_splash_640x480.bmp "
external3_splash_640x480.bmp "
firewall_splash_640x480.bmp "
firewall2_splash_640x480.bmp "
internal_splash_640x480.bmp "
v6only_splash_640x480.bmp "Finish setting up the entire lab by referring to the instructions found in Section Quick Start.
Also, ensure each virtual machine is set up to boot a serial console by adding "console=comconsole" to /boot/loader.conf.
Chapter 5. ipfw NAT
Network Address Translation (NAT) is the process of changing the source or destination address of a packet as it flows through the firewall. This is done chiefly to segregate internal networks and subnets from external networks.
FreeBSD has two capabilities for NAT - natd(8) a daemon process that can perform these translations, and in-kernel NAT with ipfw. Both of these capabilities use the libalias(3) library.
This section will focus primarily on in-kernel NAT with ipfw.
5.1. General Procedures for Working NAT Examples
In this section we will use more than two virtual machines (VMs). If you have followed the directions on Setting Up the Entire ipfw Lab, we can begin with the setup for simple NAT.
The examples in this section and later sections grow increasingly complex. Follow this standard procedure for startup with each new example:
On the host, begin by setting up the bridge and tap setup needed for the examples. Use mkbr.sh to configure bridge and tap devices on the host. Examine the figure, and run the script with all bridges and taps accounted for.
Start up the required VMs. Use runvm.sh to start up several VMs at one time.
On each VM, set up the required addressing. Check the diagram in each Section for addressing requirements.
Ensure all VMs have connectivity to their local network peers.
If there are additional scripts to load onto the firewall, external, internal, dnshost, or v6only VMs, load them.
If there are specific DNS entries that are required for an example, load them into the dnshost and test the entries from another VM.
Other VMs in some examples require adding additional routes.
On the firewall VM, unload and reload the firewall: (kldunload ipfw and kldload ipfw).
Check whether any sysctl entries are required for the example.
Follow the procedure given for each section.
Troubleshoot as necessary.
5.2. Setting Up for Simple NAT
Figure 28. Setting Up Simple NAT
Shut down the existing VMs from the previous examples and reload ipfw. To set up the correct network bridge and tap architecture as shown in Figure 7, use this command:
# sudo /bin/sh mkbr.sh reset bridge0 tap1 tap4 bridge1 tap0 tap5
Restart the desired VMs with:
# /bin/sh runvm.sh firewall external1 internal
You will have to reconfigure the network addressing. Use the above figure to set up the correct addresses for each VM and ensure you can ping adjacent interfaces.
For routing, the external1 VM should have the default route set to 203.0.113.50.
The internal VM should have its default route set to 198.51.100.50.
The firewall should have its default route set to 203.0.113.10 (external1) since we want all traffic to exit via the firewall em0 interface.
The firewall should already be set up for IP forwarding (sysctl net.inet.ip.forwarding=1), but if not, set the sysctl as indicated. You should be able to ping em0 on external1 VM from the internal VM host and vice-versa. Check all addressing, the host bridge and tap devices, and the sysctl net.inet.ip.forwarding=1 on the firewall if something is not working.
On the firewall VM, restart ipfw with
# kldload ipfw
To use in-kernel NAT, you must first load the ipfw_nat kernel module:
# kldload ipfw_nat
Running kldstat should now show output similar to:
# kldstat Id Refs Address Size Name 1 11 0xffffffff80200000 1f370e8 kernel 2 1 0xffffffff82818000 3220 intpm.ko 3 1 0xffffffff8281c000 2178 smbus.ko 4 2 0xffffffff8281f000 27450 ipfw.ko 5 1 0xffffffff82847000 42d0 ipfw_nat.ko 6 1 0xffffffff8284c000 c962 libalias.ko #
We are now ready to explore ipfw_nat.
Similar to other ipfw entities such as pipes and queues, ipfw_nat works with a NAT object. A NAT object is a single entry in the packet aliasing database.
We first create a NAT object:
# ipfw nat 25 config ip 198.51.100.50 ipfw nat 25 config ip 198.51.100.50 # # ipfw nat show config ipfw nat 25 config ip 198.51.100.50
Note that the NAT object identifier must be numeric, not alphabetic or alphanumeric.
A NAT object identifier such as foo or 25foo will be rejected by ipfw.
iNext, load two rules that will use that instance:
# ipfw add 1000 nat 25 tcp from any to any # ipfw add 2000 nat 25 icmp from any to any
Immediately listing the rulest shows the the NAT object and the rule body.
# ipfw list 01000 nat 25 tcp from any to any 02000 nat 25 icmp from any to any 65535 deny ip from any to any
#
We now have an ipfw_nat instance in the packet aliasing database and rules that will engage that instance. This is generally referred to as "static NAT".
The ipfw_nat instance will replace the IP source address of any packet exiting the firewall with 198.51.100.50, provided that packet has reached the ipfw_nat rule and matches its configuration.
To test, start tcpdump(1) on the host system monitoring bridge0. (You should again ensure that the host system is not running a firewall.)
host_system# tcpdump -n -i bridge0 -v
Then, from the firewall VM window, telnet(1) to any IP address not used in our lab:
# telnet 172.16.10.10 Trying 172.16.10.10... ^C
All you need is a few seconds to attempt the connection (which will not succeed anyway).
Examining the host tcpdump output we see the following:
host_system# tcpdump -n -i bridge0 -v
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:58:34.099782 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
198.51.100.50.62143 > 172.16.10.10.23: Flags [S], cksum 0x89d4 (correct), seq 3107170690, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 384725297 ecr 0], length 0
19:58:38.300043 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
198.51.100.50.62143 > 172.16.10.10.23: Flags [S], cksum 0x796b (correct), seq 3107170690, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 384729498 ecr 0], length 0
19:58:46.500217 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
198.51.100.50.62143 > 172.16.10.10.23: Flags [S], cksum 0x5964 (correct), seq 3107170690, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 384737697 ecr 0], length 0
^CThe source address has been changed from 203.0.113.50 to 198.51.100.50 as per our ipfw_nat instance. Note however, that with our configuration binding NAT to an IP address, as opposed to an interface, the NAT aliasing takes place on all configured interfaces, internal and external. You can verify this by repeating the above tcpdump on bridge1, and running a similar command for an existing host on the internal network. This time the destination will send a reset (RST), since the packet reached the destination but the service on the destination was not open.
# telnet 10.10.10.20
host_system# tcpdump -n -i bridge1 -v
tcpdump: listening on bridge1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:12:13.706505 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
198.51.100.50.32825 > 10.10.10.20.23: Flags [S], cksum 0x6039 (correct), seq 1314409263, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3924141446 ecr 0], length 0
20:12:13.710494 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
10.10.10.20.23 > 198.51.100.50.32825: Flags [R.], cksum 0x5774 (correct), seq 0, ack 1314409264, win 0, length 0
20:12:29.573756 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
^CTo specify just the outside interface is to be NATed, use the keyword if on the ipfw NAT configuration statement and specify the correct external interface:
# ipfw nat 25 config if em1 # ipfw nat show config ipfw nat 25 config if em1
Note that you cannot use the ip ip_addr and if interf_name options at the same time on the same NAT instance - you must use one or the other.
What happens in this case is that the NAT instance will ensure that the IP address of interface em1 will always be used on traffic exiting through that interface - even if the address changes (because of DHCP or an administrative addressing change):
Traffic destined externally from the internal VM host via:
root@internal:~ # telnet 172.16.10.10 Trying 172.16.10.10... ^C
On the FreeBSD host:
host_system# tcpdump -n -i bridge0 -v
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:24:41.147755 IP (tos 0x10, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.50.40001 > 172.16.10.10.23: Flags [S], cksum 0x5962 (correct), seq 950423268, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2027118491 ecr 0], length 0
20:24:42.189806 IP (tos 0x10, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.50.40001 > 172.16.10.10.23: Flags [S], cksum 0x554b (correct), seq 950423268, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2027119538 ecr 0], length 0
20:24:44.394747 IP (tos 0x10, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
203.0.113.50.40001 > 172.16.10.10.23: Flags [S], cksum 0x4caa (correct), seq 950423268, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2027121747 ecr 0], length 0
^CThough it does not look like it, ipfw is translating the packets as they exit the firewall.
Consider this exchange where the internal VM host pings external1:
root@internal:~ # ping 203.0.113.10 PING 203.0.113.10 (203.0.113.10): 56 data bytes 64 bytes from 203.0.113.10: icmp_seq=0 ttl=63 time=2.742 ms 64 bytes from 203.0.113.10: icmp_seq=1 ttl=63 time=2.675 ms ^C
The traffic on the internal bridge (bridge1) shows the packets from internal1:
host_system# tcpdump -n -i bridge1 -v
tcpdump: listening on bridge1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:29:27.048162 IP (tos 0x0, ttl 64, id 58916, offset 0, flags [none], proto ICMP (1), length 84)
10.10.10.20 > 203.0.113.10: ICMP echo request, id 15077, seq 0, length 64
20:29:27.052446 IP (tos 0x0, ttl 63, id 36018, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.10 > 10.10.10.20: ICMP echo reply, id 15077, seq 0, length 64
20:29:28.104133 IP (tos 0x0, ttl 64, id 58917, offset 0, flags [none], proto ICMP (1), length 84)
10.10.10.20 > 203.0.113.10: ICMP echo request, id 15077, seq 1, length 64
20:29:28.105732 IP (tos 0x0, ttl 63, id 36019, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.10 > 10.10.10.20: ICMP echo reply, id 15077, seq 1, length 64whereas the traffic on the external bridge (bridge0) shows the corrrect translation:
host_system# tcpdump -n -i bridge0 -v
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:33:19.695939 IP (tos 0x0, ttl 63, id 58919, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.50 > 203.0.113.10: ICMP echo request, id 58206, seq 0, length 64
20:33:19.696546 IP (tos 0x0, ttl 64, id 36021, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.10 > 203.0.113.50: ICMP echo reply, id 58206, seq 0, length 64
20:33:20.715148 IP (tos 0x0, ttl 63, id 58920, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.50 > 203.0.113.10: ICMP echo request, id 58206, seq 1, length 64
20:33:20.715824 IP (tos 0x0, ttl 64, id 36022, offset 0, flags [none], proto ICMP (1), length 84)
203.0.113.10 > 203.0.113.50: ICMP echo reply, id 58206, seq 1, length 64
^CThe unreg_only and unreg_cgn configuration options allow to bypass the NAT operation if the source IP of the packet is not one of the RFC 1918 addresses (unreg_only) or the RFC 6598 addresses (unreg_cgn - carrier grade NAT). In these cases, the original source address will be maintained in the packet, even though there is an ipfw_nat instance and a matching rule.
# ipfw nat 25 show config ipfw nat 25 config if em1 # # ipfw nat 25 config if em1 unreg_only ipfw nat 25 config if em1 unreg_only # # ipfw nat 25 show config ipfw nat 25 config if em1 unreg_only #
To try the unreg_only option, on the internal machine, change its IP address on em0 to a registered number, say 140.140.140.140/24, and change the corresponding link on the firewall (em1) to a compatible address - 140.140.140.1/24.
The internal machine will need a new default route: 140.140.140.1
root@internal:~ # ifconfig em0 140.140.140.140/24 root@internal:~ # route add default 140.140.140.1 add net default: gateway 140.140.140.1 root@internal:~ # and on the firewall root@firewall:~ # ifconfig em1 140.140.140.1/24
From the internal machine, try to ping an external address not in our lab:
# ping 5.5.5.5
and observe on the host system that the ipfw_nat instance did not replace the source address with the configured IP:
host_system# tcpdump -n -i bridge0 -v
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
21:07:18.154319 IP (tos 0x0, ttl 63, id 58943, offset 0, flags [none], proto ICMP (1), length 84)
140.140.140.140 > 5.5.5.5: ICMP echo request, id 38569, seq 0, length 64
21:07:19.180094 IP (tos 0x0, ttl 63, id 58944, offset 0, flags [none], proto ICMP (1), length 84)
140.140.140.140 > 5.5.5.5: ICMP echo request, id 38569, seq 1, length 64
21:07:20.194988 IP (tos 0x0, ttl 63, id 58945, offset 0, flags [none], proto ICMP (1), length 84)
140.140.140.140 > 5.5.5.5: ICMP echo request, id 38569, seq 2, length 64Not all the options available to ipfw_nat are described in the NAT section of the ipfw(8) man page.
Some of the options usable from natd(8) are avialable to ipfw_nat. These include:
redirect_port proto targetIP:targetPORT[-targetPORT]
[aliasIP:]aliasPORT[-aliasPORT]
[remoteIP[:remotePORT[-remotePORT]]]
redirect_proto proto localIP [publicIP [remoteIP]]
redirect_address localIP publicIPThe below options are used for Load Sharing NAT (LSNAT) as described in RFC 2391.
redirect_port proto targetIP:targetPORT[,targetIP:targetPORT[,...]]
[aliasIP:]aliasPORT [remoteIP[:remotePORT]]
redirect_address localIP[,localIP[,...]] publicIPWe discuss LSNAT in the next section.
5.3. Setting Up for LSNAT
For this example, we will use our three hosts "external1", "external2", and "external3" and pretend they are on the inside of the network; and our "internal" host is on the outside of the network.
The Figure below shows the architecture setup for working with LSNAT.
Figure 29. Setting Up for LSNAT
As before, shutdown all virtual machines and rebuild the network from scratch.
Use this command to set up the network bridge and tap architecture.
# sudo /bin/sh mkbr.sh reset bridge0 tap4 tap5 bridge1 tap0 tap1 tap2 tap3
Note that the host interface is not needed for this example.
Restart the virtual machines with:
# /bin/sh runvm.sh firewall internal external1 external2 external3
or start them up individually.
Configure each machine to ensure its network configuration matches the above Figure and test connectivity between adjacent systems with ping(8).
Throughout this section, remember that the "external" server VMs are now internal web servers load balancing between .10, .20, .30, and the "internal" server VM is the outside host accessing the internal webservers.
On each inside machines the following commands are necessary to perform the examples in this section:
# route delete default # # route add default 10.10.10.50
On the outside machine perform these commands:
# route delete default # # route add default 198.51.100.50
Also, on each host host, edit the primary index.html page and insert a line of text that has the actual IP address of the host - something like this:
File: /usr/local/www/nginx/index.html: <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p> This is VM EXTERNAL1 </p>
and start nginx on each inside host:
# service nginx onestart Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful Starting nginx. #
Figure 30. LSNAT Setup Showing All VMs
With no ipfw loaded on the firewall, you should be able to ping all inside addresses (10.10.10.10, .20, .30) from the outside host (198.51.100.20). You should also be able to access each web server via:
# lynx 203.0.113.10 (or .20 or .30)
We now start on LSNAT configuration.
5.3.1. Setting up LSNAT- One address (10.10.10.10)
We begin with loading ipfw and ipfw_nat on the firewall VM
# kldload ipfw # # kldload ipfw_nat
The first configuration is similar to static NAT, though from the outside to the inside. The command redirects incoming traffic from the outside host sent to destination IP 3.3.3.3 to inside host 10.10.10.10.
# ipfw nat 25 config redirect_addr 10.10.10.10 3.3.3.3 ipfw nat 25 config redirect_addr 10.10.10.10 3.3.3.3
Next create a rulset that utilizes this NAT instance:
# ipfw add 50 check-state # ipfw add 1000 nat 25 tcp from any to any # # ipfw list 00050 check-state :default 01000 nat 25 tcp from any to any 65535 deny ip from any to any #
Do not use the setup keyword on the ipfw rule referencing LSNAT. The setup keyword causes the final ACK of the TCP 3-way handshake to be never received and the connection is never established. |
From the outside host, access the web server using:
# lynx 3.3.3.3
brings up the web page on 10.10.10.10.
Figure 31. Accessing Nginx on 10.10.10.10 With LSNAT via 3.3.3.3
NAT with one address is working.
5.3.2. Engaging Multiple Hosts With LSNAT
Next, reconfigure the nat 25 instance to utilize all of the inside hosts:
# ipfw nat 25 config redirect_addr 10.10.10.10,10.10.10.20,10.10.10.30 3.3.3.3
(Note that adding a modification to a NAT instance just overwrites the existing instance. It does not create a new instance with the same number.)
On the outside host, running lynx 3.3.3.3 repeatedly retrieves the home page of each server - in round-robin fashion, without regard for any network load, or server utilization.
In the lynx browser, you can reload the current page by pressing Ctl+R. |
# ipfw nat 25 show config ipfw nat 25 config log redirect_addr 10.10.10.10,10.10.10.20,10.10.10.30 3.3.3.3 #
By adding a rule to redirect icmp traffic, both icmp and tcp will be load shared across the firewall.
# ipfw add 2000 nat 25 icmp from any to any # # ipfw list 00050 check-state :default 01000 nat 25 tcp from any to any 02000 nat 25 icmp from any to any 65535 deny ip from any to any
You can test this by running tcpdump -n -i em0 on each inside machine, and running ping -c 1 3.3.3.3 on the outside host a few times. The incoming ping will hit each inside machine in turn.
However, if you run ping 3.3.3.3, the result is that these pings hit only one server. The reason is that the aliasing engine treats ICMP differently from TCP and UDP. The aliasing engine recognized the ICMP id number, and if this number does not change, it uses the same alias. If the command ping -c 1 3.3.3.3 is used repeatedly, the ICMP id number changes, and this creates a new entry in the aliasing database resulting in redirection to a different VM.
It is common to want to balance the load across servers according to certain characteristics such as system load. This is possible - manually - by reconfiguring the NAT statement. You can add multiple instances of the same host to give that host more traffic. Consider this ruleset created with the Unix line continuation character '\' to close the space between successive IP addresses except for the last one and the alias adddress:
# ipfw nat 25 config log redirect_addr \ 10.10.10.30,\ 10.10.10.20,10.10.10.20,\ 10.10.10.10,10.10.10.10,10.10.10.10,10.10.10.10 3.3.3.3
This configuration shifts the NAT load heavily toward 10.10.10.10 and moderately toward 10.10.10.20, with 10.10.10.30 having a lot less traffic. Repeat the above single ping example above to see the result. While this works, it is a bit of a hack.
It would be better to have a range assignment feature similar to the sparse address feature already in ipfw, something like:
# ipfw nat 25 config redirect_addr 10.10.10.0/24{10,20-25,30-50} 3.3.3.3but this feature does not work with LSNAT.
However, it is possible to use the prob keyword to address load balancing. In a rule with the prob keyword, if the rule matches and the probability is "true", the action of the rule is taken and processing stops for that packet. If the rule matches, and the probability is "not true", the action is not taken, and processing continues with the next rule. You can verify this with a simple test ruleset and the ucont.sh shell rule from an external host.
03000 prob 0.200000 allow udp from any to me 5656 // set probability to 20% chance of matching 04000 count udp from any to me // count how many where not chosen by rule 3000 05000 prob 0.400000 allow udp from any to me 5656 // set probability to 40% chance of matching 06000 count udp from any to me // count how many were not chosen by 3000 and 5000 07000 prob 0.999000 allow udp from any to me 5656 // set probability to 99.9% chance of matching 08000 count udp from any to me // count how many were not chosen by all 3 rules 09000 allow udp from any to me 5656 // unconditional matching 65535 deny ip from any to any // default rule deny
Ater a run of 200 entries from sh ucont.sh 5656 1 the counts are:
03000 47 3314 prob 0.200000 allow udp from any to me 5656 04000 153 10776 count udp from any to me 05000 64 4505 prob 0.400000 allow udp from any to me 5656 06000 89 6271 count udp from any to me 07000 89 6271 prob 0.999000 allow udp from any to me 5656 08000 0 0 count udp from any to me 09000 0 0 allow udp from any to me 5656 65535 0 0 deny ip from any to any
From the above data, out of 200 packets sent from ucont.sh, 47 were matched by rule 3000, but 153 were not matched (rule 4000). Then, 64 were matched at rule 5000, but 89 were not matched. Finally, 89 where matched at rule 7000.
If you duplicate this example and find some packets hitting the default deny rule (65535), delete the host interface from the bridge and re-run the test. You are then unlikely to have any stray UDP packets hitting the default rule. |
While the above works for UDP, it does not work for TCP. The TCP 3-way handshake is broken because some packets will match, but others will not.
@@@@@@@@ NEED MORE RESEARCH HERE.
Other NAT Keywords
The other keywords in the NAT section of ipfw(8) are straightforward:
deny_in : deny incoming packets same_ports : keep the same ports after redirection reset : clear the aliasing table when the address changes reverse : reverse the direction of the NAT proxy_only : packet aliasing is not performed skip_global global tablearg
Chapter 6. IPv6 Network Address Translation (IPv6NAT)
ipfw supports both stateful and stateless IPv6 / IPv4 translation.
From the ipfw(8) man page:
Stateful translation
ipfw supports in-kernel IPv6/IPv4 network address and protocol transla-
tion. Stateful NAT64 translation allows IPv6-only clients to contact
IPv4 servers using unicast TCP, UDP or ICMP protocols. One or more IPv4
addresses assigned to a stateful NAT64 translator are shared among sev-
eral IPv6-only clients. When stateful NAT64 is used in conjunction with
DNS64, no changes are usually required in the IPv6 client or the IPv4
server. The kernel module ipfw_nat64 should be loaded or kernel should
have options IPFIREWALL_NAT64 to be able use stateful NAT64 translator.Stateful translation is suitable for deployment at the client side or at the service provider, allowing IPv6-only client hosts to reach remote IPv4-only nodes.
Stateless translation is appropriate when a NAT64 translator is used in front of IPv4-only servers to allow them to be reached by remote IPv6-only clients.
Specific requirements for these translation services are found in a collection of RFCs:
There are a couple of bugs registered for NAT64. See the following: NAT64 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255928 (NAT64 issue on 13.0)
The lab examples for all IPv6 / IPv4 translations will use two new virtual machines:
dnshost - this virtual machine runs a configured copy of BIND 9. Some experience with DNS setup with BIND 9 is helpful but not required.
v6only - this virtual machine only uses IPv6. It is not configured for IPv4 addressing at all.
Readers should have a basic understanding of IPv6 and its addressing characteristics. These resources may be helpful:
6.1. Stateful NAT64 (NAT64LSN) With DNS64
NAT64, described in RFC 6146 is one of a number of transition mechanisms that companies can take as they introduce IPv6 into their environment, or move wholesale into IPv6 locally. The idea with NAT64 is to provide a mechanism to allow an IPv6-only host to make a connection to a remote IPv4-only host. This includes the ability to do a DNS lookup on the remote host, and through the features of DNS64 (a companion transition service described in RFC 6147), translate a received IPv4 address into a special IPv6 address that provides a way to connect using the Network Address Translation variant called NAT64.
A logical view of NAT64 and DNS64 is shown in Figure 7:
Figure 32. Logical View of NAT64 and DNS64
The process works like this:
An IPv6 only host wants to access a resource from host external1.example.com which only uses IPv4. A DNS lookup for "external1.example.com" is sent to the locally configured DNS64 server. This lookup is for an "AAAA" record for the external1 VM.
The DNS64 server forwards the request to an authoritative server for "example.com".
The authoritative server returns an IPv4 address back to the DNS64 server.
The DNS64 server converts the IPv4 address into an IPv6 address using the transition service described in RFC 6147.
The IPv6 only host, receives the IPv6 address and sends a connection request (SYN) to its local IPV6 router running NAT64.
The NAT64 router converts the IPv6 packet back to IPv4 and forwards the packet to external1.example.com.
The remaining conversions between the IPv6 VM and external1 VM happen in a similar fashion.
In step 4, the DNS64 server converts the IPv4 address into IPv6 by using the "Well Known Prefix" 64:ff9b:: and encapsulating the IPv6 address into the last 4 octets of the address.
In the Figure above, "203.0.113.10" has been converted to "cb00:710a" and added as the last four octets of the new address.
Note that this is one instance of a larger selection of translation algorithms to translate an IPv4 address into an IPv6 address. In our implementation, the DNS64 server and the authoritative server are essentially merged together following the description of "Example of 'an IPv6 Network to the IPv4 Internet' Setup with DNS64 in Stub-Resolver Mode" in Section 7.2 of RFC 6147.
6.1.1. Setting Up for NAT64 / DNS64
To exercise the NAT64 capabilities of ipfw, it is first necessary to restart all lab virtual machines and reconfigure the ipfw lab.
Figure 8 shows the new configuration needed.
Figure 33. Network Setup for NAT64 and DNS64 Examples
On the FreeBSD host system, the appropriate bridge and tap setup is given by this command:
$ sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 tap7 bridge1 tap4 tap6 tap8
Start up the required virtual machines with:
$ /bin/sh runvm.sh firewall dnshost external1 v6only
As before, configure all interfaces and ensure connectivity of adjacent interfaces. The firewall VM should be set for both IPv4 forwarding and IPv6 forwarding:
# sysctl net.inet.ip.forwarding=1 # sysctl net.inet6.ip6.forwarding=1
The external1 VM’s default route should point to 203.0.113.1 and the v6only host’s default IPv6 route should point to 2001:db8:12::1 as follows:
On external1:
# route add default 203.0.113.50
On v6only:
# route -6 add default 2001:db8:12::50
RFC 5737 describes the use of the 203.0.113.0/24 network for documentation and example purposes. RFC 3849 describes the use of the 2001:db8::/32 network the same purposes. |
6.1.2. Setting Up the dnshost VM
You will have to set up the dnshost VM to provide DNS64 services. ISC’s bind9 (9.18 and above) provides this capability. Setting up bind9, while not trivial, is not impossible. You will have to install the following packages:
bind9 Use the latest supported version. The server running here is using bind 9.18.1
bind-tools Same note as above. The server here uses bind-tools-9.16.27
These packages will install a modest number of dependencies.
If you have downloaded the practice kit, see the tar file dnshost.tar for the bind9 configuration files needed. Use the following command to untar the files:
# tar xvzf dhshost_userlocaletc_namedb.tgz -C /usr/local/etc
This will install all the needed DNS files. Otherwise see the zone files in Appendix E, and try to set up DNS. Note that the files include a stub root zone. This provides a locally complete DNS setup.
Restart the named service with:
# service named restart
There should not be any errors, but if there are, track down and fix.
Test the dnshost configuration with these commands. The first lookup returns the A resource record with an IPv4 address. The second lookup returns the AAAA resource record with the DNS64 configured "Well-known Prefix" 64:ff9b that is used in this section.
root@dnshost:~ # dig @localhost external1.example.com ; <<>> DiG 9.16.27 <<>> @localhost external1.example.com ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61764 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ; COOKIE: 96f573e1a62ce4380100000062884cacf9b86d6e9f54b542 (good) ;; QUESTION SECTION: ;external1.example.com. IN A ;; ANSWER SECTION: external1.example.com. 3600 IN A 203.0.113.10 ;; Query time: 58 msec ;; SERVER: ::1#53(::1) ;; WHEN: Fri May 20 22:21:32 EDT 2022 ;; MSG SIZE rcvd: 94 # # root@dnshost:~ # dig @localhost external1.example.com aaaa ; <<>> DiG 9.16.27 <<>> @localhost external1.example.com aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5865 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ; COOKIE: 0ad6c60faab12f7f0100000062884cb0be6e6bd00f7e482f (good) ;; QUESTION SECTION: ;external1.example.com. IN AAAA ;; ANSWER SECTION: external1.example.com. 3600 IN AAAA 64:ff9b::cb00:710a #
The last test is the most important. The dnshost must return the Well-Known Prefix "64:ff9b::" with the coresponding bits for the embedded IPv4 address. If the test does not return this value, reconfigure the DNS service (named.conf and the primary forward zone "example.com") to fix.
Since the v6only machine will only ever request AAAA lookups, the configuration is complete for this section.
We now have to set up the firewall, the IPV6 only host v6only, and the external VM host, external1. Boot all three VMs and test connectivity without any ipfw running on the firewall host.
We now proceed with the installation of NAT64 on the ipfw firewall.
On the firewall VM:
# kldload ipfw The next line loads the NAT64 module. # kldload ipfw_nat64
Configuring NAT64 is similar to configuring NAT. You must create an instance of the NAT64 translator first.
# ipfw nat64lsn foo create prefix4 203.0.112.0/24 allow_private
The use of the "allow_private" keyword is required. The ipfw(8) manual page notes that the NAT64 translator, by defualt, will not handle addresses whose destination matches those listed in RFC 1918. The addressing scheme in this lab uses special purpose addresses as noted in RFC 68901 which are also considered "private addresses" by the ipfw NAT64 translator.
Note that the prefix4 address pool (203.0.112.0/24 above) should not be manually configured as an alias on any interface. These addresses are used internally by ipfw. The only requirement is that they be reserved from deployment elsewhere in the local network so they do not cause a routing conflict with ipfw. This allows for 254 simultaneous NAT64 addresses. If more are needed due to high volume, add another prefix4, or increase the existing prefix4 address space.
Continue configuring the NAT64 / DNS64 translator:
# ipfw add allow log ipv6-icmp from any to any icmp6types 135,136 # ipfw add nat64lsn foo log ip from 2001:db8:12::/64 to 64:ff9b::/96 in # ipfw add nat64lsn foo log ip from any to 203.0.112.0/24 in # ipfw add allow log ip from any to any
and the direct_output sysctl must be set to 1 (not zero):
# sysctl net.inet.ip.fw.nat64_direct_output=1
You can also set the nat64_debug sysctl and the firewall verbose sysctl:
# sysctl net.inet.ip.fw.nat64_debug=1 # sysctl net.inet.ip.fw.verbose=1
See /var/log/security for output.
With these prerequisites completed the following tests on the v6only VM should be successful:
root@v6only# ping6 -c 3 64:ff9b::203.0.113.10 PING6(56=40+8+8 bytes) 2001:db8:12::30 --> 64:ff9b::cb00:710a 16 bytes from 64:ff9b::cb00:710a, icmp_seq=0 hlim=63 time=8.401 ms 16 bytes from 64:ff9b::cb00:710a, icmp_seq=1 hlim=63 time=3.429 ms 16 bytes from 64:ff9b::cb00:710a, icmp_seq=2 hlim=63 time=3.398 ms --- 64:ff9b::203.0.113.10 ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 3.398/5.076/8.401/2.351 ms
and using lynx to grab the external1.example.com home page should be successful:
[root@v6only ~]# lynx external1.example.com
Figure 34. Viewing the IPv4 Webpage of external1 VM from IPv6 v6only VM.
6.1.3. Setting Up for Stateless NAT64 - NAT64STL
The previous ipfw_nat64 examples used "stateful" address translation. ipfw is also capable of performing "stateless" address translation.
Stateless translation is appropriate when a NAT64 translator is used in front of IPv4-only servers to allow them to be reached by remote IPv6-only clients. Stateful translation is suitable for deployment at the client side or at the service provider, allowing IPv6-only client hosts to reach remote IPv4-only nodes.
Stateless configuration of NAT64 is possible with the same architecture as the previous stateful example. Configuration details however, are different. In the stateless case, ipfw uses two tables for translating addresses in either direction: IPv4 → IPv6 and IPv6 → IPv4. A typical configuration is shown below.
Start fresh # kldunload ipfw_nat64 # kldunload ipfw # kldload ipfw # kldload ipfw_nat64 Create the tables used for ipfw_nat64stl # ipfw table T4to6 create type addr valtype ipv6 # ipfw table T6to4 create type addr valtype ipv4 # ipfw table T4to6 add 203.0.112.1 2001:db8:12::6 # ipfw table T6to4 add 2001:db8:12::6 203.0.112.1 # ipfw nat64stl NAT64 create table4 T4to6 table6 T6to4 allow_private Add rules for ipfw_nat64stl # ipfw add allow log icmp6 from any to any icmp6types 135, 136 # ipfw add nat64stl NAT64 log ip from any to 'table(T4to6)' # ipfw add nat64stl NAT64 log ip6 from 'table(T6to4)' to 64:ff9b::/96 # ipfw add allow log ip from any to any Adjust sysctls # sysctl net.inet.ip.fw.verbose=1 # sysctl net.inet.ip.fw.nat64_debug=1 # sysctl net.inet.ip.fw.nat64_direct_output=1
Only the net.inet.ip.fw.nat64_direct_output sysctl is required.
Use the same tests as in the stateful NAT64 example:
[root@v6only ~]# ping6 -c 3 64:ff9b::203.0.113.10 and [root@v6only ~]# lynx external1.example.com
Both tests should be successful.
It may seem limiting to have to use tables to effect communication for stateless NAT64. However, if you look at the architecture involved, the above statements about stateless translation being appropriate when a NAT64 translator is used in front of IPv4-only servers to allow them to be reached by remote IPv6-only clients makes sense. The entire IPv6 cloud can reach a specified server.
This can be accomplished by, for example, changing the T4to6 and T6to4 tables to read:
# ipfw table T4to6 add 203.0.112.0/31 2001:db8:12::30 # ipfw table T6to4 add 2000:0000:0000::/8 203.0.112.0 # ipfw table T6to4 add 2100:0000:0000::/8 203.0.112.1
The T4to6 table allocates two addresses in the address pool: 203.0.112.0 and 203.0.112.1. These are used separately in the T6to4 table to cover vast ranges of IPv6 address space.
Certainly using just one IPv4 pool address is not going to be sufficient to translate such a large range of IPv6 addresses. The point here is that by carefully constructing the translation pool addresses and the T4to6 and T6to4 address tables, ipfw can manage translation to as many IPv6 addresses as needed.
Note that stateless NAT64 shares the same limitations of stateful NAT64.
We move on to the next, and most important IPv6 / IPv4 translation mechanism NAT64 CLAT.
6.2. XLAT464
ipfw supports 464XLAT (RFC 6877) calling it "XLAT464 CLAT". This transition mechanism provides connectivity for IPv4 edge devices across an IPv6 only network. It does this by combining stateful translation in the core and stateless translation at the edge. 464XLAT only suports IPv4 in the client-server model, so it does not support IPv4 peer-to-peer communication or inbound IPv4 connections.
See diagrams and explanation here: https://www.juniper.net/documentation/us/en/software/junos/interfaces-adaptive-services/topics/topic-map/ipv4-connect-ipv6-464xlat.html#id-464xlat-overview The RFC for this mechanism is more enlightening. See RFC 6877 |
The discussion on Wikipedia1 is somewhat sparse:
464XLAT 464XLAT (RFC 6877) allows clients on IPv6-only networks to access IPv4-only Internet services, such as Skype.[13][14] The client uses a SIIT translator to convert packets from IPv4 to IPv6. These are then sent to a NAT64 translator which translates them from IPv6 back into IPv4 and on to an IPv4-only server. The client translator may be implemented on the client itself or on an intermediate device and is known as the CLAT (Customer-side transLATor). The NAT64 translator, or PLAT (Provider-side transLATor), must be able to reach both the server and the client (through the CLAT). The use of NAT64 limits connections to a client-server model using UDP, TCP, and ICMP.
Figure @@@ shows a diagram for implementing 464XLAT in our lab.
Figure 35. ipfw 464XLAT Design
As earlier, shutdown all virtual machines and for this example we will reset all the bridge and tap devices to the new architecture.
This example will require two firewalls. The firewall VM and firewall2 VM will both be used as shown in the diagram. In this example, the firewall VM is the CLAT translator (stateless translation) and the firewall2 VM is the PLAT translator (stateful translation).
To start, set up the bridge and tap interfaces with this command on the FreeBSD host:
$ sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 tap11 bridge1 tap4 tap6 tap7 tap9 \ bridge2 tap2 tap8 tap10
Start up the virtual machines with:
$ /bin/sh runvm.sh external1 firewall firewall2 dnshost v6only external2
As earlier, configure all interfaces according to the diagram and ensure connectivity with adjacent interfaces.
This example is more complex than past examples. There are a number of additional configuration steps needed as follows:
external1.example.com
On external1.example.com: route add default 192.168.1.1 echo "nameserver 192.168.1.53" > /etc/resolv.conf echo "nameserver 203.0.113.53" >> /etc/resolv.conf
firewall.example.com
On firewall.example.com: /bin/sh /root/bin/bsdclat464.sh echo "nameserver 2001:db8:12::53" > /etc/resolv.conf echo "nameserver 192.168.1.53" >> /etc/resolv.conf route -6 add 2001:db8:bbbb::/96 2001:db8:12::1 sysctl net.inet.ip.forwarding=1 sysctl net.inet6.ip6.forwarding=1 sysctl net.inet.ip.fw.nat64_direct_output=1
firewall2.example.com
On firewall2.example.com /bin/sh /root/bin/bsdplat464.sh route -6 add 2001:db8:aaaa::/96 2001:db8:12::2 echo "nameserver 2001:db8:12::53" > /etc/resolv.conf sysctl net.inet.ip.forwarding=1 sysctl net.inet6.ip6.forwarding=1 sysctl net.inet.ip.fw.nat64_direct_output=1
external2.example.com
On external2.example.com route add default 203.0.113.1 echo "nameserver 203.0.113.53" > /etc/resolv.conf service nginx onestart
dnshost.example.com
On dnshost.example.com: echo "nameserver 127.0.0.1" > /etc/resolv.conf echo "nameserver 2001:db8:12::53" >> /etc/resolv.conf service named onestart route add default 203.0.113.1 route add -net 192.168.1.0/24 192.168.1.1 sysctl net.inet.ip.forwarding=0 sysctl net.inet6.ip6.forwarding=0
v6only.example.com
On v6only.example.com: echo "nameserver 2001:db8:12::53" > /etc/resolv.conf route -6 add 2001:db8:bbbb::/96 2001:db8:12::1 route -6 add 2001:db8:aaaa::/96 2001:db8:12::2
Due to the complex nature of these firewall configurations, a script is available for copy and paste. However, do try to understand the configration details.
Once all the above commands are entered on their respective VMs, test the configuration with the following command:
# ping -c 2 external2.example.com
The snippets below show at each step, how the request was transformed.
The examples below are taken from multiple different invocations of the ping command. However, the data transformations are correct. |
On interface em0 on the firewall VM: root@firewall:~/bin # tcpdump -n -i em0 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 14:38:07.254114 IP 192.168.1.2 > 203.0.113.20: ICMP echo request, id 46395, seq 0, length 64 14:38:07.256893 IP 203.0.113.20 > 192.168.1.2: ICMP echo reply, id 46395, seq 0, length 64 14:38:08.322597 IP 192.168.1.2 > 203.0.113.20: ICMP echo request, id 46395, seq 1, length 64 14:38:08.326715 IP 203.0.113.20 > 192.168.1.2: ICMP echo reply, id 46395, seq 1, length 64 On interface em1 on the firewall VM: root@firewall:~/bin # tcpdump -n -i em1 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on em1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 14:54:45.140746 IP6 2001:db8:aaaa::c0a8:102 > 2001:db8:bbbb::cb00:7114: ICMP6, echo request, id 38233, seq 0, length 64 14:54:45.142995 IP6 2001:db8:bbbb::cb00:7114 > 2001:db8:aaaa::c0a8:102: ICMP6, echo reply, id 38233, seq 0, length 64 14:54:46.171754 IP6 2001:db8:aaaa::c0a8:102 > 2001:db8:bbbb::cb00:7114: ICMP6, echo request, id 38233, seq 1, length 64 14:54:46.173925 IP6 2001:db8:bbbb::cb00:7114 > 2001:db8:aaaa::c0a8:102: ICMP6, echo reply, id 38233, seq 1, length 64 On interface em0 on the firewall2 VM: root@firewall2:~/bin # tcpdump -n -i em0 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 14:57:32.519334 IP6 2001:db8:aaaa::c0a8:102 > 2001:db8:bbbb::cb00:7114: ICMP6, echo request, id 17270, seq 0, length 64 14:57:32.529066 IP6 2001:db8:bbbb::cb00:7114 > 2001:db8:aaaa::c0a8:102: ICMP6, echo reply, id 17270, seq 0, length 64 14:57:33.560392 IP6 2001:db8:aaaa::c0a8:102 > 2001:db8:bbbb::cb00:7114: ICMP6, echo request, id 17270, seq 1, length 64 14:57:33.561596 IP6 2001:db8:bbbb::cb00:7114 > 2001:db8:aaaa::c0a8:102: ICMP6, echo reply, id 17270, seq 1, length 64 On interface em1 on the firewall2 VM: root@firewall2:~/bin # tcpdump -n -i em1 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on em1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 14:58:37.139612 IP 203.0.112.22 > 203.0.113.20: ICMP echo request, id 1025, seq 0, length 64 14:58:37.141043 IP 203.0.113.20 > 203.0.112.22: ICMP echo reply, id 1025, seq 0, length 64 14:58:38.187477 IP 203.0.112.22 > 203.0.113.20: ICMP echo request, id 1025, seq 1, length 64 14:58:38.188308 IP 203.0.113.20 > 203.0.112.22: ICMP echo reply, id 1025, seq 1, length 64 On interfae em0 on the external2 VM: root@external2:~ # tcpdump -n -i em0 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 15:00:44.171439 IP 203.0.112.22 > 203.0.113.20: ICMP echo request, id 1024, seq 0, length 64 15:00:44.172313 IP 203.0.113.20 > 203.0.112.22: ICMP echo reply, id 1024, seq 0, length 64 15:00:45.200883 IP 203.0.112.22 > 203.0.113.20: ICMP echo request, id 1024, seq 1, length 64 15:00:45.201035 IP 203.0.113.20 > 203.0.112.22: ICMP echo reply, id 1024, seq 1, length 64
The firewall log sysctl was reset to log to syslogd(8) and captured these logs:
Logs on the firewall VM: root@firewall:~/bin # cat /var/log/security Dec 2 15:14:04 firewall kernel: ipfw: 150 Eaction nat64clat ICMP:8.0 192.168.1.2 203.0.113.20 in via em0 Dec 2 15:14:04 firewall kernel: ipfw: 150 Eaction nat64clat ICMPv6:129.0 [2001:db8:bbbb::cb00:7114] [2001:db8:aaaa::c0a8:102] in via em1 Dec 2 15:14:04 firewall kernel: ipfw: 150 Eaction nat64clat ICMP:8.0 192.168.1.2 203.0.113.20 in via em0 Dec 2 15:14:05 firewall kernel: ipfw: 150 Eaction nat64clat ICMPv6:129.0 [2001:db8:bbbb::cb00:7114] [2001:db8:aaaa::c0a8:102] in via em1 Dec 2 15:14:08 firewall kernel: ipfw: 100 Accept ICMPv6:135.0 [2001:db8:12::1] [2001:db8:12::2] in via em1 Dec 2 15:14:08 firewall kernel: ipfw: 100 Accept ICMPv6:136.0 [2001:db8:12::2] [2001:db8:12::1] out via em1 Dec 2 15:14:09 firewall kernel: ipfw: 100 Accept ICMPv6:135.0 [2001:db8:12::2] [2001:db8:12::1] out via em1 Dec 2 15:14:09 firewall kernel: ipfw: 100 Accept ICMPv6:136.0 [2001:db8:12::1] [2001:db8:12::2] in via em1 Logs on the firewall2 VM: root@firewall2:~/bin # cat /var/log/security Dec 2 15:10:25 firewall2 kernel: ipfw: 300 Eaction nat64lsn ICMPv6:128.0 [2001:db8:aaaa::c0a8:102] [2001:db8:bbbb::cb00:7114] in via em0 Dec 2 15:10:25 firewall2 kernel: ipfw: 400 Eaction nat64lsn ICMP:0.0 203.0.113.20 203.0.112.22 in via em1 Dec 2 15:10:26 firewall2 kernel: ipfw: 300 Eaction nat64lsn ICMPv6:128.0 [2001:db8:aaaa::c0a8:102] [2001:db8:bbbb::cb00:7114] in via em0 Dec 2 15:10:26 firewall2 kernel: ipfw: 400 Eaction nat64lsn ICMP:0.0 203.0.113.20 203.0.112.22 in via em1 Dec 2 15:10:29 firewall2 kernel: ipfw: 100 Accept ICMPv6:135.0 [2001:db8:12::1] [2001:db8:12::2] out via em0 Dec 2 15:10:29 firewall2 kernel: ipfw: 100 Accept ICMPv6:136.0 [2001:db8:12::2] [2001:db8:12::1] in via em0 Dec 2 15:10:30 firewall2 kernel: ipfw: 100 Accept ICMPv6:135.0 [2001:db8:12::2] [2001:db8:12::1] in via em0 Dec 2 15:10:30 firewall2 kernel: ipfw: 100 Accept ICMPv6:136.0 [2001:db8:12::1] [2001:db8:12::2] out via em0
Finally, a webpage request was made with:
# lynx external2.example.com
as shown below:
Figure 36. Retrieving the Webpage at external2.example.com
Chapter 7. See Also
IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also
IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also
IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also
IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also IPFW See Also
Appendix A: Appendix A: QEMU Setup
This appendix contains helpful information for getting QEMU installed on FreeBSD and instructions for common use.
QEMU can be invoked to display a vt(4) style virtual terminal console using SDL.
Because of this the user environment, either X Window or Wayland, must allow for use of the DISPLAY variable in the local environment.
It can also be invoked in -nographics mode, resulting is the current screen or window immediately becoming the console for the QEMU instance.
The examples in this book utilize the SDL based vt(4) based console and also include a using a FreeBSD serial console (described Adding a Serial Console).
Additional resources for understanding and using QEMU include:
The QEMU Virtualization chapter in the FreeBSD Handbook
The qemu(1) manual page
The QEMU Home Page
QEMU is available as a package or a port. There are a large number of build options on the port, so in most cases it is best to install the package:
# pkg install qemu sudo
Perform the following steps to get a QEMU virtual machine started. The Quick Start section in the Introduction has a suggested layout.
Designate a destination directory for the virtual machine(s). This procedure is an example that does not use the directory layout in this book. We will call this directory QVM.
Create the image files that will become the QEMU virtual machines. Use qemu-img(1) to create the images. Using the
qcow2format is recommended as the scripts in this book use that format.# cd QVM # qemu-img create -f qcow2 -o preallocation=full MyQemuVM 4G
Download or copy a FreeBSD installation DVD into the QVM directory and rename it to fbsd.iso. You can also just link that name to an existing DVD.
Set up networking You can use QEMU without any networking, with an internal legacy based SLiRP protocol that uses an internal DHCP server, or you can use the FreeBSD tap(4) and if_bridge(4) interfaces to connect to your local network. In this example, we will install without a network interface, and add it later.
Once the image file is complete, you can enter these commands to start up the virtual machine:
/usr/local/bin/qemu-system-x86_64 -monitor stdio \ -cpu qemu64 \ -vga std \ -m 4096 \ -smp 2 \ -cdrom fbsd.iso \ -boot order=cd,menu=on \ -blockdev driver=file,aio=threads,node-name=fbsdimg,filename=MyQemuVM.qcow2 \ -blockdev driver=qcow2,node-name=drive0,file=fbsdimg \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -name \"FreeBSD\"
The virtual machine should start in a console window and load the FreeBSD installation DVD. Install the system as usual, but select UFS as the filesystem. If you want to use ZFS, increase the amount of memory (
-m parameter).To add networking, configure a tap(4) device as
tap0, and an if_bridge(4) device asbridge0as shown in the Quick Start section. You can add thetap0andbridge0devices to the above configuration with these two lines (add above the-nameline).-netdev tap,id=nd0,ifname=tap0,script=no,downscript=no,br=bridge0 \ -device e1000,netdev=nd0,mac=02:55:33:12:34:56 \
If you want the virtual machine to use DHCP on your local network, add your host interface (em0, bge0, etc.) to the bridge. For example, to add an
em0host interface use:# ifconfig bridge0 addm em0
If you are having trouble, check the other resources noted above. The given examples should work.
Appendix B: Appendix B: Scripts and Code for QEMU Lab
The listing below shows how the scripts are organized on the GitHub ipfw-primer/SCRIPTS site.
.
|-- VM_SCRIPTS
| |-- IPFW_root_bin.tgz : common scripts for all VMs (see bin list)
| |-- Manifest_IPFW_root_bin.txt : Manifest document for IPFW_root_bin.tgz
| |-- Manifest_index.txt : Manifest document for index.html files
|-- bin
| |-- tcon.sh : TCP connection script
| |-- tconr.sh : TCP connect with random port script
| |-- tcont.sh : TCP continuous connection script
| |-- tserv.sh : TCP server script for one port
| |-- tserv3.sh : TCP server script for three ports
| |-- ucon.sh : UDP connection script
| |-- uconr.sh : UDP connect with random port script
| |-- ucont.sh : UDP continuous connection script
| |-- userv.sh : UDP server script for one port
| |-- userv3.sh : UDP server script for three ports
| `-- userv5.sh : UDP server script for five ports
| |-- dnshost :
| | |-- Manifest_namedb.txt : Manifest for dnshost /usr/local/etc/namedb
| | |-- dnshost_usrlocaletc_namedb.tgz : Files for the above
| | `-- index.html : Nginx index.html file for dnshost VM
| |-- external1 :
| | `-- index.html : Nginx index.html file for external1 VM
| |-- external2 :
| | `-- index.html : Nginx index.html file for external2 VM
| |-- external3 :
| | `-- index.html : Nginx index.html file for external3 VM
| |-- firewall :
| | |-- bsdclat464.sh : Script for Section 6.2 XLAT464 CLAT
| | `-- index.html : Nginx index.html file for firewall VM
| |-- firewall2 :
| | |-- bsdplat464.sh : Script for Section 6.2 XLAT464 CLAT
| | `-- index.html : Nginx index.html file for firewall2 VM
| |-- internal :
| | `-- index.html : Nginx index.html file for internal VM
| `-- v6only :
| `-- index.html : Nginx index.html file for v6only VM
|-- _CreateAllVMs.sh : Script to create all VMs used in book
|-- dnshost.sh : QEMU startup script for dnshost VM
|-- external1.sh : QEMU startup script for external1 VM
|-- external2.sh : QEMU startup script for external2 VM
|-- external3.sh : QEMU startup script for external3 VM
|-- firewall.sh : QEMU startup script for firewall VM
|-- firewall2.sh : QEMU startup script for firewall2 VM
|-- internal.sh : QEMU startup script for internal VM
|-- mkbr.sh : Script to make host bridge and tap devices
|-- runvm.sh : XFCE4 script to start VMs
|-- v6only.sh : QEMU startup script for v6only VM
|-- vm_envs.sh : IPFW lab environment variables
`-- CODE
`-- divert.c : C code for working with divert keywordAll scripts are shown below in lexicographic order:
SCRIPT: VM_SCRIPTS/firewall/bsdclat464.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD 464XLAT CLAT script for firewall VM.
#
# bsdclat464.sh: FreeBSD IPFW script for 464XLAT CLAT. See Section 6.2
# Usage: # /bin/sh bsdclat464.sh (run script as root)
set -x
kldunload ipfw_nat64
kldunload ipfw
sleep 1
kldload ipfw
kldload ipfw_nat64
# Create the nat64clat instance
ipfw nat64clat CLAT create clat_prefix 2001:db8:aaaa::/96 plat_prefix 2001:db8:bbbb::/96 allow_private log
# Allow neighbor discovery
ipfw add 100 allow log icmp6 from any to any icmp6types 135,136
# pass any ip through the nat64clat instance
ipfw add 150 nat64clat CLAT log ip from any to any
# pass any ip through the nat64plat instance
ipfw add 200 nat64clat CLAT log ip from any to 2001:db8:bbbb::/96
# allow ipv6 from any to any
ipfw add 300 allow log ip6 from any to any
# allow ipv4 from any to any
ipfw add 400 allow log ip from any to any
# 0=log with ipfwlog0, 1=log with syslog
sysctl net.inet.ip.fw.verbose=0
sysctl net.inet.ip.fw.nat64_debug=1
# direct output: 1 enable, 0 disable (packet goes back into ruleset)
sysctl net.inet.ip.fw.nat64_direct_output=1
==========================================================
SCRIPT: VM_SCRIPTS/firewall2/bsdplat464.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD 464XLAT PLAT script for firewall2 VM.
#
# bsdplat464.sh: FreeBSD IPFW script for 464XLAT CLAT. See Section 6.2
# Usage: # /bin/sh bsdplat464.sh (run script as root)
set -x
kldunload ipfw_nat64
kldunload ipfw
sleep 1
kldload ipfw
kldload ipfw_nat64
# create the nat64 stateful instance
ipfw nat64lsn NAT64 create log prefix4 203.0.112.0/24 prefix6 2001:db8:bbbb::/96 allow_private
# Allow neighbor discovery
ipfw add allow log icmp6 from any to any icmp6types 135,136
# Allow the nat64 outbound
ipfw add nat64lsn NAT64 log ip from 2001:db8:12::/64 to 2001:db8:bbbb::/96 in
ipfw add nat64lsn NAT64 log ip from any to 2001:db8:bbbb::/96 in
# Allow the nat64 inbound
ipfw add nat64lsn NAT64 log ip from any to 203.0.112.0/24 in
# Allow ipv4 from any to any
ipfw add allow log ip from any to any
# Allow ipv6 from any to any
ipfw add allow log ip6 from any to any
# Logging: 0 interfaces, 1 syslog
sysctl net.inet.ip.fw.verbose=0
# Debug nat64
sysctl net.inet.ip.fw.nat64_debug=1
# Direct output: 1 enable, 0 disable (packet goes back into ruleset)
sysctl net.inet.ip.fw.nat64_direct_output=1
==========================================================
SCRIPT: _CreateAllVMs.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# _CreateAllVM.sh : Create VMs for the IPFW Primer lab. Files are created in ../VM/
#
echo "Running _CreateAllVMs.sh"
echo
echo "This script will create 8 virtual machines in ../VM/"
echo
read -p "DO YOU REALLY WANT TO CREATE NEW QEMU IMAGES OVERWRITING ANY EXISTING IMAGES? Answer YES to continue. " junk
echo [${junk}]
if [ "X${junk}" != "XYES" ]
then
echo "Response was [${junk}]"
echo "bailing out..."
exit 1
fi
echo "Response was [${junk}]"
echo "Ok, continuing..."
#exit
for i in dnshost external1 external2 external3 firewall firewall2 internal v6only
do
echo "Creating ${i} VM"
echo qemu-img create -f qcow2 -o preallocation=full ../VM/${i}.qcow2 4G
qemu-img create -f qcow2 -o preallocation=full ../VM/${i}.qcow2 4G
done
echo
echo "Done."
==========================================================
SCRIPT: tcon.sh
#!/bin/sh
#
# location: external Vms
#
# sh tcon.sh PORTNUM - start up 1 connection over TCP
#
usage() {
echo "sh tcon.sh PORTNUM"
exit 1
}
#echo $#
if [ $# -ne 1 ]
then
usage
else
export PORT1=$1
fi
# echo "PORT1 = [$PORT1]"
export CONN="203.0.113.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export MYNAME="external1"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
export PREVIOUS_PORT=$PORT1
while :
do
COUNT=expr $COUNT + 1
read -p "ncat [$COUNT] ready. Enter a valid PORTNUM: " PORT1
if [ "X$PORT1" = "X" ]
then
PORT1=$PREVIOUS_PORT
fi
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
if [ $? -ne 0 ]
then
echo "TCP connection [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
PREVIOUS_PORT=$PORT1
done
==========================================================
SCRIPT: tconr.sh
#!/bin/sh
#
# location: external Vms
#
# sh tconr.sh PORTNUM SLEEPVAL (randomized port numbers) - start up 1 connection over TCP
#
usage() {
echo "sh tconr.sh PORT1NUM SLEEPVAL (randomized port numbers)"
exit 1
}
# echo $#
if [ $# -ne 2 ]
then
usage
fi
PORT1=$1
SLEEPVAL=$2
echo "PORT1 = [$PORT1]"
echo "SLEEPVAL = [$SLEEPVAL]"
export CONN="203.0.113.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export MYNAME="external1"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
while :
do
COUNT=expr $COUNT + 1
# use jot(1) to get a random port between 5656 and 5659.
# Connection to 5659 has no listener on firewall and will thus fail.
PORT1=jot -r 1 5656 5659 $RANDOM
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
if [ $? -ne 0 ]
then
echo "TCP connection [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
sleep $SLEEPVAL
done
==========================================================
SCRIPT: tcont.sh
#!/bin/sh
#
# location: external Vms
#
# sh tcont.sh PORT1NUM SLEEPVAL - keep hammering same TCP port every SLEEPVAL
#
usage() {
echo "sh tcont.sh PORT1NUM SLEEPVAL"
exit 1
}
#echo $#
if [ $# -ne 2 ]
then
usage
fi
export PORT1=$1
export SLEEPVAL=$2
echo "PORT1 = [$PORT1]"
echo "SLEEPVAL = [$SLEEPVAL]"
export CONN="203.0.113.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
while :
do
COUNT=expr $COUNT + 1
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"
echo "TCP connection from [$MYIP],[$PORT1],[$COUNT]"| ncat $CONN $PORT1
if [ $? -ne 0 ]
then
echo "TCP connection [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
sleep $SLEEPVAL
done
==========================================================
SCRIPT: tserv.sh
#!/bin/sh
#
# location: firewall VMs
#
# tserv.sh - start up 1 listener over TCP
zapall() {
kill -TERM $PID1
}
trap zapall SIGINT
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export PORT1=5656
echo "Starting TCP listener on [$PORT1]"
ncat -l -4 -k $MYIP $PORT1 &
PID1=$!
wait
exit
==========================================================
SCRIPT: tserv3.sh
#!/bin/sh
#
# location: firewall VMs
#
# tserv3.sh - start up 3 listeners over TCP
zapall() {
kill -TERM $PID1 $PID2 $PID3
}
trap zapall SIGINT
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export PORT1=5656
export PORT2=5657
export PORT3=5658
echo "Starting TCP listeners on [$PORT1],[$PORT2],[$PORT3]"
ncat -l -4 -k $MYIP $PORT1 &
PID1=$!
ncat -l -4 -k $MYIP $PORT2 &
PID2=$!
ncat -l -4 -k $MYIP $PORT3 &
PID3=$!
wait
exit
==========================================================
SCRIPT: ucon.sh
#!/bin/sh
#
# location: external Vms
#
# sh ucon.sh PORTNUM - start up 1 transfer over UDP
#
usage() {
echo "sh ucon.sh PORTNUM"
exit 1
}
#echo $#
if [ $# -ne 1 ]
then
usage
else
export PORT1=$1
fi
# echo "PORT1 = [$PORT1]"
export CONN="203.0.113.50"
# export CONN="10.10.10.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export MYNAME="external1"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
export PREVIOUS_PORT=$PORT1
while :
do
COUNT=expr $COUNT + 1
read -p "ncat [$COUNT] ready. Enter a valid PORTNUM: " PORT1
if [ "X$PORT1" = "X" ]
then
PORT1=$PREVIOUS_PORT
fi
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
if [ $? -ne 0 ]
then
echo "UDP packet [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
PREVIOUS_PORT=$PORT1
done
==========================================================
SCRIPT: uconr.sh
#!/bin/sh
#
# location: external Vms
#
usage() {
echo "sh uconr.sh PORT1NUM SLEEPVAL (randomized port numbers)"
exit 1
}
# echo $#
if [ $# -ne 2 ]
then
usage
fi
PORT1=$1
SLEEPVAL=$2
echo "PORT1 = [$PORT1]"
echo "SLEEPVAL = [$SLEEPVAL]"
export CONN="203.0.113.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export MYNAME="external1"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
while :
do
COUNT=expr $COUNT + 1
# use jot(1) to get a random port between 5656 and 5659.
# Packet on 5659 has no listener on firewall and will thus fail.
PORT1=jot -r 1 5656 5659 $RANDOM
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
if [ $? -ne 0 ]
then
echo "UDP packet [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
sleep $SLEEPVAL
done
==========================================================
SCRIPT: ucont.sh
#!/bin/sh
#
# location: external Vms
#
# sh ucont.sh PORT1NUM SLEEPVAL - keep hammering same UDP port every SLEEPVAL
#
usage() {
echo "sh ucont.sh PORT1NUM SLEEPVAL"
exit 1
}
#echo $#
if [ $# -ne 2 ]
then
usage
fi
export PORT1=$1
export SLEEPVAL=$2
echo "PORT1 = [$PORT1]"
echo "SLEEPVAL = [$SLEEPVAL]"
# export CONN="10.10.10.50"
export CONN="203.0.113.50"
export COUNT=1
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
while :
do
COUNT=expr $COUNT + 1
# echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
# echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"
echo "UDP packet from [$MYIP],[$PORT1],[$COUNT]"| ncat -u $CONN $PORT1
if [ $? -ne 0 ]
then
echo "UDP packet [$MYIP],[$PORT1],[$COUNT] FAILED"
fi
sleep $SLEEPVAL
done
==========================================================
SCRIPT: userv.sh
#!/bin/sh
#
# location: firewall VMs
#
# userv.sh PORTNUM - start up 1 listener over UDP
#
usage() {
echo "sh userv.sh PORTNUM"
exit 1
}
#echo $#
if [ $# -ne 1 ]
then
usage
else
PORT1=$1
fi
echo "PORT1 = [$PORT1]"
zapall() {
kill -TERM $PID1
}
trap zapall SIGINT
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
echo "Starting UDP listener on [$MYIP],[$PORT1]"
# echo nc -l -k -u $MYIP $PORT1
nc -l -k -u $MYIP $PORT1 &
PID1=$!
wait
exit
==========================================================
SCRIPT: userv3.sh
#!/bin/sh
#
# location: firewall VMs
#
# userv3.sh - start up 3 listeners over udp
zapall() {
kill -TERM $PID1 $PID2 $PID3
}
trap zapall SIGINT
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export PORT1=5656
export PORT2=5657
export PORT3=5658
echo "Starting UDP listeners on [$PORT1],[$PORT2],[$PORT3]"
nc -l -k -u $MYIP $PORT1 &
PID1=$!
nc -l -k -u $MYIP $PORT2 &
PID2=$!
nc -l -k -u $MYIP $PORT3 &
PID3=$!
wait
exit
==========================================================
SCRIPT: userv5.sh
#!/bin/sh
#
# location: firewall VMs
#
# userv5.sh - start up 5 listeners over udp
zapall() {
kill -TERM $PID1 $PID2 $PID3 $PID4 $PID5
}
trap zapall SIGINT
export MYIP=ifconfig em0 | grep inet | grep -v inet6 | awk '{print $2}'
export PORT1=5656
export PORT2=5657
export PORT3=5658
export PORT4=5659
export PORT5=5660
echo "Starting UDP listeners on [$PORT1],[$PORT2],[$PORT3],[$PORT4],[$PORT5]"
nc -l -k -u $MYIP $PORT1 &
PID1=$!
nc -l -k -u $MYIP $PORT2 &
PID2=$!
nc -l -k -u $MYIP $PORT3 &cd
PID3=$!
nc -l -k -u $MYIP $PORT4 &
PID4=$!
nc -l -k -u $MYIP $PORT5 &
PID5=$!
wait
exit
==========================================================
SCRIPT: dnshost.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for dnshost VM.
#
# dnshost.sh: FreeBSD QEMU VM startup script for dnshost VM.
# Usage: sudo /bin/sh dnshost.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_DNSHOST_ISO}]
echo [mem=${_DNSHOST_mem}]
echo [hdsize=${_DNSHOST_hdsize}]
echo [img=${_DNSHOST_img}]
echo [mac1=${_DNSHOST_mac1}]
echo [mac2=${_DNSHOST_mac2}]
echo [name=${_DNSHOST_name}]
echo [tap7=${_DNSHOST_tap7}]
echo [tap8=${_DNSHOST_tap8}]
echo [tap11=${_DNSHOST_tap11}]
echo [telnetport=${_DNSHOST_telnetport}]
#exit
# Note - the dnshost has two interfaces - em0 and em1.
# em0 is considered the ipv4 interface and
# em1 is considered the ipv6 interface.
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_DNSHOST_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor stdio \
-serial telnet:localhost:${_DNSHOST_telnetport},server=on,wait=on \
-cpu qemu64 \
-vga cirrus \
-m ${_DNSHOST_mem} \
-cdrom ${_DNSHOST_ISO} \
-boot order=cd,menu=on,splash=${_DNS_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_DNSHOST_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_DNSHOST_tap7},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_DNSHOST_mac1} \
-netdev tap,id=nd1,ifname=${_DNSHOST_tap8},script=no,downscript=no \
-device e1000,netdev=nd1,mac=${_DNSHOST_mac2} \
-netdev tap,id=nd2,ifname=${_DNSHOST_tap11},script=no,downscript=no \
-device e1000,netdev=nd2,mac=${_DNSHOST_mac3} \
-name \"${_DNSHOST_name}\"
==========================================================
SCRIPT: external1.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for external1 VM.
#
# external1.sh: FreeBSD QEMU VM startup script for external1 VM.
# Usage: sudo /bin/sh external1.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# external1.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [$_EXTERNAL1_ISO]
echo [$_EXTERNAL1_mem]
echo [$_EXTERNAL1_hdsize]
echo [$_EXTERNAL1_img]
echo [$_EXTERNAL1_mac]
echo [$_EXTERNAL1_name]
echo [$_EXTERNAL1_tap1]
echo [$_EXTERNAL1_telnetport]
#
#exit
#
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_EXTERNAL1_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor vc \
-serial telnet:localhost:${_EXTERNAL1_telnetport},server=on,wait=on \
-cpu qemu64 \
-vga cirrus \
-m ${_EXTERNAL1_mem} \
-cdrom ${_EXTERNAL1_ISO} \
-boot order=cd,menu=on,splash=${_EX1_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_EXTERNAL1_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_EXTERNAL1_tap1},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_EXTERNAL1_mac} \
-name \"${_EXTERNAL1_name}\"
==========================================================
SCRIPT: external2.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for external2 VM.
#
# external2.sh: FreeBSD QEMU VM startup script for external2 VM.
# Usage: sudo /bin/sh external2.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD qemu vm startup script
#
# external2.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_EXTERNAL2_ISO}]
echo [mem=${_EXTERNAL2_mem}]
echo [hdsize=${_EXTERNAL2_hdsize}]
echo [img=${_EXTERNAL2_img}]
echo [mac=${_EXTERNAL2_mac}]
echo [name=${_EXTERNAL2_name}]
echo [tap2=${_EXTERNAL2_tap2}]
echo [telnetport=${_EXTERNAL2_telnetport}]
#
#exit
#
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_EXTERNAL2_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor stdio \
-serial telnet:localhost:${_EXTERNAL2_telnetport},server=on,wait=on \
-cpu qemu64 \
-vga cirrus \
-m ${_EXTERNAL2_mem} \
-cdrom ${_EXTERNAL2_ISO} \
-boot order=cd,menu=on,splash=${_EX2_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_EXTERNAL2_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_EXTERNAL2_tap2},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_EXTERNAL2_mac} \
-name \"${_EXTERNAL2_name}\"
==========================================================
SCRIPT: external3.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for external3 VM.
#
# external3.sh: FreeBSD QEMU VM startup script for external3 VM.
# Usage: sudo /bin/sh external3.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# external3.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_EXTERNAL3_ISO}]
echo [mem=${_EXTERNAL3_mem}]
echo [hdsize=${_EXTERNAL3_hdsize}]
echo [img=${_EXTERNAL3_img}]
echo [mac=${_EXTERNAL3_mac}]
echo [name=${_EXTERNAL3_name}]
echo [tap3=${_EXTERNAL3_tap3}]
echo [telnetport=${_EXTERNAL3_telnetport}]
#
#exit
#
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_EXTERNAL3_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor vc \
-serial telnet:localhost:${_EXTERNAL3_telnetport},server=on,wait=on \
-cpu qemu64 \
-vga cirrus \
-m ${_EXTERNAL3_mem} \
-cdrom ${_EXTERNAL3_ISO} \
-boot order=cd,menu=on,splash=${_EX3_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_EXTERNAL3_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_EXTERNAL3_tap3},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_EXTERNAL3_mac} \
-name \"${_EXTERNAL3_name}\"
==========================================================
SCRIPT: firewall.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for firewall VM.
#
# firewall.sh: FreeBSD QEMU VM startup script for firewall VM.
# Usage: sudo /bin/sh firewall.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# firewall.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [$_FIREWALL_ISO]
echo [$_FIREWALL_mem]
echo [$_FIREWALL_hdsize]
echo [$_FIREWALL_img]
echo [$_FIREWALL_mac1]
echo [$_FIREWALL_mac2]
echo [$_FIREWALL_name]
echo [$_FIREWALL_tap0]
echo [$_FIREWALL_tap4]
echo [$_FIREWALL_telnetport]
#exit
# Note - the firewall has two interfaces - em0 and em1.
# em0 is considered the 'external' interface and
# em1 is considered the 'internal' interface.
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_FIREWALL_telnetport "
echo
/usr/local/bin/qemu-system-x86_64 -monitor none \
-serial telnet:localhost:${_FIREWALL_telnetport},server=on,wait=on \
-cpu qemu64 \
-display gtk \
-vga cirrus \
-m ${_FIREWALL_mem} \
-cdrom ${_FIREWALL_ISO} \
-boot order=cd,menu=on,splash=${_FW_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_FIREWALL_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_FIREWALL_tap0},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_FIREWALL_mac1} \
-netdev tap,id=nd1,ifname=${_FIREWALL_tap4},script=no,downscript=no \
-device e1000,netdev=nd1,mac=${_FIREWALL_mac2} \
-name \"${_FIREWALL_name}\"
==========================================================
SCRIPT: firewall2.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for firewall2 VM.
#
# firewall2.sh: FreeBSD QEMU VM startup script for firewall2 VM.
# Usage: sudo /bin/sh firewall2.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# firewall2.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_FIREWALL2_ISO}]
echo [mem=${_FIREWALL2_mem}]
echo [hdsize=${_FIREWALL2_hdsize}]
echo [img=${_FIREWALL2_img}]
echo [mac1=${_FIREWALL2_mac1}]
echo [mac2=${_FIREWALL2_mac2}]
echo [name=${_FIREWALL2_name}]
echo [tap9=${_FIREWALL2_tap9}]
echo [tap10=${_FIREWALL2_tap10}]
echo [telnetport=${_FIREWALL2_telnetport}]
#exit
# Note - the firewall has two interfaces - em0 and em1.
# em0 is considered the 'external' interface and
# em1 is considered the 'internal' interface.
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_FIREWALL2_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor none \
-serial telnet:localhost:${_FIREWALL2_telnetport},server=on,wait=on \
-cpu qemu64 \
-display gtk \
-vga cirrus \
-m ${_FIREWALL2_mem} \
-cdrom ${_FIREWALL_ISO} \
-boot order=cd,menu=on,splash=${_FW2_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_FIREWALL2_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_FIREWALL2_tap9},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_FIREWALL2_mac1} \
-netdev tap,id=nd1,ifname=${_FIREWALL2_tap10},script=no,downscript=no \
-device e1000,netdev=nd1,mac=${_FIREWALL2_mac2} \
-name \"${_FIREWALL2_name}\"
==========================================================
SCRIPT: internal.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for internal VM.
#
# internal.sh: FreeBSD QEMU VM startup script for internal VM.
# Usage: sudo /bin/sh internal.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# internal.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_INTERNAL_ISO}]
echo [mem=${_INTERNAL_mem}]
echo [hdsize=${_INTERNAL_hdsize}]
echo [img=${_INTERNAL_img}]
echo [mac=${_INTERNAL_mac}]
echo [name=${_INTERNAL_name}]
echo [tap5=${_INTERNAL_tap5}]
echo [telnetport=${_INTERNAL_telnetport}]
#
#exit
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_INTERNAL_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor none \
-serial telnet:localhost:${\_INTERNAL_telnetport},server \
-cpu qemu64 \
-vga cirrus \
-m ${_INTERNAL_mem} \
-cdrom ${_INTERNAL_ISO} \
-boot order=cd,menu=on,splash=${_INT_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_INTERNAL_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_INTERNAL_tap5},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_INTERNAL_mac} \
-name \"${_INTERNAL_name}\"
==========================================================
SCRIPT: mkbr.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD startup script for bridge and tap devices.
#
# mkbr.sh: FreeBSD startup script for bridge and tap devices.
# EXAMPLE Usage: sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 bridge1 tap2 bridge2 tap3 tap4 tap5 em0
#
# mkbr.sh - manage bridge and tap interfaces for FreeBSD.
#
# Have fun, but don't blame me if it smokes your machine.
#
# This script is used to start the bridge and tap interfaces.
#
# To create one bridge, two tap interfaces, and connect the
# local ethernet interace (here em0), run under sudo as follows:
# sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 em0
#
# The script can be used to create any number of bridges and taps
# for any internal network design:
# sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 bridge1 tap2 tap3 tap4 bridge2 tap5 em0 ... etc.
#
# To add other taps to existing bridges, do not specify the "reset" parameter.
# sudo /bin/sh mkbr.sh bridge0 tap10 tap11 bridge1 tap12 tap13 ... etc.
#
# To delete all bridge and tap devices:
# sudo /bin/sh mkbr.sh reset
#
#
#set -x
usage() {
echo "Usage: mkbr.sh ["reset"] <bridgeN> <tapA> [[<bringeM>] <tapB> <tapC> ...]"
echo "You must be root to run this script."
exit 1
}
if [ "X0" != "X`id -u`" ]
then
usage
fi
if [ $# = 0 ]
then
usage;
fi
if [ $1 = "reset" ]
then
echo
echo "Note - if_bridge and/or if_tap may be compiled into the kernel and can't be unloaded. Adjust interfaces manually if necessary."
echo
echo "unloading..."
kldunload if_bridge
kldunload if_tap
echo
echo "Deleting any remaining bridge and tap devices:"
for i in ifconfig -l
do
echo "Interface: ${i}"
case ${i} in
bridge*)
echo " ... destroying bridge ${i}"
ifconfig ${i} destroy
;;
tap*)
echo " ... destroying tap ${i}"
ifconfig ${i} destroy
;;
esac
done
sleep 1
echo "loading..."
kldload if_bridge
kldload if_tap
shift
RESET="Y"
echo "RESET=Y"
# Before using the tap devices in QEMU, two sysctls require adjustment:
sysctl net.link.tap.user_open=1
sysctl net.link.tap.up_on_open=1
else
RESET="N"
echo "RESET=N"
fi
PARAM=$1
while [ "X${PARAM}" != "X" ]
do
# echo "PARAM=[$PARAM]"
case $PARAM in
bridge*) BRIDGE=$1
# if [ "$RESET" = "Y" ]
# then
echo ifconfig $BRIDGE create
ifconfig $BRIDGE create
echo ifconfig $BRIDGE
ifconfig $BRIDGE
# fi
echo ifconfig $BRIDGE up
ifconfig $BRIDGE up
;;
tap*) TAP=$1
# if [ "$RESET" = "Y" ]
# then
echo ifconfig $TAP create
ifconfig $TAP create
# fi
echo "ifconfig $BRIDGE addm $TAP "
ifconfig $BRIDGE addm $TAP
;;
*) echo "*** Checking to see if $1 is a valid interface"
TMPINT=$1
RESULT="IS NOT"
for i in ifconfig -l
do
# echo $i
if [ "${i}X" = "${TMPINT}X" ]
then
echo "Found a valid interface: ${TMPINT} Adding it to the bridge. Check results."
echo "ifconfig $BRIDGE addm $TMPINT"
ifconfig $BRIDGE addm $TMPINT
RESULT="IS"
break;
else
echo -n "."
fi
done
echo "Interface ${TMPINT} $RESULT a valid interface."
;;
esac
shift
PARAM=$1
done
exit 0
==========================================================
SCRIPT: runvm.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for multiple VMs at once.
#
# runvm.sh: FreeBSD QEMU VM startup script for multiple VMs.
# EXAMPLE Usage: /bin/sh runvm.sh firewall external1 external2 internal
#
# location: FreeBSD Host
#
# runvm.sh - run virtual machines specified on the command line.
#
# To use this script, run mkbr.sh first to set up the bridge and
# tap configurations for the desired network architecture.
#
# NOTE: this script works best on XFCE4 desktop as it takes advantage of the
# xfce4-terminal and it's ability to use multiple tabs.
#
# >>>> It is unlikely to work on another desktop. <<<<
#
# Essentially, this script is a big case statement. It gets the
# command line names of the virtual machines and calls a function
# that starts the virtual machine.
#
# pick up environment for this run
. ./vm_envs.sh
#set -x
#WKDIR=$HOME/LAB/SCRIPTS
export WKDIR=$HOME/ipfw
echo "[${WKDIR}]"
usage() {
echo "Usage: /bin/sh runvm.sh vmname [vmname ...]"
echo "Each virtual machine opens up on xfce4-terminal with two tabs -"
echo " one for the qemu virtual machine, and one for the serial"
echo " terminal interface."
echo ""
exit 1
}
CURDIR=pwd
if [ "X${CURDIR}" != "X${WKDIR}/SCRIPTS" ]
then
usage;
fi
if [ $# = 0 ]
then
usage;
fi
# Functions for each VM
dnshost_vm () {
# DNS host
echo "in function: [${_DNSHOST_telnetport}]"
xfce4-terminal --window --geometry="80x24+50+50" --zoom="-1" \
-T "${_DNSHOST_name}" -e "bash -c \"cd ${WKDIR}/SCRIPTS && sudo /bin/sh dnshost.sh ; bash\"" \
--tab -T "${_DNSHOST_name}" -e "bash -c \"cd ${WKDIR}/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet -4 localhost ${_DNSHOST_telnetport}); bash\""
return
}
external1_vm () {
# external1
echo "in function: [${_EXTERNAL1_telnetport}]"
xfce4-terminal --window --geometry="80x24+75+75" --zoom="-1" \
-T "${_EXTERNAL1_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh external1.sh ; bash\"" \
--tab -T "${_EXTERNAL1_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_EXTERNAL1_telnetport}); bash\""
return
}
external2_vm () {
# external2
echo "in function: [${_EXTERNAL2_telnetport}]"
xfce4-terminal --window --geometry="80x24+100+100" --zoom="-1" \
-T "${_EXTERNAL2_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh external2.sh ; bash\"" \
--tab -T "${_EXTERNAL2_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_EXTERNAL2_telnetport}); bash\""
return
}
external3_vm () {
# external3
echo "in function: [${_EXTERNAL3_telnetport}]"
xfce4-terminal --window --geometry="80x24+125+125" --zoom="-1" \
-T "${_EXTERNAL3_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh external3.sh ; bash\"" \
--tab -T "${_EXTERNAL3_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_EXTERNAL3_telnetport}); bash\""
return
}
firewall_vm () {
# Firewall
echo "in function: [${_FIREWALL_telnetport}]"
xfce4-terminal --window --geometry="80x24+150+150" --zoom="-1" \
-T "${_FIREWALL_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh firewall.sh ; bash\"" \
--tab -T "${_FIREWALL_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh; telnet localhost ${_FIREWALL_telnetport}); bash\""
return
}
firewall2_vm () {
# Firewall2
echo "in function: [${_FIREWALL2_telnetport}]"
xfce4-terminal --window --geometry="80x24+175+175" --zoom="-1" \
-T "${_FIREWALL2_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh firewall2.sh ; bash\"" \
--tab -T "${_FIREWALL2_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_FIREWALL2_telnetport}); bash\""
return
}
internal_vm () {
# internal
echo "in function: [${_INTERNAL_telnetport}]"
xfce4-terminal --window --geometry="80x24+200+200" --zoom="-1" \
-T "${_INTERNAL_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh internal.sh ; bash\"" \
--tab -T "${_INTERNAL_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_INTERNAL_telnetport}); bash\""
return
}
v6only_vm () {
# v6only
echo "in function: [${_V6ONLY_telnetport}]"
xfce4-terminal --window --geometry="80x24+225+225" --zoom="-1" \
-T "${_V6ONLY_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sudo /bin/sh v6only.sh ; bash\"" \
--tab -T "${_V6ONLY_name}" -e "bash -c \"cd $WKDIR/SCRIPTS && sleep 2 && (. ./vm_envs.sh;telnet localhost ${_V6ONLY_telnetport}); bash\""
return
}
#
# Startup the requested VMs
#
PARAM=$1
while [ "X${PARAM}" != "X" ]
do
echo "PARAM = [${PARAM}]"
case ${PARAM} in
dnshost)
echo "dnshost ..."
echo "_DNSHOST_telnetport = [${_DNSHOST_telnetport}]"
dnshost_vm
;;
external1)
echo "external1 ..."
echo "_EXTERNAL1_telnetport = [${_EXTERNAL1_telnetport}]"
external1_vm
;;
external2)
echo "external2 ..."
echo "_EXTERNAL2_telnetport = [${_EXTERNAL2_telnetport}]"
external2_vm
;;
external3)
echo "external3 ..."
echo "_EXTERNAL3_telnetport = [${_EXTERNAL3_telnetport}]"
external3_vm
;;
firewall)
echo "firewall ..."
echo "_FIREWALL_telnetport = [${_FIREWALL_telnetport}]"
firewall_vm
;;
firewall2)
echo "firewall2 ..."
echo "_FIREWALL2_telnetport = [${_FIREWALL2_telnetport}]"
firewall2_vm
;;
internal)
echo "internal ..."
echo "_INTERNAL_telnetport = [${_INTERNAL_telnetport}]"
internal_vm
;;
v6only)
echo "v6only ..."
echo "_V6ONLY_telnetport = [${_V6ONLY_telnetport}]"
v6only_vm
;;
*)
echo ""
echo "*** ERROR: NO VM NAMED [$PARAM]"
echo ""
;;
esac
shift
sleep 3
PARAM=$1
done
exit 0
==========================================================
SCRIPT: v6only.sh
#!/bin/sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM startup script for v6only VM.
#
# v6only.sh: FreeBSD QEMU VM startup script for v6only VM.
# Usage: sudo /bin/sh v6only.sh
# Note: Set up for serial console. Start another session and telnet to the port shown.
#
# FreeBSD QEMU VM startup script
#
# v6only.sh
#
#set -x
# pick up environment for this run
. ./vm_envs.sh
echo [ISO=${_V6ONLY_ISO}]
echo [mem=${_V6ONLY_mem}]
echo [hdsize=${_V6ONLY_hdsize}]
echo [img=${_V6ONLY_img}]
echo [mac=${_V6ONLY_mac}]
echo [name=${_V6ONLY_name}]
echo [tap6=${_V6ONLY_tap6}]
echo [telnetport=${_V6ONLY_telnetport}]
#
#exit
echo
echo "NOTE!!! telnet server running! To start QEMU telnet to localhost $_V6ONLY_telnetport"
echo
/usr/local/bin/qemu-system-x86_64 -monitor none \
-serial telnet:localhost:${_V6ONLY_telnetport},server \
-cpu qemu64 \
-vga cirrus \
-m ${_V6ONLY_mem} \
-cdrom ${_V6ONLY_ISO} \
-boot order=cd,menu=on,splash=${_V6_splash},splash-time=3000 \
-drive if=none,id=drive0,cache=none,aio=threads,format=raw,file=${_V6ONLY_img} \
-device virtio-blk,drive=drive0 \
-netdev tap,id=nd0,ifname=${_V6ONLY_tap6},script=no,downscript=no \
-device e1000,netdev=nd0,mac=${_V6ONLY_mac} \
-name \"${_V6ONLY_name}\"
==========================================================
SCRIPT: vm_envs.sh
# IPFW Primer
# License: 3-clause BSD
# Author: Jim Brown, jpb@jimby.name
# Code: https://github.com/jimmyb-gh/ipfw-primer
#
# FreeBSD QEMU VM environment script.
#
# vmenv.sh: FreeBSD QEMU VM environment setup script.
# Usage: ./bin/sh vmenv.sh
#
# vm_envs.sh - environment for setting up virtual machines
# for the IPFW examples lab.
#
# Set the environment variables below (or keep the defaults)
# Note that the default disk size for each virtual machine is
# 4GB - so all five VMs will take up about 32GB if you preallocate
# space.
#
# In brief:
#
# Install FreeBSD on the host machine and update to latest patch level.
# Install desktop software.
# Install QEMU (latest)
# Install nmap (needed for ncat)
# Install sudo
#
#
# The script mkbr.sh should be run before starting
# the virtual machines. mkbr.sh sets up the bridge and tap
# devices needed by the VMs.
#
# sudo /bin/sh ./mkbr.sh reset bridge0 tap0 tap1 tap2 tap3 em0 bridge1 tap4 tap5
#
# This will set up the devices needed by QEMU.
#
#
#The file directory layout for the examples is:
#
# ~/ipfw
# /SCRIPTS
# _CreateAllVMs.sh (create Qemu disks images)
# dnshost.sh (run script for dns server VM)
# external1.sh (run scripts for external VMs)
# external2.sh "
# external3.sh "
# firewall.sh (run script for firewall VM)
# firewall2.sh (run script for firewall2 VM)
# internal.sh (script to setup internal host)
# v6only.sh (run script for IPv6 only VM)
# mkbr.sh (script to create bridge and tap devices)
# vm_envs.sh (script to manage all parameters)
# runvm.sh (script to manage all virtual machines)
# /BMP
# dns_splash_640x480.bmp
# external1_splash_640x480.bmp
# external2_splash_640x480.bmp
# external3_splash_640x480.bmp
# internal_splash_640x480.bmp
# ipfw2_splash_640x480.bmp
# ipfw_splash_640x480.bmp
# v6only_splash_640x480.bmp
# dnshost_splash_640x480.bmp
# /ISO
# fbsd.iso (latest FreeBSD install iso)
# /VM
# dnshost.qcow2 (Qemu disk image for dns host)
# external1.qcow2 (Qemu disk image for external hosts)
# external2.qcow2 "
# external3.qcow2 "
# firewall.qcow2 (Qemu disk image for firewall)
# firewall2.qcow2 (Qemu disk image for firewall2)
# internal.qcow2 (Qemu disk image for an internal host)
# v6only.qcow2 (Qemu disk image for an ipv6only host)
#
#
# Start the VMs and install / test one at a time.
#
# sudo /bin/sh firewall.sh
# sudo /bin/sh firewall2.sh
# sudo /bin/sh external1.sh
# sudo /bin/sh external2.sh
# sudo /bin/sh external3.sh
# sudo /bin/sh internal.sh
# sudo /bin/sh v6only.sh
# sudo /bin/sh dnshost.sh
#
# Each install should first utilize DHCP to get a valid IP address
# After install, proceed to update FreeBSD with "freebsd-update fetch install"
# Install packages:
# Use whatever shell you prefer. Bash is listed below.
# Firewall - pkg install bash cmdwatch lynx iperf3 nmap hping3 nginx
# All others - pkg install bash cmdwatch lynx iperf3 nmap hping3 nginx
# DNS host - pkg install bind918 dual-dhclient bash cmdwatch lynx nginx
#
# Reset all IP addresses for static usage:
#
# Host interface: add 172.16.10.100/24 alias
# Disable any firewall (pf, ipfw, etc.) on the host.
# BE SURE this is Ok for your environment.
#
# Firewall em0 172.16.10.50/24, default gateway 172.16.10.100
# em1 10.10.10.50/24
#
# Firewall2 em0 as needed
# em1 as needed
#
# External1: em0 172.16.10.10/24, default gateway 172.16.10.100
# External2: em0 172.16.10.20/24, default gateway 172.16.10.100
# External3: em0 172.16.10.30/24, default gateway 172.16.10.100
# Internal: em0 10.10.10.200/24, default gateway 10.10.10.50
#
# v6only as needed
# dnshost as needed
#
#
export _BASE=/home/jpb/ipfw
# Bridge and tap info
export _FIREWALL_tap0=tap0
export _EXTERNAL1_tap1=tap1
export _EXTERNAL2_tap2=tap2
export _EXTERNAL3_tap3=tap3
export _FIREWALL_tap4=tap4
export _INTERNAL_tap5=tap5
export _V6ONLY_tap6=tap6
export _DNSHOST_tap7=tap7
export _DNSHOST_tap8=tap8
export _FIREWALL2_tap9=tap9
export _FIREWALL2_tap10=tap10
export _DNSHOST_tap11=tap11
export _bridge0_=bridge0
export bridge1=bridge1
export bridge2=bridge2
# Disk sizes
export _EXTERNAL1_hdsize=4G
export _EXTERNAL2_hdsize=4G
export _EXTERNAL3_hdsize=4G
export _FIREWALL_hdsize=4G
export _FIREWALL2_hdsize=4G
export _INTERNAL_hdsize=4G
export _V6ONLY_hdsize=4G
export _DNSHOST_hdsize=4G
# Is this needed anymore?
export _FBSD_ISO=${_BASE}/ISO/fbsd.iso
# Boot iso locations
export _DNSHOST_ISO=${_BASE}/ISO/fbsd.iso
export _EXTERNAL1_ISO=${_BASE}/ISO/fbsd.iso
export _EXTERNAL2_ISO=${_BASE}/ISO/fbsd.iso
export _EXTERNAL3_ISO=${_BASE}/ISO/fbsd.iso
export _FIREWALL_ISO=${_BASE}/ISO/fbsd.iso
export _FIREWALL2_ISO=${_BASE}/ISO/fbsd.iso
export _INTERNAL_ISO=${_BASE}/ISO/fbsd.iso
export _V6ONLY_ISO=${_BASE}/ISO/fbsd.iso
# Memory sizes
export _DNSHOST_mem=1024
export _EXTERNAL1_mem=1024 # lower all to 512 if necessary
export _EXTERNAL2_mem=1024
export _EXTERNAL3_mem=1024
export _FIREWALL_mem=1024
export _FIREWALL2_mem=1024
export _INTERNAL_mem=1024
export _V6ONLY_mem=1024
# Qemu disk image locations.
export _DNSHOST_img=${_BASE}/VM/dnshost.qcow2
export _EXTERNAL1_img=${_BASE}/VM/external1.qcow2
export _EXTERNAL2_img=${_BASE}/VM/external2.qcow2
export _EXTERNAL3_img=${_BASE}/VM/external3.qcow2
export _FIREWALL_img=${_BASE}/VM/firewall.qcow2
export _FIREWALL2_img=${_BASE}/VM/firewall2.qcow2
export _INTERNAL_img=${_BASE}/VM/internal.qcow2
export _V6ONLY_img=${_BASE}/VM/v6only.qcow2
# MAC addresses
export _DNSHOST_mac1=02:49:53:53:53:53
export _DNSHOST_mac2=02:49:53:53:54:54
export _DNSHOST_mac3=02:49:53:53:55:55
export _EXTERNAL1_mac=02:45:58:54:31:10
export _EXTERNAL2_mac=02:45:58:54:32:20
export _EXTERNAL3_mac=02:45:58:54:33:30
export _FIREWALL_mac1=02:49:50:46:57:41
export _FIREWALL2_mac1=02:49:50:00:22:22
export _FIREWALL_mac2=02:49:50:46:57:42
export _FIREWALL2_mac2=02:49:50:22:22:22
export _INTERNAL_mac=02:49:4E:54:0a:42
export _V6ONLY_mac=02:49:de:ad:be:ef
# VM names
export _DNSHOST_name=DNSHOST
export _EXTERNAL1_name=EXTERNAL1
export _EXTERNAL2_name=EXTERNAL2
export _EXTERNAL3_name=EXTERNAL3
export _FIREWALL_name=FIREWALL
export _FIREWALL2_name=FIREWALL2
export _INTERNAL_name=INTERNAL
export _V6ONLY_name=V6ONLY
# Slash images
export _DNS_splash=${_BASE}/BMP/dns_splash_640x480.bmp
export _EX1_splash=${_BASE}/BMP/external1_splash_640x480.bmp
export _EX2_splash=${_BASE}/BMP/external2_splash_640x480.bmp
export _EX3_splash=${_BASE}/BMP/external3_splash_640x480.bmp
export _FW_splash=${_BASE}/BMP/ipfw_splash_640x480.bmp
export _FW2_splash=${_BASE}/BMP/ipfw2_splash_640x480.bmp
export _INT_splash=${_BASE}/BMP/internal_splash_640x480.bmp
export _V6_splash=${_BASE}/BMP/ipv6_splash_640x480.bmp
#
# Telnet ports
export _DNSHOST_telnetport=4453
export _EXTERNAL1_telnetport=4410
export _EXTERNAL2_telnetport=4420
export _EXTERNAL3_telnetport=4430
export _FIREWALL_telnetport=4450
export _FIREWALL2_telnetport=4250
export _INTERNAL_telnetport=44200
export _V6ONLY_telnetport=4460
# Bridge and Tap configurations.
#
# Note: em0 is used for the host interface.
# Change as needed.
#
# Two bridge configuration
# Standard examples
#
# em0
# |
# External1(tap1) -----bridge0------(tap0)Firewall
# External2(tap2) -----+ | (tap4)
# External3(tap3) -------+ |
# bridge1
# |
# Internal(tap5) -----------------------------+
#
# sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 tap2 tap3 em0
#
#
#
# Two bridge configuration
# NAT & LSNAT examples
#
#
#
# (firewall does LSNAT load balancing)
# External1(tap1) -----bridge0------(tap0)Firewall
# External2(tap2) -----+ | (tap4)
# External3(tap3) -------+ |
# (these function as internal machines) bridge1----em0
# |
# Internal(tap5) -----------------------------+
# (this functions as an external machine)
#
# sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 tap2 tap3 bridge1 tap4 tap5 em0
#
#
#
# Two bridge configuration
# NAT64/DNS64 example
#
# ipv4 only NAT64 Translator
# External1(tap1) ------bridge0-----(tap0)Firewall
# (ipv4 only) + (tap4)
# (webserver) | +
# dnshost(tap7) |
# (DNS server) |
# (running DNS64) |
# (tap8) |
# | |
# + |
# ipv6 only |
# v6only(tap6) --------bridge1----------------+
# (v6 only host)
#
# sudo /bin/sh mkbr.sh reset bridge0 tap0 tap1 tap7 bridge1 tap4 tap6 tap8
#
#
==========================================================
==========================================================
CODE: divert.c
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <string.h>
#include <err.h>
#include <sys/systm.h>
#define DIVERT_PORT 700
void hexdump(void *ptr, int length, const char *hdr, int flags);
int
main(int argc, char *argv[])
{
int fd, s;
struct sockaddr_in sin;
socklen_t sin_len;
printf("Opening divert on port %d\n",DIVERT_PORT);
fd = socket(PF_DIVERT, SOCK_RAW, 0);
if (fd == -1)
err(1, "socket");
memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_port = htons(DIVERT_PORT);
sin.sin_addr.s_addr = 0;
sin_len = sizeof(struct sockaddr_in);
s = bind(fd, (struct sockaddr *) &sin, sin_len);
if (s == -1)
err(1, "bind");
for (;;) {
ssize_t n;
char packet[IP_MAXPACKET];
struct ip *ip;
struct tcphdr *th;
int hlen;
char src[64], dst[64], printbuff[12];
memset(src, 0, sizeof(src));
memset(dst, 0, sizeof(dst));
memset(printbuff, 0, sizeof(printbuff));
memset(packet, 0, sizeof(packet));
n = recvfrom(fd, packet, sizeof(packet), 0,
(struct sockaddr *) &sin, &sin_len);
if (n == -1) {
warn("recvfrom");
continue;
}
if (n < sizeof(struct ip)) {
warnx("packet is too short");
continue;
}
ip = (struct ip *) packet;
hlen = ip->ip_hl << 2;
if (hlen < sizeof(struct ip) || ntohs(ip->ip_len) < hlen ||
n < ntohs(ip->ip_len)) {
warnx("invalid IPv4 packet");
continue;
}
th = (struct tcphdr *) (packet + hlen);
if (inet_ntop(AF_INET, &ip->ip_src, src,
sizeof(src)) == NULL)
(void)strlcpy(src, "?", sizeof(src));
if (inet_ntop(AF_INET, &ip->ip_dst, dst,
sizeof(dst)) == NULL)
(void)strlcpy(dst, "?", sizeof(dst));
printf("%s:%u -> %s:%u\n",
src,
ntohs(th->th_sport),
dst,
ntohs(th->th_dport)
);
/*
* dump the packet in hex and ascii with hexdump(3)
*/
hexdump((void *)packet, n, "|",0);
n = sendto(fd, packet, n, 0, (struct sockaddr *) &sin,
sin_len);
if (n == -1)
warn("sendto");
}
return 0;
}Appendix C: Appendix C: Networking References
References for understanding IP based commmunications and building firewalls.
From /etc/rc.firewall:
Building Internet Firewalls, 2nd Edition Brent Chapman and Elizabeth Zwicky O'Reilly & Associates, Inc ISBN 1-56592-871-7 http://www.ora.com/ http://www.oreilly.com/catalog/fire2/ For a more advanced treatment of Internet Security read: Firewalls and Internet Security: Repelling the Wily Hacker, 2nd Edition William R. Cheswick, Steven M. Bellowin, Aviel D. Rubin Addison-Wesley / Prentice Hall ISBN 0-201-63466-X http://www.pearsonhighered.com/ http://www.pearsonhighered.com/educator/academic/product/0,3110,020163466X,00.html
Additional references:
TCP/IP Illustrated, Volume 1: The Protocols Author: W. Richard Stevens Publisher: Addison-Wesley Professional Publisher Website: Addison-Wesley Professional Date Published: November 1994 ISBN: 978-0201633467 The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols Reference Author: Charles M. Kozierok Publisher: No Starch Press Publisher Website: No Starch Press Date Published: October 2005 ISBN: 978-1593270476 Internetworking with TCP/IP Volume One: Principles, Protocols, and Architecture Author: Douglas E. Comer Publisher: Pearson Publisher Website: Pearson Date Published: 6th Edition, 2013 ISBN: 978-0136085300 Computer Networks: A Systems Approach Authors: Larry L. Peterson and Bruce S. Davie Publisher: Morgan Kaufmann Publisher Website: Morgan Kaufmann Date Published: 6th Edition, 2021 ISBN: 978-0128182000
Additional resources regarding firewalls:
Network Security, Firewalls, and VPNs Authors: J. Michael Stewart, Denise Kinsey Publisher: Jones & Bartlett Learning Publisher Website: Jones & Bartlett Learning Date Published: October 2020 ISBN: 978-1284183696 Firewall Fundamentals Author: David W. Chapman Jr. Publisher: Cisco Press Publisher Website: Cisco Press Date Published: June 2006 ISBN: 978-1587052213 The Best Damn Firewall Book Period Author: Thomas W. Shinder Publisher: Syngress Publisher Website: Syngress Date Published: January 2008 ISBN: 978-1597492188 Guide to Firewalls and Network Security: Intrusion Detection and VPNs Authors: Michael E. Whitman, Herbert J. Mattord, Richard Austin, Greg Holden Publisher: Cengage Learning Publisher Website: Cengage Learning Date Published: 2008 ISBN: 978-1435420168
Appendix D: Appendix D: Et Cetera
IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD
IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD
IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD IPFW Appendix DDDD
Appendix E: Appendix E: DNS Server Configuration
DNS configuration for IPFW Primer book.
Manifest of dnshost scripts and file.
File: dnshost_usrlocaletc_namedb.tgz
Description: Contains the configuration for the BIND 9 DNS services that run on this machine.
Installation:
Install bind9 package first, then untar this collection as follows:
# cd /usr/local/etc
# tar xvzf dnshost_usrlocaletc_namedb.tgz
Contents:
$ tar tvzf dnshost_usrlocaletc_namedb.tgz
drwxr-xr-x 0 root wheel 0 Nov 19 12:00 namedb/
-rw-r--r-- 0 bind bind 2403 Nov 19 11:59 namedb/bind.keys
drwxr-xr-x 0 bind bind 0 Nov 19 11:59 namedb/dynamic/
-rw-r--r-- 0 bind bind 2618 Dec 2 12:34 namedb/named.conf
-rw-r--r-- 0 bind bind 21992 Nov 19 11:59 namedb/named.conf.sample
-rw-r--r-- 0 bind bind 927 Nov 19 11:59 namedb/named.root
-rw-r--r-- 0 bind bind 3317 Nov 19 11:59 namedb/named.root.SAVE
drwxr-xr-x 0 bind bind 0 Dec 2 15:35 namedb/primary/
-rw------- 0 bind bind 100 Nov 19 11:59 namedb/rndc.key
drwxr-xr-x 0 bind bind 0 Nov 19 11:59 namedb/secondary/
drwxr-xr-x 0 bind bind 0 Nov 19 11:59 namedb/working/
-rw-r--r-- 0 bind bind 297 Nov 19 11:59 namedb/working/managed-keys.bind
-rw-r--r-- 0 bind bind 355 Nov 19 11:59 namedb/primary/ptr_198.51
-rw-r--r-- 0 bind bind 465 Nov 19 11:59 namedb/primary/ptr_203.0
-rw-r--r-- 0 bind bind 693 Dec 1 19:29 namedb/primary/example.com
-rw-r--r-- 0 bind bind 148 Nov 19 11:59 namedb/primary/empty
-rw-r--r-- 0 bind bind 407 Nov 19 14:12 namedb/primary/ptr_ipv6
-rw-r--r-- 0 bind bind 287 Dec 2 15:35 namedb/primary/managed-keys.bind
-rw-r--r-- 0 bind bind 226 Nov 19 11:59 namedb/primary/localhost-reverse
-rw-r--r-- 0 bind bind 158 Nov 19 11:59 namedb/primary/localhost-forward
-rw-r--r-- 0 bind bind 351 Dec 1 19:30 namedb/primary/ptr_192.168
$
=====================================================================================
bind.keys
# Copyright (C) Internet Systems Consortium, Inc. ("ISC")
#
# SPDX-License-Identifier: MPL-2.0
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, you can obtain one at https://mozilla.org/MPL/2.0/.
#
# See the COPYRIGHT file distributed with this work for additional
# information regarding copyright ownership.
# The bind.keys file is used to override the built-in DNSSEC trust anchors
# which are included as part of BIND 9. The only trust anchors it contains
# are for the DNS root zone ("."). Trust anchors for any other zones MUST
# be configured elsewhere; if they are configured here, they will not be
# recognized or used by named.
#
# To use the built-in root key, set "dnssec-validation auto;" in the
# named.conf options, or else leave "dnssec-validation" unset. If
# "dnssec-validation" is set to "yes", then the keys in this file are
# ignored; keys will need to be explicitly configured in named.conf for
# validation to work. "auto" is the default setting, unless named is
# built with "configure --disable-auto-validation", in which case the
# default is "yes".
#
# This file is NOT expected to be user-configured.
#
# Servers being set up for the first time can use the contents of this file
# as initializing keys; thereafter, the keys in the managed key database
# will be trusted and maintained automatically.
#
# These keys are current as of Mar 2019. If any key fails to initialize
# correctly, it may have expired. In that event you should replace this
# file with a current version. The latest version of bind.keys can always
# be obtained from ISC at https://www.isc.org/bind-keys.
#
# See https://data.iana.org/root-anchors/root-anchors.xml for current trust
# anchor information for the root zone.
trust-anchors {
# This key (20326) was published in the root zone in 2017.
. initial-key 257 3 8 "AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTOiW1vkIbzxeF3
+/4RgWOq7HrxRixHlFlExOLAJr5emLvN7SWXgnLh4+B5xQlNVz8Og8kv
ArMtNROxVQuCaSnIDdD5LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF
0jLHwVN8efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7pr+e
oZG+SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLYA4/ilBmSVIzuDWfd
RUfhHdY6+cn8HFRm+2hM8AnXGXws9555KrUB5qihylGa8subX2Nn6UwN
R1AkUTV74bU=";
};
=====================================================================================
named.conf
// Refer to the named.conf(5) and named(8) man pages, and the documentation
// in /usr/local/share/doc/bind for more details.
acl trusted-queriers {
203.0.113.0/24;
2001:db8:12::/64;
127.0.0.1;
::1;
localhost;
};
acl v6only-networks {
2001:db8:12::/64;
};
options {
directory "/usr/local/etc/namedb/primary";
pid-file "/var/run/named/pid";
dump-file "/var/dump/named_dump.db";
statistics-file "/var/stats/named.stats";
listen-on { any; };
listen-on-v6 { any; };
recursion no;
allow-transfer { trusted-queriers; };
// NOTE: Remove comments when using DNS64
// dns64 64:FF9B::/96 {
// clients { any; };
// exclude { 64:FF9B::/96; ::ffff:0000:0000/96; };
// suffix ::;
// };
};
// forward zone
zone "example.com" {
type primary;
file "/usr/local/etc/namedb/primary/example.com";
allow-transfer {trusted-queriers;};
};
// reverse zones for 203.0, 198.51, 192.168, and 2001:0db8
zone "0.203.in-addr.arpa" {
type primary;
file "/usr/local/etc/namedb/primary/ptr_203.0";
allow-transfer {trusted-queriers;};
};
zone "51.198.in-addr.arpa"{
type primary;
file "/usr/local/etc/namedb/primary/ptr_198.51";
allow-transfer {trusted-queriers;};
};
zone "2.1.0.0.8.b.d.0.1.0.0.2.ip6.arpa" {
type primary;
file "/usr/local/etc/namedb/primary/ptr_ipv6";
allow-transfer {trusted-queriers; };
};
zone "168.192.in-addr.arpa" {
type primary;
file "/usr/local/etc/namedb/primary/ptr_192.168";
allow-transfer {trusted-queriers;};
};
// Block below added by BIND9
// RFCs 1912, 5735 and 6303 (and BCP 32 for localhost)
zone "localhost" { type primary; file "/usr/local/etc/namedb/primary/localhost-forward"; };
zone "127.in-addr.arpa" { type primary; file "/usr/local/etc/namedb/primary/localhost-reverse"; };
zone "255.in-addr.arpa" { type primary; file "/usr/local/etc/namedb/primary/empty"; };
// RFC 1912-style zone for IPv6 localhost address (RFC 6303)
zone "0.ip6.arpa" { type primary; file "/usr/local/etc/namedb/primary/localhost-reverse"; };
// "This" Network (RFCs 1912, 5735 and 6303)
zone "0.in-addr.arpa" { type primary; file "/usr/local/etc/namedb/primary/empty"; };
// Our own root zone file so we don't leak out to the Internet
zone "." {
type master;
file "/usr/local/etc/namedb/named.root";
allow-transfer {trusted-queriers; };
};
=====================================================================================
named.root
; This file holds the information on root name servers needed to
; initialize cache of Internet domain name servers
; (e.g. reference this file in the "cache . <file>"
; configuration file of BIND domain name servers).
;
; This file is made available by InterNIC
; under anonymous FTP as
; file /domain/named.cache
; on server FTP.INTERNIC.NET
; -OR- RS.INTERNIC.NET
;
; last update: November 16, 2017
; related version of root zone: 2017111601
;
; FORMERLY NS.INTERNIC.NET
;
$TTL 3600
. 3600 IN SOA dnshost.example.com. jpb.dnshost.example.com (
100 ; serial
14400 ; refresh
7200 ; retry
28800 ; expire
64000 ) ; min neg cache expire
. 3600 NS dnshost.example.com.
dnshost.example.com. 3600 A 203.0.113.53
dnshost.example.com. 3600 AAAA 2001:db8:12::53
=====================================================================================
rndc.key
key "rndc-key" {
algorithm hmac-sha256;
secret "wesiGsTgu7OwV44aA6C2P8XmZdW4z/YdPJ4D/vRNPTM=";
};
=====================================================================================
empty
$TTL 3h
@ SOA @ nobody.localhost. 42 1d 12h 1w 3h
; Serial, Refresh, Retry, Expire, Neg. cache TTL
@ NS @
; Silence a BIND warning
@ A 127.0.0.1
=====================================================================================
example.com
$TTL 3600
@ IN SOA example.com. jpb.example.com. (
5 ; Serial
3h ; Refresh
1h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
; name servers - NS records
@ IN NS dnshost.example.com.
; name servers - A records
dnshost IN A 203.0.113.53
;external1 IN A 203.0.113.10
external1 IN A 192.168.1.2
external2 IN A 203.0.113.20
external3 IN A 203.0.113.30
firewall IN A 203.0.113.50
firewall-em0 IN A 203.0.113.50
firewall-em1 IN A 198.51.100.50
firewall-em1 IN AAAA 2001:db8:12::50
internal IN A 198.51.100.200
; name servers - AAAA records
dnshost IN AAAA 2001:db8:12::53
v6only IN AAAA 2001:db8:12::6
=====================================================================================
localhost-forward
$TTL 3h
localhost. SOA localhost. nobody.localhost. 42 1d 12h 1w 3h
; Serial, Refresh, Retry, Expire, Neg. cache TTL
NS localhost.
A 127.0.0.1
AAAA ::1
=====================================================================================
localhost-reverse
$TTL 3h
@ SOA localhost. nobody.localhost. 42 1d 12h 1w 3h
; Serial, Refresh, Retry, Expire, Neg. cache TTL
NS localhost.
1.0.0 PTR localhost.
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR localhost.
=====================================================================================
managed-keys.bind
$TTL 0 ; 0 seconds
. IN SOA . . (
100 ; serial
0 ; refresh (0 seconds)
0 ; retry (0 seconds)
0 ; expire (0 seconds)
0 ; minimum (0 seconds)
)
KEYDATA 20241202213508 19700101000000 19700101000000 0 0 0 ; placeholder
=====================================================================================
ptr_192.168
$TTL 3600
@ IN SOA example.com. jpb.example.com. (
3 ; Serial
3h ; Refresh
1h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
; name servers - NS records
IN NS dnshost.example.com.
; PTR Records
53.1 IN PTR dnshost.example.com.
2.1 IN PTR external1.example.com.
=====================================================================================
ptr_198.51
$TTL 3600
@ IN SOA example.com. jpb.example.com. (
3 ; Serial
3h ; Refresh
1h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
; name servers - NS records
IN NS dnshost.example.com.
; PTR Records
50.100 IN PTR firewall-em1.example.com.
200.100 IN PTR internal.example.com.
=====================================================================================
ptr_203.0
$TTL 3600
@ IN SOA example.com. jpb.example.com. (
3 ; Serial
3h ; Refresh
1h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
; name servers - NS records
IN NS dnshost.example.com.
; PTR Records
53.113 IN PTR dnshost.example.com.
10.113 IN PTR external1.example.com.
20.113 IN PTR external2.example.com.
30.113 IN PTR external3.example.com.
50.113 IN PTR firewall-em0.example.com.
=====================================================================================
ptr_ipv6
$TTL 3600
@ IN SOA example.com. jpb.example.com. (
3 ; Serial
3h ; Refresh
1h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
@ IN NS dnshost.example.com.
$ORIGIN 0.0.0.0.2.1.0.0.8.b.d.0.1.0.0.2.ip6.arpa.
3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0 IN PTR dnshost.example.com.
6.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 IN PTR v6only.example.com.
=====================================================================================
managed-keys.bind
$ORIGIN .
$TTL 0 ; 0 seconds
@ IN SOA . . (
100 ; serial
0 ; refresh (0 seconds)
0 ; retry (0 seconds)
0 ; expire (0 seconds)
0 ; minimum (0 seconds)
)
KEYDATA 20220502020339 19700101000000 19700101000000 0 0 0 ; placeholder
=====================================================================================