Chapter 7. Other Keywords
Table of Contents
This section covers some other lesser used keywords.
7.1. abort / abort6
The abort and abort6 keywords interrupt the data stream between two endpoints. The effect of this keyword, is similar to the reset keyword, but there are important differences.
Figure 1. Abort and abort6 keywords
The above figure shows the effect of inserting the firewall rule:
# ipfw add 50 abort tcp from 203.0.113.30 to me
Unlike the reset keyword, there is no packet sent from the firewall to the source. What happens is that ipfw just starts dropping packets that match the rule. Since there are no more replies coming from the destination (here the firewall itself), the source endpoint issues retransmissions over and over. Eventually the source concludes that the connection is irrevocably broken and it closes the connection.
In the rule above, all TCP connections will be interrupted between the two systems. |
In a TCP connection, ipfw will use dynamic rules if a check-state rule is already in place. If this is the case, issue the abort rule at a rule number before the check-state rule. Otherwise, it will have no effect. |
7.2. mark / setmark
The setmark keyword functions similar to the tag keyword. If the packet matches the rule, ipfw applies a 32-bit identifier to the packet. This identifier (the "mark") is held with the packet internally inside ipfw. It is not sent with the packet on the wire and is not visible to any network monitoring from tools like tcpdump(1) or wireshark(1).
Like tags, a mark can be used as another filtering device with other ipfw rules to do policy based routing or filtering. Note that only one mark can be applied at a time.
A big advantage of marks over tags are their ability to be matched as a lookup key in a table. Also, a mark can have a bitmask applied to it.
To explore mark and setmark, use the architecture of Simple NAT shown in Simple NAT. Begin by creating the network with the mkbr.sh script and starting the VMs with the runvm.sh script shown in Simple NAT.
# sudo /bin/sh mkbr.sh reset bridge0 tap1 tap4 bridge1 tap0 tap5 # /bin/sh runvm.sh firewall external1 internal
Assign the IP addresses as shown, and ensure all VMs have connectivity with adjacent systems.
On the internal VM, start up the userv.sh script with port number 5656. Then, on the external1 VM, start up the ucont.sh server with the same port and a time value of 1 second. With no ipfw module loaded, the communications should succeed.
It may be necessary to examine the ucont.sh script and assign the correct address for the connection. |
Before placing a setmark value on a packet, load the ipfw module and redirect the output of the ipfw log by setting the sysctl to log to syslog:
# kldload ipfw # sysctl net.inet.ip.fw.verbose=1
Now insert the following firewall rules and examine the log file /var/log/security:
# ipfw add 1000 allow log udp from any to 10.10.10.20 dst-port 5656 01000 allow log udp from any to 10.10.10.20 5656 # # tail -f /var/log/security Dec 29 22:32:33 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:30463 10.10.10.20:5656 in via em1 Dec 29 22:32:33 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:30463 10.10.10.20:5656 out via em0 Dec 29 22:32:36 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:24588 10.10.10.20:5656 in via em1 Dec 29 22:32:36 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:24588 10.10.10.20:5656 out via em0
Now add the following rule to apply the mark value of 20 (decimal) and observe the change in the logs:
# ipfw add 500 setmark 20 log udp from any to 10.10.10.20 dst-port 5656 00500 setmark 0x14 log udp from any to 10.10.10.20 5656 # # tail -f /var/log/security Dec 29 22:41:20 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:27955 10.10.10.20:5656 in via em1 Dec 29 22:41:20 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:27955 10.10.10.20:5656 out via em0 Dec 29 22:41:23 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:37423 10.10.10.20:5656 in via em1 Dec 29 22:41:23 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:37423 10.10.10.20:5656 out via em0 Dec 29 22:41:25 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:45176 10.10.10.20:5656 in via em1 Dec 29 22:41:25 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:45176 10.10.10.20:5656 mark:0x14 in via em1 Dec 29 22:41:25 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:45176 10.10.10.20:5656 out via em0 Dec 29 22:41:25 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:45176 10.10.10.20:5656 mark:0x14 out via em0 Dec 29 22:41:27 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:21444 10.10.10.20:5656 in via em1 Dec 29 22:41:27 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:21444 10.10.10.20:5656 mark:0x14 in via em1
7.3. NPTv6
IPv6-to-IPv6 Network Prefix Translation (NPTv6) is the process of translating IPv6 header source and destination addresses. Functionally, it is similar to the more well understood Network Address Translation, but without the need to maintain state. It is only the IPv6 source and destination addresses that are translated. The idea here is to allow an edge network to have its own independent addressing scheme while being able to exchange IPv6 traffic with external IPv6 networks through the use of an NPTv6 Translator
RFC 6296 is the definitive document on NPTv6. The example in this section is taken from Sections 2.1 of that document.
The architecture for these examples is based on Simple NAT as in the previous section.
7.3.1. NPTv6 Setup
Use the setup instructions shown in Simple NAT but use the IPv6 addressing as shown below:
Figure 2. NPTv6 Simple Case
At first glance, this appears to be a simple IPv6 forwarding example. However in this case, NPTv6 changes the actual packet source and destination addresses, so no forwarding is needed.
ipfw(8) explains the syntax of the NPTv6 command and options, but there are a number of details that need to be set up correctly. Use the following as a guide:
On the FreeBSD host: $ sudo /bin/sh mkbr.sh reset bridge0 tap1 tap4 bridge1 tap0 tap5 $ /bin/sh runvm.sh external1 firewall internal Ensure all IPv6 addresses on all VMs are set up correctly. On the firewall VM: # kldunload ipfw_nptv6 # kldunload ipfw # kldload ipfw # kldload ipfw_nptv6 # sysctl net.inet.ip.fw.one_pass=0 # sysctl net.inet.ip.fw.verbose=1 # ipfw -q flush # Set up the NPTv6 instance. # ipfw nptv6 foo create int_prefix fd01:0203:0405:: ext_prefix 2001:0db8:0001:: prefixlen 48 # Rules for nptv6 # ipfw add 500 allow log ipv6-icmp from any to any icmp6types 135,136 // allow neighbor solicitation # *ipfw add 2000 nptv6 foo log ip6 from fd01:0203:0405::/48 to any # ipfw add 3000 allow ip6 from any to any
As noted in ipfw(8), the sysctl net.inet6.ip6.forwarding=1 must be applied or NPTv6 will silently stop working.
7.3.2. NPTv6 Testing
Set up a UDP listener on the external1 VM. Using the userv.sh (and its ucon.sh partner) is possible, but that would require editing the scripts to set up an IPv6 address. Try this method instead:
On the external1 VM: # Listen for a UDP packet $ ncat -l -k -u -6 2001:0db8:0001::10 5656 On the internal VM: # Set up the default route for IPv6 # route -6 add default fd01:0203:0405::0050 # # Send the desired UDP packet. $ echo "testing123" | ncat -6 -u 2001:0db8:0001::10 5656
In the setup section above, logging to syslogd was set up, so the results can be seen by examining the tail end of /var/log/security:
Dec 31 19:51:44 firewall kernel: ipfw: 2000 Eaction nptv6 UDP [fd01:203:405::20]:52451 [2001:db8:1::10]:5656 in via em0
The output of a tcpdump on external1 shows:
root@external1:~ # tcpdump -n -i em0 -X "udp"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:51:43.827543 IP6 2001:db8:1:d54f::20.52451 > 2001:db8:1::10.5656: UDP, length 11
0x0000: 600a 145a 0013 113f 2001 0db8 0001 d54f `..Z...?.......O
0x0010: 0000 0000 0000 0020 2001 0db8 0001 0000 ................
0x0020: 0000 0000 0000 0010 cce3 1618 0013 f72b ...............+
0x0030: 7465 7374 696e 6731 3233 0a testing123.The highlighted section shows the effect of the NPTv6 translation. (See RFC 6296, Section 3, for details.)
7.4. ipttl
The ipttl (Time to Live or TTL) keyword identifies packets that have specific TTL characteristics. ipfw(8) notes that the ipttl keyword will accept a single value, a list of values, or a range of values, in the same syntax as that used for the ports keyword. (Recall the discussion of lists and ranges in the Notes on Rule Numbering.)
ipttl is one of a number of ipfw keywords that work on individual fields of packets flowing through the firewall. Similar keywords include ipid, iplen, ipprecedence, etc. The ipttl keyword controls the lifetime of the packet on the network (see below).
7.4.1. ipttl Setup
Use the setup instructions shown in Simple NAT with IPv4 addressing, not IPv6.
Also, this example will use the hping3 command. (To downloaded the hping3 package, reset the internal VM for access to the Internet, and download the package with pkg install hping3. Remember to reset for Simple NAT IP addressing for this example.)
Refer to hping3(8) for details.
On the external1 VM: # Listen for a UDP packet # ncat -l -k -u 203.0.113.10 5656 On the internal VM: # Send the desired UDP packet. Now, deliberately set the initial TTL to 13. # hping3 --sign "test for ttl 13" --count 1 --udp --ttl 13 --destport 5656 203.0.113.10
Without ipfw in place, the result should be similar to:
root@external1:~ # tcpdump -n -i em0 -X -vv "udp"
tcpdump: listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:49:16.658160 IP (tos 0x0, ttl 12, id 6642, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.2472 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 19f2 0000 0c11 44a8 0a0a 0a14 E..+......D.....
0x0010: cb00 710a 09a8 1618 0017 3013 7465 7374 ..q.......0.test
0x0020: 2066 6f72 2074 746c 2031 3300 0000 .for.ttl.13...The IP Time to Live option was set up to prevent IP packets from bouncing around the Internet forever. RFC 791 initially intended that the value would be considered an actual time value (number of seconds) and that each module processing the packet would subtract processing time from the initial value. This was later changed to an integer identifying a "hop count" where the initial value (now 64) is decremented by every router or gateway or forwarding device, such as a firewall.
In this case the firewall VM, even though it is not running firewall software, is still a 'forwarding device' and decrements the count as it forwards the packet.
7.4.2. ipttl Testing
To examine the ipttl keyword follow this example:
# kldload ipfw # sysctl net.inet.ip.fw.verbose=1 # Count all packets as the flow through # ipfw add 800 count ip from any to any # Count all packets with TTL of exactly 13 as they enter em0. # ipfw add 900 count ip from any to any ipttl 13 # Allow and log packets with TTL of exactly 13 as they enter em0. # ipfw add 1000 allow log udp from any to any ipttl 13 # Just before the packet exits, the IP stack decrements the ttl, # so the following rule is also needed for the packet to exit out em1. # ipfw add 1050 allow log udp from any to any ipttl 12 # Count any other ip packets after the ipttl rule # ipfw add 1100 count ip from any to any
Below is a sample run of ncat and hping3 commands to test the above rules:
# echo "UDP with default TTL" | ncat -u 203.0.113.10 5656 # echo "UDP with default TTL" | ncat -u 203.0.113.10 5656 # hping3 --sign "UDP with TTL=13" --count 1 --udp --ttl 13 --destport 5656 203.0.113.10 # hping3 --sign "UDP with TTL=13" --count 1 --udp --ttl 13 --destport 5656 203.0.113.10
The results show the first two packets with default TTL values (64) were not passed by the firewall. The third and fourth packets were passed. The traces below show the input packet in interface em0 with ttl 13, and the output packet on em1 with ttl 12.
# tcpdump -n -i em0 -X -vvv "udp"
tcpdump: listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
03:26:42.875634 IP (tos 0x0, ttl 13, id 17385, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.1648 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 43e9 0000 0c11 1ab1 0a0a 0a14 E..+C...........
0x0010: cb00 710a 0670 1618 0017 1d07 5544 5020 ..q..p......UDP.
0x0020: 7769 7468 2054 544c 3d31 3300 0000 with.TTL=13...
#
#
# tcpdump -n -i em1 -X -vvv "udp"
tcpdump: listening on em1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
03:27:37.903863 IP (tos 0x0, ttl 12, id 33936, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.2539 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 8490 0000 0c11 da09 0a0a 0a14 E..+............
0x0010: cb00 710a 09eb 1618 0017 198c 5544 5020 ..q.........UDP.
0x0020: 7769 7468 2054 544c 3d31 3300 0000 with.TTL=13...7.5. tcpdatalen
The tcpdatalen keyword is one of several related keywords:
tcpack, tcpdatalen, tcpflags, tcpmss, tcpseq, tcpwin, tcpoptions
These keywords are not often used.
However, there is one very important use case. From time to time, an Internet worm - a malicious packet that gets resent to all local and remote hosts matching some criteria - makes its way onto the Internet. Quick thinking network security administrators can sometimes identify a unique characteristic of these malicious packets such as all packets having the same length - akin to tcpdatalen, or a certain set of tcpoptions.
In this example, the firewall VM is running the tserv.sh 5656 script.
It may be necessary to edit the tserv.sh script to listen on the correct interface (em1) for this example. |
The example below, using the Simple NAT setup and addressing, configures ipfw to deny all packets having a TCP data length of a certain value range. But, it also allows the completion of the TCP 3-way handshake. When the handshake is completed, one of these ranges will cause the malicious packet to be denied. Keep in mind, this is the length of the TCP data payload, not the overall length of the packet.
# ipfw -q flush # ipfw add 10 deny tcp from any to me tcpdatalen 10-19 # ipfw add 20 deny tcp from any to me tcpdatalen 20-29 # ipfw add 30 deny tcp from any to me tcpdatalen 30-39 # ipfw add 40 deny tcp from any to me tcpdatalen 40-49 # ipfw add 50 deny tcp from any to me tcpdatalen 50-59 # ipfw add 60 deny tcp from any to me tcpdatalen 60-69 # ipfw add 70 deny tcp from any to me tcpdatalen 70-79 # ipfw add 80 deny tcp from any to me tcpdatalen 80-89 # ipfw add 90 deny tcp from any to me tcpdatalen 90-99 # ipfw add 500 check-state # ipfw add 1000 allow tcp from any to any 5656 setup keep-state # # ipfw show 00010 0 0 deny log tcp from any to me tcpdatalen 10-19 00020 0 0 deny log tcp from any to me tcpdatalen 20-29 00030 0 0 deny log tcp from any to me tcpdatalen 30-39 00040 0 0 deny log tcp from any to me tcpdatalen 40-49 00050 0 0 deny log tcp from any to me tcpdatalen 50-59 00060 0 0 deny log tcp from any to me tcpdatalen 60-69 00070 0 0 deny log tcp from any to me tcpdatalen 70-79 00080 0 0 deny log tcp from any to me tcpdatalen 80-89 00090 0 0 deny log tcp from any to me tcpdatalen 90-99 00500 0 0 check-state :default 01000 0 0 allow log tcp from any to any setup keep-state :default 65535 0 0 deny ip from any to any
And a test using ncat directly from external1:
# echo "1234567890123456789012345678901234" | ncat 203.0.113.50 5656
The TCP 3-way handshake completes, but the packet containing the data payload is stopped by rule 30 as shown below:
# ipfw show 00010 0 0 deny log tcp from any to me tcpdatalen 10-19 00020 0 0 deny log tcp from any to me tcpdatalen 20-29 00030 13 1066 deny log tcp from any to me tcpdatalen 30-39 00040 0 0 deny log tcp from any to me tcpdatalen 40-49 00050 0 0 deny log tcp from any to me tcpdatalen 50-59 00060 0 0 deny log tcp from any to me tcpdatalen 60-69 00070 0 0 deny log tcp from any to me tcpdatalen 70-79 00080 0 0 deny log tcp from any to me tcpdatalen 80-89 00090 0 0 deny log tcp from any to me tcpdatalen 90-99 00500 0 0 check-state :default 01000 8 420 allow log tcp from any to any 5656 setup keep-state :default 65535 0 0 deny ip from any to any
The reason for the excessive number of packets denied is TCP retransmission trying to account for the dropped packet as shown in the wireshark(1) trace below.
Figure 3. Denying Packet Based on TCP Data Length
Eventually TCP gives up and shuts down the connection.
7.6. verrevpath / versrcreach / antispoof
These keywords all work to determine if an incoming packet is legitimate.
As noted in ipfw(8), verrevpath ("verify reverse path") looks up the incoming packet’s source address in the routing table.
Quoting: "If the interface on which the packet entered the system matches the outgoing interface for the route, the packet matches. If the interfaces do not match up, the packet does not match. All outgoing packets or packets with no incoming interface match."
Setting up on the FreeBSD host: % cd ~/ipfw-primer/ipfw/HOST_SCRIPTS % sudo /bin/sh mkbr.sh reset bridge0 tap0 tap5 bridge1 tap4 tap1 % /bin/sh runvm.sh external1 firewall internal
Consider the figure below (same as Simple NAT):
Figure 4. verrevpath Example
In this figure, the firewall has interface em0 directly connected to the 10.10.10.0/24 network and the em1 interface directly connected to the 203.0.113.0/24 network.
The firewall VM interfaces and routing table are shown in the text below:
root@firewall:~ # ifconfig em0
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 02:49:50:46:57:41
inet 10.10.10.50 netmask 0xffffff00 broadcast 10.10.10.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@firewall:~ #
root@firewall:~ # ifconfig em1
em1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 02:49:50:46:57:42
inet 203.0.113.50 netmask 0xffffff00 broadcast 203.0.113.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@firewall:~ #
root@firewall:~ # netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
10.10.10.0/24 link#1 U em0
10.10.10.50 link#3 UHS lo0
127.0.0.1 link#3 UH lo0
203.0.113.0/24 link#2 U em1
203.0.113.50 link#3 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 link#3 URS lo0
::1 link#3 UHS lo0
::ffff:0.0.0.0/96 link#3 URS lo0
fe80::%lo0/10 link#3 URS lo0
fe80::%lo0/64 link#3 U lo0
fe80::1%lo0 link#3 UHS lo0
ff02::/16 link#3 URS lo0
root@firewall:~ #If a packet came in on the em0 interface with a source address that was not in the 10.10.10.0/24 network, the above quote says the packet should be dropped.
The following example tests this with the ncat program which has an option to set the source IP.
First, set up ipfw on the firewall VM to allow any UDP packets as shown.
Then, set up the firewall VM to run sh userv.sh 5656, the service to receive UDP packets on the identified port. Next, send one packet from the internal VM with echo "hello from internal VM" | ncat -u 10.10.10.50 5656.
root@firewall:~/bin # ipfw add 1000 allow udp from any to me verrevpath 01000 allow udp from any to me verrevpath root@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 0 0 allow udp from any to me verrevpath 65535 0 0 deny ip from any to any root@firewall:~/bin # root@firewall:~/bin # sh userv.sh 5656 PORT1 = [5656] Starting UDP listener on [10.10.10.50],[5656] hello from internal VM ^Croot@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 1 51 allow udp from any to me verrevpath 65535 0 0 deny ip from any to any root@firewall:~/bin #
So far, so good. This is expected behavior.
Now zero the rule counts on the firewall VM and send a similar message from the internal VM, but this time spoof the source address. This requires adding an alias IP address to the interface on the internal VM:
root@internal:~/bin # ifconfig em0 4.4.4.4/32 alias root@internal:~/bin # root@internal:~/bin # echo "hello 2 from internal VM" | ncat -u -s 4.4.4.4 10.10.10.50 5656 root@internal:~/bin #
Now, rule 1000 prevents the matching of the incoming packet with a spoofed source address and no packet is received by the userv.sh service. Instead, the packet is handled by the default deny rule:
root@firewall:~/bin # sh userv.sh 5656 PORT1 = [5656] Starting UDP listener on [10.10.10.50],[5656] ^Croot@firewall:~/bin # root@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 0 0 allow udp from any to me verrevpath 65535 1 53 deny ip from any to any root@firewall:~/bin #
The other keywords in this section, versrcreach and antispoof operate in a similar manner. Check the man page for the slight differences between them.
7.7. jail
Jails are an important component of FreeBSD and have been a part of the base system since FreeBSD 4. ipfw works in tandem with jails to provide networking security. As discussed in the FreeBSD Handbook Section on Jails and Networking, there are three types of jail networking setups. This section discusses the first two:
Host Networking Setup
Virtual Networking (VNET) Setup
7.7.1. Host-based Jail Networking
In this type of networking setup, the jail shares the host networking stack. The jail has the same IP address and interface as the host.
Recall that the jail1 VM has different characteristics than the standard VMs used in this book. It has 8GB memory, a bigger disk, and is running ZFS for its filesystem.
Instructions for setting up this type of jail are found in the FreeBSD Handbook Section Creating a Thin Jail Using OpenZFS Snapshots.
% cd ~/ipfw-primer/ipfw/HOST_SCRIPTS % sudo /bin/sh mkbr.sh reset bridge0 tap12 host_interface % /bin/sh runvm.sh jail1
Set up the jail1 VM to use DHCP addressing and follow the instructions in the handbook to create a thinjail using ZFS, including creating /etc/jail.conf with the parameters shown in that section.
Once that is completed, reconfigure the FreeBSD host for this example.
Set up the external1 and jail1 VMs with these commands on the FreeBSD host:
% cd ~/ipfw-primer/ipfw/HOST_SCRIPTS % sudo /bin/sh mkbr.sh reset bridge0 tap1 tap12 % /bin/sh runvm.sh external1 jail1
and use the addressing shown in the figure below.
Figure 5. Jail With Host Based Networking
The typical jail configuration file for this setup uses the following network configuration:
jailname {
. . .
# Network
ip4 = inherit;
interface = em0;
. . .
}
Login to the jail1 VM and start the jail with:
# service jail onestart thinjail
Access the jail with the jexec command:
# jexec -u root thinjailThe jail named thinjail is now using the host /etc/jail.conf example, which uses the inherited IPv4 network stack, for the jail.
There are now three different command line environments - the FreeBSD host, the QEMU jail1 VM, and the thinjail running inside the jail1 VM. Keep track of which command line you are using by watching the shell prompt. |
Here, it is the host that controls the network stack and all ipfw commands (loading, unloading, adding/deleteing rules, etc.) must be done from the host. The jail root user does not have permission to operate ipfw inside the jail.
Figure 6. Jail With Host Based Networking
All ipfw configuration for the jail must be done on the host. ipfw provides the jail keyword for this purpose. For IP communications, this keyword applys primarily to outbound packets from the jail. Inbound packets to the jail, follow the normal host rules.
By default, if the thinjail runs nc -l 203.0.113.75 5656, it opens up a TCP socket listening on port 5656 in the jail1 VM. If instead, the jail1 VM runs the identical command in the jail1 VM, the listening socket is not visible to the thinjail.
The conditions for outside access to thinjail rely on the host network, and the jail jailname keyword is not needed.
root@jail1:# kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled root@jail1:# root@jail1:# ipfw add 100 check-state root@jail1:# ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state
This rule on the host system will allow a connection from the external1 VM to reach the above nc -l 203.0.113.75 5656 running inside thinjail.
For outbound TCP communication, rule 2000 below, using the jail thinjail keyword is required. For the most part, the ipfw rules used elsewhere in this book are applicable here with the addition of the jail jailname keyword.
# ipfw add 100 check-state # ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state # ipfw add 2000 allow tcp from me to any setup keep-state jail thinjail
Always provide the jail name rather than a numeric ID. If the jail is restarted for any reason it may get a new jail ID number and an existing rule with a jail number will be immediately out of date. The rule will have to be re-entered using the jail name. |
When entering a rule with a jail name, ipfw will lookup the name and reply with the number.
So even when listing or showing the ruleset, ipfw will always show the number not the name.
Use the |
It is a good idea to compartmentalize the rules for each jail in a file with the jail name. That way, if a jail is restarted, the specific file can be rerun to update the ipfw rules on the host. |
7.7.2. Virtual Network (VNET) Jail Networking
A more advanced setup is using the VNET networking capabilities of FreeBSD for the jail. There are many good online tutorials on setting up VNET jails. This section is focused on the use of ipfw with a VNET network for the jail.
The architecture for this setup is shown in the figure below and is similar to that used in the last section.
Figure 7. Jail With VNET Based Networking
While there are two |
For this section, create a second thinjail named vnetjail as follows:
root@jail1:# zfs clone zroot/jails/templates/14.2-RELEASE@base zroot/jails/containers/vnetjail See below for configuring /etc/jail.conf.
Configuring multiple jails can be done with separate sections in /etc/jail.conf, or by creating separate configuration files in /etc/jail.d/jailname.conf. See jail.conf(5) for details. |
The vnetjail configuration sets up a VNET network as follows:
#
# vnetjail.conf - handbook/jails - setting up a thin jail under ZFS
#
vnetjail {
# Startup / Logging
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";
# Permissions
allow.raw_sockets;
exec.clean;
mount.devfs;
devfs_ruleset = 5;
# Hostname / Path
host.hostname = "${name}";
path = "/usr/local/jails/containers/${name}";
# VNET / VIMAGE
vnet;
vnet.interface = "${epair}b";
# Network
$id = "90";
$ip = "203.0.113.${id}/24";
$gateway = "203.0.113.50";
$bridge = "bridge0";
$epair = "epair${id}";
# ADD TO bridge INTERFACE
exec.prestart = "/sbin/ifconfig ${bridge} create up";
exec.prestart += "/sbin/ifconfig ${epair} create up";
exec.prestart += "/sbin/ifconfig ${epair}a up descr jail:${name}";
exec.prestart += "/sbin/ifconfig ${bridge} addm ${epair}a up";
exec.prestart += "/sbin/ifconfig ${bridge} addm em0";
exec.start += "/sbin/ifconfig ${epair}b ${ip} up";
exec.start += "/sbin/route add default ${gateway}";
exec.poststop = "/sbin/ifconfig ${bridge} deletem ${epair}a";
exec.poststop += "/sbin/ifconfig ${bridge} deletem em0";
exec.poststop += "/sbin/ifconfig ${epair}a destroy";
exec.poststop += "/sbin/ifconfig ${bridge} destroy";
}In this instance, the network stack is completely separate from the host network stack. However, achieving and managing connectivity happens inside the vnetjail jail.
Testing connectivity with the jail can be accomplished by
Ensuring ipfw is not loaded on jail1,
Entering the vnetjail, and
Starting up a listening service using nc(1):
root@jail1:~ # kldunload ipfw IP firewall unloaded root@jail1:~ # service jail onestart vnetjail root@jail1:~ # root@jail1:~ # jexec -u root vnetjail root@vnetjail:/ # cd root@vnetjail:~ # root@vnetjail:~ # nc -l -k 5656
Connecting from external1 using nc(1):
root@external1:~ # nc 203.0.113.90 5656 hello from external1 ^C #
With no ipfw firewall in place, the test is successful.
To apply ipfw rules for the vnetjail jail, start ipfw in the jail1 VM.
In VNET jails, ipfw is started from outside the jail, but rules are added from inside the jail. ipfw is also stopped from outside the jail. |
Then, from inside the vnetjail jail, start up a listener using nc(1):
root@vnetjail:~ # nc -l -k 5656 root@vnetjail:~ #
Since the vnetjail jail has a separate IP address and network stack from the jail1 VM, orient ipfw rules around the vnetjail IP address:
root@jail1:~ # kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled root@jail1:~ # root@jail1:~ # jexec -u root vnetjail root@vnetjail:/ # cd root@vnetjail:~ # root@vnetjail:~ # ipfw show 65535 0 0 deny ip from any to any root@vnetjail:~ # root@vnetjail:~ # ipfw add 100 check-state 00100 check-state :default root@vnetjail:~ # root@vnetjail:~ # ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state 01000 allow tcp from any to me 5656 setup keep-state :default root@vnetjail:~ # root@vnetjail:~ # ipfw show 00100 0 0 check-state :default 01000 0 0 allow tcp from any to me 5656 setup keep-state :default 65535 0 0 deny ip from any to any The single rule above is enough to set up a TCP connection. From external1: root@external1:~ # nc 203.0.113.90 5656 Hello from external1 after ipfw rules have been set up. ^C root@external1:~ # After the above: root@vnetjail:~ # root@vnetjail:~ # ipfw show 00100 0 0 check-state :default 01000 18 1012 allow tcp from any to me 5656 setup keep-state :default 65535 0 0 deny ip from any to any root@vnetjail:~ #
Setting up rules in the vnetjail is left as an exercise to the reader.