Chapter 7. Other Keywords
Table of Contents
This section covers some other lesser used keywords.
7.1. abort / abort6
The abort and abort6 keywords interrupt the data stream between two endpoints. The effect of this keyword, is similar to the reset keyword, but there are important differences.
Figure 1. Abort and abort6 keywords
The above figure shows the effect of inserting the firewall rule:
# ipfw add 50 abort tcp from 203.0.113.30 to me
Unlike the reset keyword, there is no packet sent from the firewall to the source. What happens is that ipfw just starts dropping packets that match the rule. Since there are no more replies coming from the destination (here the firewall itself), the source endpoint issues retransmissions over and over. Eventually the source concludes that the connection is irrevocably broken and it closes the connection.
In the rule above, all TCP connections will be interrupted between the two systems. |
In a TCP connection, ipfw will use dynamic rules if a check-state rule is already in place. If this is the case, issue the abort rule at a rule number before the check-state rule. Otherwise, it will have no effect. |
7.2. mark / setmark
The setmark keyword functions similar to the tag keyword. If the packet matches the rule, ipfw applies a 32-bin identifier to the packet. This identifier (the "mark") is held with the packet internally inside ipfw. It is not sent with the packet on the wire and is not visible to any network monitoring from tools like tcpdump(1) or man:[wireshark].
Like tags, a mark can be used as another filtering device with other ipfw rules to do policy base routing or filtering. Note that only one mark can be applied at a time.
A big advantage of marks over tags are their ability to be matched as a lookup key in a table. Also, a mark can have a bitmask applied to it.
To explore mark and setmark we will use the architecture of Simple NAT shown in Simple NAT. Begin by creating the network with the mkbr.sh script and starting the VMs with the runvm.sh script shown in Simple NAT.
Assign the IP addresses as shown, and ensure all VMs have connectivity with adjacent systems.
On the internal VM, start up the userv.sh script with port number 5656. Then, on the external1 VM, start up the ucont.sh server with the same port and a time value of 1 second.
Before we place a setmark value on a packet, start the data communications scripts and examine the output of the ipfw log by setting the sysctl to log to syslog:
# sysctl net.inet.ip.fw.verbose=1
Now insert the following firewall rules and examine the log file /var/log/security:
# ipfw add 1000 allow log udp from any to 10.10.10.20 dst-port 5656 01000 allow log udp from any to 10.10.10.20 5656 # # tail -f /var/log/security Dec 29 22:32:33 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:30463 10.10.10.20:5656 in via em1 Dec 29 22:32:33 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:30463 10.10.10.20:5656 out via em0 Dec 29 22:32:36 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:24588 10.10.10.20:5656 in via em1 Dec 29 22:32:36 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:24588 10.10.10.20:5656 out via em0
Now add the following rule to apply the mark value of 20 (decimal) and observe the change in the logs:
# ipfw add 500 setmark 20 log udp from any to 10.10.10.20 dst-port 5656 00500 setmark 0x14 log udp from any to 10.10.10.20 5656 # # # tail -f /var/log/security Dec 29 22:41:20 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:27955 10.10.10.20:5656 in via em1 Dec 29 22:41:20 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:27955 10.10.10.20:5656 out via em0 Dec 29 22:41:23 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:37423 10.10.10.20:5656 in via em1 Dec 29 22:41:23 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:37423 10.10.10.20:5656 out via em0 Dec 29 22:41:25 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:45176 10.10.10.20:5656 in via em1 Dec 29 22:41:25 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:45176 10.10.10.20:5656 mark:0x14 in via em1 Dec 29 22:41:25 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:45176 10.10.10.20:5656 out via em0 Dec 29 22:41:25 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:45176 10.10.10.20:5656 mark:0x14 out via em0 Dec 29 22:41:27 firewall kernel: ipfw: 500 SetMark 0x14 UDP 203.0.113.10:21444 10.10.10.20:5656 in via em1 Dec 29 22:41:27 firewall kernel: ipfw: 1000 Accept UDP 203.0.113.10:21444 10.10.10.20:5656 mark:0x14 in via em1
7.3. NPTv6
IPv6-to-IPv6 Network Prefix Translation (NPTv6) is the process of translating IPv6 header source and destination addresses. Functionally, it is similar to the more well understood Network Address Translation, but without the need to maintain state. It is only the IPv6 source and destination addresses that are translated. The idea here is to allow an edge network to have its own independent addressing scheme while being able to exchange IPv6 traffic with external IPv6 networks through the use of an NPTv6 Translator
RFC 6296 is the definitive document on NPTv6. The example in this section is taken from Sections 2.1 of that document.
The architecture for these examples is based on Simple NAT as in the previous section.
7.3.1. NPTv6 Setup
Use the setup instructions shown in Simple NAT but use the IPv6 addressing as shown below:
Figure 2. NPTv6 Simple Case
At first glance, this appears to be a simple IPv6 forwarding example. As we will see, NPTv6 changes the actual packet source and destination addresses, so no forwarding is needed.
ipfw(8) explains the syntax of the NPTv6 command and options, but there are a number of details that need to be set up correctly. Use the following as a guide:
On the FreeBSD host: $ sudo /bin/sh mkbr.sh reset bridge0 tap1 tap4 bridge1 tap0 tap5 $ /bin/sh runvm.sh external1 firewall internal Ensure all IPv6 addresses on all VMs are set up correctly. On the firewall VM: # kldunload ipfw_nptv6 # kldunload ipfw # kldload ipfw # kldload ipfw_nptv6 # sysctl net.inet.ip.fw.one_pass=0 # sysctl net.inet.ip.fw.verbose=1 # ipfw -q flush # Set up the NPTv6 instance. # ipfw nptv6 foo create int_prefix fd01:0203:0405:: ext_prefix 2001:0db8:0001:: prefixlen 48 # Rule for outbound # ipfw add 2000 nptv6 foo log ip6 from fd01:0203:0405::/48 to any # ipfw add 3000 allow ip6 from any to any
As noted in ipfw(8), the sysctl net.inet6.ip6.forwarding=1 must be applied or NPTv6 will silently stop working.
7.3.2. NPTv6 Testing
Set up a UDP listener on the external1 VM. We could use the userv.sh (and its ucon.sh partner), but that would require editing the scripts to set up an IPv6 address. Try this method instead:
On the external1 VM: # Listen for a UDP packet $ ncat -l -k -u -6 2001:0db8:0001::10 5656 On the internal VM: # Send the desired UDP packet. $ echo "testing123" | ncat -6 -u 2001:0db8:0001::10 5656
In the setup section above, we arranged for logging to syslogd, so the results can be seen by examining the tail end of /var/log/security:
Dec 31 19:51:44 firewall kernel: ipfw: 2000 Eaction nptv6 UDP [fd01:203:405::20]:52451 [2001:db8:1::10]:5656 in via em0
The output of a tcpdump on external1 shows:
root@external1:~ # tcpdump -n -i em0 -X "udp"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:51:43.827543 IP6 2001:db8:1:d54f::20.52451 > 2001:db8:1::10.5656: UDP, length 11
0x0000: 600a 145a 0013 113f 2001 0db8 0001 d54f `..Z...?.......O
0x0010: 0000 0000 0000 0020 2001 0db8 0001 0000 ................
0x0020: 0000 0000 0000 0010 cce3 1618 0013 f72b ...............+
0x0030: 7465 7374 696e 6731 3233 0a testing123.The highlighted section shows the effect of the NPTv6 translation. (See RFC 6296, Section 3, for details.)
7.4. ipttl
The ipttl (Time to Live or TTL) keyword identifies packets that have specific TTL characteristics. ipfw(8) notes that the ipttl keyword will accept a single value, a list of values, or a range of values, in the same syntax as that used for the ports keyword. (Recall the discussion of lists and ranges in the Notes on Rule Numbering.)
ipttl is one of a number of ipfw keywords that work on individual fields of packets flowing through the firewall. Similar keywords include ipid, iplen, ipprecedence, etc.
7.4.1. ipttl Setup
Use the setup instructions shown in Simple NAT with IPv4 addressing, not IPv6.
Also, this example will use the hping3 command. (If you have not downloaded the hping3 package, reset the internal VM for access to the Internet, and download the package with pkg install hping3. Remember to reset for Simple NAT for this example.)
Refer to hping3(8) for details.
On the external1 VM: # Listen for a UDP packet $ ncat -l -k -u -6 203.0.113.10 5656 On the internal VM: # Send the desired UDP packet. We deliberate set the initial TTL to 13. $ hping3 --sign "test for ttl 13" --count 1 --udp --ttl 13 --destport 5656 203.00.113.10
Without ipfw in place, you should see something similar to:
01:50:00.765310 IP (tos 0x0, ttl 12, id 27704, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.2600 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 6c38 0000 0c11 f261 0a0a 0a14 E..+l8.....a....
0x0010: cb00 710a 0a28 1618 0017 2f93 7465 7374 ..q..(..../.test
0x0020: 2066 6f72 2074 746c 2031 3300 0000 .for.ttl.13...The IP Time to Live option was set up to prevent IP packets from bouncing around the Internet forever. RFC 791 initially intended that the value would be considered an actual time value (number of seconds) and that each module processing the packet would subtract processing time from the initial value. This was later changed to an integer identifying a "hop count" where the initial value (now 64) is decremented by every router or gateway or forwarding device, such as a firewall.
In this case the firewall VM, even though it is not running firewall software, is still a 'forwarding device' and decrements the count as it forwards the packet.
7.4.2. ipttl Testing
To examine the ipttl keyword follow this example:
# kldload ipfw # sysctl net.inet.ip.fw.verbose=1 # Count all packets as the flow through # ipfw add 800 count ip from any to any # Count all packets with TTL of exactly 13 as the flow through # ipfw add 900 count ip from any to any ipttl 13 # Allow and log packets with TTL of exactly 13. # ipfw add 1000 allow log udp from any to any ipttl 13 # Count any other ip packets after the ipttl rule # ipfw add 1100 count ip from any to any
Below is a sample run of ncat and hping3 commands to test the above rules:
# echo "UDP with default TTL" | ncat -u 203.0.113.10 5656
# echo "UDP with default TTL" | ncat -u 203.0.113.10 5656
# hping3 --sign "UDP with TTL=13" --count 1 --udp --ttl 13 --destport 5656 203.0.113.10
# hping3 --sign "UDP with TTL=13" --count 1 --udp --ttl 13 --destport 5656 203.0.113.10
The results show the first two packets with default TTL values (64) were not passed by the firewall.
The third and fourth packets were passed.
# *tcpdump -n -i em0 -X -vvv "udp"
tcpdump: listening on em0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
03:26:42.875634 IP (tos 0x0, ttl 12, id 17385, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.1648 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 43e9 0000 0c11 1ab1 0a0a 0a14 E..+C...........
0x0010: cb00 710a 0670 1618 0017 1d07 5544 5020 ..q..p......UDP.
0x0020: 7769 7468 2054 544c 3d31 3300 0000 with.TTL=13...
03:26:47.903863 IP (tos 0x0, ttl 12, id 33936, offset 0, flags [none], proto UDP (17), length 43)
10.10.10.20.2539 > 203.0.113.10.5656: [udp sum ok] UDP, length 15
0x0000: 4500 002b 8490 0000 0c11 da09 0a0a 0a14 E..+............
0x0010: cb00 710a 09eb 1618 0017 198c 5544 5020 ..q.........UDP.
0x0020: 7769 7468 2054 544c 3d31 3300 0000 with.TTL=13...But it is the ipfw show results that reveal how things really worked:
# ipfw show 00800 6 270 count ip from any to any 00900 2 86 count ip from any to any ipttl 13 01000 2 86 allow log udp from any to any ipttl 13 01100 4 184 count ip from any to any 01200 2 86 allow log udp from any to any ipttl 12 65535 2 98 deny ip from any to any
The count of 6 packets on rule 800 above accounts for the inbound and outbound packets for those that matched later rules. |
7.5. tcpdatalen
The tcpdatalen keyword is one of several related keywords:
tcpack, tcpdatalen, tcpflags, tcpmss, tcpseq, tcpwin, tcpoptions
These keywords are not often used.
However, there is one very important use case. From time to time, an Internet worm - a malicious packet that gets resent to all local and remote hosts matching some criteria - makes its way onto the Internet. Quick thinking network security administrators can sometimes identify a unique characteristic of these malicious packets such as all packets having the same length - akin to tcpdatalen, or a certain set of tcpoptions.
In this example, the firewall VM is running the tserv.sh 5656 script.
The example below configures ipfw to deny all packets having a TCP data length of a certain value range. One of these ranges will cause the malicious packet to be denied. Keep in mind, this is the length of the TCP data payload, not the overall length of the packet.
# ipfw add 10 deny tcp from any to me tcpdatalen 10-19 # ipfw add 20 deny tcp from any to me tcpdatalen 20-29 . . . root@firewall:~/bin # ipfw show 00010 0 0 deny tcp from any to me tcpdatalen 10-19 00020 0 0 deny tcp from any to me tcpdatalen 20-29 00030 0 0 deny tcp from any to me tcpdatalen 30-39 00040 0 0 deny tcp from any to me tcpdatalen 40-49 00050 0 0 deny tcp from any to me tcpdatalen 50-59 00060 0 0 deny tcp from any to me tcpdatalen 60-69 00070 0 0 deny tcp from any to me tcpdatalen 70-79 00080 0 0 deny tcp from any to me tcpdatalen 80-89 00090 0 0 deny tcp from any to me tcpdatalen 90-99 08000 0 0 check-state :default 09000 0 0 allow tcp from any to me setup keep-state :default 65535 0 0 deny ip from any to any
And a test using ncat directly from external3:
# echo "123456789012345678901234567890" | ncat 203.0.113.50 5656
The TCP 3-way handshake completes, but the packet containing the data payload is stopped by rule 30 as shown below:
root@firewall:~/bin # ipfw show 00010 0 0 deny tcp from any to me tcpdatalen 10-19 00020 0 0 deny tcp from any to me tcpdatalen 20-29 00030 13 1079 deny tcp from any to me tcpdatalen 30-39 00040 0 0 deny tcp from any to me tcpdatalen 40-49 00050 0 0 deny tcp from any to me tcpdatalen 50-59 00060 0 0 deny tcp from any to me tcpdatalen 60-69 00070 0 0 deny tcp from any to me tcpdatalen 70-79 00080 0 0 deny tcp from any to me tcpdatalen 80-89 00090 0 0 deny tcp from any to me tcpdatalen 90-99 08000 0 0 check-state :default 09000 8 420 allow tcp from any to me setup keep-state :default 65535 0 0 deny ip from any to any
The reason for the excessive number of packets denied is TCP retransmission trying to account for the dropped packet as shown in the figure below.
Figure 3. Denying Packet Based on TCP Data Length
Eventually TCP gives up and shuts down the connection.
7.6. verrevpath / versrcreach / antispoof
These keywords all work to determine if an incoming packet is legitimate.
As noted in ipfw(8), verrevpath ("verify reverse path") looks up the incoming packet’s source address in the routing table.
Quoting: "If the interface on which the packet entered the system matches the outgoing interface for the route, the packet matches. If the interfaces do not match up, the packet does not match. All outgoing packets or packets with no incoming interface match."
Consider the figure below:
Figure 4. verrevpath Example
In this figure, the firewall has interface em0 directly connected to the 10.10.10.0/24 network and the em1 interface directly connected to the 203.0.113.0/24 network.
The firewall VM interfaces and routing table are shown in the text below:
root@firewall:~ # ifconfig em0
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 02:49:50:46:57:41
inet 10.10.10.50 netmask 0xffffff00 broadcast 10.10.10.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@firewall:~ #
root@firewall:~ # ifconfig em1
em1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 02:49:50:46:57:42
inet 203.0.113.50 netmask 0xffffff00 broadcast 203.0.113.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@firewall:~ #
root@firewall:~ # netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
10.10.10.0/24 link#1 U em0
10.10.10.50 link#3 UHS lo0
127.0.0.1 link#3 UH lo0
203.0.113.0/24 link#2 U em1
203.0.113.50 link#3 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 link#3 URS lo0
::1 link#3 UHS lo0
::ffff:0.0.0.0/96 link#3 URS lo0
fe80::%lo0/10 link#3 URS lo0
fe80::%lo0/64 link#3 U lo0
fe80::1%lo0 link#3 UHS lo0
ff02::/16 link#3 URS lo0
root@firewall:~ #If a packet came in on the em0 interface with a source address that was not in the 10.10.10.0/24 network, the above quote says the packet should be dropped.
We can test this with our trusty ncat program which has an option to set the source IP.
First, we set up the firewall VM to allow any UDP packets as shown.
Then, we set up the firewall VM to run sh userv.sh 5656, our service to receive UDP packets on the identified port, and send one packet from the internal VM with echo "hello from internal VM" | ncat -u 10.10.10.50 5656.
root@firewall:~/bin # ipfw add 1000 allow udp from any to me verrevpath 01000 allow udp from any to me verrevpath root@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 0 0 allow udp from any to me verrevpath 65535 0 0 deny ip from any to any root@firewall:~/bin # root@firewall:~/bin # sh userv.sh 5656 PORT1 = [5656] Starting UDP listener on [10.10.10.50],[5656] hello from internal VM ^Croot@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 1 51 allow udp from any to me verrevpath 65535 0 0 deny ip from any to any root@firewall:~/bin #
So far, so good. This is expected behavior.
Now we restart the service on the firewall VM and send a similar message from the internal VM, but this time we spoof the source address. To do this, we must add an alias IP address to the interface on the internal VM:
root@internal:~/bin # ifconfig em0 4.4.4.4/32 alias root@internal:~/bin # root@internal:~/bin # echo "hello 2 from internal VM" | ncat -u -s 4.4.4.4 10.10.10.50 5656 root@internal:~/bin #
Now, rule 1000 prevents the matching of the incoming packet with a spoofed source address and no packet is received by the userv.sh service. Instead, the packet is handled by the default deny rule:
root@firewall:~/bin # sh userv.sh 5656 PORT1 = [5656] Starting UDP listener on [10.10.10.50],[5656] ^Croot@firewall:~/bin # root@firewall:~/bin # root@firewall:~/bin # ipfw show 01000 0 0 allow udp from any to me verrevpath 65535 1 53 deny ip from any to any root@firewall:~/bin #
The other keywords in this section, versrcreach and antispoof operate in a similar manner. Check the man page for the slight differences between them.
7.7. jail
Jails are an important component of FreeBSD and have been a part of the base system since FreeBSD 4. ipfw works in tandem with jails to provide networking security. As discussed in the FreeBSD Handbook Section on Jails and Networking, there are three types of jail networking setups. We will discuss the first two:
Host Networking Setup
Virtual Networking (VNET) Setup
7.7.1. Host-based Jail Networking
In this type of networking setup, the jail shares the host networking stack. The jail has the same IP address and interface as the host.
The typical jail configuration file for this set uses the following network configuration:
jailname {
. . .
# Network
ip4 = inherit;
interface = em0;
. . .
}Here, it is the host that controls the network stack and all ipfw commands (loading, unloading, adding/deleteing rules, etc.) must be done from the host. The jail root user does not have permission to operate ipfw inside the jail.
Figure 5. Jail With Host Based Networking
All ipfw configuration for the jail must be done on the host. ipfw provides the jail keyword. For TCP communications, this keyword applys primarily to outbound packets from the jail. Inbound packets to the jail, follow the normal host rules.
If the jail runs sh tserv.sh 5656, it opens up a TCP socket listening on port 5656 in the jail. The rule for outside access to this jail relies on the host network, and the jail jailname keyword is not needed.
# ipfw add 100 check-state # ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state
This rule will allow a connection from an external host to port 5656 in the jail.
For the most part, the ipfw rules that we have used elsewhere in this book are applicable here with the addition of the jail jailname keyword. For outbound communication, the added rule below, using the jail thinjail keyword succeeds.
# ipfw add 100 check-state # ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state # ipfw add 2000 allow tcp from me to any setup keep-state jail thinjail
You should always provide the jail name rather than a numeric ID. If the jail is restarted for any reason it may get a new jail ID number and an existing rule with a jail number will be immediately out of date. The rule will have to be re-entered using the jail name. |
When entering a rule with a jail name, ipfw will lookup the name and reply with the number.
So even when listing or showing the ruleset, ipfw will always show the number not the name.
Use the |
It is a good idea to compartmentalize the rules for each jail in a file with the jail name. That way, if a jail is restarted, the specific file can be rerun to update the ipfw rules on the host. |
7.7.2. Virtual Network (VNET) Jail Networking
A more advanced setup is using the VNET networking capabilities of FreeBSD for the jail. There are many good tutorials on setting up VNET jails. This section is focused on the use of ipfw with the vnet network for the jail.
The architecture for this setup is shown in the figure below:
Figure 6. Jail With VNET Based Networking
The architecture shows the FreeBSD host, with two QEMU virtual machines, external1 and jail1.
The two VMs are connected by bridge0 and share the 203.0.113.0/24 network.
jail1 has different characteristics than the standard VMs we have been using. It has 8GB memory, and is running ZFS for its filesystem.
The jail1 VM has set up a FreeBSD thin jail inside the VM following the directions in the FreeBSD handbook on Creating a Thin Jail Using OpenZFS Snapshots.
While there are two |
The jail configuration sets up a vnet jail as follows:
#
# vnetjail.conf - handbook/jails - setting up a thin jail under ZFS
#
vnetjail {
# Startup / Logging
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";
# Permissions
allow.raw_sockets;
exec.clean;
mount.devfs;
devfs_ruleset = 5;
# Hostname / Path
host.hostname = "${name}";
path = "/usr/local/jails/containers/${name}";
# VNET / VIMAGE
vnet;
vnet.interface = "${epair}b";
# Network
$id = "90";
$ip = "203.0.113.${id}/24";
$gateway = "203.0.113.50";
$bridge = "bridge0";
$epair = "epair${id}";
# ADD TO bridge INTERFACE
exec.prestart = "/sbin/ifconfig ${bridge} create up";
exec.prestart += "/sbin/ifconfig ${epair} create up";
exec.prestart += "/sbin/ifconfig ${epair}a up descr jail:${name}";
exec.prestart += "/sbin/ifconfig ${bridge} addm ${epair}a up";
exec.prestart += "/sbin/ifconfig ${bridge} addm em0";
exec.start += "/sbin/ifconfig ${epair}b ${ip} up";
exec.start += "/sbin/route add default ${gateway}";
exec.poststop = "/sbin/ifconfig ${bridge} deletem ${epair}a";
exec.poststop += "/sbin/ifconfig ${epair}a destroy";
}This time, the network stack is completely separate from the host network stack. However, achieving and managing connectivity happens inside the vnetjail jail.
Testing connectivity with the jail can be accomplished by
Ensuring ipfw is not loaded on jail1,
Entering the vnetjail, and
Starting up a listening service using nc(1):
root@jail1:~ # kldunload ipfw IP firewall unloaded root@jail1:~ # root@jail1:~ # jexec -u root vnetjail root@vnetjail:/ # cd root@vnetjail:~ # root@vnetjail:~ # nc -l -k 5656
Connecting from external1 using nc(1):
root@external1:~ # nc 203.0.113.90 5656 hello from external1 ^C #
With no ipfw firewall in place, the test is successful.
To apply ipfw rules for the vnetjail jail, start ipfw in the jail1 VM.
In VNET jails, ipfw is started from outside the jail, but rules are added from inside the jail. ipfw is also stopped from outside the jail. |
Then, from inside the vnetjail jail, start up a listener using nc(1):
root@vnetjail:~ # nc -l -k 5656 root@vnetjail:~ #
Since the vnetjail jail has a separate IP address and network stack from the jail1 VM, we orient ipfw rules around the vnetjail IP address:
root@jail1:~ # kldload ipfw ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled root@jail1:~ # root@jail1:~ # jexec -u root vnetjail root@vnetjail:/ # cd root@vnetjail:~ # root@vnetjail:~ # ipfw show 65535 0 0 deny ip from any to any root@vnetjail:~ # root@vnetjail:~ # ipfw add 100 check-state 00100 check-state :default root@vnetjail:~ # root@vnetjail:~ # ipfw add 1000 allow tcp from any to me dst-port 5656 setup keep-state 01000 allow tcp from any to me 5656 setup keep-state :default root@vnetjail:~ # root@vnetjail:~ # ipfw show 00100 0 0 check-state :default 01000 0 0 allow tcp from any to me 5656 setup keep-state :default 65535 0 0 deny ip from any to any The single rule above is enough to set up a TCP connection. From external1: root@external1:~ # nc 203.0.113.90 5656 Hello from external1 after ipfw rules have been set up. ^C root@external1:~ # After the above: root@vnetjail:~ # root@vnetjail:~ # ipfw show 00100 0 0 check-state :default 01000 18 1012 allow tcp from any to me 5656 setup keep-state :default 65535 0 0 deny ip from any to any root@vnetjail:~ #