Tracing WireGuard IP Routing on Linux Clients through Nftables
1. Setup
To be able to correctly reproduce the steps presented in this article, we first present an exhaustive list of the wireguard client configuration relevant to the routing process.
Network Topology
Routing Tables on the WireGuard Client
When enabling wireguard, the wg-quick tool performs the following configurations:
$ sudo wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.0.0.1/24 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] wg set wg0 fwmark 51820
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
[#] sysctl -q net.ipv4.conf.all.src_valid_mark=1
[#] nft -f /dev/fd/63$ ip rule show all
0: from all lookup local
32764: from all lookup main suppress_prefixlength 0
32765: not from all fwmark 0xca6c lookup 51820
32766: from all lookup main
32767: from all lookup default$ ip route show table main
default via 192.168.178.1 dev wlp4s0 proto dhcp metric 600
10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.1
169.254.0.0/16 dev wlp4s0 scope link metric 1000
192.168.178.0/24 dev wlp4s0 proto kernel scope link src 192.168.178.48 metric 600$ ip route show table 51820
default dev wg0 scope link$ ip route show table local
broadcast 10.0.0.0 dev wg0 proto kernel scope link src 10.0.0.1
local 10.0.0.1 dev wg0 proto kernel scope host src 10.0.0.1
broadcast 10.0.0.255 dev wg0 proto kernel scope link src 10.0.0.1
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
broadcast 192.168.178.0 dev wlp4s0 proto kernel scope link src 192.168.178.48
local 192.168.178.48 dev wlp4s0 proto kernel scope host src 192.168.178.48
broadcast 192.168.178.255 dev wlp4s0 proto kernel scope link src 192.168.178.48Nftables Host-Firewall State on the WireGuard Client
Nftables defines the following concepts:
table: A collection of chainschain: A collection of rules with a default policy, and a specific hookhook: A point in the routing process at which this chain is called. Examples: Input, Forward, Output, Postrouting.rule: Matches a rule
$ sudo nft list ruleset
table inet filter {
chain global {
ct state established,related accept comment "Accept traffic originated from us"
ct state invalid drop comment "Drop invalid connections"
}
chain input {
type filter hook input priority filter; policy drop;
iif "lo" accept comment "Accept any localhost traffic"
jump global
icmp type {
echo-reply, destination-unreachable, source-quench, echo-request,
router-advertisement, router-solicitation, time-exceeded, parameter-problem,
address-mask-request, address-mask-reply
} accept
}
chain output {
type filter hook output priority filter; policy accept;
}
chain forward {
type filter hook forward priority filter; policy drop;
}
}
table ip wg-quick-wg0 {
chain preraw {
# priority -300
type filter hook prerouting priority raw; policy accept;
iifname != "wg0" ip daddr 10.0.0.1 fib saddr type != local drop
}
chain premangle {
# priority -150
type filter hook prerouting priority mangle; policy accept;
meta l4proto udp meta mark set ct mark
}
chain postmangle {
# priority -150
type filter hook postrouting priority mangle; policy accept;
meta l4proto udp meta mark 0x0000ca6c ct mark set meta mark
}
}The firewall rules translate to the following behaviour. First starting with a default ruleset for a typical workstation machine, set up by us manually, and residing in the table called “filter” applying to IPv4 and IPv6 (nftables address family keyword “inet”):
- Chain “input”, called in Input hook:
- Allow localhost traffic
- Allow already established connections (typical for stateful firewalls)
- Allow ICMPv4 traffic
- Drop all other
- Chain “output”, called in Output hook:
- Allow all outgoing traffic
- Chain “forward”, called in Forward hook:
- Never forward traffic
When activating wireguard through the wg-quick tool, new rules are inserted
to correctly route tunneled and untunneled packets through the networking
stack. These rules reside in the table called “wg-quick-wg0” and apply to
IPv4 only as we set up wireguard for IPv4:
- Chain “preraw”, called in Prerouting hook:
- Traffic not incoming from the wireguard interface, destined for the local wireguard ip, with a non-local source address is dropped. This forces all traffic hitting the wireguard IP of this machine to go through the wireguard interface before (in this case wg0), except for localhost traffic.
- Accept all other
- Chain “premangle”, called in Prerouting hook after “preraw” chain:
- For UDP flows: apply a firewall mark (fwmark) remembered by conntrack. (See postmangle)
- Accept all traffic
- Chain “postmangle”, called in Postrouting hook:
- For all UDP traffic, marked with “0x0000ca6c”, remember the mark for this flow using conntrack (for more info on the syntax see here)
- Accept all traffic
It is important to note that the netfilter hook Prerouting applies only to incoming traffic, while the Postrouting hook deals with outgoing traffic. This can be derived from Figure 1. As we disabled all forwarding (this is a wireguard VPN client and not a server/router after all), one packet will never run through both hooks.
For more detailed information on how the nftables syntax translates to this behavior, feel free to dive into the nftables manpage and the nftables wiki.
Tracing the Routing Process
Setup Firewall Tracing
To gather logs in nftables insert trace rules into every chain to trace. We want to log packets that 1) belong to WireGuard, identifyable by the usage of UDP and in our configuration on port 51820, and 2) ICMP Echo Request and Echo Reply packets used by ping. With 1) we capture packets outside the tunnel, while 2) enables us to trace packets operating within the VPN tunnel through the linux forwarding process.
In nftables practice tracing is enabled by matching a packet and selecting
meta nftrace set 1 as the action on the packet to log.
for CHAIN in input forward output; do
sudo nft insert rule inet filter $CHAIN handle 0 meta l4proto udp meta nftrace set 1;
sudo nft insert rule inet filter $CHAIN handle 0 icmp type {echo-request, echo-reply} meta \
nftrace set 1;
done
for CHAIN in preraw premangle postmangle; do
sudo nft insert rule wg-quick-wg0 $CHAIN handle 0 meta l4proto udp meta nftrace set 1;
sudo nft insert rule wg-quick-wg0 $CHAIN handle 0 icmp type {echo-request, echo-reply} meta \
nftrace set 1;
doneAny firewall activity matching our defined packets will now show up when executing the nftables log command.
$ sudo nft monitor
trace id 6ca3695d inet filter output packet: oif "wg0" ip saddr 10.0.0.1 ip daddr 9.9.9.9 ip dscp
cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip protocol icmp ip length 84 icmp code
net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d inet filter output rule icmp type { echo-reply, echo-request } meta nftrace
set 1 (verdict continue)
trace id 6ca3695d inet filter output verdict continue
trace id 6ca3695d inet filter output policy accept
trace id 6ca3695d ip wg-quick-wg0 postmangle packet: oif "wg0" ip saddr 10.0.0.1 ip daddr 9.9.9.9
ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip length 84 icmp code net-unreachable
icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d ip wg-quick-wg0 postmangle rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 6ca3695d ip wg-quick-wg0 postmangle verdict continue
trace id 6ca3695d ip wg-quick-wg0 postmangle policy accept
...Outgoing Traffic
To generate traffic we will elicit a ping from the wireguard client machine to the public DNS resolver of Quad9.
$ ping -c 1 9.9.9.9
PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.
64 bytes from 9.9.9.9: icmp_seq=1 ttl=58 time=22.2 ms
--- 9.9.9.9 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 22.216/22.216/22.216/0.000 msThe following diagram illustrates the path the ICMP Echo Request packet takes on the way out of the machine. The whole, raw nftables log output for this procedure can be seen in the appendix section.
The following headings are tied to the numerically increasing steps in Figure 2.
Step 0
Ping builds an ICMP Echo Request packet with the IP destination given by the user.
Step 1
The outgoing packet needs to be forwarded to the correct outgoing interface
to be able to reach the IP destination. Although the only internet-capable
interface on the system is a wireless ethernet interface called wlp4s0,
the packet is forwarded to the wireguard interface. This is correct, as we
configured wireguard in a way that all traffic should run through the VPN
tunnel.
Wireguard solves this problem by manipulating the IP routing configuration on the machine in the follwing way:
$ sudo wg-quick up wg0
...
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
...- create a new routing table called 51820 (the wireguard port we chose)
- the routing table 51820 is only used by packets without a firewall mark (fwmark) of 51820. A firewall mark, or fwmark, allows the operating system to mark a specific packet with an integer. Internally this assignment is saved with the packet representation in form of Linux kernel socket buffers (sk_buff).
- We add a second entry for the default routing table “main” but suppress all matches with the default route 0.0.0.0/0 (also called “default” in linux).
We can verify the performed changes with:
$ ip rule show all
0: from all lookup local
32764: from all lookup main suppress_prefixlength 0
32765: not from all fwmark 0xca6c lookup 51820
32766: from all lookup main
32767: from all lookup default
$
$ ip route show table main
default via 192.168.178.1 dev wlp4s0 proto dhcp metric 600
10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.1
169.254.0.0/16 dev wlp4s0 scope link metric 1000
192.168.178.0/24 dev wlp4s0 proto kernel scope link src 192.168.178.48 metric 600
$
$ ip route show table 51820
default dev wg0 scope link
$
$ ip route show table local
broadcast 10.0.0.0 dev wg0 proto kernel scope link src 10.0.0.1
local 10.0.0.1 dev wg0 proto kernel scope host src 10.0.0.1
broadcast 10.0.0.255 dev wg0 proto kernel scope link src 10.0.0.1
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
broadcast 192.168.178.0 dev wlp4s0 proto kernel scope link src 192.168.178.48
local 192.168.178.48 dev wlp4s0 proto kernel scope host src 192.168.178.48
broadcast 192.168.178.255 dev wlp4s0 proto kernel scope link src 192.168.178.48As we want to route to the remote destination of 9.9.9.9, the routing decision is made:
- Checking the table “local” with priority 0 (the lower the number, the higher the priority), there is no match with 9.9.9.9 because it is a remote destination not present on the local machine. The IP addresses of this machine would match here, namely 10.0.0.1 (the wireguard VPN address), 192.168.178.48 (the LAN address) and 127.0.0.1 (the loopback address).
- Checking table “main” with priority 32764, the default route is matched,
but also suppressed by the routing table keyword
32764: from all lookup main suppress_prefixlength 0. Addresses residing in the same local networks as us would get matched here, allowing us to communicate with hosts on e.g. the local LAN without going through the wireguard interface. This resembles a Split Tunnel for the local network. - Checking with table 51820 as the packet is currently not firewall-marked
with 0xca6c (Decimal: 51820), match with the default rule
default dev wg0 scope link - Forward the packet on the wg0 wireguard interface
Additionally, we can debug the routing table configuration with the
ip route tool in the following way:
$ ip route get fibmatch to 9.9.9.9
default dev wg0 table 51820 scope linkAs we can see the packet is in fact forwarded on the wg0 interface.
Step 2
The output chain accepts all outgoing packets, including this ICMP Echo Request as can be seen in the corresponding nftables log.
...
chain output {
type filter hook output priority filter; policy accept;
}
...trace id 6ca3695d inet filter output packet: oif "wg0" ip saddr 10.0.0.1 ip daddr 9.9.9.9
ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip protocol icmp ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d inet filter output rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 6ca3695d inet filter output verdict continue
trace id 6ca3695d inet filter output policy acceptStep 3
The Postrouting hook is executed for all packets leaving the system. As the packet is not of type UDP but ICMP, conntrack is not instructed to remember the mark associated with this packet flow. All traffic here is accepted.
...
chain postmangle {
# priority -150
type filter hook postrouting priority mangle; policy accept;
meta l4proto udp meta mark 0x0000ca6c ct mark set meta mark
}
...trace id 6ca3695d ip wg-quick-wg0 postmangle packet: oif "wg0" ip saddr 10.0.0.1
ip daddr 9.9.9.9 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d ip wg-quick-wg0 postmangle rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 6ca3695d ip wg-quick-wg0 postmangle verdict continue
trace id 6ca3695d ip wg-quick-wg0 postmangle policy acceptStep 4
The wireguard process receives the packet and looks up the destination of
9.9.9.9 in its cryptokey routing table, as explained in the
wireguard whitepaper, section 2.
As all traffic is configured to
go to the VPN server, 9.9.9.9 matches with the default entry 0.0.0.0/0 and
the packet is encrypted using the secure session derived from the VPN server’s
public key. The new packet is a UDP packet carrying an encrypted payload,
destined for the VPN server at 123.123.123.123:51820.
Step 5
We know that all packets going through the wg0 wireguard interface are marked with the integer 51820.
$ sudo wg-quick up wg0
[#] ip link add wg0 type wireguard
...
[#] wg set wg0 fwmark 51820
...At this point our ICMP Echo Request resides within the tunnel, and is now encrypted and carried by a UDP packet with this machine’s internet facing IP as source IP and the wireguard VPN server as the destination IP.
Once again comparing with the routing setup described in Step 1, the routing procedure goes as follows:
- Check table “local”, no match as destination is not this machine.
- Check table “main”, match with default route but is suppressed by
32764: from all lookup main suppress_prefixlength 0. - Skip table “51820” as our packet is now indeed firewall marked with the integer 51820.
- Check table “main”, match with the default route, forward on interface wlp4s0.
Quick verification:
$ ip route get fibmatch to 123.123.123.123 mark 0xca6c
default via 192.168.178.1 dev wlp4s0 proto dhcp metric 600Step 6
Accept all outgoing packets.
trace id 2988acd2 inet filter output packet: oif "wlp4s0" @ll,0,160
22835963324624387148182108647992317033744409663 ip saddr 192.168.178.48
ip daddr 123.123.123.123 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 8817 ip protocol udp
ip length 156 udp sport 51820 udp dport 51820 udp length 136
@th,64,96 1237940052367853153958232064
trace id 2988acd2 inet filter output rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id 2988acd2 inet filter output verdict continue meta mark 0x0000ca6c
trace id 2988acd2 inet filter output policy accept meta mark 0x0000ca6cStep 7
This time in the Postrouting hook the UDP rule is matching with the wireguard packet, and conntrack is instructed to remember the firewall mark for packets associated with this flow. Then the packet is accepted, and finally sent out on the wireless interface.
trace id 2988acd2 ip wg-quick-wg0 postmangle packet: iif "wg0"
oif "wlp4s0" @ll,0,160 22835963324624387148182108647992317033744409663
ip saddr 192.168.178.48 ip daddr 123.123.123.123 ip dscp cs0 ip ecn not-ect
ip ttl 64 ip id 8817 ip length 156 udp sport 51820 udp dport 51820
udp length 136 @th,64,96 1237940052367853153958232064
trace id 2988acd2 ip wg-quick-wg0 postmangle rule meta l4proto udp
meta nftrace set 1 (verdict continue)
trace id 2988acd2 ip wg-quick-wg0 postmangle rule meta l4proto udp
meta mark 0x0000ca6c ct mark set meta mark (verdict continue)
trace id 2988acd2 ip wg-quick-wg0 postmangle verdict continue meta mark 0x0000ca6c
trace id 2988acd2 ip wg-quick-wg0 postmangle policy accept meta mark 0x0000ca6cIncoming Traffic
Step 1
In the Prerouting hook, chain “preraw” created by wireguard, we see the wireguard tunnel UDP packet arriving from the VPN server. As the packet is incoming from the wireless interface, this chain simply accepts the packet. The drop rule present in this location simply prevents addressing the local wireguard IP, in this case 10.0.0.1, without having passed the wireguard interface.
...
chain preraw {
# priority -300
type filter hook prerouting priority raw; policy accept;
iifname != "wg0" ip daddr 10.0.0.1 fib saddr type != local drop
}
...trace id ae8f7320 ip wg-quick-wg0 preraw packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48
ip dscp cs0 ip ecn not-ect ip ttl 51 ip id 62408 ip length 156
udp sport 51820 udp dport 51820 udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 ip wg-quick-wg0 preraw rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 preraw verdict continue
trace id ae8f7320 ip wg-quick-wg0 preraw policy acceptStep 2
In the “premangle” chain of the Prerouting hook the UDP packet is marked with 51820 as remembered by conntrack. Afterwards the packet is accepted.
trace id ae8f7320 ip wg-quick-wg0 premangle packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48
ip dscp cs0 ip ecn not-ect ip ttl 51 ip id 62408 ip length 156 udp sport 51820
udp dport 51820 udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 ip wg-quick-wg0 premangle rule meta l4proto udp
meta nftrace set 1 (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 premangle rule meta l4proto udp
meta mark set ct mark (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 premangle verdict continue meta mark 0x0000ca6c
trace id ae8f7320 ip wg-quick-wg0 premangle policy accept meta mark 0x0000ca6cWhile the presence of this mark does not play much a role in the following
routing process (the destination IP would match the local table either way),
it is nevertheless very essential in this setup because of
reverse path filtering.
Linux employs a full reverse path filtering (set to 1) by default, which checks
whether a response packet (with a reversed source and destination IP) would
leave on the same interface as the original packet.
The reverse path filtering setting on a machine can be checked with:
$ sudo sysctl net.ipv4.conf.all.rp_filter
net.ipv4.conf.all.rp_filter = 1where quote:
rp_filter - INTEGER
- 0 - No source validation.
- 1 - Strict mode as defined in RFC3704 Strict Reverse Path Each incoming packet is tested against the FIB and if the interface is not the best reverse path the packet check will fail. By default failed packets are discarded.
- 2 - Loose mode as defined in RFC3704 Loose Reverse Path Each incoming packet’s source address is also tested against the FIB and if the source address is not reachable via any interface the packet check will fail.
(Source)
With this set mark the reverse path lookup succeeds with the wireless wlp4s0
interface, which was also used for the incoming packet, instead of failing
with the
wireguard interface.
Source for rule interpretion: Pro Custodibus.
Step 3
The UDP packet destined for the local machine is matched with the local
routing table.
$ ip route get to 192.168.178.48 mark 0xca6c
local 192.168.178.48 dev lo table local src 192.168.178.48 mark 0xca6c uid 1000
cache <local>Step 4
The input chain called by the Input hook accepts the incoming packet due
to the traffic being originated by us.
trace id ae8f7320 inet filter input packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48
ip dscp cs0 ip ecn not-ect ip ttl 51 ip id 62408 ip protocol udp ip length 156
udp sport 51820 udp dport 51820 udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 inet filter input rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id ae8f7320 inet filter input rule jump global (verdict jump global)
trace id ae8f7320 inet filter global rule ct state established,related accept
comment "Accept traffic originated from us" (verdict accept)Step 5
Decrypt the wireguard packet to reveal the ICMP Echo Reply message from our pinged destination. The ICMP Echo Reply with IP source 9.9.9.9 and IP destination 10.0.0.1 re-enters the network stack.
Step 6
Accept the ICMP Echo Reply from 9.9.9.9 to 10.0.0.1 in the preraw chain of
the Prerouting hook.
trace id 8f0cd96a ip wg-quick-wg0 preraw packet: iif "wg0" ip saddr 9.9.9.9 ip daddr 10.0.0.1
ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip length 84 icmp code net-unreachable
icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a ip wg-quick-wg0 preraw rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a ip wg-quick-wg0 preraw verdict continue
trace id 8f0cd96a ip wg-quick-wg0 preraw policy acceptStep 7
Accept the ICMP Echo Reply from 9.9.9.9 to 10.0.0.1 in the premangle chain
of the Prerouting hook.
No need to mark the packet for reverse path filtering, as it was not done
on the outgoing packet and the wg0 interface would be correctly matched
anyway.
trace id 8f0cd96a ip wg-quick-wg0 premangle packet: iif "wg0" ip saddr 9.9.9.9
ip daddr 10.0.0.1 ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a ip wg-quick-wg0 premangle rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a ip wg-quick-wg0 premangle verdict continue
trace id 8f0cd96a ip wg-quick-wg0 premangle policy acceptStep 8
The IP destination 10.0.0.1 is the local machine and therefore matches with
the local routing table.
$ ip route get to 10.0.0.1
local 10.0.0.1 dev lo table local src 10.0.0.1 uid 1000
cache <local>Step 9
At the Input hook the ICMP Echo Reply is accepted as a stateful response to the ICMP Echo Request.
trace id 8f0cd96a inet filter input packet: iif "wg0" ip saddr 9.9.9.9 ip daddr 10.0.0.1
ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip protocol icmp ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a inet filter input rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a inet filter input rule jump global (verdict jump global)
trace id 8f0cd96a inet filter global rule ct state established,related accept
comment "Accept traffic originated from us" (verdict accept)Summary
WireGuard uses the following techniques to correctly tunnel traffic on linux machines:
- Make use of policy-based routing by adding and manipulating routing tables
- Incorporate firewall marks (fwmarks) in routing tables, the wireguard interface and in the firewall ruleset to direct packets
- Use the priority of routing tables to create a split tunnel for the local network
- Circumvent reverse-path filtering using conntrack
Appendix
Raw ping nftables log output
$ sudo nft monitor
trace id 6ca3695d inet filter output packet: oif "wg0" ip saddr 10.0.0.1 ip daddr 9.9.9.9
ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip protocol icmp ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d inet filter output rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 6ca3695d inet filter output verdict continue
trace id 6ca3695d inet filter output policy accept
trace id 6ca3695d ip wg-quick-wg0 postmangle packet: oif "wg0" ip saddr 10.0.0.1 ip daddr 9.9.9.9
ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44121 ip length 84 icmp code net-unreachable
icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 6ca3695d ip wg-quick-wg0 postmangle rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 6ca3695d ip wg-quick-wg0 postmangle verdict continue
trace id 6ca3695d ip wg-quick-wg0 postmangle policy accept
trace id 2988acd2 inet filter output packet: oif "wlp4s0"
@ll,0,160 22835963324624387148182108647992317033744409663 ip saddr 192.168.178.48
ip daddr 123.123.123.123 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 8817 ip protocol udp
ip length 156 udp sport 51820 udp dport 51820 udp length 136
@th,64,96 1237940052367853153958232064
trace id 2988acd2 inet filter output rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id 2988acd2 inet filter output verdict continue meta mark 0x0000ca6c
trace id 2988acd2 inet filter output policy accept meta mark 0x0000ca6c
trace id 2988acd2 ip wg-quick-wg0 postmangle packet: iif "wg0" oif "wlp4s0"
@ll,0,160 22835963324624387148182108647992317033744409663 ip saddr 192.168.178.48
ip daddr 123.123.123.123 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 8817 ip length 156
udp sport 51820 udp dport 51820 udp length 136 @th,64,96 1237940052367853153958232064
trace id 2988acd2 ip wg-quick-wg0 postmangle rule meta l4proto udp meta
nftrace set 1 (verdict continue)
trace id 2988acd2 ip wg-quick-wg0 postmangle rule meta l4proto udp meta mark 0x0000ca6c
ct mark set meta mark (verdict continue)
trace id 2988acd2 ip wg-quick-wg0 postmangle verdict continue meta mark 0x0000ca6c
trace id 2988acd2 ip wg-quick-wg0 postmangle policy accept meta mark 0x0000ca6c
trace id ae8f7320 ip wg-quick-wg0 preraw packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48 ip dscp cs0
ip ecn not-ect ip ttl 51 ip id 62408 ip length 156 udp sport 51820 udp dport 51820
udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 ip wg-quick-wg0 preraw rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 preraw verdict continue
trace id ae8f7320 ip wg-quick-wg0 preraw policy accept
trace id ae8f7320 ip wg-quick-wg0 premangle packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48 ip dscp cs0
ip ecn not-ect ip ttl 51 ip id 62408 ip length 156 udp sport 51820 udp dport 51820
udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 ip wg-quick-wg0 premangle rule meta l4proto udp meta
nftrace set 1 (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 premangle rule meta l4proto udp meta mark
set ct mark (verdict continue)
trace id ae8f7320 ip wg-quick-wg0 premangle verdict continue meta mark 0x0000ca6c
trace id ae8f7320 ip wg-quick-wg0 premangle policy accept meta mark 0x0000ca6c
trace id ae8f7320 inet filter input packet: iif "wlp4s0" ether saddr 22:22:22:22:22:22
ether daddr 11:11:11:11:11:11 ip saddr 123.123.123.123 ip daddr 192.168.178.48 ip dscp cs0
ip ecn not-ect ip ttl 51 ip id 62408 ip protocol udp ip length 156 udp sport 51820
udp dport 51820 udp length 136 @th,64,96 1237940045101857216057573376
trace id ae8f7320 inet filter input rule meta l4proto udp meta nftrace set 1 (verdict continue)
trace id ae8f7320 inet filter input rule jump global (verdict jump global)
trace id ae8f7320 inet filter global rule ct state established,related accept
comment "Accept traffic originated from us" (verdict accept)
trace id 8f0cd96a ip wg-quick-wg0 preraw packet: iif "wg0" ip saddr 9.9.9.9 ip daddr 10.0.0.1
ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip length 84 icmp code net-unreachable
icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a ip wg-quick-wg0 preraw rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a ip wg-quick-wg0 preraw verdict continue
trace id 8f0cd96a ip wg-quick-wg0 preraw policy accept
trace id 8f0cd96a ip wg-quick-wg0 premangle packet: iif "wg0" ip saddr 9.9.9.9 ip daddr 10.0.0.1
ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip length 84 icmp code net-unreachable
icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a ip wg-quick-wg0 premangle rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a ip wg-quick-wg0 premangle verdict continue
trace id 8f0cd96a ip wg-quick-wg0 premangle policy accept
trace id 8f0cd96a inet filter input packet: iif "wg0" ip saddr 9.9.9.9 ip daddr 10.0.0.1
ip dscp cs0 ip ecn not-ect ip ttl 58 ip id 43561 ip protocol icmp ip length 84
icmp code net-unreachable icmp id 29717 icmp sequence 1 @th,64,96 1520613278444654799249670912
trace id 8f0cd96a inet filter input rule icmp type { echo-reply, echo-request }
meta nftrace set 1 (verdict continue)
trace id 8f0cd96a inet filter input rule jump global (verdict jump global)
trace id 8f0cd96a inet filter global rule ct state established,related accept
comment "Accept traffic originated from us" (verdict accept)WireGuard Client Config File
[Interface]
Address = 10.0.0.1/24
PrivateKey = AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
ListenPort = 51820
[Peer]
# VPN Server
PublicKey = BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB=
AllowedIPs = 0.0.0.0/0
Endpoint = 123.123.123.123:51820WireGuard Server Config File
[Interface]
Address = 10.0.0.254/24
PrivateKey = DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD=
ListenPort = 51820
[Peer]
# VPN Client
PublicKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC=
AllowedIPs = 10.0.0.1/32