fip
跨节点
- fip pod <--> fip pod
- fip pod <--> underlay pod
snat
跨节点
- vpc pod <--> fip pod
- vpc pod <--> underlay pod
目前在fip场景2测试中发现，tcp带宽最大有8.6G，所以可以判断集群内部fip的流量先经过tunnel网卡过去，再经过snat包转换，实际上这个过程可以看做没有经过公网网桥（网卡千兆）。
[root@k8s-ctrl-1 ~]# k exec -it -n vpc1 poc-pod-1 -- /bin/sh
sh-4.2# ping 10.5.49.12
PING 10.5.49.12 (10.5.49.12) 56(84) bytes of data.
64 bytes from 10.5.49.12: icmp_seq=1 ttl=63 time=3.49 ms
64 bytes from 10.5.49.12: icmp_seq=2 ttl=63 time=0.966 ms
64 bytes from 10.5.49.12: icmp_seq=3 ttl=63 time=0.287 ms
64 bytes from 10.5.49.12: icmp_seq=4 ttl=63 time=0.332 ms
64 bytes from 10.5.49.12: icmp_seq=5 ttl=63 time=0.235 ms
64 bytes from 10.5.49.12: icmp_seq=6 ttl=63 time=0.286 ms
64 bytes from 10.5.49.12: icmp_seq=7 ttl=63 time=0.329 ms
64 bytes from 10.5.49.12: icmp_seq=8 ttl=63 time=0.352 ms
64 bytes from 10.5.49.12: icmp_seq=9 ttl=63 time=0.319 ms
64 bytes from 10.5.49.12: icmp_seq=10 ttl=63 time=0.370 ms
64 bytes from 10.5.49.12: icmp_seq=11 ttl=63 time=0.287 ms
64 bytes from 10.5.49.12: icmp_seq=12 ttl=63 time=0.462 ms
64 bytes from 10.5.49.12: icmp_seq=13 ttl=63 time=0.377 ms
64 bytes from 10.5.49.12: icmp_seq=14 ttl=63 time=0.352 ms
64 bytes from 10.5.49.12: icmp_seq=15 ttl=63 time=0.307 ms
# 1. 可以看到包在pod出去后直接走tunnel网卡过去了，并没有进行snat转换
[root@k8s-ctrl-2 ~]# tcpdump -i any host 10.5.49.12 and icmp -netvv
dropped privs to tcpdump
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
P 00:00:00:1f:9f:79 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 64, id 29270, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.0.9 > 10.5.49.12: ICMP echo request, id 189, seq 1, length 64
Out 00:00:00:8c:bb:46 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 29270, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.0.9 > 10.5.49.12: ICMP echo request, id 189, seq 1, length 64
P 00:00:00:55:2d:76 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 37099, offset 0, flags [none], proto ICMP (1), length 84)
10.5.49.12 > 192.168.0.9: ICMP echo reply, id 189, seq 1, length 64
Out 00:00:00:55:2d:76 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 37099, offset 0, flags [none], proto ICMP (1), length 84)
10.5.49.12 > 192.168.0.9: ICMP echo reply, id 189, seq 1, length 64
P 00:00:00:1f:9f:79 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 64, id 30144, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.0.9 > 10.5.49.12: ICMP echo request, id 189, seq 2, length 64
Out 00:00:00:8c:bb:46 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 30144, offset 0, flags [DF], proto ICMP (1), length 84)
# 2. 可以看到包在pod直接走tunnel网卡过去到目标node后，才进行snat转换
[root@k8s-ctrl-1 ~]# tcpdump -i any host 10.5.49.12 and icmp -netvv
dropped privs to tcpdump
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
P 00:00:00:8c:bb:46 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 17251, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.0.9 > 10.5.49.12: ICMP echo request, id 188, seq 37, length 64
Out 00:00:00:8c:bb:46 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 63, id 17251, offset 0, flags [DF], proto ICMP (1), length 84)
10.5.49.5 > 10.5.49.12: ICMP echo request, id 188, seq 37, length 64
P 00:00:00:bc:d2:86 ethertype IPv4 (0x0800), length 100: (tos 0x0, ttl 64, id 60186, offset 0, flags [none], proto ICMP (1), length 84)
10.5.49.12 > 10.5.49.5: ICMP echo reply, id 188, seq 37, length 64
c0c0b004f435b5f720f25ce42226ee0.png
