[docker 网络] 单主机docker容器网络隔离 VLAN

0. 关于ovs VLAN

VLAN(Virtual Local Area Network)即虚拟局域网, 按照功能, 部门等因素将网络中的机器进行划分, 使之分属不同的部分, 每一个部分形成一个虚拟的局域网络, 共享一个单独的广域网. 将一个大型交换网络划分为许多个独立的广域网, 即VLAN.

目前OVS的端口支持四种VLAN模式,分别是trunk、access、native-tagged、native-untagged,刚创建的端口默认是不设置任何的VLAN模式。

模式 说明
默认 在默认模式下(VLAN_mode没被设置),如果指定了端口的tag属性,那么这个端口就工作在access模式,并且其trunk属性的值应该保持为空。否则,这个port就工作在trunk模式下,如果trunk被指定,则使用指定的trunk值。
trunk trunk模式的端口允许传输所有在其trunk属性中指定的那些VLAN对应的数据包。其他VLAN的数据包就会被丢弃。从trunk模式的端口中进入的数据包其VLAN ID不会发生变化。如果进入的数据包不含有VLAN ID,则该数据包进入交换机后的VLAN为0。从trunk模式的端口出去的数据包,如果VLAN ID不为空,则依然保持该VLAN ID,如果VLAN ID为空,则出去后不再包含802.1Q头部
access access模式的端口只允许不带VLAN的数据包进入,不管数据包的VLAN ID是否与其tag相同,只要含有VLAN ID,这个数据包都会被端口drop。数据包进入access端口后会被打上和端口tag相同的VLAN,而再从access端口出去时,数据包的VLAN会被删除,也就是说从access的端口出去的数据包和进来时一样是不带VLAN的。

来自于https://zpzhou.com/archives/openvswitch_vlan.html

来自容器与容器云.png

当PC1向PC3发送数据时, PC1将IP包封装在以太帧中, 帧的目的地址为PC3的地址, 此时帧中并没有tag信息. 当帧到达Port1时, Port1给帧打上tag(VID=100), 帧进入switch1, 然后帧通过Port3,Port4到达Switch2(假设Port3,Port4的trunks=[100,200], 所以Port3,Port4允许VLAN ID为100,200的帧通过). 在Switch2中, Port5所标记的VID和帧相同, MAC地址也匹配,帧就发送到Port5上, Port5将帧的tag信息去掉, 然后发送给PC3. 由于PC2,PC4与PC1的VLAN不同, 因此收不到PC1发出的帧.
此部分来自于容器与容器云

1. 前言

图片.png

本文将会利用ovs-docker,(关于ovs-docker可以参考[docker 网络] ovs-docker 使用及原理) 在同一台机器启动四个容器. 然后为容器加valn tag来进行容器隔离.

1.1 docker版本

[root@vm1 ~]# docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77156
 Built:             Sat May  4 02:34:58 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 02:02:43 2019
  OS/Arch:          linux/amd64
  Experimental:     false
[root@vm1 ~]# ovs-vsctl add-br br0
[root@vm1 ~]# ifconfig br0 192.168.0.250/24
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.1"

2. 配置

创建四个容器

[root@vm1 ~]# docker run -d --name con1 --net=none --privileged=true busybox top
30bdd285fc47e047e879a83f4a36a0c303ad6a38462da464b11b6266387684cd
[root@vm1 ~]# docker run -d --name con2 --net=none --privileged=true busybox top
eb0872370afd0ac1e544ae99522c4aff76abf0716283e4a41c6da471cb82ba21
[root@vm1 ~]# docker run -d --name con3 --net=none --privileged=true busybox top
db1a0b70e84e24d616d4f01c9518312794e12477779a4f8e2405427d754b0f94
[root@vm1 ~]# docker run -d --name con4 --net=none --privileged=true busybox top
247ef66e918147694f930db9e5f92afb8d2b4654b6f9959fb25ef5253bf9c887

利用ovs-docker给这些容器配置网络, 关于ovs-docker可以参考 [docker 网络] ovs-docker 使用及原理

[root@vm1 ~]# ovs-docker add-port br0 eth0 con1 --ipaddress=192.168.0.1/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con2 --ipaddress=192.168.0.2/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con3 --ipaddress=192.168.0.3/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con4 --ipaddress=192.168.0.4/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"
[root@vm1 ~]# 

2.1 测试

测试 : 从con1可以访问br0, con1, con2, con3, con4.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.501 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.501/5.501/5.501 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.053 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.053/0.053/0.053 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=4.844 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.844/4.844/4.844 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.708 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.708/5.708/5.708 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=4.738 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.738/4.738/4.738 ms
[root@vm1 ~]# 

别的容器也可以访问con1.

[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.524 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.524/4.524/4.524 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=5.441 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.441/5.441/5.441 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.805 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.805/4.805/4.805 ms
[root@vm1 ~]# 

3 加tag

为容器打上tag

[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

[root@vm1 ~]# ovs-vsctl list interface afc212d089e44_l | grep container_id
external_ids        : {container_id="con4", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port afc212d089e44_l tag=200

[root@vm1 ~]# ovs-vsctl list interface 1a18677188d94_l | grep container_id
external_ids        : {container_id="con2", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 1a18677188d94_l tag=100

[root@vm1 ~]# ovs-vsctl list interface 07bfe3100ddd4_l | grep container_id
external_ids        : {container_id="con3", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 07bfe3100ddd4_l tag=200

[root@vm1 ~]# ovs-vsctl list interface 6bf90751c96c4_l | grep container_id
external_ids        : {container_id="con1", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 6bf90751c96c4_l tag=100

[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            tag: 200
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            tag: 100
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            tag: 200
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            tag: 100
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

3.1 测试

测试容器互通性
con1可以访问con2, 但是访问不了con3和con4.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.047 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.047/0.047/0.047 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=5.429 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.429/5.429/5.429 ms
[root@vm1 ~]# 
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

con2 可以访问con1, 但是con2 不能访问con3和con4

[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=0.062 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.062/0.062 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=3.548 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.548/3.548/3.548 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# 

con3不能访问con1和con2, 但是可以访问con4

[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=0.054 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.054/0.054/0.054 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=3.399 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.399/3.399/3.399 ms
[root@vm1 ~]# 

con4 不能访问 con1 和 con2, 但是可以访问con3.

[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.234 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.234/5.234/5.234 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=0.065 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.065/0.065 ms
[root@vm1 ~]# 

3.2 访问br0

访问br0, con1,con2,con3,con4均不能访问br0

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# 
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

为br0设置tag

[root@vm1 ~]# ovs-vsctl set port br0 tag=100
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            tag: 200
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            tag: 100
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            tag: 200
            Interface "07bfe3100ddd4_l"
        Port "br0"
            tag: 100
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            tag: 100
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

测试, 此时con1和con2可以访问br0, con3和con4不能访问br0. 同样的道理,如果把tag设置为200, con3和con4就可以访问br0, 而con1和con2就不能访问br0.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.466 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.466/5.466/5.466 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=4.242 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.242/4.242/4.242 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

4. 参考

1. Docker 容器与容器云
2. https://zpzhou.com/archives/openvswitch_vlan.html

©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 218,386评论 6 506
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 93,142评论 3 394
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 164,704评论 0 353
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 58,702评论 1 294
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 67,716评论 6 392
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 51,573评论 1 305
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 40,314评论 3 418
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 39,230评论 0 276
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 45,680评论 1 314
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 37,873评论 3 336
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 39,991评论 1 348
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 35,706评论 5 346
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 41,329评论 3 330
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 31,910评论 0 22
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 33,038评论 1 270
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 48,158评论 3 370
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 44,941评论 2 355

推荐阅读更多精彩内容