万事有商量,论沟通的重要性 (NFS篇)

主流的网络存储协议中,几乎都有一部叫做协商 (Negotiate), 其重要作用就是把连接双方的各自的情况告知对方,以便找出一个双方都满意的方式来传输数据,这好比男女结婚前会谈一阵子朋友,双方对彼此了解以后,才能更为和谐的相处,不然就会出现婚姻的危机。

主流的NAS网络存储协议在我看来就是NFS和SMB(CIFS), 双方的协商过程都叫做Negotiate, 这篇文章的目的就是要‘偷听’这两种协议分别在‘谈朋友’的过程中的悄悄话。

先说NFS, 以最新的4.1版本为例, 首先要确定协商(Negotiate)发生在哪个阶段,通过RFC知道,协商一般会在mount 的时候(还有就是在NFS服务器重启后),

二话不说,开始mount抓包吧

[root@centos ~] tshark -t ud -Y "ip.addr==<server_ip>" -r mount-nfs4-1.pcap >>mount-nfs4-1.txt; vim mount-nfs4-1.txt
871 11:52:29.533571 client -> server TCP 74 881 → 2049 [SYN] Seq=0 Win=26880 Len=0 MSS=8960 SACK_PERM=1 TSval=1968097466 TSecr=0 WS=128
872 11:52:29.558365 server -> client TCP 74 2049 → 881 [SYN, ACK] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=1686911269 TSecr=1968097466 WS=128
873 11:52:29.558557 client -> server TCP 66 881 → 2049 [ACK] Seq=1 Ack=1 Win=26880 Len=0 TSval=1968097491 TSecr=1686911269
874 11:52:29.558608 client -> server NFS 110 V4 NULL Call
876 11:52:29.585548 server -> client TCP 66 2049 → 881 [ACK] Seq=1 Ack=45 Win=29056 Len=0 TSval=1686911276 TSecr=1968097491
877 11:52:29.585626 server -> client NFS 94 V4 NULL Reply (Call In 874)
878 11:52:29.585657 client -> server TCP 66 881 → 2049 [ACK] Seq=45 Ack=29 Win=26880 Len=0 TSval=1968097518 TSecr=1686911276
879 11:52:29.585978 client -> server NFS 362 V4 Call EXCHANGE_ID
882 11:52:29.610601 server -> client NFS 238 V4 Reply (Call In 879) EXCHANGE_ID
883 11:52:29.610929 client -> server NFS 322 V4 Call CREATE_SESSION
884 11:52:29.634930 server -> client NFS 194 V4 Reply (Call In 883) CREATE_SESSION
885 11:52:29.635147 client -> server NFS 222 V4 Call RECLAIM_COMPLETE
886 11:52:29.659248 server -> client NFS 158 V4 Reply (Call In 885) RECLAIM_COMPLETE
887 11:52:29.659499 client -> server NFS 230 V4 Call SECINFO_NO_NAME
888 11:52:29.683580 server -> client NFS 258 V4 Reply (Call In 887) SECINFO_NO_NAME
889 11:52:29.690901 client -> server NFS 242 V4 Call PUTROOTFH | GETATTR
898 11:52:29.715565 server -> client NFS 378 V4 Reply (Call In 889) PUTROOTFH | GETATTR
899 11:52:29.715809 client -> server NFS 302 V4 Call GETATTR FH: 0x12166020
900 11:52:29.739614 server -> client NFS 234 V4 Reply (Call In 899) GETATTR

先看summary, #871 - #873 很明显是TCP层的三次握手(顺便说一句,NFS 从version4 开始支持吃TCP的传输方式), 接下来是一个NFS NULL的call,用具检测NFS服务是否在线,接下来便要开始协商和建立会话了, 这期间主要会经历4步

  1. Frame #879 EXCHANGE_ID
  2. Frame #883 CREATE_SESSION
  3. Frame #885 RECLAIM_COMPLETE
  4. Frame #887 SECINFO_NO_NAME
  5. Frame #889 PUTROOTFH

EXCHANGE_ID
这一步就是告知对方自己的名字,就行连个刚见面的商人互相交换名片一样,我们展开这一帧开仔细看看他们之间说了什么:

Network File System, Ops(1): EXCHANGE_ID
    [Program Version: 4]
    [V4 Procedure: COMPOUND (1)]
    Tag: <EMPTY>
        length: 0
        contents: <EMPTY>
    minorversion: 1
    Operations (count: 1): EXCHANGE_ID
        Opcode: EXCHANGE_ID (42)
            eia_clientowner
                verifier: 0x5cc7bb2c294e6346    # 提供给服务器一个verifier,用于服务器生成Client ID
                Data: <DATA>
                    length: 48
                    contents: <DATA>
                flags: 0x00000101, EXCHGID4_FLAG_BIND_PRINC_STATEID, EXCHGID4_FLAG_SUPP_MOVED_REFER
                    0... .... .... .... .... .... .... .... = EXCHGID4_FLAG_CONFIRMED_R: Not set
                    .0.. .... .... .... .... .... .... .... = EXCHGID4_FLAG_UPD_CONFIRMED_REC_A: Not set
                    .... .... .... .0.. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_DS: Not set
                    .... .... .... ..0. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_MDS: Not set
                    .... .... .... ...0 .... .... .... .... = EXCHGID4_FLAG_USE_NON_PNFS: Not set
                    .... .... .... .... .... ...1 .... .... = EXCHGID4_FLAG_BIND_PRINC_STATEID: Set
                    .... .... .... .... .... .... .... ..0. = EXCHGID4_FLAG_SUPP_MOVED_MIGR: Not set
                    .... .... .... .... .... .... .... ...1 = EXCHGID4_FLAG_SUPP_MOVED_REFER: Set
            eia_state_protect: SP4_NONE (0)
            eia_client_impl_id
                Implementor DNS domain name(nii_domain): kernel.org
                    length: 10
                    contents: kernel.org
                    fill bytes: opaque data
                Implementation product name(nii_name): Linux 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64
                    length: 75
                    contents: Linux 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64
                    fill bytes: opaque data
                Build timestamp(nii_date)
                    seconds: 0
                    nseconds: 0
    [Main Opcode: EXCHANGE_ID (42)]

可以看到这一步客户端会告诉服务器一串验证码(verifier),自己的Linux 内核版本,自己的DNS信息等等等,而服务器收到后会告知其为此客户端分配的Client ID 这个信息会存在服务器作为此客户端在服务器的唯一标识符,与客户端提供的verifier关联在一起

Network File System, Ops(1): EXCHANGE_ID
    [Program Version: 4]
    [V4 Procedure: COMPOUND (1)]
    Status: NFS4_OK (0)
    Tag: <EMPTY>
        length: 0
        contents: <EMPTY>
    Operations (count: 1)
        Opcode: EXCHANGE_ID (42)
            Status: NFS4_OK (0)
            clientid: 0x5c6a7d06a9000080    
            seqid: 0x00000001
            flags: 0x00020000, EXCHGID4_FLAG_USE_PNFS_MDS
                0... .... .... .... .... .... .... .... = EXCHGID4_FLAG_CONFIRMED_R: Not set
                .0.. .... .... .... .... .... .... .... = EXCHGID4_FLAG_UPD_CONFIRMED_REC_A: Not set
                .... .... .... .0.. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_DS: Not set
                .... .... .... ..1. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_MDS: Set
                .... .... .... ...0 .... .... .... .... = EXCHGID4_FLAG_USE_NON_PNFS: Not set
                .... .... .... .... .... ...0 .... .... = EXCHGID4_FLAG_BIND_PRINC_STATEID: Not set
                .... .... .... .... .... .... .... ..0. = EXCHGID4_FLAG_SUPP_MOVED_MIGR: Not set
                .... .... .... .... .... .... .... ...0 = EXCHGID4_FLAG_SUPP_MOVED_REFER: Not set
            eia_state_protect: SP4_NONE (0)
            eir_server_owner
                minor ID: 1
                major ID: <DATA>
                    length: 44
                    contents: <DATA>
            server scope: <DATA>
                length: 3
                contents: <DATA>
                fill bytes: opaque data
            eir_server_impl_id
                Implementor DNS domain name(nii_domain): <EMPTY>
                    length: 0
                    contents: <EMPTY>
                Implementation product name(nii_name): cycle_test_src_0
                    length: 16
                    contents: cycle_test_src_0
                Build timestamp(nii_date)
                    seconds: 0
                    nseconds: 0
    [Main Opcode: EXCHANGE_ID (42)]

CREATE_SESSION
这一步就是最重要的客户端发起建立会话请求,通过此请求告知会话中使用的BLOCK SIZE等信息,会话一旦建立就会一直保存到umount或者服务器重启为止:

Network File System, Ops(1): CREATE_SESSION
    [Program Version: 4]
    [V4 Procedure: COMPOUND (1)]
    Tag: <EMPTY>
        length: 0
        contents: <EMPTY>
    minorversion: 1
    Operations (count: 1): CREATE_SESSION
        Opcode: CREATE_SESSION (43)
            clientid: 0x5c6a7d06a9000080
            seqid: 0x00000001
            csa_flags: 0x00000003, CREATE_SESSION4_FLAG_PERSIST, CREATE_SESSION4_FLAG_CONN_BACK_CHAN
                .... .... .... .... .... .... .... ...1 = CREATE_SESSION4_FLAG_PERSIST: Set
                .... .... .... .... .... .... .... ..1. = CREATE_SESSION4_FLAG_CONN_BACK_CHAN: Set
                .... .... .... .... .... .... .... .0.. = CREATE_SESSION4_FLAG_CONN_RDMA: Not set
            csa_fore_chan_attrs
                hdr pad size: 0
                max req size: 1049620
                max resp size: 1049480
                max resp size cached: 3416
                max ops: 8
                max reqs: 64
            csa_back_chan_attrs
                hdr pad size: 0
                max req size: 4096
                max resp size: 4096
                max resp size cached: 0
                max ops: 2
                max reqs: 1
            cb_program: 0x40000000
            flavor: 1
            stamp: 693003078
            machine name: drmcyc-dengh2-1-00.cec.lab.emc.com
                length: 34
                contents: drmcyc-dengh2-1-00.cec.lab.emc.com
                fill bytes: opaque data
            uid: 0
            gid: 0
    [Main Opcode: CREATE_SESSION (43)]

服务器回复,其中最重要的便是Session ID:04000000067d6a5c7856341204000000

Network File System, Ops(1): CREATE_SESSION
    [Program Version: 4]
    [V4 Procedure: COMPOUND (1)]
    Status: NFS4_OK (0)
    Tag: <EMPTY>
        length: 0
        contents: <EMPTY>
    Operations (count: 1)
        Opcode: CREATE_SESSION (43)
            Status: NFS4_OK (0)
            sessionid: 04000000067d6a5c7856341204000000
            seqid: 0x00000001
            csr_flags: 0x00000002, CREATE_SESSION4_FLAG_CONN_BACK_CHAN
                .... .... .... .... .... .... .... ...0 = CREATE_SESSION4_FLAG_PERSIST: Not set
                .... .... .... .... .... .... .... ..1. = CREATE_SESSION4_FLAG_CONN_BACK_CHAN: Set
                .... .... .... .... .... .... .... .0.. = CREATE_SESSION4_FLAG_CONN_RDMA: Not set
            csr_fore_chan_attrs
                hdr pad size: 0
                max req size: 1049620
                max resp size: 1049480
                max resp size cached: 3416
                max ops: 8
                max reqs: 64
            csr_back_chan_attrs
                hdr pad size: 0
                max req size: 4096
                max resp size: 4096
                max resp size cached: 0
                max ops: 2
                max reqs: 1
    [Main Opcode: CREATE_SESSION (43)]

RECLAIM_COMPLETE
这一步先忽略,以后在高可靠性的文章里再详细的解释,目前忽略掉不影响我们理解。

SECINFO_NO_NAME
客户将不使用任何认证用户访问NFS共享目录。

PUTROOTFH
获取NFS根目录信息,客户端会保存这些信息以便随后的文件访问。

©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 219,753评论 6 508
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 93,668评论 3 396
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 166,090评论 0 356
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 59,010评论 1 295
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 68,054评论 6 395
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 51,806评论 1 308
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 40,484评论 3 420
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 39,380评论 0 276
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 45,873评论 1 319
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 38,021评论 3 338
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 40,158评论 1 352
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 35,838评论 5 346
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 41,499评论 3 331
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 32,044评论 0 22
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 33,159评论 1 272
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 48,449评论 3 374
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 45,136评论 2 356

推荐阅读更多精彩内容