Crush Map规则的定制

参阅：https://ceph.readthedocs.io/en/latest/rados/operations/crush-map/

（一）手动编辑Crush Map规则

生产环境中，做类似调整尽量在磁盘为空磁盘的情况下进行调整或者提前规划好CrushMap的规则，提前实施。

1.分析情况

ID CLASS WEIGHT  TYPE NAME            STATUS REWEIGHT PRI-AFF

-1      0.02939 root default

-3      0.00980    host k-master

0  hdd 0.00490        osd.0            up  1.00000 1.00000

1  hdd 0.00490        osd.1            up  1.00000 1.00000

-5      0.00980    host k-worker001

2  hdd 0.00490        osd.2            up  1.00000 1.00000

4  ssd 0.00490        osd.4            up  1.00000 1.00000

-7      0.00980    host k-worker002

3  hdd 0.00490        osd.3            up  1.00000 1.00000

5  ssd 0.00490        osd.5            up  1.00000 1.00000

[root@k-master ceph]#

可以看出：存在1种类型的规则，即default。

这里由于存在ssd和hdd两种磁盘，我们通过定制规则把2种磁盘分别制作规则分隔开来以便根据用途创建不同资源池使用。

2.导出现有的Crush map规则进行备份与分析

[root@k-master ceph]# ceph osd getcrushmap -o crushmap.bin

18

[root@k-master ceph]# file crushmap.bin

crushmap.bin: MS Windows icon resource - 16 icons, 1-colors

[root@k-master ceph]# ls

ceph.bootstrap-mds.keyring  ceph.bootstrap-rgw.keyring  ceph-deploy-ceph.log  etc        swift_openrc.sh

ceph.bootstrap-mgr.keyring  ceph.client.admin.keyring  ceph.mon.keyring      rbdmap    testrbd

ceph.bootstrap-osd.keyring  ceph.conf                  crushmap.bin          s3test.py

[root@k-master ceph]# crushtool -d crushmap.bin -o crushmap.txt

[root@k-master ceph]# cat crushmap.txt

# begin crush map

tunable choose_local_tries 0

tunable choose_local_fallback_tries 0

tunable choose_total_tries 50

tunable chooseleaf_descend_once 1

tunable chooseleaf_vary_r 1

tunable chooseleaf_stable 1

tunable straw_calc_version 1

tunable allowed_bucket_algs 54

# devices

device 0 osd.0 class hdd

device 1 osd.1 class hdd

device 2 osd.2 class hdd

device 3 osd.3 class hdd

device 4 osd.4 class ssd

device 5 osd.5 class ssd

# types

type 0 osd

type 1 host

type 2 chassis

type 3 rack

type 4 row

type 5 pdu

type 6 pod

type 7 room

type 8 datacenter

type 9 region

type 10 root

# buckets

host k-master {

        id -3          # do not change unnecessarily

        id -4 class hdd        # do not change unnecessarily

        id -9 class ssd        # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.0 weight 0.005

        item osd.1 weight 0.005

}

host k-worker001 {

        id -5          # do not change unnecessarily

        id -6 class hdd        # do not change unnecessarily

        id -10 class ssd                # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.2 weight 0.005

        item osd.4 weight 0.005

}

host k-worker002 {

        id -7          # do not change unnecessarily

        id -8 class hdd        # do not change unnecessarily

        id -11 class ssd                # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.3 weight 0.005

        item osd.5 weight 0.005

}

root default {

        id -1          # do not change unnecessarily

        id -2 class hdd        # do not change unnecessarily

        id -12 class ssd                # do not change unnecessarily

        # weight 0.029

        alg straw2

        hash 0  # rjenkins1

        item k-master weight 0.010

        item k-worker001 weight 0.010

        item k-worker002 weight 0.010

}

# rules

rule replicated_rule {

        id 0

        type replicated

        min_size 1

        max_size 10

        step take default

        step chooseleaf firstn 0 type host

        step emit

}

# end crush map

[root@k-master ceph]#

3.修改规则并应用

[1]修改后的规则文件

[root@k-master ceph]# cat crushmap.txt

# begin crush map

tunable choose_local_tries 0

tunable choose_local_fallback_tries 0

tunable choose_total_tries 50

tunable chooseleaf_descend_once 1

tunable chooseleaf_vary_r 1

tunable chooseleaf_stable 1

tunable straw_calc_version 1

tunable allowed_bucket_algs 54

# devices

device 0 osd.0 class hdd

device 1 osd.1 class hdd

device 2 osd.2 class hdd

device 3 osd.3 class hdd

device 4 osd.4 class ssd

device 5 osd.5 class ssd

# types

type 0 osd

type 1 host

type 2 chassis

type 3 rack

type 4 row

type 5 pdu

type 6 pod

type 7 room

type 8 datacenter

type 9 region

type 10 root

# buckets

host k-master {

        id -3          # do not change unnecessarily

        id -4 class hdd        # do not change unnecessarily

        id -9 class ssd        # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.0 weight 0.005

        item osd.1 weight 0.005

}

host k-worker001 {

        id -5          # do not change unnecessarily

        id -6 class hdd        # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.2 weight 0.005

}

host k-worker002 {

        id -7          # do not change unnecessarily

        id -8 class hdd        # do not change unnecessarily

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.3 weight 0.005

}

host k-worker001-ssd {

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.4 weight 0.005

}

host k-worker002-ssd {

        # weight 0.010

        alg straw2

        hash 0  # rjenkins1

        item osd.5 weight 0.005

}

root default {

        id -1          # do not change unnecessarily

        id -2 class hdd        # do not change unnecessarily

        # weight 0.029

        alg straw2

        hash 0  # rjenkins1

        item k-master weight 0.005

        item k-worker001 weight 0.005

        item k-worker002 weight 0.005

}

root ssd {

        # weight 0.029

        alg straw2

        hash 0  # rjenkins1

        item k-worker001-ssd weight 0.005

        item k-worker002-ssd weight 0.005

}

# rules

rule replicated_rule {

        id 0

        type replicated

        min_size 1

        max_size 10

        step take default

        step chooseleaf firstn 0 type host

        step emit

}

rule demo_rule {

        id 10

        type replicated

        min_size 1

        max_size 10

        step take ssd

        step chooseleaf firstn 0 type host

        step emit

}

# end crush map

[root@k-master ceph]#

[2]开始应用规则

[root@k-master ceph]# ceph osd tree

ID CLASS WEIGHT  TYPE NAME            STATUS REWEIGHT PRI-AFF

-1      0.02939 root default

-3      0.00980    host k-master

0  hdd 0.00490        osd.0            up  1.00000 1.00000

1  hdd 0.00490        osd.1            up  1.00000 1.00000

-5      0.00980    host k-worker001

2  hdd 0.00490        osd.2            up  1.00000 1.00000

4  ssd 0.00490        osd.4            up  1.00000 1.00000

-7      0.00980    host k-worker002

3  hdd 0.00490        osd.3            up  1.00000 1.00000

5  ssd 0.00490        osd.5            up  1.00000 1.00000

[root@k-master ceph]# vim crushmap.txt

[root@k-master ceph]# crushtool -c crushmap.txt -o crushmap-new.bin

[root@k-master ceph]# ll crushmap-new.bin

-rw-r--r-- 1 root root 1920 Jul 29 15:35 crushmap-new.bin

[root@k-master ceph]# ceph osd setcrushmap -i crushmap-new.bin

19

[root@k-master ceph]# ceph osd tree

ID  CLASS WEIGHT  TYPE NAME                STATUS REWEIGHT PRI-AFF

-12      0.00998 root ssd

-10      0.00499    host k-worker001-ssd

  4  ssd 0.00499        osd.4                up  1.00000 1.00000

-11      0.00499    host k-worker002-ssd

  5  ssd 0.00499        osd.5                up  1.00000 1.00000

-1      0.01497 root default

-3      0.00499    host k-master

  0  hdd 0.00499        osd.0                up  1.00000 1.00000

  1  hdd 0.00499        osd.1                up  1.00000 1.00000

-5      0.00499    host k-worker001

  2  hdd 0.00499        osd.2                up  1.00000 1.00000

-7      0.00499    host k-worker002

  3  hdd 0.00499        osd.3                up  1.00000 1.00000

[root@k-master ceph]#

可以看到，现在存在2种规则：ssd与default

4.修改资源池规则以便测试

[root@k-master ceph]# ceph osd lspools

1 rbd,3 ceph,4 k8s,6 ceph-demo,7 ceph-tt,8 rbd-test,9 .rgw.root,10 default.rgw.control,11 default.rgw.meta,12 default.rgw.log,13 default.rgw.buckets.index,14 default.rgw.buckets.data,15 cephfs_metadata,16 cephfs_data,

获取资源池的规则信息

[root@k-master ceph]# ceph osd pool get ceph-demo crush_rule

crush_rule: replicated_rule

列出现有规则

[root@k-master ceph]# ceph osd crush rule ls

replicated_rule

demo_rule

设置规则为demo_rule

[root@k-master ceph]# ceph osd pool set ceph-demo crush_rule demo_rule

set pool 6 crush_rule to demo_rule

[root@k-master ceph]# ceph osd pool get ceph-demo crush_rule

crush_rule: demo_rule

[root@k-master ceph]#

创建image进行落盘分析：

[root@k-master ceph]# rbd create ceph-demo/crush-demo.img --size 3G

[root@k-master ceph]# ceph osd map ceph-demo crush-demo.img

osdmap e255 pool 'ceph-demo' (6) object 'crush-demo.img' -> pg 6.d267742c (6.c) -> up ([4,5], p4) acting ([4,5], p4)

[root@k-master ceph]#

5.清理还原

[root@k-master ceph]# ceph osd pool set ceph-demo crush_rule replicated_rule

set pool 6 crush_rule to replicated_rule

[root@k-master ceph]# clear

还原备份规则

[root@k-master ceph]#  ceph osd setcrushmap -i crushmap.bin

20

[root@k-master ceph]# ceph osd tree

ID CLASS WEIGHT  TYPE NAME            STATUS REWEIGHT PRI-AFF

-1      0.02939 root default

-3      0.00980    host k-master

0  hdd 0.00490        osd.0            up  1.00000 1.00000

1  hdd 0.00490        osd.1            up  1.00000 1.00000

-5      0.00980    host k-worker001

2  hdd 0.00490        osd.2            up  1.00000 1.00000

4  ssd 0.00490        osd.4            up  1.00000 1.00000

-7      0.00980    host k-worker002

3  hdd 0.00490        osd.3            up  1.00000 1.00000

5  ssd 0.00490        osd.5            up  1.00000 1.00000

[root@k-master ceph]#

[root@k-master ceph]#  ceph osd crush rule ls

replicated_rule

[root@k-master ceph]#

（二）通过命令行调整Crush map规则

1.一系列添加移动操作

[root@k-master ceph]# ceph osd crush add-bucket ssd root

added bucket ssd type root to crush map

[root@k-master ceph]# ceph osd tree

ID  CLASS WEIGHT  TYPE NAME            STATUS REWEIGHT PRI-AFF

-13            0 root ssd

-1      0.02939 root default

-3      0.00980    host k-master

  0  hdd 0.00490        osd.0            up  1.00000 1.00000

  1  hdd 0.00490        osd.1            up  1.00000 1.00000

-5      0.00980    host k-worker001

  2  hdd 0.00490        osd.2            up  1.00000 1.00000

  4  ssd 0.00490        osd.4            up  1.00000 1.00000

-7      0.00980    host k-worker002

  3  hdd 0.00490        osd.3            up  1.00000 1.00000

  5  ssd 0.00490        osd.5            up  1.00000 1.00000

[root@k-master ceph]# ceph osd crush add-bucket k-worker001-ssd host

added bucket k-worker001-ssd type host to crush map

[root@k-master ceph]# ceph osd crush add-bucket k-worker002-ssd host

added bucket k-worker002-ssd type host to crush map

[root@k-master ceph]#

[root@k-master ceph]# ceph osd tree

ID  CLASS WEIGHT  TYPE NAME            STATUS REWEIGHT PRI-AFF

-15            0 host k-worker002-ssd

-14            0 host k-worker001-ssd

-13            0 root ssd

-1      0.02939 root default

-3      0.00980    host k-master

  0  hdd 0.00490        osd.0            up  1.00000 1.00000

  1  hdd 0.00490        osd.1            up  1.00000 1.00000

-5      0.00980    host k-worker001

  2  hdd 0.00490        osd.2            up  1.00000 1.00000

  4  ssd 0.00490        osd.4            up  1.00000 1.00000

-7      0.00980    host k-worker002

  3  hdd 0.00490        osd.3            up  1.00000 1.00000

  5  ssd 0.00490        osd.5            up  1.00000 1.00000

[root@k-master ceph]#

[root@k-master ceph]# ceph osd crush move k-worker002-ssd root=ssd

moved item id -15 name 'k-worker002-ssd' to location {root=ssd} in crush map

[root@k-master ceph]# ceph osd crush move k-worker001-ssd root=ssd

moved item id -14 name 'k-worker001-ssd' to location {root=ssd} in crush map

[root@k-master ceph]# ceph osd tree

ID  CLASS WEIGHT  TYPE NAME                STATUS REWEIGHT PRI-AFF

-13            0 root ssd

-14            0    host k-worker001-ssd

-15            0    host k-worker002-ssd

-1      0.02939 root default

-3      0.00980    host k-master

  0  hdd 0.00490        osd.0                up  1.00000 1.00000

  1  hdd 0.00490        osd.1                up  1.00000 1.00000

-5      0.00980    host k-worker001

  2  hdd 0.00490        osd.2                up  1.00000 1.00000

  4  ssd 0.00490        osd.4                up  1.00000 1.00000

-7      0.00980    host k-worker002

  3  hdd 0.00490        osd.3                up  1.00000 1.00000

  5  ssd 0.00490        osd.5                up  1.00000 1.00000

[root@k-master ceph]#

[root@k-master ceph]# ceph osd crush move osd.4 host=k-worker001-ssd root=ssd

moved item id 4 name 'osd.4' to location {host=k-worker001-ssd,root=ssd} in crush map

[root@k-master ceph]# ceph osd crush move osd.5 host=k-worker002-ssd root=ssd

moved item id 5 name 'osd.5' to location {host=k-worker002-ssd,root=ssd} in crush map

[root@k-master ceph]# ceph osd tree

ID  CLASS WEIGHT  TYPE NAME                STATUS REWEIGHT PRI-AFF

-13      0.00980 root ssd

-14      0.00490    host k-worker001-ssd

  4  ssd 0.00490        osd.4                up  1.00000 1.00000

-15      0.00490    host k-worker002-ssd

  5  ssd 0.00490        osd.5                up  1.00000 1.00000

-1      0.01959 root default

-3      0.00980    host k-master

  0  hdd 0.00490        osd.0                up  1.00000 1.00000

  1  hdd 0.00490        osd.1                up  1.00000 1.00000

-5      0.00490    host k-worker001

  2  hdd 0.00490        osd.2                up  1.00000 1.00000

-7      0.00490    host k-worker002

  3  hdd 0.00490        osd.3                up  1.00000 1.00000

[root@k-master ceph]#

2.创建规则

[root@k-master ceph]# ceph osd crush rule ls

replicated_rule

[root@k-master ceph]# ceph osd crush rule create-replicated ssd-demo ssd host ssd

[root@k-master ceph]# ceph osd crush rule ls

replicated_rule

ssd-demo

[root@k-master ceph]#

3.测试规则

[root@k-master ceph]# ceph osd pool set ceph-demo crush_rule ssd-demo

set pool 6 crush_rule to ssd-demo

[root@k-master ceph]# ceph osd pool get ceph-demo crush_rule

crush_rule: ssd-demo

[root@k-master ceph]# rbd create ceph-demo/ssd-demo.img --size 3G

[root@k-master ceph]# ceph osd map ceph-demo crush-demo.img

osdmap e292 pool 'ceph-demo' (6) object 'crush-demo.img' -> pg 6.d267742c (6.c) -> up ([4,5], p4) acting ([4,5], p4)

[root@k-master ceph]#

（三）编辑Crush Map的注意事项

1.编辑之前，注意备份。

2.初始时，规划好Crush Map，否则实施时会引发很大的迁移变动。

3.osd_crush_update_on_start参数调整为false

[root@k-master ceph]# ceph daemon /var/run/ceph/ceph-osd.1.asok config show | grep 'osd_crush_update_on_start'

    "osd_crush_update_on_start": "true",

[root@k-master ceph]# tail -n 2 ceph.conf

[osd]

osd crush update on start = false

[root@k-master ceph]# ceph-deploy --overwrite-conf config push k-master k-worker001 k-worker002

[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf

[ceph_deploy.cli][INFO  ] Invoked (2.0.1): /usr/bin/ceph-deploy --overwrite-conf config push k-master k-worker001 k-worker002

[ceph_deploy.cli][INFO  ] ceph-deploy options:

[ceph_deploy.cli][INFO  ]  username                      : None

[ceph_deploy.cli][INFO  ]  verbose                      : False

[ceph_deploy.cli][INFO  ]  overwrite_conf                : True

[ceph_deploy.cli][INFO  ]  subcommand                    : push

[ceph_deploy.cli][INFO  ]  quiet                        : False

[ceph_deploy.cli][INFO  ]  cd_conf                      : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f99512345a8>

[ceph_deploy.cli][INFO  ]  cluster                      : ceph

[ceph_deploy.cli][INFO  ]  client                        : ['k-master', 'k-worker001', 'k-worker002']

[ceph_deploy.cli][INFO  ]  func                          : <function config at 0x7f995124fc08>

[ceph_deploy.cli][INFO  ]  ceph_conf                    : None

[ceph_deploy.cli][INFO  ]  default_release              : False

[ceph_deploy.config][DEBUG ] Pushing config to k-master

[k-master][DEBUG ] connected to host: k-master

[k-master][DEBUG ] detect platform information from remote host

[k-master][DEBUG ] detect machine type

[k-master][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf

[ceph_deploy.config][DEBUG ] Pushing config to k-worker001

[k-worker001][DEBUG ] connected to host: k-worker001

[k-worker001][DEBUG ] detect platform information from remote host

[k-worker001][DEBUG ] detect machine type

[k-worker001][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf

[ceph_deploy.config][DEBUG ] Pushing config to k-worker002

[k-worker002][DEBUG ] connected to host: k-worker002

[k-worker002][DEBUG ] detect platform information from remote host

[k-worker002][DEBUG ] detect machine type

[k-worker002][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf

[root@k-master ceph]# systemctl restart ceph-osd.target

[root@k-master ceph]# ssh k-worker001

Last login: Wed Jul 29 14:45:39 2020 from 192.168.43.201

[root@k-worker001 ~]# systemctl restart ceph-osd.target

[root@k-worker001 ~]# exit

logout

Connection to k-worker001 closed.

[root@k-master ceph]# ssh k-worker002

Last login: Wed Jul 29 14:45:38 2020 from 192.168.43.201

[root@k-worker002 ~]# systemctl restart ceph-osd.target

[root@k-worker002 ~]# exit

logout

Connection to k-worker002 closed.

[root@k-master ceph]# ceph daemon /var/run/ceph/ceph-osd.1.asok config show | grep 'osd_crush_update_on_start'

    "osd_crush_update_on_start": "false",

[root@k-master ceph]#

©著作权归作者所有,转载或内容合作请联系作者
禁止转载，如需转载请通过简信或评论联系作者。
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 220,367评论 6赞 512
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 93,959评论 3赞 396
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 166,750评论 0赞 357
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 59,226评论 1赞 295
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 68,252评论 6赞 397
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 51,975评论 1赞 308
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 40,592评论 3赞 420
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 39,497评论 0赞 276
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 46,027评论 1赞 319
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 38,147评论 3赞 340
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 40,274评论 1赞 352
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 35,953评论 5赞 347
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 41,623评论 3赞 331
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 32,143评论 0赞 23
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 33,260评论 1赞 272
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 48,607评论 3赞 375
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 45,271评论 2赞 358