Ceph 创建 OSD 时报错 GPT headers found, they must be removed on: /dev/sdX，全文示例如下：

Failed to apply osd.all-available-devices spec DriveGroupSpec(name=all-available-devices->placement=PlacementSpec(host_pattern='*'), service_id='all-available-devices', service_type='osd', data_devices=DeviceSelection(all=True), osd_id_claims={}, unmanaged=False, filter_logic='AND'): cephadm exited with an error code: 1, stderr:/usr/bin/docker:stderr usage: ceph-volume lvm batch [-h] [--db-devices [DB_DEVICES [DB_DEVICES ...]]] /usr/bin/docker:stderr [--wal-devices [WAL_DEVICES [WAL_DEVICES ...]]] /usr/bin/docker:stderr [--journal-devices [JOURNAL_DEVICES [JOURNAL_DEVICES ...]]] /usr/bin/docker:stderr [--auto] [--no-auto] [--bluestore] [--filestore] /usr/bin/docker:stderr [--report] [--yes] /usr/bin/docker:stderr [--format {json,json-pretty,pretty}] [--dmcrypt] /usr/bin/docker:stderr [--crush-device-class CRUSH_DEVICE_CLASS] /usr/bin/docker:stderr [--no-systemd] /usr/bin/docker:stderr [--osds-per-device OSDS_PER_DEVICE] /usr/bin/docker:stderr [--data-slots DATA_SLOTS] /usr/bin/docker:stderr [--block-db-size BLOCK_DB_SIZE] /usr/bin/docker:stderr [--block-db-slots BLOCK_DB_SLOTS] /usr/bin/docker:stderr [--block-wal-size BLOCK_WAL_SIZE] /usr/bin/docker:stderr [--block-wal-slots BLOCK_WAL_SLOTS] /usr/bin/docker:stderr [--journal-size JOURNAL_SIZE] /usr/bin/docker:stderr [--journal-slots JOURNAL_SLOTS] [--prepare] /usr/bin/docker:stderr [--osd-ids [OSD_IDS [OSD_IDS ...]]] /usr/bin/docker:stderr [DEVICES [DEVICES ...]] /usr/bin/docker:stderr ceph-volume lvm batch: error: GPT headers found, they must be removed on: /dev/sda Traceback (most recent call last): File "<stdin>", line 6113, in <module> File "<stdin>", line 1300, in _infer_fsid File "<stdin>", line 1383, in _infer_image File "<stdin>", line 3613, in command_ceph_volume File "<stdin>", line 1062, in call_throws RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15 -e NODE_NAME=yc-hy08 -e CEPH_VOLUME_OSDSPEC_AFFINITY=all-available-devices -v /var/run/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033:/var/run/ceph:z -v /var/log/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033:/var/log/ceph:z -v /var/lib/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /tmp/ceph-tmplk1m2oxc:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp7vahlwql:/var/lib/ceph/bootstrap-osd/ceph.keyring:z docker.io/ceph/ceph:v15 lvm batch --no-auto /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh --yes --no-systemd Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/serve.py", line 387, in _apply_all_services if self._apply_service(spec): File "/usr/share/ceph/mgr/cephadm/serve.py", line 425, in _apply_service self.mgr.osd_service.create_from_spec(cast(DriveGroupSpec, spec)) File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 47, in create_from_spec ret = create_from_spec_one(self.prepare_drivegroup(drive_group)) File "/usr/share/ceph/mgr/cephadm/utils.py", line 68, in forall_hosts_wrapper return CephadmOrchestrator.instance._worker_pool.map(do_work, vals) File "/lib64/python3.6/multiprocessing/pool.py", line 266, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/lib64/python3.6/multiprocessing/pool.py", line 644, in get raise self._value File "/lib64/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar return list(map(*args)) File "/usr/share/ceph/mgr/cephadm/utils.py", line 62, in do_work return f(*arg) File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 43, in create_from_spec_one host, cmd, replace_osd_ids=osd_id_claims.get(host, []), env_vars=env_vars File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 63, in create_single_host code, '\n'.join(err))) RuntimeError: cephadm exited with an error code: 1, stderr:/usr/bin/docker:stderr usage: ceph-volume lvm batch [-h] [--db-devices [DB_DEVICES [DB_DEVICES ...]]] /usr/bin/docker:stderr [--wal-devices [WAL_DEVICES [WAL_DEVICES ...]]] /usr/bin/docker:stderr [--journal-devices [JOURNAL_DEVICES [JOURNAL_DEVICES ...]]] /usr/bin/docker:stderr [--auto] [--no-auto] [--bluestore] [--filestore] /usr/bin/docker:stderr [--report] [--yes] /usr/bin/docker:stderr [--format {json,json-pretty,pretty}] [--dmcrypt] /usr/bin/docker:stderr [--crush-device-class CRUSH_DEVICE_CLASS] /usr/bin/docker:stderr [--no-systemd] /usr/bin/docker:stderr [--osds-per-device OSDS_PER_DEVICE] /usr/bin/docker:stderr [--data-slots DATA_SLOTS] /usr/bin/docker:stderr [--block-db-size BLOCK_DB_SIZE] /usr/bin/docker:stderr [--block-db-slots BLOCK_DB_SLOTS] /usr/bin/docker:stderr [--block-wal-size BLOCK_WAL_SIZE] /usr/bin/docker:stderr [--block-wal-slots BLOCK_WAL_SLOTS] /usr/bin/docker:stderr [--journal-size JOURNAL_SIZE] /usr/bin/docker:stderr [--journal-slots JOURNAL_SLOTS] [--prepare] /usr/bin/docker:stderr [--osd-ids [OSD_IDS [OSD_IDS ...]]] /usr/bin/docker:stderr [DEVICES [DEVICES ...]] /usr/bin/docker:stderr ceph-volume lvm batch: error: GPT headers found, they must be removed on: /dev/sda Traceback (most recent call last): File "<stdin>", line 6113, in <module> File "<stdin>", line 1300, in _infer_fsid File "<stdin>", line 1383, in _infer_image File "<stdin>", line 3613, in command_ceph_volume File "<stdin>", line 1062, in call_throws RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15 -e NODE_NAME=yc-hy08 -e CEPH_VOLUME_OSDSPEC_AFFINITY=all-available-devices -v /var/run/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033:/var/run/ceph:z -v /var/log/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033:/var/log/ceph:z -v /var/lib/ceph/1c61cc7a-433d-11eb-9f93-2cea7f689033/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /tmp/ceph-tmplk1m2oxc:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp7vahlwql:/var/lib/ceph/bootstrap-osd/ceph.keyring:z docker.io/ceph/ceph:v15 lvm batch --no-auto /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh --yes --no-systemd

该错误关键点就是 GPT headers found, they must be removed，发生原因应该是之前磁盘被分区过，虽然删掉了分区，但是还存在 GPT 数据结构，使用 sgdisk 命令进行清除。
#sgdisk --zap-all /dev/sdX
sgdisk 命令参数如下：

Usage: sgdisk  [OPTION...] <device>
  -A, --attributes=list|[partnum:show|or|nand|xor|=|set|clear|toggle|get[:bitnum|hexbitmask]]     operate on partition attributes
  -a, --set-alignment=value                                                                       set sector alignment
  -b, --backup=file                                                                               backup GPT to file
  -c, --change-name=partnum:name                                                                  change partition's name
  -C, --recompute-chs                                                                             recompute CHS values in protective/hybrid MBR
  -d, --delete=partnum                                                                            delete a partition
  -D, --display-alignment                                                                         show number of sectors per allocation block
  -e, --move-second-header                                                                        move second header to end of disk
  -E, --end-of-largest                                                                            show end of largest free block
  -f, --first-in-largest                                                                          show start of the largest free block
  -F, --first-aligned-in-largest                                                                  show start of the largest free block, aligned
  -g, --mbrtogpt                                                                                  convert MBR to GPT
  -G, --randomize-guids                                                                           randomize disk and partition GUIDs
  -h, --hybrid=partnum[:partnum...]                                                               create hybrid MBR
  -i, --info=partnum                                                                              show detailed information on partition
  -j, --move-main-table=sector                                                                    adjust the location of the main partition table
  -l, --load-backup=file                                                                          load GPT backup from file
  -L, --list-types                                                                                list known partition types
  -m, --gpttombr=partnum[:partnum...]                                                             convert GPT to MBR
  -n, --new=partnum:start:end                                                                     create new partition
  -N, --largest-new=partnum                                                                       create largest possible new partition
  -o, --clear                                                                                     clear partition table
  -O, --print-mbr                                                                                 print MBR partition table
  -p, --print                                                                                     print partition table
  -P, --pretend                                                                                   make changes in memory, but don't write them
  -r, --transpose=partnum:partnum                                                                 transpose two partitions
  -R, --replicate=device_filename                                                                 replicate partition table
  -s, --sort                                                                                      sort partition table entries
  -S, --resize-table=numparts                                                                     resize partition table
  -t, --typecode=partnum:{hexcode|GUID}                                                           change partition type code
  -T, --transform-bsd=partnum                                                                     transform BSD disklabel partition to GPT
  -u, --partition-guid=partnum:guid                                                               set partition GUID
  -U, --disk-guid=guid                                                                            set disk GUID
  -v, --verify                                                                                    check partition table integrity
  -V, --version                                                                                   display version information
  -z, --zap                                                                                       zap (destroy) GPT (but not MBR) data structures
  -Z, --zap-all                                                                                   zap (destroy) GPT and MBR data structures

Help options:
  -?, --help                                                                                      Show this help message
      --usage                                                                                     Display brief usage message