背景介绍
需要访问ceph创建subvolume卷，因为在ceph quincy版本rest api不支持subvolume的创建（reef版本已经支持）。所以需要引入ceph的sdk库进行subvlume的创建。我选择的是go-ceph:github.com/ceph/go-ceph
前提条件
- 在ceph已创建好volume和subvolumegroup
ceph fs volume create xxx
ceph fs subvolumegroup create <vol_name> <group_name>
- 创建对接账号
ceph auth get-or-create client.test mon "allow r" mds "allow rw" osd 'allow rwx pool=cephfs.testfs.data'
#生成密钥对
ceph auth get client.test -o ceph.client.test.keyring
ceph auth print-key client.test> test.key
创建subvolume
创建go-ceph客户端(参考csi代码修改)
import (
"fmt"
"github.com/ceph/go-ceph/rados"
)
const (
cephConfigRoot = "/etc/ceph"
// CephConfigPath ceph configuration file.
CephConfigPath = "/etc/ceph/ceph.conf"
keyRing = "/etc/ceph/keyring"
)
func GetConn(monitors, user, keyfile string) (*rados.Conn, error) {
// construct and connect a new rados.Conn
args := []string{"--keyfile=" + keyfile}
conn, err := rados.NewConnWithUser(user)
if err != nil {
return nil, fmt.Errorf("creating a new connection failed: %w", err)
}
err = conn.ParseCmdLineArgs(args)
if err != nil {
return nil, fmt.Errorf("parsing cmdline args (%v) failed: %w", args, err)
}
if err = conn.ReadConfigFile(CephConfigPath); err != nil {
return nil, fmt.Errorf("failed to read config file %q: %w", CephConfigPath, err)
}
err = conn.Connect()
if err != nil {
return nil, fmt.Errorf("connecting failed: %w", err)
}
return conn, nil
}
conn, err := GetConn("", "slurm", "/etc/ceph/slurm.key")
创建ceph fs客户端, 并创建subvolume
import ca "github.com/ceph/go-ceph/cephfs/admin"
fsadmin := ca.NewFromConn(conn)
if err := fsadmin.CreateSubVolume("testfs", "data", fmt.Sprintf("mount%d", rand.Intn(100)), &ca.SubVolumeOptions{Size: 10}); err != nil {
fmt.Printf("create subvolume err, %v", err)
eturn err
}
return nil
编译
因为go-ceph 引入了ceph的c语言动态链接库，需要找一台linux机器，安装对应的动态链接库之后才能正常编译。ceph-go中已经做了相关说明GitHub - ceph/go-ceph: Go bindings for Ceph :octopus: :octopus:
The code in go-ceph is purely a library module. Typically, one will import go-ceph in another Go based project. When building the code the native RADOS, RBD, & CephFS library and development headers are expected to be installed.
On debian based systems (apt) these may be:
libcephfs-dev librbd-dev librados-dev
On rpm based systems (dnf, yum, etc) these may be:
libcephfs-devel librbd-devel librados-devel
On MacOS you can use brew to install the libraries:
brew tap mulbc/ceph-client
brew install ceph-client
