pipeline
数据标注 doccano/labelstudio
lora微调 ms-swift
模型部署
- vllm
- llama.cpp
- ollama
安装更新参考：https://github.com/ollama/ollama/blob/main/docs/linux.md- 已支持的模型架构，直接import safetensor模型
- 未支持的模型架构，需要用llama.cpp转成gguf格式
llama.cpp安装可以用homebrew参考https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md
convert_hf_to_gguf.py直接下载llama.cpp代码库zip包
mv llama.cpp-master llama.cpp
cd llama.cpp
python convert_hf_to_gguf.py /data/tsliu/checkpoint/cpms-trade-tag/cpms-trade-tag-sft-250901-mexico-2-8B-100/v0-20250911-175818/checkpoint-26200-merged/ --outfile /data/tsliu/checkpoint/cpms-trade-tag/cpms-trade-tag-sft-250901-mexico-2-8B-100/v0-20250911-175818/qwen3-8b-trade-tag.gguf
ollama
gathering model components
copying file sha256:37cd7996c99bcb361d3d048eb7116b1b19a54dafef26cd87152a4d25292826da 100%
parsing GGUF
using existing layer sha256:37cd7996c99bcb361d3d048eb7116b1b19a54dafef26cd87152a4d25292826da
writing manifest
success
- ollama modelfile配置
需要配置的地方比较多，参考：https://github.com/ollama/ollama/blob/main/docs/modelfile.md
