安装Docker
参考：https://www.jianshu.com/p/62bac0a1060a
Docker支持Nvidia驱动
参考：docker支持nvidia gpu https://www.jianshu.com/p/ed1e54c2a972
安装Ollama
# 拉取镜像
docker pull ollama/ollama:0.5.7
# Docker方式运行Ollama
mkdir -p /data/ollama
docker run -d --gpus=all -v /data/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:0.5.7
--gpus=all参数是为了在docker容器内可以使用GPU
--gpus "device=0": 使用第一个 GPU 设备--gpus "device=2,3": 使用第三和第四个 GPU 设备
由于习惯docker-compose，以docker-compose方式启动的话，上述docker run可以用如下命令代替：
mkdir -p /data/ollama/data
cat > /data/ollama/docker-compose.yml <<"EOF"
services:
ollama:
image: ollama/ollama:0.5.7
container_name: ollama
restart: always
ports:
- "11434:11434"
volumes:
- /data/ollama/data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
# count 与 device_ids 二选一
# count指定需要使用的GPU数量；值为int数组型或all；
# device_ids: ["0", "3"]
count: "all"
capabilities: ["gpu"]
EOF
# 启动Ollama
docker-compose up -d
Ollama更多环境变量，可以参考：https://zhuanlan.zhihu.com/p/712642806
运行Deepseek方式一：直接运行Ollama官方大模型
官方模型：https://ollama.com/search
# 进入容器内
docker exec -it ollama bash
# -------------以下命令都在容器内执行-------------
# 运行deepseek-r1:7b
ollama run deepseek-r1:7b
效果如下：
[root@bogon ollama]# docker exec -it ollama bash
root@e2809411e208:/# ollama run deepseek-r1:7b
pulling manifest
pulling 96c415656d37... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████ ▏ 4.7 GB/4.7 GB 14 MB/s 0s
pulling 369ca498f347... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 387 B
pulling 6e4c38e1172f... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 1.1 KB
pulling f4d24e9138dd... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 148 B
pulling 40fb844194b2... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 487 B
verifying sha256 digest
writing manifest
success
>>> 你是谁？
<think>
</think>
您好！我是由中国的深度求索（DeepSeek）公司开发的智能助手DeepSeek-R1。如您有任何任何问题，我会尽我所能为您提供帮助。
>>> Send a message (/? for help)
运行Deepseek方式二：通过离线gguf文件运行大模型
1. 下载gguf文件
文件来源：
下载方式：
- wget
wget -b https://www.modelscope.cn/models/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF/resolve/master/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf
- modelscope命令（推荐）
2. 编写 modelfile
cat > /data/ollama/data/gguf/deepseek-r1-70B-modelfile <<"EOF"
FROM ./DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf
# 设置对话终止符
PARAMETER stop "<｜begin▁of▁sentence｜>"
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER stop "<｜User｜>"
PARAMETER stop "<｜Assistant｜>"
SYSTEM """
"""
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}"""
EOF
modelfile必须与gguf文件同目录，否则需要修改modelfile第一行的相对位置
3. 创建模型
# 通过gguf创建模型(docker内执行）
ollama create deepseek-r1-custom:70B -f /root/.ollama/mygguf/deepseek-r1-70B-modelfile
4. 运行模型
# docker内执行
ollama run deepseek-r1-custom:70B
通过Ollama API 测试模型
# 通过ollama api测试模型
curl http://10.3.6.38:11434/api/chat -d '{
"model": "deepseek-r1-bidr:70b",
"messages": [
{
"role": "user",
"content": "为什么天空是蓝色的?"
}
],
"stream": true
}'
效果：
[root@bogon ~]# curl http://10.3.6.38:11434/api/chat -d '{
"model": "deepseek-r1-bidr:70b",
"messages": [
{
"role": "user",
"content": "为什么天空是蓝色的?"
}
],
"stream": true
}'
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.490403406Z","message":{"role":"assistant","content":"\u003cthink\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.62964314Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.782469849Z","message":{"role":"assistant","content":"\u003c/think\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.925424223Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.060309984Z","message":{"role":"assistant","content":"天"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.209522495Z","message":{"role":"assistant","content":"空"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.369515693Z","message":{"role":"assistant","content":"呈"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.52581505Z","message":{"role":"assistant","content":"现"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.686391838Z","message":{"role":"assistant","content":"蓝"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.82169606Z","message":{"role":"assistant","content":"色"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.969790941Z","message":{"role":"assistant","content":"是"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.105679167Z","message":{"role":"assistant","content":"因为"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.243647488Z","message":{"role":"assistant","content":"当"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.380513238Z","message":{"role":"assistant","content":"太"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.525637676Z","message":{"role":"assistant","content":"阳"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.669904633Z","message":{"role":"assistant","content":"光"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.813863747Z","message":{"role":"assistant","content":"照"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.970479134Z","message":{"role":"assistant","content":"射"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.109944242Z","message":{"role":"assistant","content":"到"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.242174673Z","message":{"role":"assistant","content":"地球"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.381833905Z","message":{"role":"assistant","content":"大"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.51949058Z","message":{"role":"assistant","content":"气"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.66205966Z","message":{"role":"assistant","content":"层"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.798480342Z","message":{"role":"assistant","content":"时"},"done":false}
