ubuntu 22.04 部署 ollama + deepseek + open webui
1. 环境：以下 kvm 虚拟机
| 系统 | CPU | 内存 | GPU |
|---|---|---|---|
| Ubuntu 22.04 | 64 core | 512GB | v100 * 3 |
2. 安装 V100 驱动
apt update aptinstall-y software-properties-common 驱动包资源
add-apt-repository ppa:graphics-drivers/ppa -yaptinstall ubuntu-drivers-common 查看可以安装的版本
ubuntu-drivers devices 删除已经安装的驱动
apt-get remove --purge'^nvidia-.*'自动安装最新版本
ubuntu-drivers install或安装指定版本
aptinstall nvidia-driver-565 重启
reboot查看 GPU 信息
nvidia-smi Wed Feb 12 09:39:33 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 565.57.01 Driver Version: 565.57.01 CUDA Version: 12.7||-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |||| MIG M. ||=========================================+========================+======================||0 Tesla V100-PCIE-16GB-LS On | 00000000:00:07.0 Off |0|| N/A 36C P0 24W / 250W | 4MiB / 16384MiB |0% Default |||| N/A | +-----------------------------------------+------------------------+----------------------+ |1 Tesla V100-PCIE-16GB-LS On | 00000000:00:08.0 Off |0|| N/A 38C P0 24W / 250W | 4MiB / 16384MiB |0% Default |||| N/A | +-----------------------------------------+------------------------+----------------------+ |2 Tesla V100-PCIE-16GB-LS On | 00000000:00:09.0 Off |0|| N/A 36C P0 26W / 250W | 4MiB / 16384MiB |0% Default |||| N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage ||=========================================================================================|| No running processes found | +-----------------------------------------------------------------------------------------+ 3. 安装 CUDA
下载 CUDA 软件包源
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb 加载资源包
dpkg -i cuda-keyring_1.1-1_all.deb 查看 CUDA 版本
apt policy cuda-toolkit 安装 CUDA
aptinstall cuda-toolkit 配置 CUDA 环境变量
exportCUDA_HOME=/usr/local/cuda exportPATH=${CUDA_HOME}/bin:${PATH}exportLD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH查看 CUDA 信息
nvcc --version 4. 安装 Ollama
安装命令
curl-fsSL https://ollama.com/install.sh |sh安装完成后查看 Ollama 状态
service ollama status 日志错误信息如下
Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.416+08:00 level=INFO source=routes.go:1238 msg="Listening on 127.0.0.1:11434 (version 0.5.7)" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.417+08:00 level=INFO source=common.go:131 msg="GPU runner incompatible with host system, CPU does not have AVX" runner=cuda_v11_avx Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.417+08:00 level=INFO source=common.go:131 msg="GPU runner incompatible with host system, CPU does not have AVX" runner=cuda_v12_avx Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.417+08:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cpu_avx2 rocm_avx cpu cpu_avx]" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.417+08:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.550+08:00 level=INFO source=gpu.go:283 msg="error looking up nvidia GPU memory" error="cuda driver library failed to get device context 801" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.553+08:00 level=INFO source=gpu.go:283 msg="error looking up nvidia GPU memory" error="cuda driver library failed to get device context 801" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.557+08:00 level=INFO source=gpu.go:283 msg="error looking up nvidia GPU memory" error="cuda driver library failed to get device context 801" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.558+08:00 level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered" Feb 11 17:50:06 i-mvlzfacx ollama[6794]: time=2025-02-11T17:50:06.558+08:00 level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant="no vector extensions" driver=0.0 total="503.7 GiB" available=> 问题原因
GPU runner incompatible with host system, CPU does not have AVX 根据错误信息，虚拟机 VCPU 缺少 AVX 指令集，导致 GPU 不能使用。
查看 CPU 是否支持 AVX
lscpu |grep avx 没有 AVX 信息。
5. 修改虚拟机 config.xml 配置
在 <cpu> 中添加如下内容：
<cpumode='custom'match='exact'check='full'><modelfallback='forbid'>Skylake-Server</model><topologysockets='4'cores='16'threads='1'/><featurepolicy='require'name='avx'/><featurepolicy='require'name='avx2'/><featurepolicy='require'name='hypervisor'/></cpu>重新定义虚拟机，查看 AVX：
lscpu |grep avx 查看输出：
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 arat 6. 再次查看 Ollama 已经正常
查看 Ollama 服务状态：
service ollama status 输出状态：
ollama.service - Ollama Service Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2025-02-12 09:32:09 CST; 10min ago Main PID: 1529 (ollama) Tasks: 27 (limit: 618662) Memory: 8.1G CPU: 1min 21.889s CGroup: /system.slice/ollama.service └─1529 /usr/local/bin/ollama serve Feb 12 09:32:10 i-mvlzfacx ollama[1529]: [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) Feb 12 09:32:10 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:10.875+08:00 level=INFO source=routes.go:1238 msg="Listening on 127.0.0.1:11434 (version 0.5.7)" Feb 12 09:32:10 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:10.885+08:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cpu_avx2 cuda_v11_avx cuda_v12_avx rocm_avx cpu cpu_avx]" Feb 12 09:32:10 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:10.886+08:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs" Feb 12 09:32:12 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:12.464+08:00 level=INFO source=types.go:131 msg="inference compute" id=GPU-745b3d31-7b14-6335-7ea8-d27ea7261802 library=cuda variant=v12 compute=7.0 driver=12.7 name="Te> Feb 12 09:32:12 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:12.464+08:00 level=INFO source=types.go:131 msg="inference compute" id=GPU-bd0014a9-9fb8-ade2-6054-a721c20dbef1 library=cuda variant=v12 compute=7.0 driver=12.7 name="Te> Feb 12 09:32:12 i-mvlzfacx ollama[1529]: time=2025-02-12T09:32:12.464+08:00 level=INFO source=types.go:131 msg="inference compute" id=GPU-5cfd0bcc-c8c5-29ec-4f8d-630adb6d33b2 library=cuda variant=v12 compute=7.0 driver=12.7 name="Te> Feb 12 09:36:42 i-mvlzfacx ollama[1529]: [GIN] 2025/02/12 - 09:36:42 | 200 | 18.869142ms | 127.0.0.1 | HEAD "/" Feb 12 09:36:42 i-mvlzfacx ollama[1529]: [GIN] 2025/02/12 - 09:36:42 | 404 | 644.305µs | 127.0.0.1 | POST "/api/show" Feb 12 09:36:45 i-mvlzfacx ollama[1529]: time=2025-02-12T09:36:45.027+08:00 level=INFO source=download.go:175 msg="downloading 6e9f90f02bb3 in 16 561 MB part(s)" 7. 使用 Ollama 下载 DeepSeek
运行命令：
# ollama run deepseek-r1:14b pulling manifest pulling 6e9f90f02bb3... 100% ▕███████████████████████████████████████████████████▏ 9.0 GB pulling 369ca498f347... 100% ▕███████████████████████████████████████████████████▏ 387 B pulling 6e4c38e1172f... 100% ▕███████████████████████████████████████████████████▏ 1.1 KB pulling f4d24e9138dd... 100% ▕███████████████████████████████████████████████████▏ 148 B pulling 3c24b0c80794... 100% ▕███████████████████████████████████████████████████▏ 488 B verifying sha256 digest writing manifest success >>>8. 监控 GPU 信息
watch-n1 nvidia-smi 输出显示：
Every 1.0s: nvidia-smi i-mvlzfacx: Wed Feb 12 09:56:13 2025 Wed Feb 12 09:56:13 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 565.57.01 Driver Version: 565.57.01 CUDA Version: 12.7 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-PCIE-16GB-LS On | 00000000:00:07.0 Off | 0 | | N/A 38C P0 38W / 250W | 10694MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 Tesla V100-PCIE-16GB-LS On | 00000000:00:08.0 Off | 0 | | N/A 37C P0 24W / 250W | 4MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 2 Tesla V100-PCIE-16GB-LS On | 00000000:00:09.0 Off | 0 | | N/A 35C P0 26W / 250W | 4MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 2151 C ...rs/cuda_v12_avx/ollama_llama_server 10690MiB | +-----------------------------------------------------------------------------------------+ 此时发现只有一张 v100 在被使用
9. 环境变量中添加 CUDA_VISIBLE_DEVICES
exportCUDA_VISIBLE_DEVICES=0,1,2 重启 Ollama：
service ollama restart 再次运行 DeepSeek，并查看 GPU 监控，发现三张 GPU 都被使用了：
Every 1.0s: nvidia-smi i-mvlzfacx: Wed Feb 12 10:19:25 2025 Wed Feb 12 10:19:25 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 565.57.01 Driver Version: 565.57.01 CUDA Version: 12.7 | |-----------------------------------------+------------------------+----------------------| | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-PCIE-16GB-LS On | 00000000:00:07.0 Off | 0 | | N/A 38C P0 38W / 250W | 14452MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 Tesla V100-PCIE-16GB-LS On | 00000000:00:08.0 Off | 0 | | N/A 39C P0 38W / 250W | 13804MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 2 Tesla V100-PCIE-16GB-LS On | 00000000:00:09.0 Off | 0 | | N/A 37C P0 38W / 250W | 14216MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 6067 C ...rs/cuda_v12_avx/ollama_llama_server 14448MiB | | 1 N/A N/A 6067 C ...rs/cuda_v12_avx/ollama_llama_server 13800MiB | | 2 N/A N/A 6067 C ...rs/cuda_v12_avx/ollama_llama_server 14212MiB | +-----------------------------------------------------------------------------------------+ 10. 安装 Open WebUI
环境安装：
Open WebUI 要求使用 Python 3.11。使用以下命令创建一个新的环境：
conda create --name open-webui python=3.11进入环境：
conda activate open-webui 使用 pip 安装 Open WebUI：
pip install open-webui 启动服务：
RAG_EMBEDDING_MODEL="" ENABLE_OPENAI_API="false" CORS_ALLOW_ORIGIN="*" open-webui serve --host 0.0.0.0 --port 5000 RAG_EMBEDDING_MODEL不加载默认嵌入的模型。ENABLE_OPENAI_API禁止请求 OpenAI。CORS_ALLOW_ORIGIN开启跨域请求。
上传文件配置：
修改内容如下：
上传后，文件一直转圈，如下图。后台查看 GPU 监控和 Ollama 进程都是正常的。等待一会儿后，可以继续提交内容。应该是模型在进行推理。
