520 to 512 sector size shenanigans

Getting the NVIDIA driver to build on Proxmox with kernel 7.0

I wanted my RTX 2080 Ti available to LXC containers on my Proxmox host (a Dell R730xd), so I installed the NVIDIA driver on the host. It didn't work the first time. Here's what went wrong and how I fixed it.

Setup: Proxmox kernel 7.0.14-23-pve, NVIDIA driver 615.78.08 (the .run installer), open (MIT/GPL) kernel module.

Error 1: kernel source tree not found

ERROR: Unable to find the kernel source tree for the currently running kernel.

Proxmox doesn't install the kernel headers by default. Install them, plus DKMS so the module gets rebuilt when the kernel updates:

apt update
apt install proxmox-default-headers proxmox-headers-$(uname -r) build-essential dkms

Error 2: the module fails to build

The second attempt failed during compilation:

ERROR: The nvidia kernel module was not created.
ERROR: Installation has failed.

The real error was further up in /var/log/nvidia-installer.log:

grep -n -m 15 -E "error:|fatal error" /var/log/nvidia-installer.log

common/inc/nv-linux.h:1270:34: error: field 'cxlds' has incomplete type

The compiler wasn't the problem: /proc/version showed the kernel was built with the same gcc (14.2.0) I had installed. The cause was in NVIDIA's code. At build time the driver checks whether certain kernel headers exist. Kernel 7.0 includes include/cxl/cxl.h, so the check sets NV_CXL_CXL_H_PRESENT and the driver compiles a struct that embeds struct cxl_dev_state. But that header doesn't define the struct at all:

grep -n "struct cxl_dev_state" /usr/src/linux-headers-$(uname -r)/include/cxl/cxl.h
# (no output)

CXL is a datacenter memory-expansion interface. A 2080 Ti in an R730 has nothing to do with it, so it's safe to skip.

The fix: stop the driver detecting the CXL header

Extract the installer, remove cxl/cxl.h from the header-presence list, and install from the extracted folder:

./NVIDIA-Linux-x86_64-615.78.08.run --extract-only
cd NVIDIA-Linux-x86_64-615.78.08
sed -i 's#cxl/cxl.h##' kernel-open/header-presence-tests.mk kernel/header-presence-tests.mk
./nvidia-installer --dkms

Without the define, all the CXL code is skipped and the module builds. The installer copies the patched source to /usr/src/nvidia-615.78.08/, so DKMS keeps the fix when it rebuilds for later 7.0.x kernels.

Other prompts during the install, all safe to dismiss on a headless host:

• X library path warning: press OK (no desktop on Proxmox).
• 32-bit compatibility libraries: press OK.
• Run nvidia-xconfig: choose No.

Result

root@pve:~# nvidia-smi
| NVIDIA-SMI 615.78.08    KMD Version: 615.78.08    CUDA UMD Version: 13.4 |
|   0  NVIDIA GeForce RTX 2080 Ti   Off | 00000000:05:00.0 Off |       N/A |
| 28%   33C    P8    14W / 260W |    1MiB / 11264MiB |   0%    Default |

root@pve:~# dkms status
nvidia/615.78.08, 7.0.14-23-pve, x86_64: installed

Notes

• This is a stopgap. When NVIDIA releases a driver with kernel 7.0 support, run ./nvidia-installer --uninstall and install the new one normally.
• After a Proxmox kernel update, run dkms status before rebooting to make sure the module rebuilt.
• If you'd rather not patch, the alternative is to pin an older 6.x kernel with proxmox-boot-tool kernel pin.