GPU (Nvidia) passthrough on Proxmox LXC container
Updated 2020-05-01: Added /dev/dri mount point in LXC container.
Recently I needed a GPU inside a LXC container. There are some good tutorials already however I found it difficult. I’m writing this post to serve as documentation for future reference.
- Proxmox 6.1
- Arch Linux Container
- Nvidia Quadro K620
Ensure the system is up to date:
Add the pve-no-subscription repository to get access to
Ensure your graphics card is present:
You should see some output like the following:
Next install the Nvidia drivers. These exists in buster-backports, therefore add the following to
Install the Nvidia driver and SMI package (you must specify
buster-backports when installing):
Ensure you load the correct kernel modules.
/etc/modules-load.d/nvidia.conf should contain the following modules (in my case
nvidia-drm was already present):
/etc/udev/rules.d/70-nvidia.rules and populate with:
These rules are required to:
- Set more permissive permissions
- Enable nvidia_uvm which isn’t started by default (at least for my card it seems)
Reboot the host and check the output of
ls -al /dev/nvidia* and
/dev/dri/* is similar to below:
Take note of the numbers in the fifth column above
226 respectively. These are required later on.
Additionally you can check the nvidia card is recognised and the drivers are working using the
nvidia-smi command which should show something similar:
Next we need to install the Nvidia drivers inside the container. It is important that the exact Nvidia driver versions match between the Proxmox host and the container.
Debian uses an older Nvidia driver
430.64, therefore I had to manually install the Nvidia driver inside my Arch Linux container.
Download the required driver (changing version numbers in the URL does work):
Execute the install without the kernel module:
Now the drivers are installed, we need to passthrough the GPU. Shutdown the container and add the following to your container configuration
/etc/pve/lxc/###.conf making sure to change the numbers we recorded earlier if they differ:
Start the container and confirm the passthrough worked by executing
ls -al /dev/nvidia* and
ls -al /dev/dri/*. Additionally
nvidia-smi should now show you an identical result to the Proxmox host:
You have now successfully setup GPU passthrough for an LXC container.
- For some reason
nvidia-smidoes not return a
CUDA Versionon the Proxmox host
nvidia-smioutputs the name of processes using the GPU however this list is blank when executed from within the container
- This setup is somewhat fragile as any update to a Nvidia driver that causes a version mismatch will break the passthrough. I have seen other posts talk about pinning / ignoring Nvidia driver updates via apt but have not set that up as yet.
Hopefully this will help someone in future. If you haven’t looked into Proxmox before I suggest you do.