Running llama-server with AMD ROCm hardware acceleration as a systemd service requires bypassing default sandboxing and namespace isolation that hide GPU sysfs trees and character devices from non-interactive system runs.

Step 1: Index Shared Libraries

Ensure custom library paths are globally indexed by the system linker so binaries find their dependencies:

echo "/usr/local/bin/llama" > /etc/ld.so.conf.d/llama.conf
ldconfig

Step 2: Create the environment file

Create /etc/environment:

TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1
FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE
LD_LIBRARY_PATH="/usr/local/bin/llama:/usr/local/lib:/usr/lib/x86_64-linux-gnu"
ROCR_VISIBLE_DEVICES="0"
HIP_VISIBLE_DEVICES="0"
HSA_OVERRIDE_GFX_VERSION="11.0.2"

Step 3: Create the Systemd Service File

Create /etc/systemd/system/llama.service with complete unmasked device permissions and environment variables:

[Unit]
Description=Llama.cpp Inference Server Engine
After=network.target network-online.target remote-fs.target
Requires=network-online.target

[Service]
Type=simple
PrivateDevices=false
ProtectSystem=false
ProtectHome=false
PrivateTmp=false
NoNewPrivileges=false
ProtectKernelTunables=false
ProtectControlGroups=false
RestrictNamespaces=false
MemoryDenyWriteExecute=false
ProtectKernelModules=false
ProtectHostname=false
ProtectClock=false
ProtectKernelLogs=false
RestrictAddressFamilies=
SystemCallFilter=
DevicePolicy=auto
LimitMEMLOCK=infinity
User=root
WorkingDirectory=/usr/local/bin/llama
EnvironmentFile=/etc/environment
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Environment=LD_LIBRARY_PATH=/usr/local/bin/llama:/usr/lib/x86_64-linux-gnu
Environment=HSA_OVERRIDE_GFX_VERSION=gfx1102
Environment=ROCR_VISIBLE_DEVICES=0
Environment=HIP_VISIBLE_DEVICES=0
Environment=CUDA_VISIBLE_DEVICES=0
ExecStartPre=/bin/mount -a
ExecStartPre=/bin/test -d /models/active
ExecStart=/bin/bash -c "source /train.venv/bin/activate && LLAMA_LOG_LEVEL=debug && /usr/local/bin/llama/llama-server --models-preset /models/models.ini --path /usr/local/bin/llama/html --port 80 --host 0.0.0.0 --verbose"
Restart=always
RestartSec=5
StandardOutput=append:/var/log/llama/access.log
StandardError=append:/var/log/llama/error.log

[Install]
WantedBy=multi-user.target

Step 4: Enable and Start the Service

Reload systemd, enable the service to start on boot, and launch it:

systemctl daemon-reload
systemctl enable llama.service
systemctl start llama.service