Trying it out.
sudo bash ./launch.sh
╔═══════════════════════════════════════════════════════════════════════════════╗
║ Launching HAProxy-vLLM Installation with Custom Configuration ║
╚═══════════════════════════════════════════════════════════════════════════════╝
Configuration:
STACK_DIR: /opt/llm-stack
MODEL_ID: eousphoros/persona_epsilon_20b_mxfp4
GPU_TYPE: auto
VLLM1_GPUS: 0,1 (backend 1)
VLLM2_GPUS: 2,3 (backend 2)
LB_PORT: 8000
ENABLE_TLS: 0
[2026-01-11T16:25:45-08:00] WARNING: This script was designed for Ubuntu 24.04. Continuing anyway.
[2026-01-11T16:25:45-08:00] Creating stack directories under /opt/llm-stack ...
[2026-01-11T16:25:46-08:00] Installing baseline packages ...
Hit:1 http://us.archive.ubuntu.com/ubuntu questing InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu questing-updates InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu questing-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu questing-security InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ca-certificates is already the newest version (20250419).
ca-certificates set to manually installed.
curl is already the newest version (8.14.1-2ubuntu1).
git is already the newest version (1:2.51.0-1ubuntu1).
jq is already the newest version (1.8.1-3ubuntu1).
jq set to manually installed.
vim is already the newest version (2:9.1.0967-1ubuntu6).
htop is already the newest version (3.4.1-5build1).
htop set to manually installed.
python3 is already the newest version (3.13.7-1).
python3-venv is already the newest version (3.13.7-1).
python3-venv set to manually installed.
python3-pip is already the newest version (25.1.1+dfsg-1ubuntu2).
build-essential is already the newest version (12.12ubuntu1).
pciutils is already the newest version (1:3.14.0-1build1).
pciutils set to manually installed.
gnupg is already the newest version (2.4.8-2ubuntu2.1).
gnupg set to manually installed.
linux-headers-6.14.0-36-generic is already the newest version (6.14.0-36.36).
linux-headers-6.14.0-36-generic set to manually installed.
Solving dependencies... Done
The following additional packages will be installed:
libevent-core-2.1-7t64
The following NEW packages will be installed:
libevent-core-2.1-7t64 pkg-config tmux
0 upgraded, 3 newly installed, 0 to remove and 8 not upgraded.
Need to get 613 kB of archives.
After this operation, 1,587 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu questing/main amd64 libevent-core-2.1-7t64 amd64 2.1.12-stable-10build1 [98.8 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu questing/main amd64 pkg-config amd64 1.8.1-4build1 [7,494 B]
Get:3 http://us.archive.ubuntu.com/ubuntu questing/main amd64 tmux amd64 3.5a-3build1 [507 kB]
Fetched 613 kB in 1s (1,050 kB/s)
Selecting previously unselected package libevent-core-2.1-7t64:amd64.
(Reading database ... 638546 files and directories currently installed.)
Preparing to unpack .../libevent-core-2.1-7t64_2.1.12-stable-10build1_amd64.deb ...
Unpacking libevent-core-2.1-7t64:amd64 (2.1.12-stable-10build1) ...
Selecting previously unselected package pkg-config:amd64.
Preparing to unpack .../pkg-config_1.8.1-4build1_amd64.deb ...
Unpacking pkg-config:amd64 (1.8.1-4build1) ...
Selecting previously unselected package tmux.
Preparing to unpack .../tmux_3.5a-3build1_amd64.deb ...
Unpacking tmux (3.5a-3build1) ...
Setting up pkg-config:amd64 (1.8.1-4build1) ...
Setting up libevent-core-2.1-7t64:amd64 (2.1.12-stable-10build1) ...
Setting up tmux (3.5a-3build1) ...
Processing triggers for man-db (2.13.1-1) ...
Processing triggers for debianutils (5.23.2) ...
Processing triggers for libc-bin (2.42-0ubuntu3) ...
[2026-01-11T16:25:53-08:00] rocm-smi found; validating GPU setup ...
[2026-01-11T16:25:53-08:00] Detected: 6 AMD GPU(s), ROCm 6.4.3, AMD GPU (ISA: unknown)
[2026-01-11T16:25:53-08:00] WARNING: Expected 4 GPUs but found 6.
[2026-01-11T16:25:53-08:00] You may need to adjust VLLM1_GPUS and VLLM2_GPUS.
[2026-01-11T16:25:53-08:00] Configuring AMD GPU power profiles for compute ...
============================ ROCm System Management Interface ============================
=================================== Set Power Profile ====================================
GPU[0] : get_power_profile, Unexpected data received
==========================================================================================
GPU[1] : get_power_profile, Unexpected data received
==========================================================================================
GPU[2] : get_power_profile, Unexpected data received
==========================================================================================
GPU[3] : get_power_profile, Unexpected data received
==========================================================================================
GPU[4] : get_power_profile, Unexpected data received
==========================================================================================
GPU[5] : get_power_profile, Unexpected data received
==========================================================================================
================================== End of ROCm SMI Log ===================================
============================ ROCm System Management Interface ============================
================================= Set Performance Level ==================================
GPU[0] : Performance level set to high
GPU[1] : Performance level set to high
GPU[2] : Performance level set to high
GPU[3] : Performance level set to high
GPU[4] : Performance level set to high
GPU[5] : Performance level set to high
==========================================================================================
================================== End of ROCm SMI Log ===================================
Created symlink '/etc/systemd/system/multi-user.target.wants/rocm-compute-profile.service' → '/etc/systemd/system/rocm-compute-profile.service'.
[2026-01-11T16:25:56-08:00] Setting CPU governor to performance mode ...
Setting cpu: 0
Setting cpu: 1
Setting cpu: 2
Setting cpu: 3
Setting cpu: 4
Setting cpu: 5
Setting cpu: 6
Setting cpu: 7
Setting cpu: 8
Setting cpu: 9
Setting cpu: 10
Setting cpu: 11
Setting cpu: 12
Setting cpu: 13
Setting cpu: 14
Setting cpu: 15
Setting cpu: 16
Setting cpu: 17
Setting cpu: 18
Setting cpu: 19
Setting cpu: 20
Setting cpu: 21
Setting cpu: 22
Setting cpu: 23
Setting cpu: 24
Setting cpu: 25
Setting cpu: 26
Setting cpu: 27
Setting cpu: 28
Setting cpu: 29
Setting cpu: 30
Setting cpu: 31
Created symlink '/etc/systemd/system/multi-user.target.wants/cpu-performance.service' → '/etc/systemd/system/cpu-performance.service'.
[2026-01-11T16:25:57-08:00] Docker already installed; skipping.
Failed to enable unit: Unit docker.service does not exist