Setup: Linux (NVIDIA)¶
Use this guide when you run the provider client natively on Linux with an NVIDIA GPU. For example, flockadoo-flock1 on a workstation. You need at least 32 GB of system memory and 24 GB of VRAM for approved vLLM models.
Quick install (recommended)¶
- Get your provider API key — see Sign up and create a flock.
- Run the install script:
The setup wizard installs vLLM if it is missing, offers to replace a release older than 0.30.0 (see Supported engine versions), and shows which models fit this GPU. Re-run anytime:
- Start the client:
Default config: ~/.config/flockadoo/config.yaml. Logs: ~/.local/state/flockadoo/log.
NVIDIA GPU¶
- At least 32 GB of system memory.
- At least 24 GB of VRAM (for example RTX 3090 or RTX 4090).
- Install the NVIDIA driver so
nvidia-smiis onPATH. The wizard will not offer vLLM ifnvidia-smiis missing. - The client uses
nvidia-smifor GPU name, VRAM, and utilization in the health report. Utilization above 20% sets the GPU busy flag in that report. A new request is rejected when an inference slot is full. See How it works. - The install script does not install the driver.
Blackwell GPUs¶
vLLM on a GPU with compute capability 10.0 or higher also needs FlashInfer kernels. That includes datacenter Blackwell and RTX 50-series. Hopper (9.0), Ada, and Ampere (for example an RTX 3090 or RTX 4090) skip this step.
When the wizard installs vLLM, it reads compute capability from nvidia-smi. At 10.0 or higher it prints Blackwell GPU detected. Downloading FlashInfer kernels... and downloads the kernels before setup continues. If that download fails, setup stops with vLLM kernel download failed.
If nvidia-smi cannot report compute capability, setup prints Skipping vLLM kernel download and continues. On a Blackwell GPU, vLLM can then fail to start. Fix nvidia-smi, then run flockadoo setup again.
Check the value yourself:
Manual install from source¶
- Go: Install Go 1.22 or higher.
- Clone and build:
- Run
flockadoo setupso it writes config and can install the engine. Do not copyconfig.yaml.exampleby hand unless you are skipping the wizard on purpose. - Run:
The client opens an outbound connection to Flockadoo. No need to install ngrok or open firewall ports.
Team Flocks¶
Linux is the only platform for a Team Flock. Setup downloads winglink, may ask for your password (winglink needs root to create winglink0), and prints Winglink address when this machine is on the private network. It does not start vLLM. See Team Flocks.