Skip to content

FAQ

General

How do I get started as a provider?

  1. Check Software and hardware requirements.
  2. Sign up and create a flock to copy a provider API key.
  3. Install the client. The setup wizard installs the LLM engine and an approved model — you do not need the engine already running.

What is the Flockadoo Provider Client?

A lightweight service that runs next to a local LLM engine (vLLM, vLLM Metal, or llama.cpp). It monitors GPU resources, reports health to Flockadoo, and handles inference requests by forwarding them to that engine.

What platforms are supported?

  • Linux with NVIDIA GPU: At least 32 GB of system memory and 24 GB of VRAM. vLLM. The health report includes utilization. See Setup: Linux.
  • macOS with Apple Silicon: At least 48 GB of unified memory for approved vLLM Metal models. The wizard also offers llama.cpp. Intel Macs are not supported. Basic GPU info (model, core count). Utilization is not reported. A request in flight still marks the client busy. See Setup: macOS.
  • Windows amd64: 32 GB of RAM plus a fixed 64 GB page file (or 64 GB of RAM), and an NVIDIA GPU with 24 GB of VRAM. llama.cpp only. It only lists catalog rows tagged llama_cpp. Encoder and embedding models such as baai/bge-m3 stay off Windows. GPU monitoring via nvidia-smi when the NVIDIA driver is installed. See Setup: Windows.

How do I configure the client?

Prefer flockadoo setup. It writes engine, model, and API key into the YAML config. Server and logging stay in that file — see Configuration. The connection to Flockadoo is opened at startup — it is not a config section.

How do I update the client?

Stop the client, run flockadoo update, then start it again. The console tells you when a newer release exists. If it says Update required, Flockadoo is no longer accepting that release. See Updating the client.

Can I earn while capability checks run?

Yes, from ordinary inference requests, if consumers send them. The checks decide whether you can also receive requests that need tools, JSON mode, or logprobs. Being online does not by itself guarantee traffic.

Why does a capability stay unverified?

The check did not finish. A timeout, an engine error, or a cancelled check is unverified, not a "no". The client retries while the GPU is idle: after 1 minute, 5 minutes, 30 minutes, then hourly.

Why do capability checks run again?

They run again when the deployment changes: model, engine, engine version, weights, context length, GPU, node count, tool parser, or engine arguments. A late result from the previous deployment is dropped.

How do I change engine or model later?

Stop the engine if it is already running, then re-run the setup wizard:

flockadoo setup

Auto-start only replays config.yaml. It does not notice that the catalog row changed, and it will not restart a process that is already listening.

Can I start vLLM (or llama.cpp) myself?

Yes. Use the same flags the wizard printed. If the catalog row has no tool_call_parser, do not pass --enable-auto-tool-choice (vLLM) or --jinja (llama.cpp). If the catalog sets max_model_len, serve at least that context or verification fails. Memory and batching flags, such as --gpu-memory-utilization and --max-num-seqs, are fine as long as the live context stays at least the catalog max_model_len. Safer: let setup start the engine. See Which flags change verification.

Setup says this machine's GPU differs from the master

The worker GPU is not the same model as the master's. Setup prints the two names and continues. Flockadoo tests flocks where every machine has the same GPU. Mixing can work, and it can also leave vLLM stuck during startup. Use the same GPU when you can. See GPUs across nodes.

Setup wants to reinstall vLLM

There are two reasons.

  • The release is older than 0.30.0. Setup prints Flockadoo needs vLLM 0.30.0 or newer. Say yes and setup installs the current release. Say no and setup leaves the old release in place, turns auto_start off, and the client does not serve through it. See Supported engine versions.
  • The flock pins another release. The flock owner set a vLLM version on Nodes, and this machine has a different release. Pipeline parallel needs the same vLLM on every machine. Say yes and setup removes ~/.venv-vllm and installs the pinned release. Say no and setup leaves the old release in place. The flock may not start.

A reinstall takes several minutes. A worker install does not ask. --yes also accepts the reinstall.

"older than Flockadoo supports (vLLM 0.30.0 or newer)"

The vLLM or vLLM Metal that is running is older than 0.30.0. Until it is upgraded, the client offers no models to Flockadoo and refuses requests that reach it, so you earn nothing from this machine. Stop the engine and run flockadoo setup; it installs the current release. vLLM before 0.30 handles the long-context (YaRN) settings differently and would serve some models past their limit with wrong output.

vLLM crashes at startup with "Could not find nvcc"

vLLM tried to compile the FlashInfer sampling kernel on first start. The machine has a GPU driver and no CUDA compiler (nvcc). Rented GPU boxes often look like this.

download-kernels installs other FlashInfer kernels. It does not install this sampler. Setup still runs it on a new install. If vLLM was already installed, run it once yourself:

~/.venv-vllm/bin/python -m flashinfer download-kernels

On a machine with no nvcc, put this in front of the command setup printed. Set it on every machine in the flock:

VLLM_USE_FLASHINFER_SAMPLER=0

For example, VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve .... Sampling is a little slower.

A machine with the CUDA toolkit installed can compile the sampler instead, so skip the variable there. nvcc --version has to succeed for the user that starts vLLM. The first start takes longer. Later starts reuse the compiled sampler. On a GPU with compute capability 10.0 or higher, install CUDA 12.9 or newer. Check the toolkit with nvcc --version. The CUDA version printed by nvidia-smi is the driver.

On a Team Flock, one crashed machine leaves the others waiting. The master often logs No available shared memory broadcast block found in 60 seconds while a worker is still compiling, or stuck waiting for that machine. That line is the wait. The worker crash is the failure. Stop vLLM on every machine, then start the master, then the workers. See Team Flocks.

Tailscale gives every machine an address from one fixed range, 100.64.0.0/10. That range is built into the program. If setup used Tailscale for the flock network, it would collide with a Tailscale network you already run on the same machine: the same addresses, the same network device, the same routes.

So Flockadoo ships winglink, its own version of Tailscale. It uses a different range, 198.18.0.0/15, and its own network device, winglink0. Your Tailscale keeps working and the two stay out of each other's way. For example, a machine can stay on your own tailnet at 100.x.x.x and join flockadoo-flock1 at 198.18.x.x.

You do not install it yourself. Setup downloads and starts it. See How the machines connect.

Setup asks for my password (winglink0)

Team Flock setup joins a private network. It prints winglink needs root to create winglink0 and runs sudo. Enter your password. If the machine has no sudo, setup stops with winglink needs root to create winglink0, and sudo is not installed. This step runs on Linux only. See How the machines connect.

"one-shot key was rejected" or "one-shot key was already used"

This machine is not on the private network, and the one-shot network key is gone.

On the master, run flockadoo setup again. Setup requests a new key.

On a worker, the join code was spent. Ask the owner for a New code. Then run:

flockadoo setup --api-key "join-…"

flockadoo setup alone prints This machine is already in the flock. and will not ask for the new code.

Can I move a machine to another flock?

Yes. Run flockadoo setup --api-key "…" with the new flock's join code or provider API key. Setup logs the machine out of the old private network before it joins the new one. You do not need to clean anything up by hand.

If the machine still shows the old address after setup, log out yourself and run setup again:

sudo /usr/local/libexec/flockadoo/winglink logout
flockadoo setup --api-key "…"

"the owner removed this machine from the flock"

The owner clicked Remove on Nodes. Run flockadoo setup and paste a new join code. See Joining a flock.

How do I delete a flock from my account?

Use the dashboard: My Flocks → select your flock → Settings → Delete flock (Danger zone). That stops inference and revokes API keys; it does not uninstall the client on your machine. See Deleting and managing flocks.

Model management

When I add new models, does the client pick them up?

Yes. New models are detected on each health check (every 3 minutes). The client advertises at most 5 approved models per flock. Marketplace routing still requires an approved model build. Re-run setup to switch the engine to a different approved model.

How does model pricing work?

Pricing is per flock, not account-wide. A new model starts at the market-average price. You can change it once the model shows Verified, whether or not it shows Serving. A model that shows Not verified keeps the market-average price. For example, flockadoo-flock1 and flockadoo-flock2 can charge different prices for the same model name.

The client advertises at most 5 approved models. Each model has a maximum price.

See Model pricing for editing prices in the dashboard.

Can I change model pricing after registration?

Yes, once the model shows Verified. Open My Flocks → select a flock → Models tab. You can also call PATCH /api/provider/models/{model_id}/pricing on https://core.flockadoo.ai with that flock’s provider API key. Prices are $/1M with up to 2 decimals. See Model pricing.

Changing prices on one flock does not change another. See Model pricing.

Which models are approved on the marketplace?

See Approved model builds. Call GET /api/models/approved-builds for the public catalog (aliases, HuggingFace locations, and engines; no hashes). The setup wizard lists rows tagged for this engine that should also fit this machine. On Windows, models without llama_cpp in engines are hidden. The dashboard says No Windows build yet.

Reaching your machine

Do I set a public URL?

No. On startup the client opens an outbound connection to Flockadoo. Flockadoo sends inference requests back over that connection. You do not install another program or open inbound ports.

What if the connection drops?

The client reconnects on its own. Check the client logs if it stays disconnected.

Can other providers SSH into my machine?

No, not over the private network a Team Flock uses. Port 22 is blocked there. That does not mean the other node is safe. Only share a flock with people you intend to share GPUs with. See Team Flock networking.

What can other nodes in a Team Flock reach?

vLLM on 8000, llama.cpp on 8081, the distributed setup port 29501, ping, and TCP 32768–60999 for GPU-to-GPU traffic. SSH, typical web admin, and database ports stay closed. The high-port window is wide on purpose. A service you bind there on all interfaces is reachable by every node in the flock. See Team Flock networking.

The client failed to start because of the API key

The provider API key in config.yaml (api_key) must be the key for that flock, copied from My Flocks → select the flock → Settings. A consumer key, an old rotated key, or a typo is rejected at startup with an “Invalid provider API key” message.

For example, after you regenerate the key in Settings, update api_key (or re-run setup) and restart the client. Do not leave the placeholder your_api_key_here.

Health and monitoring

How often does the client report health?

Every 3 minutes, and also when the client becomes busy or free. It exposes GET /api/health for on-demand status.

What is in a health report?

  • GPU info (type, driver, CUDA, memory, utilization where available).
  • Available models from the local LLM server.
  • Whether the client is connected to Flockadoo.

Flock country is not part of the health payload. Flockadoo resolves it from your connection. See Flock location.

How is “GPU busy” determined?

There are two signals. See How it works.

  • Rejecting a new request (GET /api/busy and 503): every inference slot is full. The default is one request at a time.
  • Health report GPU is_busy: on Linux and Windows with NVIDIA, utilization above 20%, or a slot is full. On macOS, a slot is full.
  • No GPU monitoring and nothing in flight: the health report does not mark the GPU busy from utilization.

Inference requests

How are inference requests handled?

  1. The request signature in X-Request-Id is validated with Flockadoo; missing or invalid requests get 401.
  2. If every inference slot is full → 503 {"error":"GPU is busy"} (Flockadoo can try another provider). Same-session follow-ups can wait for the slot instead. The 20% utilization flag is on the health report.
  3. Valid requests are proxied to the local LLM server.

What endpoints does the client expose for inference?

OpenAI-compatible:

  • POST /v1/chat/completions
  • POST /v1/completions
  • POST /v1/embeddings

Logging

How do I configure logging?

In config.yaml, under logging: set level (debug, info, warn, error), log_dir, max_size, max_backups, max_age. See Configuration. Engine stdout from auto-start goes to engine.log.

What if GPU monitoring isn’t available?

The client still runs. Health reports omit GPU utilization, so that flag stays clear unless a request is in flight. You can monitor activity via the console UI and log files.