Automatic model selection¶
You can leave the model choice to Flockadoo. Send your price and speed limits, and Flockadoo picks the cheapest approved model that can serve the request.
How to ask¶
Omit model, or set it to auto. For example:
{
"messages": [
{
"role": "user",
"content": "Write a short product summary."
}
],
"flockadoo": {
"sort": "cost",
"min_tokens_per_second": 20,
"max_input_price_per_1m": 0.5,
"max_output_price_per_1m": 1.0
}
}
The reply includes a model field. That is the model that served the request.
sort defaults to cost, which prefers a lower price. throughput prefers faster providers. See Request routing options for the other fields.
What counts as able to serve it¶
Flockadoo reads the body:
| Body | Models considered |
|---|---|
messages or prompt |
Chat models |
input |
Embedding models |
tools or tool_choice (other than none) |
Chat models that support tool calling |
Only approved models are eligible. A model a provider loaded that is not on the approved list is skipped in this mode. Name that model yourself if you want it.
Your max price (profile, key, and the request) and min_tokens_per_second still apply. Max price doesn't apply to your own flocks.
If nothing qualifies, you get 404.