How it works
AI inference on the machines your company already owns.
kibbu is private AI infrastructure that runs language models on the laptops, workstations and servers your company already owns. Your apps send requests to one OpenAI-compatible endpoint, and kibbu runs each one on an idle, plugged-in machine in your fleet, using a cloud provider only when your policy allows it and you have added that provider's key.
A small agent on each enrolled machine runs the models. A control plane keeps track of which machines are free and sends each request to one of them. Work only runs while a machine is idle and plugged in, and stops the moment its owner comes back.
The path of a request
From your app to an idle machine and back.
Four steps, the same in every deployment. What differs is who runs the control plane.
- 01Your app sends a requestAny OpenAI client, pointed at the kibbu endpoint with a kibbu key and a model ID.
- 02The control plane checks the key's policyIt decides which models, machines and countries the key may use, and whether cloud is allowed.
- 03An eligible machine runs itkibbu picks an idle, plugged-in machine that has the model and room to run it. The request travels over an encrypted mesh.
- 04The answer streams backThe same way it came. The request is logged as metadata: model, tokens, timings and which machine served it.
Hosted, kibbu runs the control plane, so requests pass through it on the way to your machines. With kibbu On-Prem you run it yourself. The security page sets out what each one means for your data.
Enrollment
You choose which machines join.
Minutes, not weeks, for a first machine. kibbu has not published a measured setup time, because model download time depends on the model and your network.
An admin creates an install code in the console, then installs the kibbu agent with it, by hand or through your MDM. kibbu ships templates for Microsoft Intune, Jamf and Group Policy, and a Windows .msi. Machines join only with your install code, and an admin decides which ones serve.
- The agent runs in the background with no setup for the person using the machine.
- A tray app lets that person pause sharing for 1 hour, 8 hours, 7 days or indefinitely.
- Updates install silently in the background, so the fleet stays current.
- An admin can remove a machine from the console at any time.
Scheduling
Only idle machines take work.
The person who uses the machine always comes first. kibbu fits AI work into the time it would otherwise sit idle.
A machine takes a request only when
- Nobody has touched it for about two minutes (in the default idle-only mode).
- It is on mains power, and the battery is not low or draining.
- It is not thermally throttled, and CPU, GPU and memory have room.
- It has the model, or can load it, and nobody has paused it.
When the fleet is busy
When every eligible machine is busy, a request waits in a queue for up to 60 seconds by default before kibbu returns fleet_busy (HTTP 503 with Retry-After), or uses cloud if your policy allows it.
When cloud is involved
kibbu does not run a cloud of its own. A request can only reach a cloud provider if your organization has added that provider's API key, the model allows cloud, and the key's policy does not deny it. With cloud denied, a request that no machine can serve returns an error (HTTP 503 with a Retry-After header) rather than leaving your fleet.
Interruptions
What happens when someone sits down.
Machines come and go all day. Your client should be ready to retry, as it would with any API.
When someone touches the keyboard or mouse, the work on that machine stops and the machine goes back to its owner. If no output has been sent yet, kibbu retries the request on another machine. If output was already streaming, the client receives a kibbu_failover_required error marked retry: true and sends the request again. kibbu does not move a running request between machines.
| Moment | What kibbu does | What your client sees |
|---|---|---|
| Before output starts | Retries on another eligible machine. | A normal response, a little later. |
| After output started | Stops the work on that machine. | A kibbu_failover_required error with retry: true. Send the request again. |
| No machine is free | Queues the request for up to 60 seconds. | A response, or fleet_busy (503) with Retry-After. |
Hardware
Macs, Windows PCs, Linux machines and servers.
A machine with a GPU or Apple Silicon is worth many CPU-only ones.
| Platform | What serves inference |
|---|---|
| macOS | Apple Silicon (M-series). |
| Windows | x64 with an NVIDIA, AMD or Intel GPU. |
| Linux | x64 with an NVIDIA GPU, on Ubuntu 24.04 or newer. |
| Servers | Linux servers and Kubernetes nodes, grouped as Servers so policies can target them. |
There is no single memory minimum: each model has its own floor, from 3 GB for small models to 20 GB and more for larger ones. More memory matters more than a faster processor. Throughput depends on the machine and the model, and kibbu has not published benchmarks.
Models
Models come from kibbu's curated catalog of open models. Hosted, machines download them from kibbu's model mirror and check each file's SHA-256 hash. With kibbu On-Prem, your control plane syncs them from kibbu's registry or imports them from an offline bundle, and machines download from your control plane.
For developers
An OpenAI-compatible API.
Base URL https://api.kibbu.io/v1. Authenticate with Authorization: Bearer kibbu_….
Three things change in an existing OpenAI client: the base URL, the API key (a kibbu key starting with kibbu_), and the model, which is an ID from GET /v1/models. Compatible means the same request and response shapes. A different model gives different output, so validate each workflow against its own acceptance criteria before you move it.
| Status | Endpoints |
|---|---|
| Supported | GET /v1/models, POST /v1/chat/completions (streaming, tools and structured output on models that support them, image input), POST /v1/completions, POST /v1/responses. |
| Available, not yet validated | POST /v1/embeddings (your fleet only), /v1/batches, /v1/files, /v1/prompts, /v1/receipts. |
| Not served | Images, audio, realtime, fine-tuning, assistants, threads and vector stores return 404 unsupported_endpoint. |
TypeScript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.kibbu.io/v1", // changed
apiKey: process.env.KIBBU_API_KEY, // changed: a kibbu_ key
});
const r = await client.chat.completions.create({
model: process.env.KIBBU_MODEL!, // changed: an ID from GET /v1/models
messages: [{ role: "user", content: "Summarize this contract clause." }],
});Python
from openai import OpenAI
import os
client = OpenAI(
base_url="https://api.kibbu.io/v1", # changed
api_key=os.environ["KIBBU_API_KEY"], # changed: a kibbu_ key
)
r = client.chat.completions.create(
model=os.environ["KIBBU_MODEL"], # changed: an ID from GET /v1/models
messages=[{"role": "user", "content": "Summarize this contract clause."}],
)curl
# The models your organization can call
curl https://api.kibbu.io/v1/models \
-H "Authorization: Bearer $KIBBU_API_KEY"
curl https://api.kibbu.io/v1/chat/completions \
-H "Authorization: Bearer $KIBBU_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "'"$KIBBU_MODEL"'", "messages": [{"role": "user", "content": "Hello"}]}'Tools built on the OpenAI API, such as n8n, Dify, Langflow or a Databricks notebook, connect the same way: set the base URL, key and model in their OpenAI settings. See workloads for what to test first.
Keep reading
Start with one workflow.
Tell us about the work you want to keep private, and we will scope a pilot on your own machines.