AI nodes and the copilot

OrderLoop uses Workers AI in three places: the AI nodes in a flow, the copilot in the flow editor, and the chat on each order page. All three go through one function, runJson (apps/engine/src/ai.ts), with the same guarantees:

• JSON mode with a schema. Every call asks for response_format: json_schema, so the result has a known shape (temperature 0.2, at most 2048 tokens).
• A 20-second cap. A slower call is abandoned.
• A deterministic fallback. When AI_MODE is mock, or the model errors, times out or returns nothing, a heuristic produces the result instead. An order never stalls on the model.
• The source is recorded. Each AI node’s output carries source: "model" or source: "heuristic"; the Extract order and Demand forecast summaries show it too.

Configuration

Set in apps/engine/wrangler.jsonc under vars (see Configuration):

Variable	Default	Effect
AI_MODE	live	mock skips the model and always uses the heuristics (tests, offline dev)
AI_MODEL	@cf/meta/llama-3.3-70b-instruct-fp8-fast	The text model for every call
AI_GATEWAY_ID	""	When set, calls go through that AI Gateway (caching, rate limits, logs)

Workers AI always calls your Cloudflare account, even in local dev. The engine’s tests run with AI_MODE=mock, so they need no network.

AI nodes

Each node’s output is in nodes.<id> for later expressions, along with source.

Node	Model task	Fallback	Result
Extract order	Read documentText (or the R2 document at documentKey) and map items to catalog SKUs	Regexes for 120 x WID-100-style lines and SKU … qty N, PO #…, From: …	When it finds known SKUs, replaces the order’s lines with them; sets customerId by name match when the order had none; titles the order PO <number>, sets vars.poNumber. With no document it passes the order through
Classify	Pick one of labels with a confidence	Keyword match on the title and lines; the first label otherwise	Sets vars.<into> (default category). A label outside the list becomes the first label
Risk score	Score fraud and credit risk 0–100 with up to three reasons	10, +30 with no customer, +30 over 20,000, +20 for a line over 1,000 units	Takes high at or above threshold (default 70), else low; sets vars.risk
Demand forecast	Suggest quantities for SKUs at or below their reorder point, over horizonDays	Each low SKU’s reorderQty	Replaces the order’s lines; sets vars.supplier from the first SKU. With nothing low it reports “Stock is healthy” without calling the model
Summarize	Two sentences on the order and its progress	<type> <id>: N lines, total …	Sets vars.<into> (default summary)
Agent router	Follow instructions and choose exactly one of routes, with a one-sentence reason	No lines → ask_customer; total over 10,000 → needs_approval; else the first route (each falling back to a route that exists)	Takes the chosen route as its branch; a route outside the list becomes the first route

The router’s routes are its outputs, so editing routes changes the handles on the canvas. The model sees the order, the customer record, the run variables and the outputs of completed nodes.

The flow copilot

The copilot proposes a whole graph. It is reachable from the flow editor’s prompt bar and from Describe it when creating a flow. The model gets the node catalog (types, outputs, config fields) and the graph rules, and returns a complete graph plus an explanation. The engine then:

1. fills each node’s missing config from its catalog defaults and gives edges ids;
2. keeps existing nodes’ positions, and lays the graph out again only when nodes were added;
3. runs validateGraph and returns the errors and warnings with the graph.

In the editor the result is a proposal: it shows the explanation and how many nodes it adds and removes, and the graph changes only if you accept it. Accepting doesn’t publish.

The offline fallback understands one sentence shape, “add a node label after node id or label” (“fraud” or “risk” mean Risk score), and splices the node into that node’s outgoing edges.

Order chat

Each order page has a chat with an operations agent that answers from the order’s live state: status, totals, waits, variables, recent timeline and each node’s status and summary. It may call tools:

Tool	Does
get_state	Reads the order summary
approve, reject	Prepare a decision for a waiting approval
cancel	Prepares a cancellation with a reason
deliver_event	Delivers a business event to the waiting node

Approvals, rejections and cancellations are only prepared. The reply carries a proposal, and nothing happens until a person confirms it on the order page; a confirmed decision still gets the 5-second undo window. Dismissing it sends nothing, and a proposal for a step that has stopped waiting can’t be confirmed. Delivering an event is not gated this way.

The agent takes up to four model turns per message and keeps the last 100 messages. The fallback reads “approve”, “reject” and “cancel” in the message, and a name.event together with “send”, “deliver”, “mark” or “fire”; otherwise it replies with the status, total and what the order is waiting on.