BUILT INTO EVERY GOODSPEED APP
LLM Client (OpenAI + Anthropic, Cost-Gated)
A provider-agnostic LLM service exposes chat(), streamChat(), embed(), and transcribe() with OpenAI and Anthropic adapters; every call routes through a cost-budget RPC that can throttle, block, or alert when per-user or per-scope spend limits are exceeded.
- Tier: Common
- Status: Config-toggled
- Config: gasConfig.llm.provider
WHY IT MATTERS
Calling a large language model is one line of code. The part that costs founders money and causes production incidents is everything around that line: choosing a provider and being locked in, forgetting to budget per-user spend until a single script-kiddie drains the account, wiring retry logic differently in four components, and losing visibility into which model and which user generated each dollar of API cost. The template treats the LLM layer as infrastructure, not a one-off import. services/llm.ts is a provider-agnostic facade that resolves the active adapter from gasConfig.llm.provider at runtime. Switching from OpenAI to Anthropic, or from gpt-4o-mini to claude-haiku-3-5, is a single config change that does not touch call sites. Both adapters implement the same LlmAdapter interface, so chat(), streamChat(), embed(), and transcribe() behave identically regardless of which provider is active. The adapter layer handles the provider-specific request shapes, token counting, and rate-limit error codes, surfacing a consistent ServiceError to callers so your UI error handling does not need to branch on provider details.
The cost-gate mechanism is the structural differentiator. Before every chat() call, the service estimates the worst-case cost of the upcoming completion using the adapter's per-million-token rates for that model and the combined character length of the messages. It then calls the consume_cost Supabase RPC, which checks the spend against a per-user, per-scope budget for a configurable period (daily by default). If the budget is exceeded, consume_cost returns allowed: false and the service throws a typed llm_over_budget ServiceError with HTTP status 429 that your UI can catch and surface gracefully. When the completion returns, the service reconciles the actual token usage against the reservation with a signed delta call: the delta is negative if actual cost was less than the estimate (refunding the unused portion), positive if it ran over. This two-phase reserve-then-reconcile pattern means the budget is never artificially exhausted by overestimates, and a user cannot stack concurrent calls that each pre-authorize the full budget. Failed adapter calls refund the full reservation so a transient API error does not permanently consume a user's quota. Every call is tracked via PostHog and Sentry for cost dashboarding and error alerting.
HOW IT IS WIRED
Real code from the GAS template
The code below is drawn from services/llm.ts in the gas-template repository. This is the code your generated app gets, not pseudocode, not a description of intent.
// services/llm.ts -- chat(): cost-gated multi-provider completion
export async function chat(messages: ChatMessage[], opts?: ChatOptions): Promise<ChatCompletion> {
const adapter = resolveAdapter(); // reads gasConfig.llm.provider
const model = opts?.model ?? gasConfig.llm.defaultChatModel ?? DEFAULT_CHAT_MODEL;
const period = gasConfig.llm.budgetPeriod ?? 'day';
const maxTokens = opts?.maxTokens ?? gasConfig.llm.defaultMaxTokens ?? DEFAULT_MAX_TOKENS;
// Phase 1: reserve worst-case cost before the network call
const charCount = messages.reduce((n, m) => n + m.content.length, 0);
const reserve = reserveEstimate(adapter, model, charCount, maxTokens);
const userId = await getUserId();
const allowed = await consumeCost(userId, reserve, period);
if (!allowed) throw new ServiceError('llm_over_budget', 429, 'LLM budget exceeded');
let completion: ChatCompletion;
try {
completion = await retryWithBackoff(
() => adapter.chat(messages, { ...opts, model }),
{ shouldRetry: isTransientNon4xxError },
);
} catch (err) {
// Refund reservation so a transient failure does not burn the user's budget
try { await recordDelta(userId, -reserve, period); } catch { /* best-effort */ }
captureException(err, { service: 'llm', provider: adapter.name, model });
throw err;
}
// Phase 2: reconcile actual tokens used against the estimate
const delta = actualCost(adapter, completion) - reserve;
await recordDelta(userId, delta, period); // negative = refund overestimate
return completion;
}Source: goodspeed-apps/gas-template → services/llm.ts
HONEST LIMITS
When LLM Client (OpenAI + Anthropic, Cost-Gated) is the wrong choice
The cost-gate overhead is designed for interactive, per-user calls where a human is waiting on screen. Each chat() invocation makes two Supabase RPC calls in sequence (reserve, then reconcile) before and after the provider call. For a user sending a one-off message in a chat UI, this adds tens of milliseconds and is invisible. For a background pipeline that processes thousands of documents in parallel, the RPC overhead compounds with concurrency and the consume_cost RPC becomes a bottleneck. The budget ledger is also keyed per user per period, which makes sense for interactive throttling but is meaningless for a server-side batch job running under a service account. If your use case is bulk generation, document ingestion, embedding pipelines, or any job-queue workload where the caller is not an end user, route those calls through a direct adapter invocation with a separate budget-tracking mechanism appropriate to your volume and cost-accounting model. Additionally, streamChat() intentionally skips the cost gate: streaming responses do not have a deterministic token count until the stream closes, so there is no reservation phase. The stream is dispatched directly to the adapter. If you need budget enforcement on streaming calls, you must implement a post-stream cost reconciliation yourself using the adapter's ratesPerMillionTokens() method after counting the streamed tokens. The embed() and transcribe() functions also have no budget gate, only retry logic and error capture. For apps where embedding or transcription costs are material, add explicit spend guards at the call site rather than relying on the service layer to enforce limits automatically.
Tier: Common · Config-toggled
Evaluate your use case
Check whether llm client (openai + anthropic, cost-gated) aligns with your target audience, platform constraints, and regulatory environment before enabling it.
Audit the config
The `gasConfig.llm.provider` flag controls this feature. Set it to false in gas.config.ts to disable the feature entirely with no residual code paths.
Seek alternatives
If the built-in implementation does not fit, the generated codebase is standard React Native + Expo code. Any library in the Expo ecosystem can replace the default.
APPS USING THIS FEATURE
Every generated Goodspeed app includes llm client (openai + anthropic, cost-gated). Browse the ideas catalog to see apps across all categories that ship with this feature wired in.
CAPABILITIES
LLM Client (OpenAI + Anthropic, Cost-Gated) capability breakdown
Concrete dimensions of what the built-in llm client (openai + anthropic, cost-gated) implementation covers. These reflect the actual template code, not a marketing summary.
| Item | Description | Strength |
|---|---|---|
| Provider support | LLM providers available via adapter swap in gasConfig.llm.provider | OpenAI + Anthropic (adapter-extensible) |
| Cost enforcement | Budget mechanism used to throttle or block calls that exceed spend limits | Per-user per-period via consume_cost RPC |
| Streaming | Whether the service supports token-streaming responses | Yes (streamChat, no budget gate) |
| Operations | LLM operations exposed by the service facade | chat, streamChat, embed, transcribe |
| Error handling | Retry and observability strategy for adapter failures | Exponential backoff + Sentry capture + budget refund on failure |
COMMON QUESTIONS
How do I switch from OpenAI to Anthropic without changing my components?
Set gasConfig.llm.provider to 'anthropic' in your gas.config.ts file. The resolveAdapter() function in services/llm.ts reads that value at call time and returns the corresponding adapter from the ADAPTERS map. Every call site that uses chat(), streamChat(), embed(), or transcribe() continues to work without modification because all adapters implement the same LlmAdapter interface. Model defaults also follow config: gasConfig.llm.defaultChatModel sets the fallback model name for the active provider, so you can set it to 'claude-haiku-3-5' when switching to Anthropic without touching individual call sites. The adapter handles the difference in request shape, response normalization, and token counting internally. If you need to run both providers simultaneously in one app, you can bypass the facade and import openaiAdapter or anthropicAdapter directly from lib/llm-adapters/ and call adapter.chat() with an explicit model string.
What happens when a user hits their daily LLM budget?
The consume_cost RPC returns allowed: false, and chat() throws a ServiceError with code 'llm_over_budget' and HTTP status 429. The error is typed, so you can catch it at the call site with a simple instanceof ServiceError check and inspect the code property. The recommended UI pattern is to catch this specific error in your component and show a contextual message explaining that the user has reached their daily limit rather than a generic error state. The budget period and per-user limit are configured via gasConfig.llm.budgetPeriod and the consume_cost RPC's threshold parameter in your Supabase migration. Default period is 'day', which resets at midnight UTC. You can set a higher threshold for premium-tier users by parameterizing the RPC call with a scope that encodes their plan, for example 'llm:pro' vs 'llm:free', and configuring different thresholds per scope in the cost_budgets table that backs the consume_cost function.
Does the cost gate work on streaming calls?
No. streamChat() dispatches directly to the adapter without a budget reservation or reconciliation step. The reason is architectural: streaming responses do not have a final token count until the stream closes, so there is no upfront estimate to reserve against, and holding open a reservation for the duration of a long stream would block other concurrent calls from the same user from running. If your app uses streaming extensively and you need to enforce per-user spend limits on those calls, the correct approach is to count the tokens you receive from the stream yourself, then call the consume_cost RPC once after the stream closes with the actual cost. Alternatively, cap stream length with a maxTokens option passed to streamChat(), which is forwarded to the adapter and enforced at the provider level, giving you a predictable worst-case cost ceiling per call even without a pre-flight budget check.
How are LLM call costs tracked for debugging and billing analytics?
Every successful chat() call emits a PostHog event via captureEvent(EVENTS.llm_call) with fields for provider, model, tokens_in, tokens_out, cost (actual USD), and status. Failed calls emit the same event with status 'failed' and an error string. This means your PostHog dashboard has a full ledger of LLM activity broken down by user, model, and outcome. Sentry receives the exception context on failed calls via captureException with a service: 'llm' tag. The consume_cost RPC writes to a cost_events table in Supabase, which you can query directly for billing reconciliation or to audit a specific user's usage history. The two-phase reserve-then-reconcile pattern means the cost_events table reflects actual token usage rather than worst-case estimates: the delta call corrects each reservation to the real cost within the same budget period. For apps where LLM costs are a significant share of operating expenses, joining cost_events against your user table in the Supabase dashboard gives you a per-user cost breakdown without any additional instrumentation.
GET IT BUILT INTO YOUR APP