Guides / culpa vs raindrop

Culpa vs Raindrop

Raindrop is agent observability that traces production runs, surfaces silent failures like loops and broken tools, and prices itself per event, where an event is one user message and the agent's reply. Culpa, a local-first LLM cost, margin, and forecast ledger, traces the money instead, and bills per tracked call rather than per conversation turn.

Why this happens

Both products watch agent runs, and they ask different questions of the same traffic. Raindrop asks whether the run was correct, so it looks for hallucinations, retries, broken tools and the failures that never raise an error. Culpa asks what the run cost and who it was for. The overlap is real, since a failing agent is often an expensive one, and it stops well short of substitution. A trace that shows you the retry loop doesn't tell you the loop cost $340 last month and belongs to one customer on a $99 plan.

What this usually looks like

You can see a failing run and can't say what the failure cost.
Retries are visible per trace and never totalled into a monthly number.
Nobody can say which customer's agent runs are the expensive ones.
A reliability fix shipped and nobody priced what it saved.

Free, no card, no account

Run the free Cost Leak Scan

It shows your most expensive conversation before you install anything.

Run a free scan

Mistakes that cost the most

Mistake	Why it hurts	Do instead
Assuming a trace viewer answers a cost question.	Per-run cost is visible and the roll-up by feature, customer and month usually isn't.	Check whether your tool totals spend by customer, not just by run.
Fixing reliability without pricing the fix.	The cheapest failures get fixed first because they're the easiest to see.	Rank failures by measured cost as well as by frequency. The order usually changes.
Running both and reconciling neither against the invoice.	Two internally consistent tools can still both sit below the bill you actually pay.	Reconcile whatever you capture against the provider invoice for the same period.

Run this check tonight

Take last month's noisiest failing agent and put a dollar figure on it.
If that took more than a minute, the cost side is the gap.
Total your retry spend for the month and compare it against your total bill.
Rank your known failure modes by cost rather than by count.
Reconcile whatever you captured against the provider invoice for the same period.

One retry loop, seen twice

Illustrative example

Raindrop's published capabilities, read from raindrop.ai on 2026-08-03, against what Culpa reports for the same incident. Both publish pricing and the units differ, so the plans are listed side by side rather than reduced to one comparison that would need an assumed traffic mix to mean anything.

Raindrop: logs every production agent run with its messages, tool calls, retries and errors

Raindrop: surfaces the silent failures, from hallucinations and loops to broken tools, into Slack

Raindrop: runs an experiment against live traffic to confirm the regression is gone

Culpa: names the single most expensive conversation and what it cost

Raindrop plans: Startup $59 a month with a 14-day trial, Pro $399, Enterprise custom

Raindrop's unit: one event is a user message plus the agent's reply, tool calls and sub-agents included

Culpa: prices the same retries and rolls them up by feature, customer and month

Culpa: forecasts the next month from your own history, as a median and a worst case

Costed: 40,000 failed runs a month on Claude Haiku 4.5 at $1.00 per million input and $5.00 output

Per attempt, 2,000 input and 300 output: (2 x $1.00) + (0.3 x $5.00) = $0.0020 + $0.0015 = $0.0035

Two wasted attempts per failure: 40,000 x 2 x $0.0035 = $280.00 a month of pure retry

Raindrop tells you the loop happened and helps you kill it. Culpa tells you those wasted attempts cost $280 a month and which customer carried them. Teams running agents in production usually want both answers, and neither tool produces the other's.

Every number, with its confidence and source

Figure	What it means	Confidence	Source
$59 to $399 per month	Raindrop's published Startup and Pro plan prices, before per-event charges	provider-reported	raindrop.ai, read 2026-08-03. Startup includes 1,000 events a month then $0.004 per event, Pro is $0.003 per event to 1M then $0.002. An event is defined there as a user message plus the agent's reply.
$280.00	modelled monthly cost of the wasted attempts in a three-try retry loop	calculated	40,000 failures x 2 wasted attempts x $0.0035 per attempt, using real Claude Haiku 4.5 rates from the price book, effective 2026-07-02. The failure volume and token counts are modelled.

What a generic answer can’t know

Whether your reliability problem is also your cost problem is a question about your own traffic, and the two rankings often disagree. Run the free scan and see whether the most expensive conversation is one you'd have flagged. Culpa keeps your prompts and responses on your infrastructure, and counts the calls to run your plan.

Questions founders ask next

Does Raindrop show cost?

Its published plans list Tracing and User Tracking from the Startup tier, and developers quoting it on Raindrop's own site describe seeing token count and cost per run. The difference is the roll-up: attributing months of spend to a feature, a customer and a margin, and forecasting forward, is a different job from inspecting a trace.

Should I run both?

Many agent teams should. Raindrop answers why a run failed and Culpa answers what your runs cost and who for. The two rankings, most broken and most expensive, are often not the same list.

Does Culpa need my prompts to attribute cost?

No. Culpa runs on your own infrastructure, so your prompts and responses stay there, and metadata-only mode is a single toggle if you'd rather store no payloads at all. Culpa counts the calls to run your plan.

On your infrastructure

Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.

How Culpa works

Find the culprit. Not just the total.

Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.

Your prompts stay local.

Culpa runs on your own infrastructure. What you send to a model reaches us at no point.

Every dollar has a name.

Follow any charge to the conversation, the user, the feature and the customer behind it.

See the bill before it lands.

Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.

Three steps to your first answer.

Change one base URL.

Or drop in the Python or TypeScript library.

Find your most expensive conversation.

In the first session, not the first week.

Cost your next feature before you ship it.

Base URLhttp://localhost:4545/v1Your traffic keeps flowing if Culpa ever stops.

Why the bill went up

Example dashboard

Calls traced

418,209

across 3 projects

Spend this week

$378.41

+ $182 vs last week

Failed calls

312

74% retried, and you paid for all of them

+ $182 this week traced to one culprit

Spend over 14 days

user_384report_generatorconv_91fprompt_v1894,220 tokens3 retries$6.81

Most expensive users

user_384$38.42

user_119$21.07

user_562$14.90

user_204$8.30

user_871$5.10

Next week forecast

Best$180

Median$240

p90$310

p99$395

Graded against reality. Accuracy shown as results land.

Run the free Cost Leak Scan Start 14-day trial

Free, no card, no account

Run the free Cost Leak Scan

It shows your most expensive conversation before you install anything.

Run a free scan

Keep reading

Sources: Raindrop. Last reviewed 2026-08-03. Plain text version.