# Culpa vs Humanloop > Humanloop closed on 2025-09-08 and deleted account data that day. Its cost model is still documented, and worth reading before you pick a replacement. URL: https://getculpa.com/culpa-vs-humanloop Last reviewed: 2026-08-03 ## Answer Humanloop was an LLM evals platform whose plaform closed on 2025-09-08, with account data deleted that day. Its docs still describe a real cost model. Culpa, a local-first LLM cost, margin, and forecast ledger, prices each call from a versioned price book in exact decimal and forecasts next month, so this page compares two designs rather than two products you can buy. ## Why this happens Take the date first, because it decides how to read everything else. Humanloop's own migration guide says the platform closed on 2025-09-08 and that all Files, Versions, Logs, Evaluations and account settings were permanently deleted on that date. Billing stopped on 2025-07-30 and the notice went out on 2025-07-17, so users had 53 days to export through an open-source tool the team published. That's orderly conduct, and it isn't the interesting part. The interesting part is that the docs are still up and they describe a cost model more careful than most live products. Humanloop computed cost, tokens and latency for every Prompt log across all model providers from 2024-04-16. It counted reasoning tokens as their own field rather than folding them into output. It rolled cost up a trace, so an agent run carried the total of its children. And it exposed Cost as an evaluator you could threshold in CI, which is the thing almost nobody ships. Read that list as a specification for whatever you buy next. ## What this usually looks like - Your replacement tool reports one cost number per call and nothing per agent run. - Reasoning tokens are folded into output tokens, so a thinking model looks cheap and bills otherwise. - You can fail a build on latency and you can't fail one on cost. - Your provider stops returning usage when you stream, and your tool quietly stops counting. - Nobody can say what next month costs, because the tool that held your history was switched off. ## Common mistakes - Treating an evals platform's cost column as a finance number. Why it hurts: Humanloop typed cost as a double. That's right for a per-call display and it drifts once you sum a million rows. Do instead: Ask what type your tool stores money in, and prefer exact decimal anywhere totals are reported. - Assuming the replacement counts reasoning tokens separately. Why it hurts: Humanloop had a reasoning_tokens field. Fold that into output and a reasoning workload misprices in the direction that hurts. Do instead: Send one reasoning-heavy call and check the fields your tool actually stored. - Exporting the data and never opening it. Why it hurts: 53 days sounds generous until the export lands in a format nothing reads and the deadline is real. Do instead: Export early and load it somewhere, while the source system still exists to check against. - Reading the shutdown as a verdict on the category. Why it hurts: The team went to Anthropic and recommended Langfuse and Braintrust on the way out. That's a good outcome, not a collapse. Do instead: Judge the replacement on its cost model, which is what this page is for. ## Self-check - Ask your current tool for cost on an agent run, not on a single call. - Check whether reasoning tokens are a separate field or folded into output. - Try to fail a CI build on a cost threshold, the way Humanloop let you. - Stream one call and confirm the token counts still landed. - Ask what type money is stored in, and whether totals are computed in it. ## Where a double stops being good enough for money Humanloop's published OpenAPI schema types prompt_cost and output_cost as number, format double. A double carries about 15 to 17 significant digits, which is ample for one call and isn't the problem. The problem is summation. Price 40M output tokens on a rate of $0.0012 per 1k, a real shape for a high-volume cheap model, and compare exact decimal against float. 40,000,000 output tokens / 1,000 = 40,000 billable units 40,000 units x $0.0012 = $48.00 exactly, in decimal the same product in IEEE-754 double evaluates to 47.99999999999999 one row is off by $0.00000000000001, and the error compounds with every row you add Nobody loses money on that row. You lose the ability to say a total is correct, which is the whole job of a ledger. Culpa stores money as integer pico-dollars and parses rates from decimal strings, so a total is exact by construction rather than by rounding at the end. ## Cost figures Every figure carries its confidence and its source. For each provider-reported figure below, its own source line names the published record it came from, the date that record was read, and any aggregation applied to it. - $48.00, exact decimal cost of 40M output tokens at $0.0012 per 1k, against 47.99999999999999 in float [calculated] Source: Arithmetic shown in full on this page. The rate shape is drawn from the price book's cheap high-volume tier and the token volume is modelled. The float result is what IEEE-754 double multiplication returns for the same product, which is the point of the figure rather than an incidental detail. Humanloop's own OpenAPI schema at humanloop.com/docs types prompt_cost and output_cost as number, format double, read 2026-08-03. - 53 days, notice period between Humanloop's shutdown email and permanent deletion of account data [provider-reported] Source: humanloop.com/docs/v5/guides/migrating-from-humanloop, read 2026-08-03. The guide says the notice email went to all users on 2025-07-17, billing stopped on 2025-07-30, and the platform was sunset with all data permanently deleted on 2025-09-08. 2025-07-17 to 2025-09-08 is 53 days. Reported because it's the realistic answer to how long you get, not because it was ungenerous. ## FAQ Q: When did Humanloop shut down? A: The platform closed on 2025-09-08. Its own migration guide says accounts and all associated data, including Files, Versions, Logs, Evaluations and settings, were permanently deleted on that date. Billing had stopped on 2025-07-30 and the notice email went out on 2025-07-17. The docs remain online. Q: What did Humanloop track for cost? A: Per log it stored prompt_tokens, output_tokens, reasoning_tokens, prompt_cost and output_cost, computed for all Prompt logs across every model provider from 2024-04-16. A Flow log rolled cost, tokens and latency up the trace as child logs were added, so an agent run carried its own total. Q: What did Humanloop recommend that users move to? A: Its migration guide names Langfuse and Braintrust, and says to survey the options rather than take a single recommendation. Culpa isn't on that list and this page doesn't pretend otherwise. Culpa is a cost and margin ledger rather than an evals platform, so for most Humanloop users it's a complement to that choice. Q: Why does it matter that cost was stored as a double? A: It doesn't for one call. It matters when you total a month. A double is a binary fraction, so a decimal rate like $0.0012 has no exact representation and small errors accumulate across rows. Culpa stores money as integer pico-dollars, so a monthly total is exact rather than nearly right. Q: Is a page about a dead product useful? A: The product is gone and the design isn't. Humanloop shipped cost as a CI-thresholdable evaluator, reasoning tokens as their own field, and trace-level cost rollup. Most live tools do none of those. Treat the list as a specification to hold your replacement against. ## Sources - Humanloop: https://humanloop.com - Migrating from Humanloop: https://humanloop.com/docs/v5/guides/migrating-from-humanloop - Humanloop Prompt Log API reference: https://humanloop.com/docs/v5/api-reference/prompts/log Run the free Cost Leak Scan: https://app.getculpa.com/scan?source=pseo&slug=culpa-vs-humanloop&cluster=competitor Machine-readable index of every guide: https://getculpa.com/api/pages Human-readable index of every guide: https://getculpa.com/guides Site overview: https://app.getculpa.com/llms.txt Privacy: Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.