This dependency-free browser calculator estimates the monthly cost of LLM-as-a-judge evaluation and the observability platform that stores its telemetry. Version 1.1.0 preserves a five-platform, twelve-plan pricing model verified on 2026-08-13. It combines judge-model token charges with the selected platform's published billing rules while keeping workload assumptions visible.
\nPharos Production, an AI software development company, developed the calculator to make evaluation-cost assumptions inspectable before teams choose a monitoring plan. The interactive LLM-as-a-judge cost calculator exposes the inputs and component costs. This Zenodo record preserves the source release that implements the model.
\nStart with monthly application traces, then specify telemetry sampling and the proportion of sampled traces evaluated. Evaluation happens after sampling. Judge calls per evaluated trace and stored scores per evaluated trace are separate inputs, so running several judges does not silently imply an identical number of stored scores.
\nThe calculation derives ingested traces from application traffic and sampling, evaluated traces from ingested traffic and evaluation percentage, and judge calls from evaluated traces and judge-call multiplicity. Input and output tokens have separate editable rates. Observations, bytes per trace, retention and seats determine the remaining platform-specific conversions.
\nLangSmith, Langfuse, Braintrust, Arize AX and Confident AI use different combinations of trace, observation, score, span, data-volume and retention meters. Decimal gigabytes are calculated from ingested traces and bytes per trace. GB-months also include retention. The method uses a normalized 30-day month when comparing requested retention with published day limits. The companion LLM observability cost analysis explains why these billing units require explicit workload conversions. It provides company-authored architectural context, not independent verification of the archived prices.
\nAn exact result means every required component is computable under the declared assumptions and recorded plan constraints. It is not a guaranteed invoice. Results separate judge-model cost from platform fees, usage and retention charges, with a total and cost per 1,000 application traces where the calculation supports them.
An unpublished overage, an exceeded fixed retention window or an incompatible seat constraint can produce not_computable. Known subtotals remain visible, but the total is withheld. Confident AI Free above its retained-capacity allowance produces lower_bound because the plan drops excess spans. A low monetary charge therefore does not establish that the requested workload will be fully retained.
The archive preserves source commit 67b6d1d9d741214a986ea4d7ce7111a99605bbf3. It includes the static application, pure calculation module, tests and normalized data registries. Model presets and vendor plans live in data/models.json and data/vendors.json; data/sources.json and data/claims.json connect recorded claims to first-party sources. Published data mirrors are included under docs/data/.
With Node.js 20 or newer, run npm test from the extracted repository root to check calculation behavior and release metadata. The project declares no package dependencies. The calculation methodology documents meter mappings, retention behavior, worked examples and exports. The separate npm run check:sources command checks source reachability. It neither revalidates prices semantically nor advances their verification date.
Use the release for reproducible planning against its dated pricing snapshot. Check current first-party terms before making a purchasing decision. Enterprise agreements and negotiated terms are outside the model. The related MLOps and LLMOps services from Pharos Production cover deployment, monitoring and ongoing operations around model workloads. Those services have a broader scope than the calculator's inference and observability charges.
\nResearch, drafting, implementation and quality checks were AI-assisted. Independent external review is not claimed. Source code and the static application retain the MIT license. Normalized pricing registries retain CC BY 4.0 under DATA-LICENSE.md. Vendor names and marks belong to their respective owners, and inclusion implies no endorsement.