FinOps & Cloud Economics · Sep 2026 · 15 min read

Testing My Own Multi-Cloud FinOps CLI Against a Real Azure Account — and Finding Two Real Bugs

A provider-neutral FinOps data platform I built — extraction, normalization, SQL-first governance policies. I pointed it at a real Azure Cost Management export and found the governance layer was completely broken against any live cloud source. Here is exactly what broke, why, and the fix.

Building It Is One Thing. Pointing It at Real Data Is Another.

cloudcost-cli is a provider-neutral FinOps data platform: declarative YAML pipelines, Apache Arrow in-memory data movement, DuckDB/Postgres/BigQuery destinations, and a SQL-first governance engine for writing FinOps policies against normalized cost data. I built it end-to-end — sources for Azure, AWS, OCI, and Alibaba Cloud, a FOCUS-schema normalization layer, a policy runner. It passed its own bundled quickstart against sample data.

Then I pointed it at a real Azure Cost Management export from a real resource group, and the governance layer — the actual point of the tool — broke completely.

Before Step 1, one term this walkthrough leans on:

Step 1 — Set up a real Azure Cost Management export

No sample CSV this time. A real export, from a real Azure subscription, into a real storage account:

az group create --name cloudcost-cli-test-rg --location eastus
az storage account create --name <name> --resource-group cloudcost-cli-test-rg --sku Standard_LRS --kind StorageV2
az storage container create --name cost-exports --account-name <name> --auth-mode login

az provider register --namespace Microsoft.CostManagementExports

az costmanagement export create --name cloudcost-cli-test-export \
  --scope "subscriptions/<sub-id>" \
  --storage-account-id "<storage-account-resource-id>" \
  --storage-container cost-exports --storage-directory exports \
  --type Usage --timeframe MonthToDate \
  --dataset-configuration columns="UsageDateTime" columns="ResourceGroup" columns="ResourceType" columns="ServiceName" columns="MeterCategory" columns="PreTaxCost" columns="Currency" columns="InstanceId" \
  --recurrence Daily --recurrence-period from="<today>" to="<+7d>" --schedule-status Active

A real wall hit immediately: RP Not Registered — the Microsoft.CostManagementExports resource provider isn't registered by default on a subscription that's never used cost exports before. One az provider register and about a minute's wait fixed it. Then a second real wall: the --dataset-configuration columns=... values I first tried (Date, ResourceId) aren't valid for this export type — Azure's error message conveniently listed the actual valid column names, which is how I got the schema right on the second attempt.

The az costmanagement export command group also has no run subcommand to trigger an export on demand — I called the underlying REST endpoint directly (POST .../exports/{name}/run) to avoid waiting for the daily schedule.

Step 2 — Extraction actually worked, cleanly

cloudcost sync cloudcost.azure-real.yml

Extracting from source: azure_billing
  -> Extracted 215 rows
Transforming data: normalize_costs
  -> Transformed 215 rows
Loading data to destination: local_duckdb
  -> Loaded 215 rows into local_duckdb
Pipeline real-azure-test-pipeline execution completed successfully!

215 real cost line items, authenticated via DefaultAzureCredential picking up my existing az login session, downloaded from real blob storage, parsed into Arrow, loaded into DuckDB. This part — the hard, cloud-specific part — worked exactly as designed on the first real attempt.

Step 3 — The governance engine, completely broken

cloudcost policy run cloudcost.azure-real.yml

Policy execution failed: IO Error: Cannot open database
"/private/tmp/cloudcost-cli/data/cloudcost.duckdb" in read-only mode: database
does not exist

Bug 1: cli.py instantiated PolicyRunner() with zero arguments, and PolicyRunner.__init__ hardcoded db_path: str = "data/cloudcost.duckdb". It never read the pipeline's own destinations[].config.path — the file sync had just written to (data/cloudcost-azure-real.duckdb, a perfectly reasonable name for a real pipeline distinct from the bundled sample). sync succeeds, policy run fails against a file that was never created, because it's looking in the wrong place entirely.

Working around that by renaming the file to match the hardcoded default surfaced the real, deeper bug:

Binder Error: Referenced column "billed_cost" not found in FROM clause!
Candidate bindings: "normalized_at", "PreTaxCost"

Bug 2: focus.normalize — the transform whose entire job is producing the FOCUS schema the policies query against — was a pure passthrough. Its own comment admitted it: "For the MVP, we act as a passthrough." It only appended a normalized_at timestamp. Azure's real export produces ServiceName/PreTaxCost; the bundled policies (policies/unallocated_costs.sql, policies/zero_utilization.sql) query service_name/billed_cost/provider/resource_id. Nothing ever renamed one to the other. The SQL-first governance engine — the tool's headline feature — could not function against any real cloud source. It only ever worked against the bundled sample CSV, which happened to already be pre-formatted to match.

Step 4 — Fixing both, honestly

For Bug 1, the fix is unambiguous: read the real destination path.

duckdb_destinations = [d for d in pipeline.destinations if d.type == "duckdb"]
db_path = duckdb_destinations[0].config.get("path")
runner = PolicyRunner(db_path=db_path)

For Bug 2, the honest fix isn't guessing at AWS/OCI/Alibaba's exact real export column names — I only had verified ground truth for Azure's, from the CSV actually sitting in blob storage. So the mapping is explicit, in config, using a TransformConfig.config field that already existed in the schema but was never read:

transforms:
  - name: normalize_costs
    type: focus.normalize
    input: azure_billing
    config:
      provider: azure
      column_map:
        ServiceName: service_name
        PreTaxCost: billed_cost
        InstanceId: resource_id

Step 5 — Re-run against the same real data, no workaround

cloudcost sync cloudcost.azure-real.yml
cloudcost policy run cloudcost.azure-real.yml

Evaluating policy: unallocated-costs
  -> Generated 0 findings.
Evaluating policy: zero-utilization
  -> Generated 6 findings.
Policy execution complete. Total findings: 6

Six real findings, against real Virtual Machines cost line items, with dollar amounts ($0.4888, $0.9984) that match the actual per-day VM costs from that resource group's real billing history — not fabricated, not sample data. No manual database-renaming workaround needed. Both fixes verified against the same live export that broke them.

Closing Thoughts

The extraction layer being solid and the governance layer being completely non-functional against real data is a real, common shape for a tool built provider-first and tested against its own sample fixtures: the hard cloud-integration work gets exercised early and often, while the "glue" — a hardcoded path, a placeholder transform that quietly never got finished — survives because the bundled quickstart never stresses it. Pointing a tool at genuinely real data, from a resource group with real billing history, is what surfaced both bugs in about twenty minutes of testing; neither would show up running the README's own quickstart forever.

GitHub Repository: cloudcost-cli — both fixes, the real Azure pipeline config (cloudcost.azure-real.yml) that found and then verified them, and the README section documenting the column-mapping pattern for pointing this at any real cloud source.

Reviewed against a real Azure Cost Management export as of 2026-09-20.

FinOps · Azure · Python · DuckDB · Apache Arrow · Data Engineering