Support triage

A user writes in: credits are missing, their instance stopped answering, a charge surprised them. This runbook goes from the user’s email to a verdict — which tenant is theirs, what Stripe actually holds, and whether anything failed or the gates worked as designed.

Related: Stripe debugging · tenant-debugging · where-to-look · Billing reference

---

Look up the user and their orgs

Start from the email in the ticket and find the user’s row, then every org they belong to. Everything hangs off the orchestrator DB. Connect directly (Where to look) or run the query inside a gateway pod, which uses the service’s own credentials so nothing secret leaves the cluster:

kubectl --context gke_plastic-labs-prod_us-east4_operations -n groudon \
  exec deploy/groudon-gateway -- python -c "
import asyncio
from sqlalchemy import text
from groudon.db import SessionLocal
async def main():
    async with SessionLocal() as db:
        r = await db.execute(text(\"select id, email, welcome_credit_tenant_id, welcome_credit_granted_at from groudon.users where email ilike :e\"), {'e': 'user@example.com'})
        print([tuple(map(str, row)) for row in r])
asyncio.run(main())
"

Match the email case-insensitively: Supabase lowercases at signup, and the address the user types in a support thread may not.

From the user id, list their orgs:

select t.id, t.name, t.instance_state, t.stripe_customer_id, t.billing_email, uta.role
from groudon.tenants t
join groudon.user_tenant_association uta on uta.tenant_id = t.id
where uta.user_id = '<user_id>';

Two things to check before going further:

• A user can hold several orgs, and org names are not unique. A real case: the same user created two orgs with the identical name five days apart, put their card on the second, and asked why it had no free credits (the credits were on the first). Never assume the org in the ticket is the user’s only one.
• instance_state is on the tenant row. If the complaint is “my instance is down”, take the tenant id to groudon.tenant_batch_allocations for its cluster and batch, then follow tenant-debugging.

Going the other way — a Stripe customer id and nothing else — the customer’s tenant_id metadata key is the join back, or select * from groudon.tenants where stripe_customer_id = 'cus_…'.

---

The Stripe side

tenants.stripe_customer_id names the customer. Reading and changing anything Stripe-side — finding the customer from the dashboard, credit balances, granting credits, invoices and reissues, coupon codes — is Stripe debugging. The one read this runbook’s verdicts need:

stripe get /v1/billing/credit_grants -d customer=<cus_id> -d limit=20 --live

Trust the grant list, not the customer metadata — the cached totals there are recomputed by backfills and prove nothing about what ran.

---

Welcome credit: failed or never owed

Two promotional grants, both one-shots (groudon/stripe/credits.py):

• $20 welcome at org creation, no card needed. claim_welcome_credit stamps users.welcome_credit_tenant_id and welcome_credit_granted_at once per user, ever. Only the stamped tenant is eligible. Additional orgs get nothing, by design.
• $80 billing unlock when a card is added — only on the designated welcome tenant, and only once per physical card (groudon.card_fingerprint_redemptions, keyed on the card fingerprint, so a card reused across accounts unlocks nothing the second time).

A missing welcome grant has two causes, and Stripe cannot tell them apart — a failed grant call leaves a Sentry event, not a Stripe event. The stamp is the discriminator:

select id, email from groudon.users where welcome_credit_tenant_id = '<tenant_id>';

Grants in Stripe	Stamp on this tenant	Verdict	Action
no welcome grant	nobody stamped	Never owed. The member’s own welcome_credit_tenant_id says where their credit went; confirm that org holds its $20.	Explain. Nothing to fix.
no welcome grant	a user stamped here	The claim won and the grant call failed. Sentry has it under action:welcome_credit_grant.	Re-run StripeClient.create_initial_credit_grant(customer, tenant, name). The grant-list guard makes the re-run idempotent, and it also repairs the customer metadata.
grant exists, metadata stale	either	The grant landed and the totals write failed.	The same re-run repairs it.

Sentry tags for the whole path: action:welcome_credit_claim, action:welcome_credit_grant, action:welcome_credit_retry, action:signup_provision_gate — each carries tenant_id, the grant ones also stripe_customer_id, and the event’s recovery extra states the fix.

---

Balance, pauses, and emails

What the user experiences as “my instance stopped working” is often billing (gateway/billing_controls.py, groudon/stripe/notifications.py):

• Derivers pause at zero balance and resume when credits land. The pause state is a Redis key (billing_paused_key(app_name), 30-day TTL), and both transitions send the user an email. A paused deriver means the queue stops draining; the API keeps serving.
• Threshold emails go out at the $10and$0 crossings, deduplicated by the metadata flags above and re-armed when the balance recovers past the threshold.
• Welcome-only customers — total granted exactly $20,nocardonfile—getanadd-cardnudge(pointingatthe$80 unlock) instead of the top-up emails, since they have nothing to top up from.
• Auto top-up has a $10minimumthreshold;acustomerwithitenabledshouldhavetoppedupbeforethe$10 email could fire.

---

Watch out for

• Same user, several orgs, identical names. Resolve by tenant id, never by org name.
• The card put on a non-designated org forfeits the $80 unlock silently. Working as designed, but it reads as a bug from the user’s side.
• Stripe-side pitfalls — metadata that lies, duplicate customers, the events firehose, the wrong CLI account — are listed in Stripe debugging.