The payment succeeded, the provisioning failed, and the invoice says paid. It is unavoidable — the question is whether your platform notices, or the customer does first.
Mario Marin6 min readThe payment succeeded. The provisioning failed. The customer has been charged, the invoice says paid, and there is no server. This is the single most damaging state a hosting platform can be in, because everything downstream — support, billing, capacity, trust — is now built on a record that is wrong.
It is also unavoidable. Hypervisors time out, panels reject a username, IP pools run dry, hardware fails its post-install check. The question is not whether it happens but whether your platform notices.
Clean failure. The provisioning call returns an error. The platform marks the service failed and tells somebody. Annoying, entirely survivable.
Timeout with unknown outcome. The call did not return in time. The VM may exist, may be half-built, or may not exist at all. Retrying blindly creates a second machine; not retrying may leave the customer with nothing. This is the case that needs an explicit answer.
Partial success. The machine was created, the IP was not assigned. Or the account exists on the panel but the customer never got credentials. The service looks active and does not work.
Silent success-then-drift. Provisioning worked, and later something changed outside the platform — the VM was deleted directly on the hypervisor, or the account was removed on the panel. Billing continues indefinitely for something that no longer exists. This is the worst one because nothing ever errored.
If you take one thing from this: a retry must not be able to create a second resource. Every provisioning request should carry an identifier derived from the service — not a random value generated at call time — so that a retried request against a provider that already fulfilled it returns the existing resource rather than building another.
Without this, a timeout plus an automatic retry is how you end up with two VMs for one paid service, one of which nobody knows about and both of which consume capacity. The orphan is usually discovered months later during a capacity audit, and by then nobody can safely say which one the customer is using.
A service needs a state between "ordered" and "active" and it needs a state for "we do not know". A binary active flag cannot express a half-built machine, so build these explicitly:
The distinction between "the API returned success" and "the resource exists and works" is where most of the pain lives. Where you can, verify after provisioning rather than trusting the response.
A failed provision that only writes a log line is a failed provision nobody knows about. The customer finds out first, which is the worst possible order, and by then they have been charged.
Failures need to reach a human through a channel that person actually reads — a ticket, a chat notification, an email that is not one of forty. And the customer needs an honest message too: something went wrong, we know, we are on it. Silence after a payment is what turns a technical failure into a chargeback, and we covered where that leads in chargebacks and payment disputes.
The drift case — billing something that no longer exists, or running something nobody is billed for — cannot be caught by error handling, because nothing errored. It needs a periodic sweep that compares two lists: what the platform thinks is active, and what the provider says exists.
Three questions that sweep should answer:
Run it weekly and the discrepancies are a short list. Run it never and the first time you look will be during a capacity crisis.
If provisioning failed and cannot be recovered promptly, the customer should not stay charged. That means a credit note or a refund, not an informal apology — the invoice needs to be corrected as a document, and the mechanics are in credit notes, refunds and clawbacks.
If it failed and was then fixed within a reasonable window, the honest adjustment is usually to shift the billing period start to when the service actually worked, rather than charging from the payment date.
FluxBilling treats provisioning failure as a first-class state rather than an exception. Services carry distinct lifecycle states through the order-to-active path, so a service that is paid but not yet provisioned is not indistinguishable from one that is running. A dedicated provisioning-failure notifier routes failures to a human rather than only to a log, and a scheduled provisioning-recovery job retries and reconciles services stuck in an incomplete state rather than leaving them for someone to notice.
Because IP allocation and hardware inventory live in the same schema as the service rather than in a separate DCIM product, the reconciliation questions above are queries against one database rather than a comparison between two systems that each believe they are authoritative. That is the practical form of the argument in the all-in-one post.
An honest limit: reconciliation against a third-party provider is only as good as what that provider's API will tell you, and integrations differ in how completely they can answer "does this still exist". Verify the behaviour for the specific control plane you run.
See provisioning and IPAM.
Test this deliberately before it happens by accident. Break the connection to your hypervisor and place a paid order. Watch what the platform does: does the service land in a failed state, does anyone get told, does the customer see something honest, and can you retry without creating a second machine? The answer to that last question is the one that decides whether your capacity numbers mean anything a year from now.
A chargeback is the customer’s bank deciding for you, taking the money, and charging a fee whether you win or not. Most are communication failures, and three small fixes remove them.
The gap between "this invoice is overdue" and "this server is gone" is a policy, not a feature. Most businesses never write it down, so it exists anyway as whatever the software defaults to.
Getting money in is built carefully. Getting it back out is built in a hurry, and that is where the accounting goes wrong. Four operations that look like one.