System onboarding: the decision at this stage
System-onboarding path, step 4: Accept the failure paths. Confirm applicability, evidence, and owners before continuing.
Shipping Integration Guide
How to Test a WMS-Shipping API Integration Before Go-Live: 15 Failure Scenarios
A shipping API integration looks finished the day every happy-path test passes. The quote comes back, the label prints, the tracking number shows up. Then go-live week happens: a retry fires twice and a customer gets charged for two labels, an address typo stalls a whole wave of orders, and the carrier's rate file goes stale at 2 p.m. This is where most integrations earn their reputation, and it has nothing to do with the happy path.
The stakes are concrete. Duplicate charges trigger refunds and chargebacks. Mis-shipped orders create return loops that eat margin. Voids and reprints pile up as a reconciliation mess at month end. Carrier penalties for bad labels or canceled pickups land on your invoice with no one to dispute them against. Every one of these failures is discoverable before go-live, if you deliberately test the failure paths.
This guide covers exactly that. It walks through 15 failure scenarios in five groups, each with a trigger, the expected behavior, an acceptance criterion, and an owner. The scenarios are written for operations decision-makers who will run the tests and sign off on go-live, with the API-level detail kept where it matters. They assume a setup like EasyShippingX: a channel-neutral shipping layer that compares US domestic carrier options and executes the shipment through the Portal, Excel, or the API. You can run every scenario against whichever channel your operation actually depends on.
This illustration is a reference acceptance scenario, not proof of sandbox, idempotency, event-delivery, or automatic-refund capabilities. Define pass criteria against verified features.
Why Test Failure Paths Before Go-Live
Happy-path tests pass everywhere. They validate that your integration works when everything cooperates: the network is up, the carrier's rate file is fresh, the address is valid, and the request arrives exactly once. None of those conditions hold reliably in production. The failure paths are where money leaks, and they only reveal themselves when you deliberately break the normal flow and check that recovery works.
The cost of one bad go-live is easy to underestimate. A single duplicate-label bug that survives for two weeks means hundreds of double charges, each requiring a refund and a ledger correction. A timeout-handling gap that re-creates paid labels can turn into a week of manual reconciliation just to find which charges were real. Add lost customer trust and carrier penalties, and the price of the test week looks very small by comparison.
The operating philosophy behind these scenarios is simple: assume the network will fail, assume the carrier will fail, assume your own code will fail, and then verify recovery, not just success[1]. A timeout is not an edge case, it is a routine event. An out-of-order tracking event is not a data glitch, it is Tuesday. When you design tests around that assumption, the integration that survives is the one you can operate.
The 15 scenarios below are grouped by integration stage, so you can test them in the order your operation touches them: order submission and idempotency, fulfillment and address handling, carrier availability, label lifecycle, tracking and events, and finally reconciliation and go-live governance.
This illustration is a reference acceptance scenario, not proof of sandbox, idempotency, event-delivery, or automatic-refund capabilities. Define pass criteria against verified features.
Every scenario in this guide follows the same four-part structure so a test can be executed and signed off without ambiguity. The trigger describes how you reproduce the failure. The expected behavior states what the integration should do, not just what it should avoid. The acceptance criterion is the concrete, checkable outcome that defines a pass. The owner names who is accountable for running the test and signing the result, typically the operations lead, the integration engineer, or finance depending on the scenario.
Test Setup: Environments and Acceptance Baseline
Treat this checklist as procurement and acceptance requirements, not a list of activated platform features. A 3PL records client and charge ownership; a 4PL records execution and recovery owners per warehouse; a brand checks order-to-parcel relationships. Use current documentation and account confirmation to verify test environments, idempotency, event delivery, branding, and finance fields. Keep undocumented capabilities pending and retain manual recovery.
The onboarding scope here is U.S. domestic transportation and API; customers or partners own clearance and warehousing. Use the API product and onboarding evaluation with order samples, tender arrangements, and capability requirements. For quotes and full charges, request a rate review.
Before any scenario makes sense, you need the right environment. Test and production credentials are separate, and the test environment must be used for every scenario in this guide. Test data carries environment markers so a stray test submission can never be mistaken for a production order in a report, and so monitoring can catch it if it leaks through. If your setup lets you submit from a test credential set, run every scenario there first; production stays untouched until the checklist is signed.
Build a small test catalog before you start. You need a handful of sample orders with realistic SKUs, a mix of valid and deliberately invalid addresses, weight and box profiles that include the edge cases your operation actually ships (heavy items, oversize boxes, multi-box orders), and at least two service classes to compare. This catalog is reused across all 15 scenarios, which keeps the tests comparable and fast to run.
Define the acceptance baseline before testing, not after. For each scenario, write down what pass means in observable terms: no double charge appears in the ledger, the order lands in the correct state, the event is recorded with the right timestamp, and the reconciliation difference is zero or explicitly explained. Without a baseline, a scenario that "mostly works" gets signed off, and mostly works is exactly what leaks money.
EasyShippingX runs three channels in parallel: the Portal, Excel upload, and the API. Portal and API are both live go-live channels, so verify each independently and then verify they agree. The last group of scenarios includes the sampling check that proves the two channels return the same quote and the same order result, and that sampling continues for the first 30 days after go-live. For how to choose between the channels, see our guide to Portal vs API onboarding; for the API capability list and credential separation, see the Shipping API page.
Group 1: Order Submission and Idempotency
This group tests the most expensive class of failure: the same order being submitted more than once and turning into two shipments and two charges. Order submission is where retries happen, where operators double-click, and where the WMS resends a request it thinks was lost. The integration has to make "submit exactly once" a guarantee, not a hope[2].
Scenario 1: Duplicate Order Submission
Trigger: submit the same order twice with the same order identifier, either through a WMS resend, a retry after a suspected failure, or an operator double-submit. The scenario reproduces exactly what happens in production when a request succeeds but the response is slow, and the caller sends it again.
Expected behavior: the idempotency key deduplicates the second submission. The first request creates one shipment and returns a shipment code; the second request returns the existing shipment instead of creating a new one. The carrier is only charged once, and the order maps to exactly one shipment code.
Acceptance criterion: the second submission returns the existing shipment code, and the charge ledger shows exactly one line for the order. Owner: integration engineer, with the operations lead confirming the ledger.
Scenario 2: Idempotent Label Retry
Trigger: a label request times out after the label was actually created on the carrier side. The response is lost in transit, so the caller does not know the label exists. This is the classic double-charge trap, and it is more common than most teams expect.
Expected behavior: the integration queries by the order identifier or shipment key before re-creating anything. If the shipment already exists, it returns the existing shipment code and does not buy a second label. The retry path is documented and testable, not something an engineer improvises at 3 a.m.
Acceptance criterion: after any timeout, the integration queries by order identifier before re-creating, and the retry never produces a second paid label. Owner: integration engineer.
Group 2: Fulfillment, Address, and Carrier Availability
The second group covers what happens between "order accepted" and "label purchased." These failures stall fulfillment instead of duplicating it, which means they surface as blocked waves, stuck orders, and operators improvising around the system. Each scenario checks that the failure is visible, explainable, and recoverable by the operator who is standing in front of the screen.
Scenario 3: Invalid Address or No-Rate Response
Trigger: an order carries a malformed or undeliverable address, or a service constraint that no carrier can satisfy, so the quote request comes back with no rate. In production this is not rare; it is a daily occurrence in any operation that accepts addresses as free text.
Expected behavior: the integration returns a clear error with a reason, blocks the order from shipping, and routes it to a manual review queue. The operator sees what is wrong, not a generic failure. No label is purchased for an order that cannot be priced.
Acceptance criterion: the error is surfaced to the operator with an actionable next step, and the order stays blocked until someone fixes the address or constraint. Owner: operations lead.
Scenario 4: Incorrect Ship-From Mapping
Trigger: the ship-from address or warehouse code on the order is mapped to the wrong value, for example a stale warehouse code, a template with an old address, or a field mapping that swaps the ship-from and bill-to. The order quotes with a rate that looks normal, so nothing flags it.
Expected behavior: the ship-from address and warehouse code are validated at quote time, and a mismatch is rejected before any label is created. A wrong-warehouse order cannot proceed to label, because shipping from the wrong location is worse than shipping late.
Acceptance criterion: an order with an incorrect ship-from mapping is rejected before label purchase, with the mismatch identified in the error. Owner: operations lead, with integration engineer for the field mapping.
Scenario 5: Carrier or Service Unavailable
Trigger: a carrier or service class is temporarily unavailable, either because the carrier's rate feed is down, a service is suspended, or the carrier API is failing. The quote request hits the failure and the integration has to decide what the operator sees.
Expected behavior: the quote fails gracefully with no hang, a clear message identifies the carrier and the reason, and an alternative is offered where one exists. The operator can act without opening a support ticket.
Acceptance criterion: the operator sees the failure reason and can act on it directly, and no request hangs indefinitely waiting for a carrier that will not answer. Owner: operations lead.
Scenario 6: Carrier Fallback
Trigger: the primary carrier is down and the operation needs to ship anyway. The fallback path selects a secondary carrier or service, but a fallback is only safe if the operator knows what changed.
Expected behavior: the fallback carrier and service are documented and selectable, and any change in rate or service level is surfaced in the quote before purchase. The operator confirms the fallback with full visibility of what it costs.
Acceptance criterion: the operator can confirm the fallback with the cost difference visible in the quote, and the shipment is created under the fallback carrier with correct tracking. Owner: operations lead.
Group 3: Label Lifecycle
The third group tests what happens to labels after they are created. Labels get split, voided, refunded, and occasionally created twice by accident. This is the group where money actually moves, and where the ledger has to stay consistent with what the carrier did. The money events around unused labels are covered in detail in our guide to unused and voided shipping labels.
Scenario 7: Partial and Split Shipments
Trigger: an order ships in multiple boxes, or a line item ships separately from the rest of the order. The integration has to create a shipment per box or per line, and each one needs its own tracking.
Expected behavior: partial and split shipments create one shipment per box or line, each with its own tracking number, and the order maps to all of the resulting shipment codes. No box ships without a traceable label.
Acceptance criterion: each box has its own tracking number, and the order record maps to every shipment code created for it. Owner: operations lead.
Scenario 8: Label Void and Refund
Trigger: a label is purchased and then voided, because of an order cancellation, a data fix, or a reprint. The void has to come back as a refund that is linked to the original shipment, and the refund has to be traceable in the ledger.
Expected behavior: the void returns a refund identifier linked to the original shipment code, and the refund status is traceable through to completion. The original shipment stays identifiable even after the label is voided, so the ledger can always connect a refund to what it cancelled[3].
Acceptance criterion: the refund appears in reconciliation linked to the original shipment code, and the voided label is identifiable in the shipment history. Owner: finance, with operations lead for the workflow.
Scenario 9: Timeout After Successful Label Creation
Trigger: the label was created and paid for, but the response timed out. Without a recovery path, a retry buys a second label and the order ends up with two paid labels and two charges. This scenario is the label-lifecycle twin of Scenario 2 and it deserves its own test because the cost is identical.
Expected behavior: the integration queries by order identifier before re-creating. If the label exists, it returns the existing shipment code and no second label is purchased. The query-before-create flow is the documented recovery path for every timeout.
This illustration is a reference acceptance scenario, not proof of sandbox, idempotency, event-delivery, or automatic-refund capabilities. Define pass criteria against verified features.
Acceptance criterion: after any timeout, the integration queries by order identifier before re-creating, and the recovery path never yields two paid labels. Owner: integration engineer.
Scenario 10: Order Cancellation After Label Purchase
Trigger: an order is cancelled after the label has been purchased. In this setup the cancellation originates from WMS-side logic, for example a stock correction or a customer service decision, not from a marketplace native connection. EasyShippingX is a channel-neutral layer, so this scenario tests the WMS-side cancel flow that then executes a void, not a marketplace integration.
Expected behavior: the cancellation triggers the void and refund flow, and the order state and the charge ledger stay consistent. The cancelled order shows its true state to the operator, with the label voided and the refund linked.
Acceptance criterion: the cancelled order's label is voided, the refund is linked to the original shipment code, and the ledger shows no unresolved charge. Owner: operations lead, with finance confirming the ledger.
Group 4: Tracking and Events
The fourth group tests the data stream that your customers actually see. Tracking events arrive out of order, duplicated, or not at all, and the integration has to produce a correct final state regardless. This is also where webhook reliability gets proven, because a lost callback is invisible until a customer asks where their package is.
Scenario 11: Tracking Events Arriving Out of Order
Trigger: tracking events arrive out of sequence, with duplicates. A "delivered" event can arrive before an earlier "in transit" event, and the same event can be delivered twice. This is normal carrier behavior, not a bug.
Expected behavior: the integration orders events by timestamp, tolerates duplicates, and lets the latest state win. The final tracking state shown to the customer is correct even when the raw event stream was reordered or duplicated.
Acceptance criterion: the final tracking state is correct after a deliberately reordered and duplicated event sequence, and no duplicate event corrupts the displayed status. Owner: integration engineer, with operations lead confirming the customer-facing state.
Scenario 12: Lost or Duplicate Webhooks, Signature Verification Failure
Trigger: a tracking callback is lost, delivered twice, or arrives with a wrong signature, either from a carrier side failure or a tampered payload. The integration has to handle all three without breaking the tracking pipeline.
Expected behavior: callbacks retry with backoff[5], every callback is signature-checked before it is trusted[4], and events that cannot be matched to an order go to a manual queue instead of being silently dropped. A wrong-signature callback is rejected, and a duplicate callback does not double-process the event.
Acceptance criterion: a failed callback is retried, a tampered or wrong-signature callback is rejected, and unmatched events land in the manual queue. Owner: integration engineer.
Group 5: Reconciliation and Go-Live Governance
The fifth group is about trust after the traffic is real. Quotes become labels, labels become final charges, and every difference has to be explainable. This group also covers the discipline that keeps the integration honest: sampling between channels and keeping test data out of production.
Scenario 13: Final Charge Reconciliation
Trigger: the final charge from the carrier differs from the quote shown at purchase. This happens routinely because of dimensional weight, zone changes, and surcharges[6]. The difference itself is not the failure; an unexplained difference is.
Expected behavior: every final-charge difference is explainable, with the reason visible next to the amount, and refund and void offsets are linked to the original shipment. The reconciliation report shows a running list where each line either reconciles to zero or carries an explicit explanation. For the full discipline of matching settled funds back to shipments, see our shipping reconciliation guide; for how to dispute a charge adjustment, see our guide to rate quote vs final charge.
This illustration is a reference acceptance scenario, not proof of sandbox, idempotency, event-delivery, or automatic-refund capabilities. Define pass criteria against verified features.
Acceptance criterion: the reconciliation report shows zero unexplained differences across the test batch, and every refund and void offset traces to its original shipment code. Owner: finance.
Scenario 14: Portal vs API Results Sampling Check
Trigger: the same order is submitted through the Portal and through the API, and the results have to agree. Divergence between channels means the operation cannot trust one channel while the other is in use, which is a governance problem, not a cosmetic one.
Expected behavior: the same quote and the same order return the same result in both channels, and a periodic sampling check runs during go-live. Any divergence beyond a documented tolerance is investigated before it becomes a pattern.
Acceptance criterion: sampling finds no divergence beyond the documented tolerance across a defined sample size, and any divergence found is investigated and resolved. Owner: operations lead.
Scenario 15: Test Data Leaking Into Production
Trigger: a test order, a test label, or a test credential submission makes it into the production environment. It can happen through a misconfigured environment variable, a copy-paste from a test session, or a stale credential set on a workstation.
Expected behavior: environment markers are checked by monitoring, and a test submission in production is blocked or quarantined before it creates a paid label. Production accounts never carry charges for test data.
Acceptance criterion: a deliberately planted test submission in the production environment is caught, blocked, or quarantined, and no test label is charged to a production account. Owner: integration engineer.
Turn failure scenarios into repeatable input, outcome, and sign-off records
Keep all fifteen scenarios as a test library and select applicability by actual mode, account capabilities, and service scope. Portal/file workflows still test duplicates and recovery without assuming API headers. If webhooks are not used, test the actual query/reconciliation recovery path and record why a scenario is not applicable and how its risk is covered.
| Acceptance record | What to document | What is insufficient |
|---|---|---|
| Scope and authorization | Test client, facility, mode, service, environment, charge boundary, and execution owner. | A demo-account success does not establish the production account’s capabilities. |
| Trigger and input | Failure trigger, business identity, input version, and test-record marking. | One successful label does not prove duplicate/timeout recovery. |
| Outcome and recovery | Observed status, query/manual-review evidence, recovery action, and charge changes. | A local timeout does not prove no upstream creation; do not blindly repurchase. |
| Return flow and invoice | Source-system confirmation, tracking identity, invoice/credit records, and client outcome. | A successful response does not prove source confirmation or billing closure. |
| Sign-off and open items | Passed, failed, or pending; evidence location, owner, and retest conditions. | No visible error or a non-reproducible verbal promise is not acceptance. |
Exercise failures in an authorized isolated environment or agreed test arrangement. Test credentials alone do not prove the absence of real purchases, charges, notifications, or refunds; confirm those side effects in the arrangement. These additions define recommended evidence and ownership, not completed tests of any provider or production operation.
Continue to the limited pilot once failure evidence is sufficient. Return to data alignment for identity/charge mapping gaps instead of finding them by expanding live volume.
Go-Live Checklist and Monitoring
Before the switch flips, run all 15 scenarios against the test environment with a sign-off per owner. Each scenario gets a pass or a documented deviation, and no scenario is waived because it "probably works." The checklist is the go-live gate, and it is only closed when every owner has signed. For a wider rollback plan, including how to stage the switch, see our low-risk shipping rollout guide; to run a load pass before go-live, see our guide to shipping API load testing for peak season.
Once traffic is real, watch five things. Request success rate tells you whether the integration is fundamentally healthy. Latency tells you whether it will stay healthy under load. Retry volume tells you how often the recovery paths are actually firing. Unmatched tracking events tell you whether the tracking pipeline is losing events to the manual queue. Manual queue depth tells you whether operators are drowning in exceptions they have to resolve by hand. Each has a suggested threshold you set before go-live, and each has an owner who responds when it is crossed.
This illustration is a reference acceptance scenario, not proof of sandbox, idempotency, event-delivery, or automatic-refund capabilities. Define pass criteria against verified features.
| Monitor | What it catches | Suggested threshold | Owner |
|---|---|---|---|
| Request success rate | Fundamental integration health | Above 99% of requests succeed | Integration engineer |
| Latency (p95) | Degradation under load | Below 5 seconds per request | Integration engineer |
| Retry volume | Recovery paths firing too often | Below 1% of requests trigger retry | Integration engineer |
| Unmatched tracking events | Events lost to the manual queue | Queue drains within one shift | Operations lead |
| Manual queue depth | Operator overload on exceptions | Queue drains within one shift | Operations lead |
Define pause and rollback criteria before you need them. Decide what triggers a halt to API traffic, and make the Portal the fallback when that happens. EasyShippingX runs the Portal as a parallel go-live channel, which means a local API failure does not have to stop shipping; the operation can fall back to the Portal while the API issue is fixed. The rollback test is itself a scenario: verify that switching to the Portal mid-stream produces no duplicate orders and no lost shipments.
For the first 30 days after go-live, keep the sampling check from Scenario 14 running between Portal and API results. Sampling is cheap, it catches divergence while it is still small, and it builds the month-end reconciliation habit the whole operation will rely on. Go-live is not the finish line; the first month of reconciliation discipline is what decides whether the integration is trusted or just tolerated.
Run these 15 scenarios before go-live, keep the monitors on, and keep the fallback ready. The integration that survives its failure paths is the one that stops being a project and starts being the shipping layer your operation can build on.
Next: carry the evidence forward
Carry this stage’s evidence and open items forward. Keep unconfirmed capabilities pending rather than replacing verification with defaults or more volume.
Ready to put your shipping integration through its failure paths?
These scenarios evaluate execution and recovery for rates, labels, tracking, and charges. For EasyShippingX or another provider, confirm environments/capabilities against the actual account and current documentation, then agree which tests can cause real purchases, charges, or notifications. Credential names or reference examples do not prove that all fifteen scenarios avoid production effects. Use a demo to clarify boundaries before planning acceptance.
FAQ
Duplicate submission and timeout recovery. Happy-path tests never trigger them: the request arrives exactly once and the response is always on time. Idempotency keys, query-before-create, and void-to-refund linking each deserve their own scenario, because when they are missing the failure shows up as double charges, the most expensive and hardest-to-trace class of failure there is.
Keep the fifteen scenarios as a risk library and assess applicability by mode/capability. Document non-applicability and alternative coverage; absence of errors does not pass an uncovered critical risk.
Environment/credential separation reduces misuse but does not alone prove the absence of real charges or notifications. Confirm the provider’s arrangement, account capabilities, and side effects before running scenarios with environment markers and evidence.
Not necessarily. Dimensional-weight adjustments, zone changes, and surcharges make final charges differ from quotes by design. The real problem is the unexplained difference: every line in the reconciliation report either nets to zero or carries an explicit reason. Scenario 13 accepts "zero unexplained differences", not "zero differences".
The Portal is a live go-live channel in parallel with the API, and the fallback when a local API failure hits: operations can keep shipping through the Portal while the API issue is fixed, and the rollback itself is verified as a scenario (the switch produces no duplicate orders and no lost shipments). Scenario 14's sampling check exists specifically to prove Portal and API results agree.
Must every mode run all fifteen scenarios in exactly the same way?
Choose applicable scenarios by actual capability and document non-applicability and alternative risk coverage. Still validate duplicates, unknown outcomes, identity, and charges even when a particular interface is unused.