物流指南Logistics Guide

USPS API 宕机应急手册:3PL 如何让面单、追踪与退货继续运转USPS API Outage Playbook: How 3PLs Keep Labels, Tracking, and Returns Moving

3PL 打印的每一张面单都是一份承诺:报价决定了客户付多少钱,面单意味着包裹会走,manifest 意味着承运商已接收,追踪号意味着所有人都看得见进度。当 USPS Web Tools API 抽风时,所有这些承诺同时被质疑,而故障并不会出现在 API 控制台里,它出现在仓库地板上:订单卡在出单环节、包裹堆在暂存区没有面单、客户追问为什么货没了动静。Every label a 3PL prints is a promise. The rate quote says what the customer pays, the label says the parcel will move, the manifest says the carrier accepted it, and the tracking number says everyone can watch. When the USPS Web Tools API blips, all of those promises come into question at once, and the failure shows up not in an API console but on the warehouse floor: orders stuck at label generation, parcels staged without labels, and customers asking why their shipment has gone silent.

这篇手册填补的是另外两篇指南之间的空档。《物流问题诊断框架》回答「这是承运商的错、仓库的错还是软件的错」;《物流异常管理》处理「已经在途的包裹出问题怎么办」。两篇都没有覆盖那个最麻烦的中间地带:上游承运商 API 暂时不可用,而仓库仍然要发货,还不能破坏交易完整性。This playbook fills the gap between two questions we have covered elsewhere. The Shipping Problem Diagnostic Framework answers whose fault it is, the carrier, the warehouse, or the software. Shipment Exception Management answers what to do with a parcel that is already in motion and went wrong. Neither covers the messy middle: the upstream carrier API is temporarily unavailable, and the warehouse still has to keep shipping without breaking transaction integrity.

下面就是这片中间地带的操作手册。它建立在六个阶段之上:检测(Detect)、分类(Classify)、止血(Contain)、继续(Continue)、恢复(Recover)、对账(Reconcile)。整个过程中你会反复用到四个出口:等待、重试、人工处理、切换承运商。目标不是炫技,而是保持吞吐不停、面单唯一,并在 API 恢复后让账单和追踪对得上账。So here is the operating manual for that middle. It is built on six phases: Detect, Classify, Contain, Continue, Recover, and Reconcile. Along the way you will use four exits over and over: wait, retry, handle manually, and switch carrier. The goal is not to be clever. The goal is to keep throughput moving, keep labels unique, and make sure the billing and tracking story reconciles when the API comes back.

运营团队在履约中心盯着展示承运商 API 状态的大屏,身后传送带上的包裹在排队
故障从来不是先出现在控制台,而是先出现在仓库地板Outages show up on the warehouse floor before they show up in a console

六阶段框架一览The Six-Phase Framework at a Glance

六个阶段对应运营团队在故障中真正会问的六个问题。检测:这是承运商的故障,还是我们自己的?分类:我们哪些流程受伤了,伤得多重?止血:现在立刻停止做什么,才不会把事情搞得更糟?继续:没有正常 API 的情况下,怎么让仓库继续运转?恢复:API 回来之后,怎么安全地消化积压?对账:怎么证明每张面单、每份 manifest、每张账单都对得上?The six phases map to the six questions an operations team actually asks during an outage. Detect: is this the carrier, or is this us? Classify: which of our flows are hurt, and how badly? Contain: what do we stop doing right now so we do not make things worse? Continue: how do we keep the warehouse moving without the normal API? Recover: how do we drain the backlog safely when the API returns? Reconcile: how do we prove that every label, every manifest, and every invoice lines up?

四个决策出口出现在团队判断力所在的地方。等待是故障范围窄或时间短时的默认答案;重试是故障属于瞬时性、且幂等性让重试安全时的答案;人工处理是吞吐比自动化更重要时的答案;切换承运商是故障看起来会持续很久、且备用通道已经就绪时的答案。下图展示了每个出口在链条中的位置。The four decision exits appear where the team's judgment lives. Wait is the default answer when the outage is narrow or short. Retry is the answer when failures are transient and idempotency makes it safe. Manual handling is the answer when throughput matters more than automation. Switch carrier is the answer when the outage looks long and a backup lane is ready. The diagram below shows where each exit sits in the chain.

六阶段流程图:检测、分类、止血、继续、恢复、对账,各阶段间标出等待、重试、人工模式、切换承运商四个出口Flowchart of the six phases: Detect, Classify, Contain, Continue, Recover, Reconcile, with the wait, retry, manual mode and switch-carrier exits marked between stages
六个阶段是一条带逃生口的链条Six phases, a chain with escape hatches

把这张图读成一条带逃生口的链条。大多数事故根本走不出前两格:你检测到一个超时,把它分类为低风险,然后靠等待或重试就扛过去了。昂贵的出口,人工处理和切换承运商,只留给那些等待就会让仓库停摆的事故。Read the diagram as a chain with escapes. Most incidents never leave the first two boxes: you detect a timeout, you classify it as low risk, and you wait or retry your way through. The expensive exits, manual handling and carrier switching, are reserved for the incidents where waiting would stall the warehouse.

检测:是它们还是我们?Detect: Is It Them or Us?

第一个阶段也是团队最浪费时间的地方,因为任何 API 故障时的默认假设都是「承运商挂了」。这个假设出错的频率高得惊人。故障的症状,坏凭证、写坏的集成、配错的防火墙同样会产生,而修法完全不同。如果日志有序,分辨两者的差别只需要六十秒。The first phase is also where teams waste the most time, because the default assumption during any API failure is that the carrier is down. That assumption is wrong surprisingly often. The same symptoms an outage produces, a bad credential, a broken integration, or a misconfigured firewall can produce too, and the fix is completely different. Detecting the difference takes sixty seconds if your logs are in order.

先从失败的样子入手。所有端点同时超时和连接重置,通常意味着承运商一侧。HTTP 500 和 503 意味着承运商的应用或负载均衡在挣扎[^1]。HTTP 429 意味着限流,而限流往往是你自己的重试风暴触发的,不是承运商挂了[^2]。认证失败,401 和无效凭证错误,几乎从来不是故障;是凭证轮换、过期或被吊销了。只有一个端点挂、其他端点都健康,指向部分承运商故障,或者你自己那个调用的集成 bug。延迟在几分钟内逐渐爬升然后跌进超时,是经典的过载特征。Start with the shape of the failure. Timeouts and connection resets that hit every endpoint at once usually mean the carrier side. HTTP 500s and 503s mean the carrier's application or load balancer is struggling[^1]. HTTP 429 means rate limiting, which is often triggered by your own retry storm rather than by the carrier being down[^2]. Authentication failures, 401s and invalid-credential errors, are almost never an outage; the credential rotated, expired, or was revoked. A single endpoint failing while others stay healthy points to a partial carrier problem, or to your own integration bug in that specific call. Latency that creeps up over minutes and then tips into timeouts is the classic overload signature.

症状Symptom可能原因Likely cause第一步检查First check
所有端点超时和重置Timeouts and resets across all endpoints承运商故障或网络路径Carrier outage or network path承运商状态页,然后自己的日志Carrier status page, then your own logs
多个端点 500 / 503500 / 503 on many endpoints承运商应用过载Carrier application overload状态页,然后重试策略Status page, then your retry policy
429 限流429 rate limiting你的请求量,往往是自找的Your request volume, often self-inflicted重试循环和并发Your retry loop and concurrency
401 / 认证错误401 / auth errors凭证轮换或过期Credential rotated or expired凭证库和令牌续期Credential store and token renewal
只有单个端点失败One endpoint failing only部分承运商故障或你的代码 bugPartial carrier issue or your code bug该调用的日志和错误码That call's logs and error codes

确认协议:先看承运商的状态页和 Web Tools 通告[^3],但别停在那里,因为状态页比现实慢几分钟。然后查自己的请求日志,看响应码、延迟和错误在各端点的分布。如果失败在所有端点、以及测试和生产环境里都一样,那是承运商。如果只局限在某个端点或某份凭证,那是你自己。再加一条规则:把开始时间和第一次观察到的错误码记录到一个共享的地方,ITSM 工单或故障频道。后面每个阶段,从客户沟通到复盘,都需要这个时间戳。The confirmation protocol: check the carrier's status page and Web Tools advisories[^3], but do not stop there, because status pages lag reality by minutes. Then check your own request logs for response codes, latency, and the error distribution across endpoints. If failures are uniform across all endpoints and across both test and production environments, it is the carrier. If they are scoped to one endpoint or one credential, it is you. One more rule: record the start time and the first observed error code somewhere shared, an ITSM ticket or an incident channel. Every later phase, from customer comms to the post-incident review, needs that timestamp.

分类:按端点评估爆炸半径Classify: Endpoint-by-Endpoint Blast Radius

一旦确认大概率是承运商的问题,就评估每个受影响端点能造成多大破坏。典型 3PL 流程里的六个端点,报价、认证、面单、manifest 或 SCAN、追踪、退货面单,爆炸半径非常不同。这里的爆炸半径有两层意思:你的吞吐有多依赖这个端点,以及一次错误的重试在经济上会伤你多深。Once you know it is probably the carrier, decide how much damage each affected endpoint can do. The six endpoints in a typical 3PL flow, rate, auth, label, manifest or SCAN, tracking, and return label, have very different blast radii. Blast radius here means two things: how much of your throughput depends on this endpoint, and how badly a bad retry can hurt you financially.

报价是低风险。报价是一份可以重算、缓存或被兜底价替代的承诺,它绝不应该阻塞结账或下单。认证是中等风险,但正如检测阶段看到的,它通常是自己造成的。追踪是中等风险:包裹已经有面单、在途,追踪故障只是降低客户体验,不威胁任何交易。退货面单是中等风险:可以推迟,或者通过 Portal 或备用承运商发放,不会破坏发货承诺。manifest 和 SCAN 是中高风险:面单已经发放,物理风险低,但承运商交接记录缺失,威胁揽收和账单对账。面单是高风险的:每次创建面单的调用都是一笔财务承诺,响应丢失后的重试可能产生一张重复面单,之后要作废、退款、审计。Rates are low risk. A rate quote is a promise that can be recomputed, cached, or replaced by a fallback rate, and it should never block checkout or order entry. Auth is medium risk but usually self-inflicted, as we saw in Detect. Tracking is medium risk: parcels already have labels and are in motion, so a tracking outage degrades the customer experience but threatens no transactions. Return labels are medium risk: they can be deferred, or issued through the portal or the backup carrier, without breaking a shipping commitment. Manifest and SCAN are medium-high: labels are already issued, so the physical risk is low, but the carrier-handoff record is missing, which threatens pickup and billing reconciliation. Labels are high risk: each create-label call is a financial commitment, and a retry after a lost response can mint a duplicate label that later has to be voided, refunded, and audited.

端点Endpoint典型故障Typical failures风险Risk默认动作Default action
报价Rate超时、429、坏费率数据timeout, 429, bad rate dataLow缓存最近一次好报价,回退到比价结果,绝不停下单Cache last good rates, fall back to comparison rates, never block checkout
认证Auth401、令牌过期401, token expiryMedium先查凭证,别先假设故障Check credentials first, not an outage assumption
面单Label超时、5xx、响应丢失timeout, 5xx, response lostHigh快速熔断,仅在幂等保护下重试,否则人工Circuit-break fast, retry with idempotency only, else manual
Manifest / SCANManifest / SCAN超时、5xxtimeout, 5xx中高Medium-high保留面单,推迟 manifest,揽收时对账Keep labels, defer manifest, reconcile at pickup
追踪Tracking超时、5xxtimeout, 5xxMedium优雅降级,显示「承运商状态暂不可用」,不要重试风暴Degrade gracefully, show carrier status unavailable, no retry storm
退货面单Return label超时、5xxtimeout, 5xxMedium推迟,或通过 Portal / 备用承运商发放Defer, or issue via portal or backup carrier

贯穿整本手册的经验法则:端点越接近创造财务或发货承诺,它的重试就必须越谨慎。报价和追踪以读为主,重试可以大方些。面单和 manifest 以写为主,重试必须抠门。The rule of thumb that drives the whole playbook: the closer an endpoint is to creating a financial or shipping commitment, the more careful its retries must be. Rates and tracking are read-mostly, so you can be generous with retries. Labels and manifests are write-mostly, so you must be stingy.

止血:停止失血,保护交易完整性Contain: Stop the Bleeding, Protect Integrity

止血阶段要做的是停止那些会让故障恶化的动作。第一步是熔断器:某个端点上连续 N 次失败后,停止调用它,打开一个快速失败窗口。窗口内每个请求不经网络立即失败,熔断器按计划半开,测试承运商是否恢复[^4]。给每次重试加上带抖动的指数退避,避免重试同步成看起来像攻击的波次[^5]。一条硬规则:没有无限重试循环。三次带退避的重试预算通常够了,再失败就进队列或交给人工。Containment is where you stop doing the things that make the outage worse. The first step is a circuit breaker: after N consecutive failures on an endpoint, stop calling it and open a fail-fast window. Inside that window every request fails immediately without touching the network, and the breaker half-opens on a schedule to test whether the carrier recovered[^4]. Add exponential backoff with jitter to every retry, so retries do not synchronize into waves that look like an attack[^5]. One hard rule: no infinite retry loops. A budget of three attempts with backoff is usually enough; after that the failure goes to a queue or to a human.

经典的完整性陷阱是重复面单。创建面单的请求超时了,但承运商其实处理了。客户端盲目重试,现在一票货有了两张面单、两笔运费,而一个包裹不可能同时贴两张面单。下面的时序图展示幂等带来的差别。The classic integrity trap is the duplicate label. A create-label request times out, but the carrier actually processed it. The client retries blindly, and now two labels exist for one order, two shipping charges, and one parcel that cannot wear both labels. The sequence below shows the difference idempotency makes.

幂等重试时序图:不带幂等键重试生成重复面单,带幂等键重试返回原面单Sequence diagram of idempotent retry: a blind retry mints a duplicate label, a same-key retry returns the original label
带幂等键的重试返回原面单,而不是再造一张A retry with an idempotency key returns the original label instead of minting a new one

每次创建面单的调用都应该带一个由订单或包裹派生的幂等键,比如订单 ID 加后缀[^6]。当承运商支持时,用同一个键重试会返回原面单,而不是生成新面单。不支持时,就自己维护请求台账:发送前把请求写入持久化存储,超时后先把台账和承运商的面单列表核对,再决定是否重试。这个检查,「这张面单到底创建了没有」,是重复事件和虚惊一场的区别。Every create-label call should carry an idempotency key derived from the order or the parcel, for example the order ID plus a suffix[^6]. When the carrier supports it, a retry with the same key returns the original label instead of minting a new one. When it does not, keep your own request ledger: write the request to durable storage before sending, and on timeout reconcile the ledger against the carrier's label list before retrying. That check, did this label actually get created, is the difference between a duplicate and a non-event.

队列管理是第二个止血杠杆。给重试队列设上限:待处理数超过阈值就停止入队,把失败置为 pending-carrier-api 这类驻留状态。驻留不是失败,是主动决定先挂起并通知人工,而不是继续堆积。最后是沟通。内部故障更新模板(什么挂了、我们在做什么、下次什么时候更新)能省掉大量混乱;面向客户的 SLA 通知要早发、诚实发:「承运商 API 正在经历延迟,您的货件会延迟」好过沉默,而且就算故障很短,这话也不花成本。Queue management is the second containment lever. Cap the retry queue: when the pending count passes a threshold, stop enqueueing and park the failure with a status like pending-carrier-api. Parking is not failure; it is a conscious decision to hold work and alert a human instead of piling up. Finally, communicate. A standing template for internal incident updates, what is down, what we are doing, when we next update, saves chaos, and customer-facing SLA notifications should go out early and honestly: the carrier API is experiencing delays and your shipment will be delayed beats silence, and it costs nothing when the outage turns out to be short.

继续:降级模式下保持仓库运转Continue: Keep the Warehouse Moving in Degraded Mode

止血完成后,问题变成吞吐:面单 API 黑着的时候,怎么继续拣货、打包、发货?第一个答案是 Portal。大多数多承运商平台,包括 EasyShippingX Portal,都可以在浏览器里对任何健康的承运商创建面单。人工出单慢,所以要选择性使用:低货量、高价值或时间敏感的包裹走人工出单,其余留在驻留队列里等。给人工通道加护栏:只有受过培训的员工、单独的队列以便人工面单可审计、以及每小时人工面单数量的硬上限,超过就整体切换通道。Once the bleeding is contained, the question is throughput: how do we keep picking, packing, and shipping while the label API is dark? The first answer is the portal. Most multi-carrier platforms, including the EasyShippingX Portal, can create labels through a browser against whatever carriers are healthy. Manual label entry is slow, so use it selectively: low-volume, high-value, or time-critical parcels get manual labels; everything else waits in the parked queue. Put guardrails on the manual lane: only trained staff, a separate queue so manual labels are auditable, and a hard cap on how many manual labels per hour you will tolerate before switching lanes entirely.

第二个答案是备用承运商。多承运商配置的回报恰恰体现在这一刻。EasyShippingX 支持在承运商之间切换面单、费率和追踪,所以应急通道是一个配置决策,不是一次重建。用明确的触发条件启用它:面单 API 宕机超过你的阈值(30 到 60 分钟是合理的起点)、积压越过第二个阈值、且备用承运商对受影响目的地的费率和时效预估可接受。切换之前告诉客户:换承运商可能改变送达日期、追踪链接,有时还有费用,一个突然出现的陌生追踪链接比一句提前告知更糟。The second answer is the backup carrier. A multi-carrier setup earns its keep in exactly this moment. EasyShippingX supports switching labels, rates, and tracking across carriers, so the emergency lane is a configuration decision, not a rebuild. Enable it on explicit triggers: the label API has been down past your threshold, 30 to 60 minutes is a sane starting point, the backlog has passed a second threshold, and the backup carrier's rates and delivery-time estimates are acceptable for the affected destinations. Before you switch, tell the customer: a carrier change can change delivery dates, tracking URLs, and sometimes cost, and a surprise tracking link is worse than a heads-up.

备用承运商切换决策流程图:宕机超阈值、备用承运商已批准、费率和时效可接受三个判断依次通过后才切换,否则走人工降级模式Decision flowchart for switching to a backup carrier: past threshold, carrier approved for the lane, and acceptable rates and delivery times gate the switch, otherwise manual degraded mode
切换备用承运商是一系列显式判断,不是一个开关Switching to a backup carrier is a series of explicit judgments, not a toggle

物理层面,仓库需要一套不依赖 API 的降级流程。给驻留包裹划一个暂存区,让它们离开实时流程,防止重复拣货。先拣后补是可行的:包裹可以先拣、先包,放在补单工位,等 API 恢复后补打面单;只有「没贴面单就发货」才会让它失败。这就说到不能做的事:绝不发没审计过的人工面单,绝不在没有追踪的情况下发货。一个没有追踪号就离开仓库的包裹,是等着发生的理赔和拒付,而且无论故障多短,它都会毒害对账阶段。On the physical side, the warehouse needs a degraded workflow that does not depend on the API. A staging area for parked parcels keeps them out of the live flow and prevents double-picking. Pick-without-label is viable when a parcel can be picked, packed, and held at a label-print station that backfills labels when the API returns; it fails only if the parcel ships without a label. Which brings us to the don'ts: never ship parcels with un-audited manual labels, and never ship without tracking. A parcel that leaves the building without a tracking number is a claim and a chargeback waiting to happen, and it poisons the reconciliation phase no matter how short the outage was.

仓库员工把已拣包裹放入标有明确标记的暂存区,旁边是补单工位,主管在核对清单
降级模式的暂存区:包裹离开实时流程,等待补单A degraded-mode staging area: parcels parked out of the live flow, waiting for backfilled labels

恢复:安全回放积压Recover: Safe Replay of the Backlog

API 回来时,本能是把积压一口气灌过去。忍住。故障刚结束时承运商是脆弱的,所有 3PL 同时排水形成的惊群效应,正是引发第二次故障的东西[^7]。恢复阶段是设计上的限速:用低于正常的并发(从一半开始)排空驻留队列,盯着错误率,只有错误率保持平稳才逐步提高节流。When the API comes back, the instinct is to slam the backlog through as fast as possible. Resist it. The carrier is fragile right after an outage, and a thundering herd of 3PLs all draining at once is exactly what triggers a second outage[^7]. Recover is throttled by design: drain the parked queue at a concurrency below normal, half is a good start, watch error rates, and step the throttle up only when the error rate stays flat.

回放还需要一步费率复核。故障期间,有些包裹可能按兜底价或缓存价报了价,客户也按那个价付了款。用这些报价创建面单之前,用实时 API 重新核对费率:如果实时费率不同,就要有意识地决定是吃下差额还是重新计费,绝不能让一张面单悄悄带着客户从未批准过的费率。Replay also needs a rate-recheck step. During the outage, some parcels may have been quoted with fallback or cached rates, and the customer may have been billed on those. Before creating labels from those quotes, recheck the rate against the live API: if the live rate differs, decide deliberately whether to eat the difference or re-bill, but never let a label silently carry a rate the customer never approved.

最后扫一遍重复。止血挡住了大部分,但人工通道和先拣后补流程仍可能漏出零星重复。列出故障时间窗口内创建的面单,和订单台账交叉核对,及时作废重复项。作废的面单是便宜的,被遗忘的重复面单会在几个月后变成拒付和对账的痛苦。Finally, sweep for duplicates. Containment prevented most of them, but the manual lane and the pick-without-label flow can still produce strays. List the labels created during the incident window, cross-check them against the order ledger, and void the duplicates promptly. Voided labels are cheap; forgotten duplicates become chargebacks and reconciliation pain months later.

对账:闭环Reconcile: Close the Loop

对账是人人都跳过、却让下一次故障更便宜的那个阶段。三组对账很重要。第一,面单对 manifest:故障期间发放的每张面单必须出现在 manifest 或 SCAN 记录里,manifest 的每一行必须能追溯到一张面单。第二,追踪:降级窗口内发出、从未被扫描的包裹需要人工追查,因为一个永远显示「已创建面单」的包裹,是一起无声的客户投诉。第三,账单:故障窗口的承运商账单必须和你实际创建的面单对上,包括作废的,这样重复面单的退款和人工通道的调整才能落在正确的月份。Reconciliation is the phase everyone skips and the one that makes the next incident cheaper. Three reconciliations matter. First, labels versus manifest: every label issued during the incident must appear on a manifest or SCAN record, and every manifest line must trace back to a label. Second, tracking: parcels shipped during the degraded window that never got scanned need a manual chase, because a parcel that shows label created forever is a silent customer complaint. Third, billing: the carrier invoice for the incident window must match the labels you actually created, including the voids, so refunds for duplicates and adjustments for the manual lane land in the right month.

复盘要调手册里的数字,而不是只叙述发生了什么。熔断阈值设对了吗?人工出单上限是不是太宽松?备用承运商切换是太早还是太晚?更新阈值、决策表和通知模板,反复跑复盘,直到这些数字无聊到不值得看。The post-incident review should tune the playbook's numbers, not just narrate what happened. Were the circuit-breaker thresholds right? Was the manual-label cap too generous? Did the backup-carrier switch fire too early or too late? Update the thresholds, the decision table, and the notification templates, and keep rerunning the review until the numbers feel boring.

用三个指标跟踪历次故障,才能看出手册是否有效:MTTR[^8],从第一次报错到恢复的时间;积压排空时间,驻留队列清空花了多久;准点率影响,多少包裹因为这次故障错过了承诺的送达时间。跨几次故障看趋势,这三个数字会告诉你应急架构是否真的在变好。Track three metrics across incidents so you can see whether the playbook is working: MTTR[^8], how long from first error to recovery; backlog drain time, how long the parked queue took to clear; and on-time rate impact, how many parcels missed their promised delivery because of the incident. Trended over a few incidents, these three numbers tell you whether your emergency architecture is actually getting better.

EasyShippingX 在应急架构中的位置 + 一份可以直接抄的清单Where EasyShippingX Fits + A Checklist You Can Steal

EasyShippingX 在这本手册里的角色是刻意克制的。它是一个多承运商平台,提供这本手册依赖的通道:USPS 报价不可用时的费率比价兜底、触发条件满足时把面单和追踪切换到备用承运商的能力、以及 Portal/API 双通道,让人工降级模式和自动化流程跑在同样的数据上。它不会自动监控 USPS 健康状态,也不会自动切换承运商;那些决策是你的,用这本手册的检测和分类阶段来做。任何声称能做到这些的产品,都在卖你一个无法审计的东西。EasyShippingX's role in this playbook is deliberately modest. It is a multi-carrier platform, so it provides the lanes this playbook depends on: rate comparison to fall back on when USPS pricing is unavailable, the ability to switch labels and tracking to a backup carrier when a trigger fires, and a Portal/API dual channel so the manual degraded mode runs on the same data as the automated flow. It does not auto-monitor USPS health, and it does not auto-failover; those decisions are yours, made with the Detect and Classify phases of this playbook. Any product that claims otherwise is selling you something you cannot audit.

下面的清单把整本手册压成一页。打印出来,贴在故障频道旁边,边处理边填时间戳。The checklist below is the whole playbook on one page. Print it, put it next to the incident channel, and fill in the timestamps as you go.

  1. 检测:记录第一次报错时间和错误码;查承运商状态页和自己的日志;判断是它们还是我们。Detect: record the first error time and code; check the carrier status page and your logs; decide them or us.
  2. 分类:标记每个受影响的端点(报价、认证、面单、manifest、追踪、退货);记录风险和默认动作。Classify: mark each affected endpoint (rate, auth, label, manifest, tracking, return); note risk and default action.
  3. 止血:触发熔断器;退避加幂等地重试;队列设上限并驻留;发内部和客户通知。Contain: trip the circuit breaker; retry with backoff and idempotency; cap and park the queue; send internal and customer notifications.
  4. 继续:把部分包裹分流到人工出单;评估备用承运商触发条件;布置暂存区和先拣后补。Continue: route select parcels to manual labels; evaluate the backup-carrier triggers; set up staging and pick-without-label.
  5. 恢复:半速排空积压;建单前复核兜底费率;作废重复面单。Recover: drain the backlog at half throttle; recheck fallback rates before label creation; void duplicates.
  6. 对账:面单、manifest、账单三方核对;追查未扫描包裹;复盘并调阈值。Reconcile: match labels to manifest to billing; chase unscanned parcels; run the post-incident review and tune thresholds.

围绕它的决策,这本手册站在我们其他指南之间。《物流问题诊断框架》教你怎么分清承运商、仓库和软件。《物流异常管理》处理已经在途的包裹。《多承运商策略》是让切换通道成为可能的长效结构。《低风险上线新流程》教你怎么引入新通道和新承运商而不破坏现有运营。API 黑着的时候读这一篇,在那之前读其他的。For the surrounding decisions, this playbook sits between our other guides. The Shipping Problem Diagnostic Framework tells you how to tell carrier, warehouse, and software apart. Shipment Exception Management handles parcels already in motion. Multi-Carrier Strategy is the long-term structure that makes switching lanes possible. And Low-Risk Shipping Rollout is how you introduce new lanes and new carriers without breaking the operation you have. Read this one when the API is dark, and read the others before it happens.

常见问题FAQ

看失败形态。所有端点同时超时和重置、500/503 遍布多个端点,多半是承运商;429 限流通常是你自己的重试风暴;401 认证失败几乎永远是凭证问题,不是故障。先查承运商状态页,再查自己日志里响应码、延迟和错误分布。失败在所有端点、测试和生产环境都一致,就是承运商;只局限在某个端点或某份凭证,就是你自己。Look at the failure shape. Timeouts and resets across all endpoints, or 500/503s on many endpoints, usually mean the carrier; 429 rate limiting is usually your own retry storm; 401 auth failures are almost always a credential problem, not an outage. Check the carrier status page, then your own logs for response codes, latency, and error distribution. Uniform failures across all endpoints in test and production mean the carrier; failures scoped to one endpoint or one credential mean you.
创建面单的请求超时时,承运商可能其实已经处理了;盲目重试就会产生第二张面单和两笔运费。解法是幂等键:每次创建面单调用都带一个由订单派生的键,承运商支持时,同键重试返回原面单。不支持时,维护自己的请求台账,超时后先核对台账和承运商面单列表再重试。When a create-label request times out, the carrier may have actually processed it; a blind retry mints a second label and a second charge. The fix is idempotency: every create-label call carries a key derived from the order, and when the carrier supports it, a same-key retry returns the original label. When it does not, keep your own request ledger and reconcile it against the carrier's label list before retrying.
三个触发条件同时满足:面单 API 宕机超过你的阈值(30 到 60 分钟是合理起点)、积压越过第二个阈值、备用承运商对受影响目的地的费率和时效可接受。切换前通知客户,因为送达日期和追踪链接会变。不满足这些条件时,用 Portal 或人工降级模式顶住,而不是急着切。When three triggers line up: the label API has been down past your threshold (30 to 60 minutes is a sane starting point), the backlog has crossed a second threshold, and the backup carrier's rates and delivery times are acceptable for the affected destinations. Tell the customer before you switch, because delivery dates and tracking links will change. Until those triggers fire, hold with portal or manual degraded mode instead of switching.
诊断框架回答「是谁的错」,异常管理处理「已在途的包裹出问题」,这本手册填的是中间地带:上游承运商 API 暂时不可用时,如何在保住交易完整性的前提下维持仓库吞吐。六阶段框架(检测、分类、止血、继续、恢复、对账)就是为这个场景设计的。The diagnostic framework answers whose fault it is, exception management handles parcels already in motion, and this playbook fills the middle: how to keep warehouse throughput moving without breaking transaction integrity while an upstream carrier API is temporarily unavailable. The six-phase framework (Detect, Classify, Contain, Continue, Recover, Reconcile) is built for exactly that scenario.

想把这套应急能力落进你的发货流程?Ready to put this incident playbook into your shipping workflow?

EasyShippingX 把多承运商比价、智能路由和运费核对固化到发货流程里,让「USPS 挂了」从一次事故变成手册里的一行配置。EasyShippingX bakes multi-carrier comparison, intelligent routing, and freight verification into your shipping flow, so "USPS is down" becomes a line in your playbook instead of an incident.

Get Rate Comparison
Get Rate Comparison