Claim State Infrastructure Selection

Selecting the messaging and state infrastructure for a high-throughput adjudication path is the architectural decision that either protects or forfeits the sub-2-second B1 budget under load. The choice splits into three distinct sub-problems that engineers routinely conflate: which transport carries claim events between services (a durable partitioned log versus a classic broker), where in-flight claim state lives while a claim walks its finite state machine (an ephemeral in-memory store versus a durable key-value database), and which validation library enforces the canonical contract at the edge. Each has a different latency profile, a different failure surface, and a different Protected Health Information (PHI) exposure. This area sits directly on the hot path defined by PBM Architecture & Taxonomy Foundations, and getting it wrong shows up not as a crash but as tail-latency drift, replay gaps during a payer audit, and cache evictions that silently drop a claim mid-adjudication.

Problem Framing #

The adjudication lifecycle — INGESTED → NORMALIZED → TAXONOMY_RESOLVED → ELIGIBILITY_VERIFIED → PRICING_CALCULATED → PA_EVALUATED → RESPONSE_DISPATCHED — is a finite state machine, and every transition is both a service boundary and an audit event. That framing forces two infrastructure commitments the rest of the platform inherits. First, the event transport must guarantee ordering per member so that a reversal (B2) never overtakes the original billing (B1) it reverses, and it must support replay so a claim can be re-adjudicated identically months later. Second, the in-flight state between two transitions must be reachable with single-digit-millisecond latency by whichever stateless worker picks up the next stage, because a synchronous database round-trip on every transition would consume the entire wall-clock budget before pricing even runs.

The engine itself must not care which backend satisfies those commitments. A pricing worker should not import a Kafka client or a Redis client directly; it should depend on an abstraction that any conforming backend can implement. That backend-agnostic boundary is what lets a platform start on RabbitMQ and DynamoDB and migrate to Kafka and Redis without rewriting adjudication logic — and it is what lets the same code run against in-memory fakes in a test suite with no broker at all. This page specifies the requirement-to-infrastructure mapping, the Python abstraction that encodes it, the data-plane topology, and the failure modes that make each choice load-bearing. The three guides below it drill into the specific head-to-head decisions.

Prerequisites #

Infrastructure selection is a downstream consequence of throughput and correctness targets, not a greenfield preference. Before choosing anything, four numbers and one boundary must be fixed:

• Throughput and latency targets. Size to peak, not average. A mid-market plan sponsor pushes 300–1,200 point-of-sale claims per second during the morning refill window and open-enrollment spikes; a national processor sees 8,000–15,000 per second. The wall-clock budget from B1 request to B1 response is sub-2-second at p99, of which the transport plus state round-trips must consume well under 100ms so the clinical and pricing stages keep their headroom. These numbers decide whether a single-broker queue suffices or a partitioned log is mandatory.
• Ordering and exactly-once needs. Decide the ordering key before the transport. Adjudication needs ordering-per-member (or per 302-C2 Cardholder ID token), not global ordering, because two unrelated members’ claims are independent and serializing them globally throttles throughput needlessly. Exactly-once is a correctness requirement for financial side effects (a rebate accrual or a paid response must not double-post), and the cheapest way to get it is an idempotency key — the tokenized 402-D2 Prescription/Service Reference # — rather than a transport-level exactly-once mode that costs latency.
• PHI boundary placement. Cardholder identifiers (302-C2) and patient name (310-CA) are tokenized at ingress, before any claim event reaches the transport, exactly as required by Security & Compliance Boundaries for Claims Data. Raw NCPDP D.0 bytes never land on a topic, a queue, or a state-store value. The event payload carries a correlation_id and tokens, never the wire transaction. This constraint eliminates transports and stores that cannot encrypt at rest or that would tempt a raw-payload dump into a dead-letter queue.
• Replay and retention windows. Audit replay drives the durable-log retention decision; ephemeral state drives the time-to-live (TTL) decision. These are different clocks — a claim’s in-flight state is worthless 30 seconds after RESPONSE_DISPATCHED, but the event that recorded the transition must survive for the audit retention window (often seven-plus years, tiered to cold storage).
• Library baseline. Python 3.11+, pydantic>=2.6 for the canonical contract, structlog or stdlib logging for JSON telemetry, decimal.Decimal for every money-bearing field, and a messaging client chosen per transport (confluent-kafka or pika). The validation library choice itself — v1 versus v2 semantics and the pydantic-core performance floor — is its own decision covered below.

Requirement-to-Infrastructure Mapping #

Each functional requirement maps to a concrete infrastructure property, and each property points at one side of a specific decision. The table is the specification the three child guides implement in detail; it is deliberately prescriptive so an engineer can read a requirement off the left and land on a defensible default on the right.

Two rules wrap the table. First, ephemeral and durable are not interchangeable: the event log is the source of truth for what happened (audit, replay), while the ephemeral store holds what is happening now (the current FSM node and its scratch data). Never invert them — reconstructing in-flight state by replaying the whole log on every transition is a latency non-starter, and treating an evictable cache as the audit ledger loses claims. Second, the ordering key and the partition key must be the same field, or per-member ordering silently degrades to per-partition ordering with cross-member interleaving.

Reference Python: A Backend-Agnostic State Plane #

The engine depends on two Protocols — a ClaimStateStore and an EventPublisher — plus a Pydantic v2 canonical envelope. Concrete backends (Redis, DynamoDB, Kafka, RabbitMQ) implement the Protocols; the adjudication code below imports neither a broker nor a database client. Every method carries only tokenized identifiers, and the audit event is emitted with no raw claim bytes.

from __future__ import annotations
import json
import logging
from decimal import Decimal
from datetime import datetime, timezone
from typing import Optional, Protocol, runtime_checkable
from pydantic import BaseModel, ConfigDict, field_validator

logging.basicConfig(format="%(message)s", level=logging.INFO)
log = logging.getLogger("claim_state_plane")

# NCPDP reject codes surfaced at the infrastructure boundary.
REJ_SYSTEM_UNAVAILABLE = "99"  # transport/state backend unreachable
REJ_TOO_SOON = "79"            # 79 Refill Too Soon (idempotent replay collision)


class ClaimState(BaseModel):
    """Ephemeral in-flight state for one claim between two FSM transitions.

    PHI GUARDRAIL: carries a correlation_id and tokenized 302-C2/402-D2 only.
    Raw NCPDP D.0 bytes, 310-CA Patient Name, and raw 302-C2 never appear here.
    """
    model_config = ConfigDict(str_strip_whitespace=True, extra="forbid")

    correlation_id: str            # internal UUID, safe to log
    cardholder_token: str          # 302-C2 Cardholder ID, ALREADY tokenized
    rx_ref_token: str              # 402-D2 Prescription/Service Reference # (idempotency key)
    ndc_407d7: str                 # 407-D7 Product/Service ID (NDC), normalized
    fsm_node: str                  # e.g. TAXONOMY_RESOLVED
    ingredient_cost_409d9: Decimal # 409-D9 Ingredient Cost -- Decimal, never float
    version: int = 0               # optimistic-concurrency guard

    @field_validator("ndc_407d7")
    @classmethod
    def _guard_payload_leak(cls, v: str) -> str:
        if len(v) > 15:  # a normalized NDC is <= 11 digits; longer means a blob leaked
            raise ValueError("407-D7 too long; possible raw payload leak")
        return v


@runtime_checkable
class ClaimStateStore(Protocol):
    """Ephemeral in-flight state. Redis or DynamoDB implements this."""

    def put_new(self, state: ClaimState, ttl_seconds: int) -> bool:
        """Create only if absent (idempotency). False = key already exists."""
        ...

    def get(self, correlation_id: str) -> Optional[ClaimState]:
        ...

    def advance(self, state: ClaimState, expected_version: int) -> bool:
        """Optimistic compare-and-set. False = a concurrent writer won."""
        ...


@runtime_checkable
class EventPublisher(Protocol):
    """Durable, ordered, replayable event transport. Kafka or RabbitMQ implements this."""

    def publish(self, ordering_key: str, event: dict) -> None:
        ...


class StatePlane:
    """Backend-agnostic coordination surface used by every adjudication worker."""

    def __init__(self, store: ClaimStateStore, publisher: EventPublisher,
                 ttl_seconds: int = 30) -> None:
        self._store = store
        self._publisher = publisher
        self._ttl = ttl_seconds

    def begin(self, state: ClaimState) -> bool:
        # Idempotent create: a retried B1 with the same 402-D2 token is a no-op.
        created = self._store.put_new(state, ttl_seconds=self._ttl)
        if not created:
            self._audit(state, outcome="duplicate_suppressed", reject=REJ_TOO_SOON)
            return False
        self._audit(state, outcome="ingested", reject=None)
        return True

    def transition(self, state: ClaimState, next_node: str) -> bool:
        prior = state.version
        moved = state.model_copy(update={"fsm_node": next_node, "version": prior + 1})
        if not self._store.advance(moved, expected_version=prior):
            # Lost the CAS race: another worker already advanced this claim.
            self._audit(state, outcome="cas_conflict", reject=None)
            return False
        # Ordering key = member token so a member's events stay per-partition ordered.
        self._publisher.publish(ordering_key=state.cardholder_token, event={
            "event": "fsm_transition",
            "correlation_id": moved.correlation_id,
            "from": state.fsm_node,
            "to": next_node,
            "version": moved.version,
        })
        self._audit(moved, outcome="advanced", reject=None)
        return True

    def _audit(self, state: ClaimState, outcome: str, reject: Optional[str]) -> None:
        # Tokenized correlation only -- no 302-C2, no 310-CA, no raw claim bytes.
        log.info(json.dumps({
            "event": "state_plane",
            "correlation_id": state.correlation_id,
            "rx_ref_token": state.rx_ref_token,   # tokenized 402-D2
            "fsm_node": state.fsm_node,
            "outcome": outcome,
            "reject_code": reject,
            "ingredient_cost": str(state.ingredient_cost_409d9),  # str(Decimal)
            "timestamp": datetime.now(timezone.utc).isoformat(),
        }))

The design intent is that StatePlane is the only thing adjudication workers import. Swapping RabbitMQ for Kafka is a change to which class satisfies EventPublisher; swapping DynamoDB for Redis is a change to which class satisfies ClaimStateStore. Because both are Protocols, a test suite supplies dict-backed fakes and the entire adjudication path runs with no infrastructure, which keeps unit tests deterministic and fast. The advance method encodes optimistic concurrency: two workers racing to move the same claim resolve deterministically because only the writer whose expected_version matches wins, and the loser re-reads rather than clobbering. The idempotency put_new collapses a retried B1 into a no-op that surfaces reject 79 rather than double-posting a paid response.

Figure: The adjudication data plane — POS ingress tokenizes PHI before the durable partitioned log; stateless workers drive FSM transitions against an ephemeral, TTL-bounded state store via atomic compare-and-set, and every transition feeds the append-only audit ledger.

Engineering Constraints & Failure Modes #

Each infrastructure choice fails in a characteristic way under load, and the mitigation belongs in the design, not in an incident retro.

• Rebalance storms. When a consumer group’s membership changes — a worker deploy, a health-check flap, a scale event — the partitioned log reassigns partitions, and every consumer pauses to re-join. On a busy topic this can freeze consumption for seconds and blow the B1 budget for in-flight claims. Mitigate with cooperative-sticky assignment, generous session.timeout.ms, static group membership across restarts, and processing that commits offsets frequently so a reassigned partition resumes near the head. The same discipline underpins Asynchronous Batch Adjudication Workflows.
• Hot partitions. Partitioning by 302-C2 member token is correct for ordering but skews when one high-volume group or a synthetic load test hammers a single key; that partition’s consumer saturates while others idle. Detect with per-partition lag metrics, and where ordering permits, sub-key by GPI so a single member’s unrelated drug lines spread across partitions. Never re-key mid-stream, which reorders in flight.
• Cache eviction of in-flight state. An ephemeral store with an aggressive TTL or memory-pressure eviction can drop a claim’s FSM state between two transitions, stranding it. The TTL must exceed the worst-case end-to-end adjudication time with margin (a 30s TTL against a 2s budget is safe; a 3s TTL is not), and eviction policy must be noeviction or volatile-ttl, never allkeys-lru, on the adjudication keyspace. A dropped state is recovered by replaying from the durable log, never by defaulting to approve.
• DLQ without PHI discipline. A poison message routed to a dead-letter destination must carry the tokenized envelope and the reject reason, never the raw D.0 payload, or the DLQ becomes an unencrypted PHI spill. Sizing and replay of that queue is specified in Sizing Dead-Letter Queues for Claim Replay.
• Backend outage on the hot path. A transport or state-store outage must degrade to a deterministic, plan-compliant response, not a hang. A read timeout surfaces reject 99 and routes through Fallback Routing Logic Design rather than blocking the adjudication thread. Circuit breakers cap the blast radius the same way they do for upstream calls under PBM API Sync & Rate Limiting.
• Split-brain snapshot skew. If two workers hold different validation-schema or crosswalk snapshot versions, the same claim adjudicates differently by node. Every event and state record stamps the schema and snapshot version so failover reconciles on the stamp, not on wall-clock arrival.

Performance & Correctness Tuning #

• Partition by the ordering key, and only that key. Per-member ordering requires partition key == 302-C2 token. If you need more parallelism than members allow, sub-partition by GPI where the two lines are genuinely independent, and document that reversals must re-key to the member partition.
• Idempotency over transactional exactly-once. A compare-and-set on the tokenized 402-D2 reference gives exactly-once financial effect at a fraction of the latency of transport-level exactly-once. The put_new/advance pattern above is the idempotency contract; a retried or replayed claim re-derives the identical outcome.
• Backpressure, not buffering. Stateless workers consuming a partitioned log lag their partition under pressure instead of dropping claims; an unbounded in-process queue that buffers to absorb a spike just moves the failure to an out-of-memory kill. Bound the in-flight window and let lag be the signal.
• Decimal money end to end. 409-D9 Ingredient Cost, 412-DC Dispensing Fee, and every accrual cross the state plane as decimal.Decimal serialized to a string. A float in a state-store value corrupts copay rounding downstream and is a payer audit finding — see the Python decimal module.
• Validate once, at the edge. The canonical ClaimState contract is enforced with pydantic v2 at ingress so no unvalidated payload reaches a worker; the v1-versus-v2 performance and API tradeoffs are quantified in the guide below.
• Audit before response. Serialize the transition event to the append-only ledger before the B1 response returns, so any claim is replayable during an examination — the obligation carried from Security & Compliance Boundaries for Claims Data.

In This Section #

• Kafka vs RabbitMQ for PBM Adjudication — the transport decision: partitioned-log replay and per-member ordering versus broker routing and per-message acknowledgment, with throughput numbers and a dead-letter story.
• Redis vs DynamoDB for Ephemeral Claim State — where the in-flight FSM state lives: in-memory TTL and atomic ops versus durable conditional writes, weighed on latency, durability, and PHI-at-rest.
• Pydantic v1 vs v2 for NCPDP Validation — the validation-library decision for parsed NCPDP D.0 claims: API differences, the pydantic-core performance floor, and migration cost.

Related #

• PBM Architecture & Taxonomy Foundations — the topology, canonical model, and lifecycle state machine this infrastructure carries.
• Asynchronous Batch Adjudication Workflows — the async worker-pool and queue model that consumes the transport chosen here.
• Fallback Routing Logic Design — the deterministic degradation path when a transport or state backend times out.
• PBM API Sync & Rate Limiting — the ingress smoothing and circuit-breaker discipline that protects the state plane from bursts.
• Security & Compliance Boundaries for Claims Data — the PHI tokenization and audit-ledger obligations every event and state record must satisfy.

← Back to PBM Architecture & Taxonomy Foundations