Redis vs DynamoDB for Ephemeral Claim State

Where a claim’s in-flight state lives while it walks the finite state machine between INGESTED and RESPONSE_DISPATCHED is a latency-versus-durability decision, and Redis and DynamoDB anchor the two ends of it. Every stateless adjudication worker that picks up the next lifecycle transition must read the current node — the resolved GPI, the pricing scratch, the FSM cursor — and conditionally advance it, thousands of times per second, inside a sub-2-second B1 budget. Redis holds that state in memory with sub-millisecond reads, native TTL eviction, and atomic operations; DynamoDB holds it durably with conditional writes that survive a node loss but cost single-digit-to-low-double-digit milliseconds per call. This is the state-store half of Claim State Infrastructure Selection, and the choice hinges on whether losing an in-flight claim to an evicted key is recoverable from the durable event log or must never happen at all.

The Exact Decision #

The ephemeral store is not the audit ledger and not the referential database — it holds what is happening now: the claim’s current FSM node, an idempotency marker on the 402-D2 Prescription/Service Reference # token, and small scratch values like the resolved GPI and the running 409-D9 Ingredient Cost. Its access pattern is a hot read plus a conditional (compare-and-set) write on every transition. Redis optimizes that pattern to memory speed and gives TTL for free, but a memory-pressure eviction or a node failure can drop a key mid-flight, so it is only safe when the durable event log can replay a stranded claim. DynamoDB never loses the key — conditional writes are durable and survive an availability-zone loss — but every transition pays a network round-trip and a write-capacity cost, and TTL deletion is best-effort and delayed. The decision is: is a dropped in-flight key recoverable by log replay (choose Redis for latency), or must the store itself be the durable authority (choose DynamoDB)?

Decision Matrix #

For the point-of-sale hot path, Redis is the default: the ephemeral state is genuinely ephemeral, the durable event log is the authority for replay, and the latency budget cannot absorb a network round-trip on every transition. DynamoDB earns the state store when there is no replayable log behind it, when the in-flight window is long (a prior-authorization pend that lingers for minutes to hours), or when regulatory posture wants durable, encrypted-by-default state with no separate persistence tuning. A common split runs Redis for sub-second POS FSM state and DynamoDB for longer-lived pended-claim state.

Figure: Redis serves FSM state at memory speed with TTL eviction backed by log replay, while DynamoDB makes each transition a durable, AZ-replicated conditional write — the latency-versus-durability crux of the ephemeral-state choice.

Step-by-Step: A State Store Abstraction with TTL and Optimistic Version #

Both backends implement one ClaimStateStore Protocol. The engine gets idempotent create, a hot read, and an optimistic compare-and-set, and never learns which store it is.

1. Model the state and the contract. The version integer is the optimistic-concurrency guard; the 402-D2 token is the idempotency key.

from __future__ import annotations
from decimal import Decimal
from typing import Optional, Protocol
from pydantic import BaseModel, ConfigDict


class ClaimState(BaseModel):
    model_config = ConfigDict(extra="forbid")
    correlation_id: str            # internal UUID
    rx_ref_token: str              # tokenized 402-D2, idempotency key
    fsm_node: str                  # e.g. PRICING_CALCULATED
    ingredient_cost_409d9: Decimal # 409-D9 Ingredient Cost -- Decimal, never float
    version: int = 0


class ClaimStateStore(Protocol):
    def put_new(self, s: ClaimState, ttl_seconds: int) -> bool: ...
    def get(self, correlation_id: str) -> Optional[ClaimState]: ...
    def advance(self, s: ClaimState, expected_version: int) -> bool: ...

2. Implement the Redis backend. SET NX gives idempotent create; a Lua script gives an atomic check-version-then-write so two workers cannot both advance the same claim. TTL is set on create.

import redis  # redis-py; TLS + encryption-at-rest configured on the server


_CAS = """
if redis.call('HGET', KEYS[1], 'version') == ARGV[1] then
  redis.call('HSET', KEYS[1], 'version', ARGV[2], 'node', ARGV[3], 'cost', ARGV[4])
  return 1
end
return 0
"""


class RedisClaimStore:
    def __init__(self, client: redis.Redis) -> None:
        self._r = client
        self._cas = client.register_script(_CAS)

    def put_new(self, s: ClaimState, ttl_seconds: int) -> bool:
        # NX = create only if absent -> a replayed B1 with same 402-D2 token is a no-op.
        ok = self._r.set(f"claim:{s.correlation_id}:lock", s.rx_ref_token,
                         nx=True, ex=ttl_seconds)
        if ok:
            self._r.hset(f"claim:{s.correlation_id}",
                         mapping={"version": s.version, "node": s.fsm_node,
                                  "cost": str(s.ingredient_cost_409d9)})
            self._r.expire(f"claim:{s.correlation_id}", ttl_seconds)
        return bool(ok)

    def get(self, correlation_id: str) -> Optional[ClaimState]:
        h = self._r.hgetall(f"claim:{correlation_id}")
        if not h:
            return None
        return ClaimState(correlation_id=correlation_id, rx_ref_token="",
                          fsm_node=h[b"node"].decode(),
                          ingredient_cost_409d9=Decimal(h[b"cost"].decode()),
                          version=int(h[b"version"]))

    def advance(self, s: ClaimState, expected_version: int) -> bool:
        won = self._cas(keys=[f"claim:{s.correlation_id}"],
                        args=[str(expected_version), str(s.version),
                              s.fsm_node, str(s.ingredient_cost_409d9)])
        return bool(won)

3. Implement the DynamoDB backend. A ConditionExpression makes the create and the version check atomic and durable; attribute_not_exists is the idempotent create, and version = :expected is the optimistic CAS.

import boto3
from botocore.exceptions import ClientError


class DynamoClaimStore:
    def __init__(self, table_name: str) -> None:
        self._t = boto3.resource("dynamodb").Table(table_name)

    def put_new(self, s: ClaimState, ttl_seconds: int) -> bool:
        import time
        try:
            self._t.put_item(
                Item={"pk": s.correlation_id, "version": s.version,
                      "node": s.fsm_node, "cost": str(s.ingredient_cost_409d9),
                      "expires_at": int(time.time()) + ttl_seconds},  # TTL attribute
                ConditionExpression="attribute_not_exists(pk)",       # idempotent create
            )
            return True
        except ClientError as e:
            if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
                return False  # duplicate: item already exists
            raise

    def get(self, correlation_id: str) -> Optional[ClaimState]:
        item = self._t.get_item(Key={"pk": correlation_id}).get("Item")
        if not item:
            return None
        return ClaimState(correlation_id=correlation_id, rx_ref_token="",
                          fsm_node=item["node"],
                          ingredient_cost_409d9=Decimal(item["cost"]),
                          version=int(item["version"]))

    def advance(self, s: ClaimState, expected_version: int) -> bool:
        try:
            self._t.update_item(
                Key={"pk": s.correlation_id},
                UpdateExpression="SET version = :nv, node = :n, cost = :c",
                ConditionExpression="version = :ev",  # optimistic CAS
                ExpressionAttributeValues={":nv": s.version, ":ev": expected_version,
                                           ":n": s.fsm_node,
                                           ":c": str(s.ingredient_cost_409d9)},
            )
            return True
        except ClientError as e:
            if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
                return False  # lost the race; re-read and retry
            raise

4. Inject one backend. The adjudication service depends on ClaimStateStore; the swap is configuration.

Verification: Idempotent Create and CAS Under Contention #

The two load-bearing properties are that a duplicate create is suppressed and that only one of two racing writers advances a claim. Drive them against a dict-backed fake that models both semantics.

import pytest


class FakeStore:
    def __init__(self) -> None:
        self._d: dict[str, ClaimState] = {}

    def put_new(self, s: ClaimState, ttl_seconds: int) -> bool:
        if s.correlation_id in self._d:
            return False
        self._d[s.correlation_id] = s
        return True

    def get(self, correlation_id):
        return self._d.get(correlation_id)

    def advance(self, s: ClaimState, expected_version: int) -> bool:
        cur = self._d.get(s.correlation_id)
        if cur is None or cur.version != expected_version:
            return False
        self._d[s.correlation_id] = s
        return True


def _state(v=0, node="INGESTED"):
    return ClaimState(correlation_id="c1", rx_ref_token="tok_402D2_x",
                      fsm_node=node, ingredient_cost_409d9=Decimal("42.10"), version=v)


def test_duplicate_create_is_suppressed():
    store = FakeStore()
    assert store.put_new(_state(), ttl_seconds=30) is True
    assert store.put_new(_state(), ttl_seconds=30) is False  # replayed B1 -> no-op


def test_cas_lets_only_one_writer_win():
    store = FakeStore()
    store.put_new(_state(v=0), ttl_seconds=30)
    first = store.advance(_state(v=1, node="NORMALIZED"), expected_version=0)
    second = store.advance(_state(v=1, node="NORMALIZED"), expected_version=0)
    assert first is True and second is False  # second lost the race, must re-read

Gotchas & PHI Guardrails #

• Encryption at rest is not optional. DynamoDB encrypts at rest by default with KMS; Redis does not — you must enable encryption at rest and TLS in transit explicitly, or the ephemeral store becomes an unencrypted PHI surface, violating Security & Compliance Boundaries for Claims Data.
• Never store raw PHI in state values. The value holds a correlation_id, a tokenized 402-D2, the FSM node, and Decimal scratch — never 302-C2 Cardholder ID, 310-CA Patient Name, or raw claim bytes. Keys are correlation-scoped, not member-scoped.
• TTL is eviction, not audit retention. The ephemeral TTL (seconds) governs in-flight state; the audit-retention clock (years) lives on the durable event log. Do not conflate them — a claim’s state can expire the instant it is dispatched, but its audit event must survive. DynamoDB TTL is also best-effort and can lag deletion by up to ~48 hours, so never treat an expired-but-present item as authoritative.
• Redis eviction policy matters. Set noeviction or volatile-ttl on the adjudication keyspace; allkeys-lru will silently drop in-flight FSM state under memory pressure. A dropped key is recovered by log replay, never by defaulting to a paid response.
• Money is Decimal, serialized as a string. 409-D9 Ingredient Cost and any accrual cross the store as str(Decimal); a float in a Redis hash or a Dynamo Number attribute corrupts copay rounding. DynamoDB’s SDK also rejects Python float for numbers — pass strings, per the Python decimal module.
• A pended PA claim is not POS-ephemeral. State that must survive minutes to hours — a claim pended for prior authorization — belongs in the durable store, and its race conditions are handled in Resolving PA Pend Race Conditions, not by stretching a Redis TTL.

Related #

• Claim State Infrastructure Selection — the parent decision framing transport, state store, and validation together.
• Kafka vs RabbitMQ for PBM Adjudication — the durable-log transport that replays a claim when an ephemeral key is evicted.
• Resolving PA Pend Race Conditions — the long-lived pended-state contention that pushes state toward the durable store.
• Fallback Routing Logic Design — the deterministic degradation when the state store times out on the hot path.
• Security & Compliance Boundaries for Claims Data — the encryption-at-rest and PHI obligations both stores must meet.

← Back to Claim State Infrastructure Selection