NATS KV and Object Store: Beyond Messaging

When I first discovered that NATS JetStream could serve as a key-value store, I was skeptical. We had been running a small Redis instance solely for runtime configuration and feature flags, and the idea of replacing it with “just another NATS feature” felt risky. But after spending a weekend migrating our config service to NATS KV and seeing the Watch API push configuration changes to all services in under 50 milliseconds, I was convinced. The real surprise came later when I tried the Object Store for passing intermediate PDF files between pipeline stages — it eliminated an entire S3 dependency and shaved seconds off our document processing workflow.

Introduction

Most people think of NATS as just a message broker. But since JetStream provides a persistence engine, the NATS team built two powerful abstractions on top of it: Key-Value (KV) Store and Object Store.

[NATS Key-Value Store Documentation] — Synadia Communications , 2024-07-20

In our project, we use these to eliminate the need for Redis (for caching/state) and simplify configuration management.

Architecture Overview

Both KV and Object Stores are just Streams under the hood.

• KV: A Stream with LastValue retention. The key is the Subject.
• Object Store: A Stream that chunks large files into messages.

[NATS JetStream Architecture] — Synadia Communications , 2024-06-15

flowchart TB
    subgraph App["🖥️ Application"]
        Config["⚙️ Config Service"]
        Worker["⚙️ Worker"]
    end

    subgraph NATS["⚡ NATS JetStream"]
        subgraph KV["🔑 KV Store"]
            KVStream["Stream: KV_config\n(LastValue retention)"]
            Keys["maintenance_mode\nlog_level\nfeature_flags"]
        end
        
        subgraph OBJ["📦 Object Store"]
            OBJStream["Stream: OBJ_temp\n(Chunked files)"]
            Files["doc_123.pdf\nimage_456.png"]
        end
    end

    Config -->|"Put/Watch"| KV
    Worker -->|"Get/Put (50MB files)"| OBJ
    KV -.->|"Real-time push"| Config

    classDef primary fill:#7c3aed,color:#fff
    classDef secondary fill:#06b6d4,color:#fff
    classDef db fill:#f43f5e,color:#fff
    classDef warning fill:#fbbf24,color:#000

    class App primary
    class NATS secondary
    class KV,OBJ db

NATS Key-Value Store

The KV store is perfect for:

1. Dynamic Configuration: Change log levels or feature flags at runtime.
2. Simple State: Store “Last Processed ID” for a worker.
3. Caching: Store ephemeral data.

Creating a KV Bucket

// Connect
var js = new NatsJSContext(conn);
var kv = js.CreateKVContext();

// Create bucket (if not exists)
var store = await kv.CreateStoreAsync("config_bucket");

[NATS.Net Client Library] — NATS Authors , 2024-10-12

Watching for Changes (Real-time Config)

This is the killer feature. Unlike Redis where you poll, NATS pushes changes to you instantly.

// Put a value
await store.PutAsync("maintenance_mode", "false");

// Watch for updates
await foreach (var entry in store.WatchAsync("maintenance_mode"))
{
    var mode = entry.ValueAsString(); // "true" or "false"
    Console.WriteLine($"Maintenance mode changed to: {mode}");
        UpdateSystemState(mode);
}

NATS Object Store

Sending a 50MB PDF over a message bus is a bad idea. But storing it in S3 just to pass it to the next worker is slow. NATS Object Store sits in the middle. It’s great for intermediate payloads.

[NATS Object Store Documentation] — Synadia Communications , 2024-08-05

var objClient = js.CreateObjectStoreContext();
var store = await objClient.CreateStoreAsync("processing_temp");

// Upload a file
await store.PutAsync("doc_123.pdf", File.OpenRead("local.pdf"));

// Download
await store.GetAsync("doc_123.pdf", File.OpenWrite("downloaded.pdf"));

When to use what?

Feature	Redis/S3	NATS KV/OBJ
Deployment	Separate Service	Built-in (Single Binary)
Latency	Low	Very Low
Atomic Updates	Yes	Yes (Optimistic Concurrency)
Watch/Observe	Pub/Sub (Separate)	Native (Watch API)
Use Case	Global Cache, Long-term Storage	Config, Event State, Temp Data

[Designing Data-Intensive Applications] — Martin Kleppmann , 2017-03-16 [NATS By Example] — Synadia Communications , 2024-09-30

Conclusion

By leveraging NATS KV and Object Store, you can significantly reduce the number of moving parts in your infrastructure. For many microservices, NATS is the only infrastructure dependency they need.

After running NATS KV and Object Store in production for several months, I can say with confidence that these features have fundamentally simplified our architecture. Removing Redis as a separate dependency eliminated an entire class of operational concerns — connection management, persistence configuration, and failover logic — that we no longer have to think about. The Object Store, while not a replacement for a proper blob storage system for long-term data, has proven invaluable for our document processing pipeline where files need to move between stages quickly and reliably. If you are already running NATS with JetStream, these capabilities are essentially free infrastructure waiting to be used.

Next Steps

• Explore how we handle event-driven messaging patterns with NATS JetStream in the earlier articles of this series.
• Review our Homelab Infrastructure for the full cluster topology.
• Experiment with KV TTL (time-to-live) for ephemeral caching use cases.
• Consider using Object Store with NATS headers for metadata tagging on stored objects.

Further Reading

[NATS Official Documentation] — NATS Authors , 2024 [Designing Data-Intensive Applications by Martin Kleppmann] — Martin Kleppmann , 2024 [NATS By Example] — NATS Authors , 2024