Beta

Sluice

Publish gateway for event buses

Validate. Order. Publish. One HTTP endpoint.

Sluice sits between your applications and your event bus. It validates schemas through partial deserialization, guarantees write-time ordering per key, and gives you per-topic publish control. Works with Kafka, Pub/Sub, and more.

Get started See how it works
2 bus backends
<5ms validation overhead
1 HTTP API

How it works

One gateway. Every topic governed.

Configure

Define topics in YAML with schema sources, validation rules, ordering keys, and producer settings. One config file controls your entire publish surface. Topics map logical names to bus-native targets.

Validate

Payloads are checked against proto and Avro schemas using partial deserialization. Only the fields that matter are decoded. Bad data gets a 400 before it ever reaches the bus. Required fields, type checks, enum constraints.

Publish

Messages are dispatched to the bus with write-time ordering guarantees per key. Per-topic producers isolate throughput-sensitive topics. Kafka, Pub/Sub, or any backend behind the same HTTP API.

Bus-agnostic by design.

Your applications publish to a single HTTP endpoint. Sluice handles the bus-specific details: partitioning for Kafka, ordering keys for Pub/Sub. Switch backends or add new ones. Client code stays the same.

Get started
Kafka

Apache Kafka. Per-topic producers, idempotent writes, partition key routing

Pub/Sub

Google Pub/Sub. Native ordering keys, ADC auth, emulator support

+

More coming. Kinesis, EventHub. Same API, same validation, same ordering

Capabilities

Data quality enforcement at the publish boundary.

Schema validation

  • Proto wire-format tag scanning, no full deserialization
  • Avro schema-guided projection reading
  • Required field checks, type validation, enum constraints
  • Local schema files included free; registry sync requires a license key

Write-time ordering

  • Per-key actor pool guarantees wire-arrival order
  • Lazy actor spawn, scales with active key cardinality
  • Idle eviction keeps memory bounded
  • Works across Kafka, Pub/Sub, and future backends

Per-topic control

  • Topic-level validation, ordering, and routing config
  • Logical topic names decoupled from bus-native targets
  • Dedicated Kafka producers and Pub/Sub batching overrides (license key)
  • One default producer handles the long tail

Operational

  • Single static binary. Deploy as container, sidecar, or standalone
  • YAML config with environment variable overrides
  • Health checks, structured JSON logging, graceful shutdown
  • Prometheus /metrics + OTLP tracing (license key)

No bad data hits your topics.

Partial deserialization means Sluice validates payloads at gateway scale without the overhead of full deserialization. Proto's tag-based format is scanned in a single pass. Avro's positional format is navigated with a schema-guided projection reader. Only the fields that matter are decoded.

Proto validation

Wire-format tag scanning reads field numbers and types directly from the binary payload. Required fields are checked, values are type-verified, and everything else is skipped at wire speed.

Avro validation

Schema-guided projection uses the writer schema to navigate to required fields and verify they are present and non-null, without materializing the entire record.

JSON schema validation

JSON payloads are validated against a JSON Schema definition. Required fields are checked for presence and non-null values. Schema is loaded from the registry or local files.

Architecture

Runs in your infrastructure.

Producers
  • HTTP clients
  • Microservices
  • Mobile / IoT
  • Webhooks
HTTP
Sluice Gateway
▸ HTTP ingress axum
✔ Schema validation · Partial deser
⇅ Ordering gateway · Actor pool tokio
■ EventBus trait · Producer pool
publish
Event buses
  • Apache Kafka
  • Google Pub/Sub
  • Kinesis (coming soon)
  • EventHub (coming soon)

Data never leaves your network. The gateway runs in your infrastructure as a container or sidecar.

Install

Get the gateway.

helm install sluice oci://ghcr.io/data-tier/charts/sluice --version 0.2.2 -f values.yaml
docker run -v ./sluice.yaml:/etc/sluice/sluice.yaml \
  -p 8080:8080 ghcr.io/data-tier/sluice:0.2.2
services:
  sluice:
    image: ghcr.io/data-tier/sluice:0.2.2
    ports: ["8080:8080"]
    volumes: ["./sluice.yaml:/etc/sluice/sluice.yaml"]

Quick start.

Configure your topics, start the gateway, and publish via HTTP.

1

Configure sluice.yaml

bus:
  type: kafka
  bootstrap_servers: "localhost:9092"

topics:
  my-events: {}
bus:
  type: pubsub
  project_id: "my-gcp-project"

topics:
  my-events: {}
2

Start

docker run -v ./sluice.yaml:/etc/sluice/sluice.yaml -p 8080:8080 ghcr.io/data-tier/sluice:0.2.2
3

Publish

curl -X POST http://localhost:8080/v1/publish/my-events -d '{"hello":"world"}'

Pricing

Free binary. License key unlocks more.

The gateway binary is free with local schemas and validation. A license key unlocks registry sync, metrics, tracing, and per-topic tuning.

Free

No license key required

  • Publish API (Kafka + Pub/Sub)
  • Proto, Avro, and JSON validation
  • Per-key write-time ordering
  • Local schema files
  • Health check endpoint
  • Structured JSON logs
  • Graceful shutdown with drain timeout
Get started
Pro

License key for production operations

  • Everything in Free
  • Confluent Schema Registry sync (Kafka)
  • GCP topic-attached schema sync (Pub/Sub)
  • Schema hot-reload (polling + file watch)
  • Per-topic producer tuning
    • Kafka dedicated producers
    • Pub/Sub batching overrides
  • Prometheus /metrics endpoint
  • OTLP distributed tracing
  • Request ID / trace ID correlation in responses
Contact sales
Enterprise
Coming soon

License key + control plane

  • Everything in Pro
  • Control plane (topic management UI, fleet observability)
  • RBAC and audit logging
  • Multi-cluster routing
  • Dedicated support and SLAs
Contact sales

Bad data shouldn't reach your topics.

Deploy the gateway in your infra and start publishing with validation and ordering.

Get started Talk to the team