Skip to Content
eventkit is in active development. APIs may change.
Advanced

Advanced

Advanced patterns, production deployment, and performance tuning for eventkit.

Topics

Architecture Patterns

  • Dual-Path Architecture - Process events through multiple coordinators (coming soon)
  • Performance Tuning - Optimize for throughput and latency (coming soon)
  • Monitoring & Observability - Prometheus metrics and logging (coming soon)

Production Deployment

  • Cloud Run - Serverless deployment (coming soon)
  • GKE/Kubernetes - Container orchestration (coming soon)
  • Horizontal Scaling - Multi-instance patterns (coming soon)

Performance

eventkit is designed for high throughput with low latency:

Throughput:

  • 10,000+ events/second validated (single instance)
  • Horizontal scaling with PubSubQueue mode
  • Adaptive batching reduces storage writes

Latency:

  • p50: less than 25ms (collection endpoint)
  • p95: less than 100ms (target)
  • Ring buffer adds less than 1ms overhead

Benchmarks:

pytest tests/performance/ --benchmark-only

See specs/performance-benchmarks/ for detailed results.

Ring Buffer (Write-Ahead Log)

eventkit uses a ring buffer for durability - events are never lost even if the service crashes.

Architecture:

Why SQLite?

  • Local durability (no network on hot path)
  • WAL mode for concurrent reads/writes
  • Zero dependencies
  • Production-proven pattern

Configuration:

export EVENTKIT_RING_BUFFER_DB_PATH="./eventkit_ring_buffer.db"
export EVENTKIT_RING_BUFFER_MAX_SIZE="100000"
export EVENTKIT_RING_BUFFER_RETENTION_HOURS="24"

Monitoring

eventkit exposes Prometheus metrics on port 9090:

curl http://localhost:9090/metrics

Key Metrics:

  • Event throughput (events/sec)
  • Queue depth and processing lag
  • Storage write latency
  • Error rates

Grafana Dashboards:

# Event processing rate
rate(eventkit_events_processed_total[5m])

# Queue backlog
eventkit_queue_depth

# Error rate
rate(eventkit_events_failed_total[5m]) / rate(eventkit_events_received_total[5m])

Custom Adapters

Implement custom validation and transformation logic:

from eventkit.adapters.base import SchemaAdapter, AdapterResult
from eventkit.schema.raw import RawEvent
from eventkit.schema.events import TypedEvent
 
class MyCustomAdapter(SchemaAdapter):
    def adapt(self, raw: RawEvent) -> AdapterResult[TypedEvent]:
        # Custom validation
        if not self._is_valid(raw.payload):
            return AdapterResult.failure("Invalid format")
 
        # Custom transformation
        event = self._transform(raw.payload)
        return AdapterResult.success(event)
 
    def _is_valid(self, payload: dict) -> bool:
        # Your validation logic
        return True
 
    def _transform(self, payload: dict) -> TypedEvent:
        # Your transformation logic
        return TypedEvent(...)

Error Handling

eventkit never drops events - validation failures go to the error store:

# Invalid events are stored for debugging
errors = await error_store.query_errors(limit=100)
for error in errors:
    print(f"Error: {error['error']}")
    print(f"Payload: {error['payload']}")
    print(f"Timestamp: {error['timestamp']}")

Common Error Patterns:

  • Missing required fields
  • Type mismatches
  • Invalid timestamps
  • Unknown event types

Security

Authentication & Authorization: eventkit doesn’t provide built-in auth. Use a reverse proxy (Cloud Run, API Gateway) for authentication.

PII Handling:

  • Never log full event payloads
  • Use structured logging with allowlists
  • Consider field-level encryption for sensitive data

Network Security:

  • Run behind load balancer (TLS termination)
  • Use VPC for internal services
  • Enable Cloud Armor for DDoS protection

Next Steps

Last updated on
IntroductionAPI Reference