Kafka Trigger
Trigger workflows from Kafka topics
Kafka Trigger
The Kafka trigger allows you to initiate workflows by consuming messages from Kafka topics. It supports consumer groups for distributed processing and includes support for SSL and SASL authentication.
Installation
pnpm add @stepflowjs/trigger-kafkaUsage
import { KafkaTrigger } from "@stepflowjs/trigger-kafka";
const trigger = new KafkaTrigger({
brokers: ["localhost:9092"],
topic: "orders",
groupId: "stepflow-orders",
fromBeginning: false,
});
await trigger.start(async (event) => {
console.log("Received Kafka message:", event.data);
await stepflow.trigger("process-order", event.data);
});Consumer Groups
By specifying a groupId, you can distribute message processing across multiple instances of your application. Kafka will ensure that each partition in the topic is assigned to only one consumer in the group.
Configuration
| Option | Type | Description |
|---|---|---|
brokers | string[] | Kafka broker URLs |
topic | string | Topic to consume from |
groupId | string | Consumer group ID |
clientId | string | Optional client identifier |
fromBeginning | boolean | Optional start from beginning (default: false) |
ssl | boolean | Optional enable SSL |
sasl | object | Optional SASL authentication |
sasl.mechanism | string | plain, scram-sha-256, or scram-sha-512 |
sasl.username | string | SASL username |
sasl.password | string | SASL password |
Event Metadata
The event metadata contains Kafka-specific information:
{
topic: string;
partition: number;
offset: string;
key?: string;
headers?: Record<string, string>;
timestamp: string;
}