Skip to content

About

Apache Pulsar Rust client library

Topics

Resources

Code of conduct

Security policy

Stars

2 stars

Watchers

1 watching

Forks

Repository files navigation

Apache Pulsar Rust client

A Rust client for Apache Pulsar scalable topics, built from the Scalable Topics client specification on the compio runtime.

Status: milestones 3 to 5. Producers publish to scalable topics, keeping per-key order across segment splits and merges, and to regular persistent:// topics through the migration bridge. Queue consumers read every segment of a topic, following its splits and merges, and acknowledge each message. Stream consumers read the segments the topic's controller assigns them, each key in order, share segments by entry bucket when they outnumber them, and acknowledge cumulatively. Checkpoint consumers read every segment, parents before their children, or as members of a group the segments the controller assigns them, and resume from checkpoints the application keeps, in the Java client's bytes. All work in the thread-safe, local and blocking APIs, with token and mTLS authentication, directly or through a proxy. Producers batch messages by entry bucket and compress them, as Java's do. The design is in DESIGN.md, the milestones in PLAN.md.

let client = PulsarClient::builder().service_url("pulsar://localhost:6650").build().await?;
let producer = client.new_producer(Schema::string()).topic("topic://public/default/orders").create().await?;
let id = producer.new_message().key("user-42").value("hello".to_string()).send().await?;
producer.close().await?;

let consumer = client.new_queue_consumer(Schema::string())
    .topic("topic://public/default/orders")
    .subscription_name("workers")
    .subscribe().await?;
let message = consumer.receive().await?;
consumer.acknowledge(message.id()).await?;
client.close().await?;

Layout

Path What it is
crates/apache-pulsar The client: producers, Queue, Stream and Checkpoint consumers, layout watches, connections, the pool, lookup, authentication, TLS, the memory limit, and the thread-safe, local and blocking APIs.
crates/apache-pulsar-protocol The sans-IO protocol core: generated protobuf code, framing, the connection, producer and consumer state machines, entry decoding, layouts and routing, message ids. Internal; not a stable API.
crates/apache-pulsar-testkit The simulated broker and cluster, and the deterministic tests of the protocol state machines. Not published.
checks/ Milestone 0 checks: compio behavior, and the handshake against a real broker.
tools/java-vectors Generates test vectors from the Java client.
xtask Repository tasks: protobuf code generation.

Building and testing

cargo test --workspace

The generated protobuf code is committed, so building needs no protoc. After changing the vendored crates/apache-pulsar-protocol/proto/PulsarApi.proto, regenerate it (needs protoc 33.1, the version CI checks with):

cargo run -p xtask -- codegen

The handshake against a real broker:

docker run -d --rm --name pulsar -p 6650:6650 -p 8080:8080 apachepulsar/pulsar:5.0.1 bin/pulsar standalone
cargo run -p first-contact -- 127.0.0.1:6650

The integration tests start a broker and a proxy in Docker, with TLS, token and mTLS authentication, and remove them when they finish. A Java tool they run in the broker container is compiled in an eclipse-temurin JDK container. cargo test skips them; they need Docker running:

cargo test -p apache-pulsar --test integration

License

Apache License 2.0.

About

Apache Pulsar Rust client library

Topics

Resources

Code of conduct

Security policy

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages