Skip to content

feat: add the Streams API for client, Activity and Workflow code - #2129

Draft
moetemp wants to merge 7 commits into
moe/streams-generated-codefrom
moe/streams-api
Draft

moetemp wants to merge 7 commits into
moe/streams-generated-codefrom
moe/streams-api

Conversation

@moetemp

@moetemp moetemp commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What changed?

This PR adds the hand-written part of the Streams API to the Python SDK. It's experimental.

A Stream is an ordered log of Records, for example the tokens of a model call. A client, an Activity or a Workflow writes it, and any of them reads it.

from temporalio_streams_redis import RedisStreamStore

store = RedisStreamStore("redis://localhost:6379")
client = await Client.connect("localhost:7233", stream_config=StreamConfig(stores=[store]))

stream = await client.create_stream(id="chat-43", body_type=str)
await stream.append("Hello", " world")
async for record in stream.read():
    print(record.body)
await stream.close()

The public API is the code that nexgen generates from the Stream WIT, which is in #2128. This PR adds what that code calls.

  • Stores. StreamConfig gives a client its Stores, in the shape of ExternalStorage, and a Worker uses the config of its client. It also has the limits of a batch (max_record_bytes, max_batch_bytes, max_batch_records), as ExternalStorage has payload_size_threshold. StreamStore is the interface of a Store, an abstract class with the five operations. The SDK has MemoryStreamStore for tests. The Redis Store is a separate package, in feat: implement the Stream Store trait on Redis temporal-community/sdk-streams#12.
  • Client and Activity code calls a Store directly. The SDK checks an append against the limits of the config before that call, through Core's check of an append.
  • Workflow code sends an append, a close or a read to Core as a command, and Core calls the same Store through the bridge. A read_page is one read command of Core with one job. A create and a describe run in a Local Activity of the SDK.
  • Fencing. An Activity writes as its attempt (feat: add the dispatching Workflow Task to activity.Info for a Local Activity #2127), so a Store that fences refuses the late writes of an earlier attempt. Each append has a request id, so a batch that is sent again is written once.
  • Errors. StreamError has the reason of a failed call. With StreamReplayError in workflow_failure_exception_types, a replay whose Records are gone fails the Workflow.
  • Helpers. temporalio.contrib.streams has BufferedAppender, which collects the appends of client and Activity code into one write, and current_producer() with as_producer() for an Activity that goes on from its heartbeat details.

It depends on Core's Stream PRs (temporalio/sdk-rust#1817, temporalio/sdk-rust#1801 and temporalio/sdk-rust#1819), which have no release yet. The submodule points to the head of temporalio/sdk-rust#1825, so CI's proto check is expected to fail until a separate PR bumps Core.

Why?

An agent application shows partial output, such as the tokens of a model call, while the work still runs. Workflow History isn't the place for that data, so a Stream keeps its Records in a Store outside History.

These points are open.

  • A Stream call of a client goes through start_nexus_operation of the client interceptors and never reaches the server. So a tracing interceptor sees a Nexus operation for each append.
  • The check of an append is the only check that the SDK runs in front of a Store call of client code. It encodes the batch once more to cross the bridge.
  • The payload visitor marks a Stream body with COMMAND_TYPE_UNSPECIFIED, since Core's Stream commands have no command type of the server.
  • A create and a describe of Workflow code run in a Local Activity of the SDK. An interceptor that changes every Local Activity changes this one too.
  • A read that waits holds its Workflow Task open, so a Signal can wait until the heartbeat of the task.
  • The last commit turns chunking of Workflow Tasks off for every new Workflow, with Core's feature disable_workflow_task_chunking of Add opt-in workflow task chunking shutoff sdk-rust#1659. With chunking, a replay can give two results in one activation that the first run gave in two. A read that waits past the heartbeat makes that common, and the released SDK fails the replay of a Workflow with two Local Activities in that shape. A Workflow that started before keeps chunking. The commit is separate, so it can move to a separate PR.
  • The docstring of StreamStore repeats the contract of the Rust trait in temporalio-streams, which is its source.

How did you test it?

The tests are in tests/nexus/test_streams*.py, tests/worker/test_workflow_streams.py, tests/testing/test_stream_store.py and tests/contrib/streams/. A property runs over seeded inputs against a model, and the Worker tests run on the test server with a replay.

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

@moetemp
moetemp added this pull request to stack #2130 November 14, 2026 01:00
@moetemp
moetemp removed this pull request from stack #2130 November 15, 2026 08:27
@moetemp moetemp changed the title feat: add the Streams API, with Stream Stores on Redis and in memory feat: add the Streams API for Activity and client code, with Stores on Redis and in memory Nov 17, 2026
@moetemp
moetemp added this pull request to stack #2139 November 18, 2026 01:19
@moetemp moetemp changed the title feat: add the Streams API for Activity and client code, with Stores on Redis and in memory feat: add the Streams API for Activity and client code Nov 18, 2026
@moetemp moetemp changed the title feat: add the Streams API for Activity and client code feat: add the Streams API for client, Activity and Workflow code Nov 22, 2026
@moetemp
moetemp force-pushed the moe/streams-api branch 2 times, most recently from 4f48bab to 066b280 Compare November 22, 2026 03:31
@moetemp
moetemp force-pushed the moe/streams-api branch 2 times, most recently from c7c3f7c to 824202b Compare November 22, 2026 10:25
The Workflow path of Streams needs the Stream commands and jobs of this
Core. The regenerated payload visitor reaches their payloads, so the
rule for a Stream body is in this commit.
A Stream Store is the handler side of the Stream service, which the
generated code doesn't have. The memory Store lets a test use Streams
with no other service, behind the checks that Core runs for a Store of
Workflow code.
The server has no Streams yet, so the last client interceptor sends each
Stream call to a Store of the client's Stream config. The generated
Stream code also needs its helpers, with the three that build a
Producer.
Workflow code can't call a Store itself, since a replay must get the
same answers. Core makes the append and the close inside the Workflow
Task and records each result in a marker. A create and a describe have
no command in Core, so they run in a Local Activity of the SDK.
A read of Workflow code is one command of Core with one job, as a Local
Activity is, so no state of a read depends on where an activation ends.
A replay reads the Records again from the Store, so a replay Worker gets
the event loop that its Python Stores need.
An Activity that appends each model token pays one Store call for each
token. The appender batches them with one write in flight, and stops at
the first failed batch so that a reader sees no gap.
A Stream read that waits past the heartbeat of its Workflow Task
completes that task with no command, and with chunking a replay takes
that task and the next one as one. Workflow code then gets two results
in one activation that the first run gave in two, and can run in another
order, as it can with Local Activities alone. Core records the choice in
the first completed Workflow Task of a Workflow, so a Workflow that
started before keeps chunking.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant