Repository navigation
Conversation
The Workflow path of Streams needs the Stream commands and jobs of this Core. The regenerated payload visitor reaches their payloads, so the rule for a Stream body is in this commit.
A Stream Store is the handler side of the Stream service, which the generated code doesn't have. The memory Store lets a test use Streams with no other service, behind the checks that Core runs for a Store of Workflow code.
The server has no Streams yet, so the last client interceptor sends each Stream call to a Store of the client's Stream config. The generated Stream code also needs its helpers, apart from the two that build a Producer.
Workflow code can't call a Store itself, since a replay must get the same answers. Core makes the append and the close inside the Workflow Task and records each result in a marker. A create and a describe have no command in Core, so they run in a Local Activity of the SDK.
A read of Workflow code is one command of Core with one job, as a Local Activity is, so no state of a read depends on where an activation ends. A replay reads the Records again from the Store, so a replay Worker gets the event loop that its Python Stores need.
An Activity that appends each model token pays one Store call for each token. The appender batches them with one write in flight, and stops at the first failed batch so that a reader sees no gap.
Owner
Author
|
Moved to temporalio#2129. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the hand-written part of the Streams API, in six commits to read in order.
It's open on the fork so that CI runs on the whole stack. The base has the generated API. The Core submodule points at a commit of
moetemp/sdk-rust, since the Stream calls of Core aren't merged yet.StreamStoreis the interface of a Store, andStreamConfigon the client says which Stores a client and its Workers have.MemoryStreamStoreis a Store for tests.read_pageis one read command of Core with one job.temporalio.contrib.streamshasBufferedAppender, which batches the appends of client and Activity code.