Repository navigation
Conversation
renkun-ken
left a comment
There was a problem hiding this comment.
Two cases still need protection before merge. A reconnected older agent can terminate an adopted terminal process, and a reused socket path can direct shutdown to a different R session. The inline comments also cover queue recovery, concurrent Stop requests, and agent cleanup after a delayed exit.
TypeScript compiled and 93 focused Node tests passed at f0c985d. I verified the upgrade case separately with both Stop entry points against the previous agent, intercepting its process signals.
Stop of an attached arf fell back to SIGTERM and SIGKILL when arf rejected shutdown, as an interactive arf does, and after shutdown was accepted but arf had not exited. An interactive arf then did not restore its terminal or finish R's exit, and the PID could have been reused by another process. An adopted arf is now only asked to shut down. Stop is refused when arf rejects the request, when the endpoint is unknown, and when it is forced; the session, its queue and its status stay as they were, and a workspace refresh deferred while stopping resumes. When arf accepts but does not exit in time, Stop reports that, and the later exit is still recorded. Queued work is cancelled once Stop is not refused, instead of before it is attempted. Stop of several sessions waits long enough for an adopted arf to answer.
…sed Stop An agent from an earlier extension version still terminates an adopted arf on Stop, so clients now ask an agent to stop an adopted session only when it advertises that it does not. Shutdown went to the saved socket path, which another arf session can bind while the adopted R is still alive. It is now sent over a new attach connection that reached the same session. When that connection cannot be opened, nothing was sent, so Stop is refused and the queue is kept, as after `:ipc stop` in the terminal. Overlapping Stop requests now share the outcome of the one in flight. After a refused Stop, the workspace is read before the next queued cell runs. An agent whose explicitly stopped runtime exits after every client has left shuts down, as the last client would have made it.
f0c985d to
cb70fdb
Compare
renkun-ken
left a comment
There was a problem hiding this comment.
I rechecked the six earlier findings on cb70fdb. Both Stop commands now protect sessions served by older agents. The shutdown request uses an identity-verified connection. With real arf 0.7.2 on macOS, terminal arf refused Stop and stayed alive, and headless arf exited with code 0 and no signal. The new tests also cover overlapping requests, workspace refresh ordering and cleanup after a late exit.
I’m requesting changes to keep cells queued after a verification failure and to make both initial and repeated Stop requests report the actual shutdown outcome. TypeScript compiled, and all 99 focused Node tests passed.
| // An interactive arf rejects shutdown; stop it as a terminal would. | ||
| await this.kill('SIGTERM', 3000); | ||
| return; | ||
| throw new StopRefusedError( |
There was a problem hiding this comment.
[P2] Hold queued cells while the original arf endpoint is unavailable. This catch turns a verified session-ID mismatch or a failed connection into the same refusal as a live terminal declining Stop. runStop() consequently calls pump(). If the old attachment closed, the first queued cell is removed, reattachment fails before evaluate is sent, and the agent records the cell as unknown. I verified that queue-to-unknown transition with the real agent and a mocked pre-send dispatch failure. Please keep the cell queued until the original session can be verified again. The replacement-endpoint test could also assert the execution record after refusal.
| } | ||
| if (this.stopping || this.manifest.status === 'exited') { | ||
| return; | ||
| return Promise.resolve(); |
There was a problem hiding this comment.
[P2] Report that shutdown is still pending on a later Stop. Once an accepted shutdown misses the exit deadline, finally clears stopRequest but this.stopping stays true. A second Stop from the open notebook then takes this branch and resolves immediately. I reproduced a successful second reply with R still running, the manifest still stopping, and just one backend Stop call. Return a pending-exit error or wait for the monitored exit. Immediate success is appropriate once the manifest records exited.
| await client.request('stop', {}, 5000); | ||
| // The agent answers once the runtime has exited or declined. An adopted | ||
| // arf can take up to 5 s to answer shutdown and 5 s more to exit. | ||
| await client.request('stop', {}, 15000); |
There was a problem hiding this comment.
[P2] Give both Stop commands time to receive the agent’s result. The new path has separate five-second waits for attach, shutdown acknowledgement, and process exit. Both this call and single-session Stop use a 15-second client timeout. With an already-ready agent and a fixture answering each IPC request after 4,995 ms, the client reported a generic timeout at 15,003 ms. The backend reported that arf had accepted shutdown but was still running at 15,007 ms. Please use a shared Stop timeout with room for all three waits, exit polling and reply delivery, and update the comment to include the attach handshake.
…report pending Stop A cell whose dispatch failed before its code was sent, because the attach connection could not be opened again or the endpoint no longer reached the adopted session, was recorded as unknown. It now stays first in the queue and is retried with a bounded backoff. After a Stop found that the endpoint did not reach the adopted session, or after a send could not open its connection, code is sent only once a new connection reaches the same session again. A cell found unsent after Stop cancelled the queue is cancelled. Once an accepted shutdown missed its exit deadline, a later Stop reported success while R was still running. It now reports that R has not exited yet, and succeeds once the exit is recorded. Clients waited 15 seconds for Stop, which an adopted arf can need for verifying a new attach connection, answering shutdown and exiting. All Stop requests now share a 20-second timeout.
renkun-ken
left a comment
There was a problem hiding this comment.
The pending-Stop and client-timeout findings are fixed at 1869d2b. Single and bulk Stop both returned the backend’s pending-exit error after 15.1 seconds, a later Stop continued to report it while R was alive, and both commands succeeded after exit. TypeScript compiled and all 104 focused Node tests passed.
I’m requesting changes on the automatic retry path. The connection that receives a queued cell must be the one whose identity was verified, retries need to find the original process after its IPC path changes, and cancelling queued work must prevent a later retry from executing it.
| try { | ||
| await this.attached(); | ||
| if (this.reverify) { | ||
| (await this.verifiedConnection()).close(); |
There was a problem hiding this comment.
[P1] Send the queued cell over the verified connection. This closes the connection whose session ID just matched. arfRequest() then opens a fresh connection to this.endpoint for evaluate. If the path changes owners between those operations, the queued cell can reach a different R process. Please keep the verified connection through delivery, as the Stop path already does for shutdown. arf supports visible evaluate on that stream. A regression test can assert that the verified connection receives the evaluation and that dispatch opens no second connection. Upstream protocol test
| // state; the session must still be the same arf process. | ||
| await this.attached(); | ||
| try { | ||
| await this.attached(); |
There was a problem hiding this comment.
[P2] Rediscover the socket when the original arf restarts IPC. I tried :ipc stop and :ipc start in one real arf 0.7.2 terminal on macOS with the default IPC settings. The PID and session ID stayed the same, but arf registered a new socket path and the old one disappeared. Every retry here still opens the saved this.endpoint, so the queue can wait indefinitely after R is reachable again. Use the discovery record to locate the current socket, verify its session ID, and resume on that connection. The existing fixture only resumes listening at the old path.
| // R has not received the cell, so it stays first in the queue | ||
| // until the session can be reached again. | ||
| this.current = undefined; | ||
| this.queue.unshift(id); |
There was a problem hiding this comment.
[P2] Honor Cancel Queued Cells during a retry. pump() temporarily removes the cell from queue while it connects, so cancelQueued() misses it. With the real agent and a mocked backend, I cancelled during the second attempt while the journal still said queued and no code had been sent. That attempt failed with DispatchNotSentError, this line put the cell back, and a third attempt executed it. Please remember the cancellation decision for the in-flight attempt and finish the cell as cancelled if the backend confirms it was unsent.
The same accounting matters on exit. If R exits before that confirmation arrives, the exit handler clears current and leaves the cell unknown, and the guard above drops the definitive unsent result. Please cover both orderings while preserving unknown for a genuinely uncertain send.
…and honor cancellation while sending A cell for an adopted arf was evaluated over a new connection to the saved socket path, which another arf can serve once the attach connection has verified the session. It is now evaluated on that attach connection. A session that must be verified again is reached by a new attach, which checks the session before anything is sent. A cell is taken off the queue while it is being sent, so Cancel Queued Cells, Stop and R's exit missed it, and a retry could then run it. Its cancellation is now kept for the attempt in flight, and a cell found unsent is cancelled. A cell whose send R's exit leaves uncertain is still recorded as unknown. After R exits, the agent closes once that outcome is recorded, for up to 5 seconds, whether the last client leaves, shutdown is requested or no client remains.
renkun-ken
left a comment
There was a problem hiding this comment.
Adopted cells now evaluate on the connection that verified the arf session. With real arf 0.7.2 on macOS, an ordinary cell ran, a running cell accepted an interrupt, and headless Stop exited cleanly. TypeScript compiled and all 108 focused Node tests passed.
The earlier socket rediscovery finding is still open. I repeated the test on this commit with one real terminal arf. After :ipc stop and :ipc start, the PID and session ID matched at the new socket, but two dispatch attempts both returned DispatchNotSentError for the deleted original path. Please rediscover the current endpoint and verify the original session ID before resuming. Existing review thread
The inline findings cover cancellation when reconnection succeeds, interrupt routing, and two event orderings during agent cleanup.
| return; | ||
| } | ||
| if (error instanceof DispatchNotSentError) { | ||
| if (this.cancelledDispatch === id) { |
There was a problem hiding this comment.
[P2] Honor cancellation before a successful reconnect sends the cell. cancelledDispatch is only checked when dispatch rejects. In a same-session test, I held the retry at attached(), invoked Cancel Queued Cells while the record was queued and zero evaluations had been sent, then let the handshake succeed. The adapter sent evaluate and the cancelled cell finished successfully. Carry cancellation into the pre-send phase and check it immediately before writing evaluate. A regression test should let the handshake succeed and assert that the backend sends zero evaluations.
| this.abort.signal, | ||
| ), | ||
| this.process.adopted | ||
| ? await connection.request('evaluate', params, { signal: this.abort.signal }) |
There was a problem hiding this comment.
[P2] Route Interrupt to the connection that owns the evaluation. Adopted evaluate now stays on this connection, but sendInterrupt() still opens a fresh HTTP connection to this.endpoint. When the saved path changes owners while this attachment remains open, Interrupt can miss the running cell or interrupt an unrelated execution with the same process-local ID. Keep the verified connection with the adopted execution and use it for both immediate and deferred interrupts. The attach protocol supports concurrent requests. arf protocol
| const current = this.current && this.journal.executions.get(this.current); | ||
| // A cell whose dispatch is in flight may not have been sent; | ||
| // the dispatch settles its outcome. | ||
| if (current && !(this.dispatching && current.state === 'queued')) { |
There was a problem hiding this comment.
[P2] Keep a confirmed process exit terminal while recording late cell events. Retaining current here lets a buffered started or finished event update the session status after exited was published. I reproduced a heartbeat reporting unknown with an exit timestamp already recorded. During Stop, the same update can leave the manifest at stopping, so a later Stop reports that R is still alive. Keep exited as the session status while using late events to settle the cell’s journal record. Cover the exit-before-started and exit-before-finished orderings.
| if (current && !this.stopped) { | ||
| this.finish(current, 'unknown'); | ||
| } | ||
| return this.close(); |
There was a problem hiding this comment.
[P2] Recheck for clients before automatic cleanup closes the agent. During this wait a new client can still connect and open the retained transcript. I verified that a second client connected, subscribed, and read the exited snapshot after the first client left. When dispatch settled, this unconditional close() disconnected it. Recheck clients.size after the wait for automatic last-client cleanup, while keeping explicitly requested agent shutdown unconditional.
…on, and decide each send at its write Requests to an adopted arf went partly over the attach connection that verified the session and partly over new connections to its socket path, which another arf can serve. Interrupt, the R environment read and the bootstrap now go over the attach connection too. An adopted execution keeps the connection that sent it, and its interrupt, sent at once or once arf reports its ID, uses that connection, or fails if it has closed. Whether a cell is sent was decided in several places after the fact, so each combination of Cancel Queued Cells, Stop, exit and retry could send or lose it. A dispatch now carries a cancellation signal that the backend checks just before writing the code. Cancelling, or R's exit, cancels a cell not yet written; Stop holds it, and it runs again if Stop is refused. A confirmed exit is now final for the session's status: later events of a cell only settle its record. After R exits, the agent closes on its own only if no client has connected while it waited for a dispatch.
…opted session again after its IPC restarts Discovery read arf's session files from the Linux and macOS cache directories, so it found nothing on Windows, and an adopted session whose IPC restarted could not be reached again: arf then registers a new socket path, keeping its PID and session ID, and the saved path is gone. Running sessions are now listed with `arf ipc list`, without a shell, with a deadline that closes the output pipes and with an output limit. A listing that cannot be obtained is reported instead of showing no sessions. A listed session is offered only when its endpoint answers for the listed PID and start time, and its session ID and PID go to the agent, whose attach must reach the same arf. The agent keeps the arf CLI and the directory that listed the session. When the saved path cannot be attached, the session is listed again and queued code resumes at a listed path whose attach reports the same PID and session ID. Stop does the same on a connection of its own, within 5 seconds, so that its shutdown and exit fit within its client timeout.
Stop of an attached arf session fell back to SIGTERM and then SIGKILL when arf rejected
shutdown, as an interactive arf does, and when arf accepted it but had not exited after a few seconds. An interactive arf then could not restore its terminal or finish R's exit, and the PID could already belong to another process.Requests to an attached arf also went to its saved socket path, which another arf session can bind while the attached R is still running. Discovery read arf's session files from the Linux and macOS cache directories, so it found no sessions on Windows, and a session whose IPC restarted at a new socket path could not be reached again.
An attached arf process is now never sent a signal or killed, and every request to it goes over a connection that reached the verified session.
Stop
shutdowngoes over a new attach connection that reached the same arf session. When that connection cannot be opened, nothing was sent, so Stop is refused and the queue is kept.Requests to an attached arf
Sending cells
Discovery
arf ipc list, run without a shell, with a deadline and an output limit. The deadline holds even when the CLI handles SIGTERM and keeps running, and the output pipes are closed when it passes. A listing that cannot be obtained is reported instead of showing no sessions, and the session picker then offers to configure arf; the R Interactive output shows the details. An untrusted workspace does not run the CLI.ARF_IPC_SESSIONS_DIRmust be set to the same absolute path for arf and for the extension host. The documentation asks for a custom sessions directory to be private to the user, as arf requires.:ipc stopand:ipc start, keeping its PID and session ID. When the saved path cannot be attached, an attached session is listed again with the arf CLI and directory that listed it, and queued code resumes at a listed path whose attach reports the same PID and session ID. Stop does the same on a connection of its own, within 5 seconds.This changes behavior on Linux and macOS too: an attached arf that runs in a terminal can no longer be stopped from VS Code.
This is also a step toward connecting to existing arf sessions on Windows.
Where Node can only terminate a process forcibly, so the previous fallback would have killed arf outright on Windows.