Skip to content

[Heap2Local2] Add a graph-based escape analysis pass - #9391

Open
tlively wants to merge 21 commits into
mainfrom
heap2local-rewrite
Open

tlively wants to merge 21 commits into
mainfrom
heap2local-rewrite

Conversation

@tlively

@tlively tlively commented Nov 13, 2026

Copy link
Copy Markdown
Member

Add a new Heap2Local2 pass (--heap2local2) structured around a four-phase dataflow graph analysis over all allocations in a function at once: constructing an AST dataflow graph, computing a smallest fixed point over Flat origins, propagating escapes through identity merges and conditional cmpxchg edges, and rewriting non-escaping allocations to local variables in a single post-order pass. Also register --heap2local2 in the pass registry and fuzzer alongside unit and lit test suites for general flows, custom descriptors, and atomic RMW/cmpxchg operations.

Add a new Heap2Local2 pass (--heap2local2) structured around a four-phase dataflow graph analysis over all allocations in a function at once: constructing an AST dataflow graph, computing a smallest fixed point over Flat<AllocId> origins, propagating escapes through identity merges and conditional cmpxchg edges, and rewriting non-escaping allocations to local variables in a single post-order pass. Also register --heap2local2 in the pass registry and fuzzer alongside unit and lit test suites for general flows, custom descriptors, and atomic RMW/cmpxchg operations.
@tlively
tlively requested a review from kripken November 13, 2026 20:14
@tlively tlively changed the title [Heap2Local2] Prototype graph-based escape analysis [Heap2Local2] Add a graph-based escape analysis pass Nov 13, 2026
@tlively

tlively commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

To see a slightly nicer diff of the tests, look at just the first two commits (i.e. exclude Revert "in-place test changes") to see how the new output differs from the old output.

Comment thread src/passes/Heap2Local2.cpp Outdated
// Metadata and lowered local variable assignments for a candidate allocation.
struct AllocInfo {
Expression* expr = nullptr;
Type type;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is type not just expr->type?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good point. I removed the field.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I further removed the descHeapType field.

Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Outdated
// Casts do not escape their inputs, but depending on the input origin,
// descriptor reachability, and target subtype check they either propagate an
// allocation origin, trap (Bot), or produce Top, so Phase 2 must re-evaluate
// them as input origins are discovered rather than using plain identity edges.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a significant benefit to this handling of casts? I mean that a cast flows through a value, like the last element of a Block, and we could in theory treat casts that way. Looking at casts in more detail, like seeing if it must trap given its inputs, can help in some cases, but is that common?

If it does help, I wonder if it could be left to a followup, keeping the original PR more minimal?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary to reach parity with the existing Heap2Local. Casts and cmpxchg handling could be left to follow-on PRs if we preferred that over reaching parity right away, but I don't think it would make much of a difference. Let me know if you would prefer that.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think reaching parity immediately is a top goal? So long as we know we can reach it after a series of PRs. But if it adds significant work to split it up, I can review this larger PR.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split out cast and cmpxchg handling.

Comment thread src/passes/Heap2Local2.cpp Outdated
struct CmpxchgSite {
NodeId expNode;
NodeId repNode;
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar feelings about cmpxchng - can be useful, but I'm not sure how much, and if it were me I'd've left it to a followup?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread src/passes/Heap2Local2.cpp Outdated
return it != exprNodes.end() ? it->second : TopNode;
}

void seedOrigin(Expression* curr, Origin val) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the word "seed" refer to here? I'm not sure if it is a metaphor or a technical term.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it does a join, could this be joinOrigin?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"seed" in the sense of "initialize."

I could change this to setOrigin, joinOrigin, or probably even just join.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to join and updated comments to avoid using the term "seed."

Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Outdated

// Lightweight pre-pass that stops as soon as any candidate allocation exists.
// Most functions contain no candidate allocations, so this avoids paying
// ControlFlowWalker and AnalysisState hash-map overhead for them.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that significant overhead? I would guess it's not that bad, and this code can be saved, but I could be wrong...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the same question, and I was still surprised at how helpful this was. On fully optimized calcworker, this bails out early on 69% of functions and decreases pass latency by about 6%. That being said, the absolute difference there is only about 20ms. The effect is much higher on unoptimized binaries or when running on just one core, but I'd be fine going either way on this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is "pass latency" - the speed of this pass, or the entire pipeline?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just this pass.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this pass's speed compare to others? I would be interested in a 6% speedup of one of our slower passes, but Heap2Local wasn't slow iirc. If this is one of our faster passes, 6% doesn't seem worth a bunch of code.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is still a fairly fast pass. I've removed this code in favor of a simpler early bailout.

Comment thread src/passes/Heap2Local2.cpp Outdated
Function* func;
Module& wasm;
// Whether any local.set writes a reference, used to skip LazyLocalGraph
// construction when no reference locals are written.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constructing a LazyLocalGraph is essentially free (that is the benefit of laziness), so this may be a premature optimization?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry, the comment was inaccurate here. The savings came not from skipping LazyLocalGraph construction, but from avoiding calls to getSets() when we already knew all reference-typed gets must read from default values or params. However, the effect here was really tiny, so I've removed this fast path.

Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Outdated

AllocId addCandidateAlloc(Expression* curr, const FieldList& fields) {
assert(state.allocs.size() < std::numeric_limits<AllocId>::max());
AllocId id = AllocId(state.allocs.size());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
AllocId id = AllocId(state.allocs.size());
auto id = AllocId(state.allocs.size());

To avoid repetition?

Also, should we have a helper getAllocId, rather than baking the logic of "the next id is the current size" in each place?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to auto. This is the only place that allocates AllocIds, so I don't think a helper would help.

// TODO: Track field, element, and descriptor edges to optimize nested
// allocations.
for (auto* op : curr->operands) {
state.escapeIfRef(op);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this handle the case where the struct.new does not escape, so the operands do not either? It looks like it marks the operands as escaping, regardless.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we always mark operands as escaping. There will be a follow-up that handles nested allocations, but that's not necessary for parity with Heap2Local.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heap2Local can handle nested allocations by optimizing them in sequence, so I think it is necessary for parity? But sgtm to leave that for later.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, fair. Heap2Local optimizes nested allocations in very particular cases where the outer allocation is an array.new and the inner allocation is a struct.new or when both allocations are struct.news but the outer one comes first and the inner one is written into it later. This pass will handle nested allocations much more generally once we get there.


// Block and Loop fallthroughs are wired directly rather than via
// Properties::getImmediateFallthrough, which conservatively rejects all named
// blocks and loops.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have a "dataflow" helper that handles this, which e.g. GUFA could use too. Maybe we can look into that later sometime.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that might make sense.

Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Resolved
Comment thread src/passes/Heap2Local2.cpp Outdated
for (auto* op : curr->operands) {
state.escapeIfRef(op);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than require a full list of all instructions that escape, how about making the default behavior to escape? That would handle all new instructions, and avoid risks of bugs if we forget something.

Look at how Checker does this in Heap2Local: the default is escapes=true and all non-escaping cases must be added, safely.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already the default; see visitExpression.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused - if that is the default, why do we need visitCall, visitReturn, etc here? Don't they do the same as the default?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, good point. Implementing those directly was just a micro-optimization to avoid the overhead of ChildIterator. Removed.

// Phase 2: Solve smallest fixed point over Flat<AllocId> origins.
void computeOrigins(AnalysisState& state) {
size_t numNodes = state.origins.size();
// Build a compressed-sparse-row (CSR) forward-adjacency table from

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this, Goose code? 😉

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More seriously, if we want to optimize data structures at this level, can we add a helper for it rather than put a bunch of Data Structures code into this pass? Could be a followup though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll investigate the benefit of a shared utility as a potential follow-up.

Comment thread src/passes/Heap2Local2.cpp Outdated
}
};

// Phase 2: Solve smallest fixed point over Flat<AllocId> origins.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Phase 2: Solve smallest fixed point over Flat<AllocId> origins.
// Phase 2: Solve smallest fixed point over Flat<AllocId> origins. That is,
// find which allocation each Expression belongs to (possibly none,
// possibly many in which case we give up).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an edited version of this.

Comment thread src/passes/Heap2Local2.cpp Outdated
}

// Phase 3: Compute escaped allocations and return whether any remain
// unescaped and can be optimized.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// unescaped and can be optimized.
// unescaped and can be optimized. Return true if we found something to
// optimize (i.e. which does not escape).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an edited version.

Comment thread src/passes/Heap2Local2.cpp Outdated
// non-escaping allocation, lowers allocations to local sets returning a bottom
// ref.null, rewrites uses, and finishes with ReFinalize (which widens
// control-flow and br_if types to allow the ref.null values) and
// handleNonDefaultableLocals.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is maybe too specific in details. Each of the sentences
after the first could appear inside the function? But I don't feel strongly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edited.

return PostWalker<Optimizer>::replaceCurrent(rep);
}

std::optional<AllocId> getOptimizedAlloc(Expression* curr) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment to this function, I'm not quite sure what it does.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread src/passes/Heap2Local2.cpp Outdated
}

// If `curr` is an optimizable allocation, return its ID; otherwise return
// nullopt.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't an "optimizable allocation" exactly "evaluates exclusively to a single non-escaping candidate allocation", which is what the previous function does? I'm not clear on the difference between these two.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function checks whether the arg is an optimizable allocation. The previous function checks whether the runtime value of the arg comes from an optimizable allocation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, then can this function contain assert(getOptimizedAlloc()) in the case of returning a non-nullopt result? That is, one is a subset of the other?

Also perhaps can we find a different name? "candidate" vs "alloc" do not clarify enough the difference here, I feel?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading the below, these are accessed disjointly and it seems safe to merge the two? Or do any of the callers care about the distinction?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged them, and that even let us get rid of the separate allocIDs map!

}

void visitRefAs(RefAs* curr) {
if (curr->op == RefAsNonNull && getOptimizedAlloc(curr->value)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (curr->op == RefAsNonNull && getOptimizedAlloc(curr->value)) {
if (getOptimizedAlloc(curr->value)) {
assert(curr->op == RefAsNonNull);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current code does the cheap check first before doing map lookups, etc, and I don't think there's much difference in clarity between the two. Do you think this change would be worth it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel strongly. It just seems slightly less good to seem to appear to allow a non-RefAsNonNull to be optimized. sgtm as is.

Comment thread test/lit/passes/heap2local2-nogc.wast Outdated
;; CHECK-NEXT: )
;; CHECK-NEXT: )
(func $non-gc (param $x i32) (result i32)
;; Without GC enabled, Heap2Local2 returns immediately.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test seems unnecessary. I guess it adds coverage but other tests will do that anyhow, once it is in the main pipeline (any test with GC enabled and -O3 etc.)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, removed.

@kripken kripken left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(finished reading code, only minor comments. next for tests)

;; HP2L2-NEXT: (local.get $desc)
;; HP2L2-NEXT: )
;; HP2L2-NEXT: )
;; HP2L2-NEXT: )

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does heap2local optimize here but not heap2local2?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we removed cast handling to make the pass smaller :)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see... my fault then 😄

;; HP2L2-NEXT: )
;; HP2L2-NEXT: (local.get $2)
;; HP2L2-NEXT: )
;; HP2L2-NEXT: )
(func $two-branches-b (result f64)
(local $0 (ref null $struct.A))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment in this testcase can be updated - heap2local2 does better, implementing a TODO

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it generally makese sense to update comments now or later when we remove the original Heap2Local. I wouldn't want to audit and update all the comments now to reflect the state of both passes, then have to update them again when we remove Heap2Local.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I was just thinking that in a racing comment below... fair enough.

(f64.const 1.5)
)
)
;; Merging the same allocation across both arms of an if still optimizes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
;; Merging the same allocation across both arms of an if still optimizes.
;; Merging the same allocation across both arms of an if still optimizes (only in heap2local2).

(f64.const 2.5)
)
)
;; Merging the same allocation across select optimizes cleanly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here

;; HP2L2-NEXT: )
;; HP2L2-NEXT: )
(func $if-unreachable-arms (param $x i32) (result anyref)
;; An unreachable if arm with a ref-producing sibling should optimize.

@kripken kripken Nov 19, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here - or were you thinking none of these comments are worth adding in the interim?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants