Repository navigation
Conversation
Add a new Heap2Local2 pass (--heap2local2) structured around a four-phase dataflow graph analysis over all allocations in a function at once: constructing an AST dataflow graph, computing a smallest fixed point over Flat<AllocId> origins, propagating escapes through identity merges and conditional cmpxchg edges, and rewriting non-escaping allocations to local variables in a single post-order pass. Also register --heap2local2 in the pass registry and fuzzer alongside unit and lit test suites for general flows, custom descriptors, and atomic RMW/cmpxchg operations.
|
To see a slightly nicer diff of the tests, look at just the first two commits (i.e. exclude |
| // Metadata and lowered local variable assignments for a candidate allocation. | ||
| struct AllocInfo { | ||
| Expression* expr = nullptr; | ||
| Type type; |
There was a problem hiding this comment.
Yes, good point. I removed the field.
There was a problem hiding this comment.
I further removed the descHeapType field.
| // Casts do not escape their inputs, but depending on the input origin, | ||
| // descriptor reachability, and target subtype check they either propagate an | ||
| // allocation origin, trap (Bot), or produce Top, so Phase 2 must re-evaluate | ||
| // them as input origins are discovered rather than using plain identity edges. |
There was a problem hiding this comment.
Is there a significant benefit to this handling of casts? I mean that a cast flows through a value, like the last element of a Block, and we could in theory treat casts that way. Looking at casts in more detail, like seeing if it must trap given its inputs, can help in some cases, but is that common?
If it does help, I wonder if it could be left to a followup, keeping the original PR more minimal?
There was a problem hiding this comment.
This is necessary to reach parity with the existing Heap2Local. Casts and cmpxchg handling could be left to follow-on PRs if we preferred that over reaching parity right away, but I don't think it would make much of a difference. Let me know if you would prefer that.
There was a problem hiding this comment.
I don't think reaching parity immediately is a top goal? So long as we know we can reach it after a series of PRs. But if it adds significant work to split it up, I can review this larger PR.
There was a problem hiding this comment.
Split out cast and cmpxchg handling.
| struct CmpxchgSite { | ||
| NodeId expNode; | ||
| NodeId repNode; | ||
| }; |
There was a problem hiding this comment.
Similar feelings about cmpxchng - can be useful, but I'm not sure how much, and if it were me I'd've left it to a followup?
| return it != exprNodes.end() ? it->second : TopNode; | ||
| } | ||
|
|
||
| void seedOrigin(Expression* curr, Origin val) { |
There was a problem hiding this comment.
What does the word "seed" refer to here? I'm not sure if it is a metaphor or a technical term.
There was a problem hiding this comment.
As it does a join, could this be joinOrigin?
There was a problem hiding this comment.
"seed" in the sense of "initialize."
I could change this to setOrigin, joinOrigin, or probably even just join.
There was a problem hiding this comment.
Changed to join and updated comments to avoid using the term "seed."
|
|
||
| // Lightweight pre-pass that stops as soon as any candidate allocation exists. | ||
| // Most functions contain no candidate allocations, so this avoids paying | ||
| // ControlFlowWalker and AnalysisState hash-map overhead for them. |
There was a problem hiding this comment.
Is that significant overhead? I would guess it's not that bad, and this code can be saved, but I could be wrong...
There was a problem hiding this comment.
I had the same question, and I was still surprised at how helpful this was. On fully optimized calcworker, this bails out early on 69% of functions and decreases pass latency by about 6%. That being said, the absolute difference there is only about 20ms. The effect is much higher on unoptimized binaries or when running on just one core, but I'd be fine going either way on this.
There was a problem hiding this comment.
What is "pass latency" - the speed of this pass, or the entire pipeline?
There was a problem hiding this comment.
How does this pass's speed compare to others? I would be interested in a 6% speedup of one of our slower passes, but Heap2Local wasn't slow iirc. If this is one of our faster passes, 6% doesn't seem worth a bunch of code.
There was a problem hiding this comment.
Yeah, this is still a fairly fast pass. I've removed this code in favor of a simpler early bailout.
| Function* func; | ||
| Module& wasm; | ||
| // Whether any local.set writes a reference, used to skip LazyLocalGraph | ||
| // construction when no reference locals are written. |
There was a problem hiding this comment.
Constructing a LazyLocalGraph is essentially free (that is the benefit of laziness), so this may be a premature optimization?
There was a problem hiding this comment.
Yeah, sorry, the comment was inaccurate here. The savings came not from skipping LazyLocalGraph construction, but from avoiding calls to getSets() when we already knew all reference-typed gets must read from default values or params. However, the effect here was really tiny, so I've removed this fast path.
|
|
||
| AllocId addCandidateAlloc(Expression* curr, const FieldList& fields) { | ||
| assert(state.allocs.size() < std::numeric_limits<AllocId>::max()); | ||
| AllocId id = AllocId(state.allocs.size()); |
There was a problem hiding this comment.
| AllocId id = AllocId(state.allocs.size()); | |
| auto id = AllocId(state.allocs.size()); |
To avoid repetition?
Also, should we have a helper getAllocId, rather than baking the logic of "the next id is the current size" in each place?
There was a problem hiding this comment.
Changed to auto. This is the only place that allocates AllocIds, so I don't think a helper would help.
| // TODO: Track field, element, and descriptor edges to optimize nested | ||
| // allocations. | ||
| for (auto* op : curr->operands) { | ||
| state.escapeIfRef(op); |
There was a problem hiding this comment.
How does this handle the case where the struct.new does not escape, so the operands do not either? It looks like it marks the operands as escaping, regardless.
There was a problem hiding this comment.
Yes, we always mark operands as escaping. There will be a follow-up that handles nested allocations, but that's not necessary for parity with Heap2Local.
There was a problem hiding this comment.
Heap2Local can handle nested allocations by optimizing them in sequence, so I think it is necessary for parity? But sgtm to leave that for later.
There was a problem hiding this comment.
Yes, fair. Heap2Local optimizes nested allocations in very particular cases where the outer allocation is an array.new and the inner allocation is a struct.new or when both allocations are struct.news but the outer one comes first and the inner one is written into it later. This pass will handle nested allocations much more generally once we get there.
|
|
||
| // Block and Loop fallthroughs are wired directly rather than via | ||
| // Properties::getImmediateFallthrough, which conservatively rejects all named | ||
| // blocks and loops. |
There was a problem hiding this comment.
I wonder if we should have a "dataflow" helper that handles this, which e.g. GUFA could use too. Maybe we can look into that later sometime.
There was a problem hiding this comment.
Yeah, that might make sense.
| for (auto* op : curr->operands) { | ||
| state.escapeIfRef(op); | ||
| } | ||
| } |
There was a problem hiding this comment.
Rather than require a full list of all instructions that escape, how about making the default behavior to escape? That would handle all new instructions, and avoid risks of bugs if we forget something.
Look at how Checker does this in Heap2Local: the default is escapes=true and all non-escaping cases must be added, safely.
There was a problem hiding this comment.
This is already the default; see visitExpression.
There was a problem hiding this comment.
I'm confused - if that is the default, why do we need visitCall, visitReturn, etc here? Don't they do the same as the default?
There was a problem hiding this comment.
Heh, good point. Implementing those directly was just a micro-optimization to avoid the overhead of ChildIterator. Removed.
| // Phase 2: Solve smallest fixed point over Flat<AllocId> origins. | ||
| void computeOrigins(AnalysisState& state) { | ||
| size_t numNodes = state.origins.size(); | ||
| // Build a compressed-sparse-row (CSR) forward-adjacency table from |
There was a problem hiding this comment.
More seriously, if we want to optimize data structures at this level, can we add a helper for it rather than put a bunch of Data Structures code into this pass? Could be a followup though.
There was a problem hiding this comment.
I'll investigate the benefit of a shared utility as a potential follow-up.
| } | ||
| }; | ||
|
|
||
| // Phase 2: Solve smallest fixed point over Flat<AllocId> origins. |
There was a problem hiding this comment.
| // Phase 2: Solve smallest fixed point over Flat<AllocId> origins. | |
| // Phase 2: Solve smallest fixed point over Flat<AllocId> origins. That is, | |
| // find which allocation each Expression belongs to (possibly none, | |
| // possibly many in which case we give up). |
There was a problem hiding this comment.
Added an edited version of this.
| } | ||
|
|
||
| // Phase 3: Compute escaped allocations and return whether any remain | ||
| // unescaped and can be optimized. |
There was a problem hiding this comment.
| // unescaped and can be optimized. | |
| // unescaped and can be optimized. Return true if we found something to | |
| // optimize (i.e. which does not escape). |
There was a problem hiding this comment.
Added an edited version.
| // non-escaping allocation, lowers allocations to local sets returning a bottom | ||
| // ref.null, rewrites uses, and finishes with ReFinalize (which widens | ||
| // control-flow and br_if types to allow the ref.null values) and | ||
| // handleNonDefaultableLocals. |
There was a problem hiding this comment.
I think this comment is maybe too specific in details. Each of the sentences
after the first could appear inside the function? But I don't feel strongly.
| return PostWalker<Optimizer>::replaceCurrent(rep); | ||
| } | ||
|
|
||
| std::optional<AllocId> getOptimizedAlloc(Expression* curr) { |
There was a problem hiding this comment.
Please add a comment to this function, I'm not quite sure what it does.
| } | ||
|
|
||
| // If `curr` is an optimizable allocation, return its ID; otherwise return | ||
| // nullopt. |
There was a problem hiding this comment.
Isn't an "optimizable allocation" exactly "evaluates exclusively to a single non-escaping candidate allocation", which is what the previous function does? I'm not clear on the difference between these two.
There was a problem hiding this comment.
This function checks whether the arg is an optimizable allocation. The previous function checks whether the runtime value of the arg comes from an optimizable allocation.
There was a problem hiding this comment.
I see, then can this function contain assert(getOptimizedAlloc()) in the case of returning a non-nullopt result? That is, one is a subset of the other?
Also perhaps can we find a different name? "candidate" vs "alloc" do not clarify enough the difference here, I feel?
There was a problem hiding this comment.
Reading the below, these are accessed disjointly and it seems safe to merge the two? Or do any of the callers care about the distinction?
There was a problem hiding this comment.
Merged them, and that even let us get rid of the separate allocIDs map!
| } | ||
|
|
||
| void visitRefAs(RefAs* curr) { | ||
| if (curr->op == RefAsNonNull && getOptimizedAlloc(curr->value)) { |
There was a problem hiding this comment.
| if (curr->op == RefAsNonNull && getOptimizedAlloc(curr->value)) { | |
| if (getOptimizedAlloc(curr->value)) { | |
| assert(curr->op == RefAsNonNull); |
There was a problem hiding this comment.
The current code does the cheap check first before doing map lookups, etc, and I don't think there's much difference in clarity between the two. Do you think this change would be worth it?
There was a problem hiding this comment.
I don't feel strongly. It just seems slightly less good to seem to appear to allow a non-RefAsNonNull to be optimized. sgtm as is.
| ;; CHECK-NEXT: ) | ||
| ;; CHECK-NEXT: ) | ||
| (func $non-gc (param $x i32) (result i32) | ||
| ;; Without GC enabled, Heap2Local2 returns immediately. |
There was a problem hiding this comment.
This test seems unnecessary. I guess it adds coverage but other tests will do that anyhow, once it is in the main pipeline (any test with GC enabled and -O3 etc.)
kripken
left a comment
There was a problem hiding this comment.
(finished reading code, only minor comments. next for tests)
| ;; HP2L2-NEXT: (local.get $desc) | ||
| ;; HP2L2-NEXT: ) | ||
| ;; HP2L2-NEXT: ) | ||
| ;; HP2L2-NEXT: ) |
There was a problem hiding this comment.
Why does heap2local optimize here but not heap2local2?
There was a problem hiding this comment.
Because we removed cast handling to make the pass smaller :)
| ;; HP2L2-NEXT: ) | ||
| ;; HP2L2-NEXT: (local.get $2) | ||
| ;; HP2L2-NEXT: ) | ||
| ;; HP2L2-NEXT: ) | ||
| (func $two-branches-b (result f64) | ||
| (local $0 (ref null $struct.A)) |
There was a problem hiding this comment.
The comment in this testcase can be updated - heap2local2 does better, implementing a TODO
There was a problem hiding this comment.
I wonder if it generally makese sense to update comments now or later when we remove the original Heap2Local. I wouldn't want to audit and update all the comments now to reflect the state of both passes, then have to update them again when we remove Heap2Local.
There was a problem hiding this comment.
Ah, I was just thinking that in a racing comment below... fair enough.
| (f64.const 1.5) | ||
| ) | ||
| ) | ||
| ;; Merging the same allocation across both arms of an if still optimizes. |
There was a problem hiding this comment.
| ;; Merging the same allocation across both arms of an if still optimizes. | |
| ;; Merging the same allocation across both arms of an if still optimizes (only in heap2local2). |
| (f64.const 2.5) | ||
| ) | ||
| ) | ||
| ;; Merging the same allocation across select optimizes cleanly. |
| ;; HP2L2-NEXT: ) | ||
| ;; HP2L2-NEXT: ) | ||
| (func $if-unreachable-arms (param $x i32) (result anyref) | ||
| ;; An unreachable if arm with a ref-producing sibling should optimize. |
There was a problem hiding this comment.
Also here - or were you thinking none of these comments are worth adding in the interim?
Add a new Heap2Local2 pass (--heap2local2) structured around a four-phase dataflow graph analysis over all allocations in a function at once: constructing an AST dataflow graph, computing a smallest fixed point over Flat origins, propagating escapes through identity merges and conditional cmpxchg edges, and rewriting non-escaping allocations to local variables in a single post-order pass. Also register --heap2local2 in the pass registry and fuzzer alongside unit and lit test suites for general flows, custom descriptors, and atomic RMW/cmpxchg operations.