Repository navigation
tests: Add fix and test for regression with compiler-generated buffers - #3108
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3108 +/- ##
==========================================
- Coverage 84.24% 84.20% -0.05%
==========================================
Files 260 260
Lines 59431 59481 +50
Branches 5019 5023 +4
==========================================
+ Hits 50068 50085 +17
- Misses 8524 8561 +37
+ Partials 839 835 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| f = alias or self.target.c0 | ||
| neighborhood = f.grid.distributor.neighborhood | ||
|
|
||
| # The ranks are taken from the `nb` provided at runtime by the (possibly |
There was a problem hiding this comment.
are u sure that the proper fix ain't in _arg_values? _arg_values is where u should pull the override
see for example MPIMsg._arg_values
| # The ranks are taken from the `nb` provided at runtime by the (possibly | ||
| # overriding) Grid or SubDomain, as `f` may be a compiler-generated Array | ||
| # still attached to the Grid the Operator was built with | ||
| nb = f.grid.distributor._obj_neighborhood |
There was a problem hiding this comment.
I still don't get why not args.grid, the grid is supposed to be unique to the operator.
There was a problem hiding this comment.
Because f.grid may be a subdomain with it's own distributor and _obj_neighborhood which differs from the base grid.
| # The ranks are taken from the `nb` provided at runtime by the (possibly | ||
| # overriding) Grid or SubDomain, as `f` may be a compiler-generated Array | ||
| # still attached to the Grid the Operator was built with | ||
| nb = f.grid.distributor._obj_neighborhood |
There was a problem hiding this comment.
this whole thing looks like it's doing what nb._arg_defaults should be doing by hand
There was a problem hiding this comment.
seconded (related to what I wrote above)
| # `alias` may be a compiler-generated Array with no override | ||
| name = alias.grid.distributor._obj_neighborhood.name | ||
| neighborhood = None | ||
| for i in getattr(args, 'discretizations', ()): |
There was a problem hiding this comment.
why do u need getattr?
There was a problem hiding this comment.
Good spot, it doesn't need it
| return alias | ||
|
|
||
| def _arg_values(self, args=None, **kwargs): | ||
| return self._arg_defaults(alias=self._arg_alias(**kwargs), args=args) |
There was a problem hiding this comment.
That's the oposite of the alias logic. alias is the object used to build the operator it should be new._arg_defaults(alias=self, ..
There was a problem hiding this comment.
Wait, but that's not how it's used here in current main. We have alias = kwargs[f.name] above, so alias here is coming from the runtime value, not the build time value. This is essentially just refactoring of what was already here to avoid duplication in MPIMsgEnriched
There was a problem hiding this comment.
Then it's incorrect in main too
| neighborhood = i.distributor.neighborhood | ||
| break | ||
|
|
||
| return self._arg_defaults(alias=alias, args=args, neighborhood=neighborhood) |
There was a problem hiding this comment.
same should new._arg_defaults(alias=self probably should rename the new _arg_alias as _arg_new or something
3b60c57 to
45ea066
Compare
| entry = self.value[i] | ||
| f = self.target.c0 | ||
| for i, halo in enumerate(alias.halos): | ||
| entry = alias.value[i] |
There was a problem hiding this comment.
that still should be self.value, this process the rutime values for self given alias's name
| # Allocate the send/recv buffers | ||
| ctype, datasize = infer_datasize(f.dtype, shape) | ||
| size = datasize * dtype_len(self.target.dtype) | ||
| size = datasize * dtype_len(alias.target.dtype) |
| # The `memfree_args` will be used to deallocate the buffer upon | ||
| # returning from C-land | ||
| self._memfree_args.extend([bufg_memfree_args, bufs_memfree_args]) | ||
| alias._memfree_args.extend([bufg_memfree_args, bufs_memfree_args]) |
| alias._memfree_args.extend([bufg_memfree_args, bufs_memfree_args]) | ||
|
|
||
| return {self.name: self.value} | ||
| return {alias.name: alias.value} |
| return self | ||
|
|
||
| return self._arg_defaults(alias=alias, args=args) | ||
| return self._rebuild(target=new) |
There was a problem hiding this comment.
why do you need a rbeuil at runtime?
There was a problem hiding this comment.
In current main, alias isn't an MPIMsg, it's the payload of the message (or a component thereof), and so _arg_defaults acts on self and simply sets the buffers to the correct size accordingly (i.e. to match alias if it exists) as far as I can tell. The _rebuild here is a product of making _arg_values get the runtime object and then call runtime_obj._arg_defaults(alias=compiletime_obj). All it is doing is making a duplicate MPIMsg which points at the correct target to make this approach work.
Iirc, there are rebuilds at runtime for other MPI-related objects in the codebase already
There was a problem hiding this comment.
That being said, I do think that current main is actually the correct approach, although alias should probably be given a new name like payload
| for i, halo in enumerate(self.halos): | ||
| entry = self.value[i] | ||
| for i, halo in enumerate(alias.halos): | ||
| entry = alias.value[i] |
| ncomp = self.target.ncomp | ||
| component_indices = self.target.component_indices | ||
| if isinstance(alias.target, BundleView): | ||
| ncomp = alias.target.ncomp |
| entry.components = (c_int*ncomp)(*range(ncomp)) | ||
|
|
||
| return {self.name: self.value} | ||
| return {alias.name: alias.value} |
| alias = f | ||
| return self | ||
|
|
||
| if not getattr(new, 'is_AbstractFunction', False): |
There was a problem hiding this comment.
Going to push back on this - whilst we try to avoid getattr, in this case, it simplifies the code and makes it much easier to read
45ea066 to
ea4f3f1
Compare
ea4f3f1 to
5d1dcff
Compare
Previous merge broke some decoupler tests in PRO due to compiler-generated arrays taking their neighbouring ranks from the
Gridwith which they were built rather than the one supplied at apply time. This specifically appeared indiag2/overlap2/fullmode.