Add boundary integral support (BdIntegral) with PETSc ghost facet fix#70
Add boundary integral support (BdIntegral) with PETSc ghost facet fix#70lmoresi wants to merge 7 commits intodevelopmentfrom
Conversation
Wrap PETSc's DMPlexComputeBdIntegral to provide standalone boundary/surface
integral capability. In 2D these are line integrals over boundary edges;
in 3D they are surface integrals over boundary faces (both codimension-1).
The integrand may reference the outward unit normal via mesh.Gamma / mesh.Gamma_N.
Internal boundaries (e.g. AnnulusInternalBoundary, BoxInternalBoundary) work
out of the box through the same DMLabel infrastructure.
New API:
bd_int = uw.maths.BdIntegral(mesh, fn=1.0, boundary="Top")
length = bd_int.evaluate()
Tests cover: constant/coordinate/sympy/mesh-variable integrands, normal-vector
integrands, perimeter checks, BoxInternalBoundary, and AnnulusInternalBoundary
circumference.
Underworld development team with AI support from Claude Code
Use the same team attribution wording for both commits and PR descriptions. No emoji in PR bodies. Underworld development team with AI support from Claude Code
There was a problem hiding this comment.
Pull request overview
Adds PETSc-backed boundary (surface/line) integration to Underworld3 via a new uw.maths.BdIntegral API, enabling integrals over named external and internal mesh boundaries (Issue #47).
Changes:
- Introduces
BdIntegralinpetsc_maths.pyx, compiling scalar integrands (optionally using outward normals) and evaluating via PETSc/DMPlex. - Adds a thin C helper
UW_DMPlexComputeBdIntegralplus Cython declarations to simplify PETSc’s multi-field boundary-integral interface. - Adds a dedicated boundary-integral test suite covering external boundaries and internal boundaries (box + annulus), and exports
BdIntegralfromunderworld3.maths.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/underworld3/cython/petsc_compat.h |
Adds UW_DMPlexComputeBdIntegral wrapper around PETSc DMPlexComputeBdIntegral. |
src/underworld3/cython/petsc_extras.pxi |
Declares the new C wrapper for use from Cython. |
src/underworld3/cython/petsc_maths.pyx |
Implements BdIntegral class and unit propagation logic. |
src/underworld3/maths/__init__.py |
Exports BdIntegral in the public maths module. |
tests/test_0502_boundary_integrals.py |
Adds test coverage for boundary integrals on external and internal boundaries. |
CLAUDE.md |
Updates AI attribution guidance to include PR descriptions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| cdef PetscInt label_val = boundary_enum.value | ||
| cdef PetscInt num_vals = 1 | ||
|
|
||
| c_label = mesh.dm.getLabel(self.boundary) |
There was a problem hiding this comment.
mesh.dm.getLabel(self.boundary) can return None even when boundary exists in mesh.boundaries (e.g., if the DM only has the consolidated UW_Boundaries label). In that case dmlabel.dmlabel will segfault. Consider fetching the stacked label (mesh.dm.getLabel("UW_Boundaries")) and using label_val to select the stratum, or at minimum check c_label is not None and raise a ValueError with a clear message.
| c_label = mesh.dm.getLabel(self.boundary) | |
| c_label = mesh.dm.getLabel(self.boundary) | |
| if c_label is None: | |
| # Some meshes store boundary information only in the consolidated | |
| # "UW_Boundaries" label. In that case, fall back to that label | |
| # and use `label_val` to select the appropriate stratum. | |
| c_label = mesh.dm.getLabel("UW_Boundaries") | |
| if c_label is None: | |
| raise ValueError( | |
| f"Boundary label '{self.boundary}' not found on mesh DM, and " | |
| "no 'UW_Boundaries' label is present. Cannot compute boundary integral." | |
| ) |
| PetscDS ds; | ||
| PetscSection section; | ||
| PetscInt Nf; | ||
|
|
||
| PetscFunctionBeginUser; | ||
|
|
||
| PetscCall(DMGetLocalSection(dm, §ion)); | ||
| PetscCall(PetscSectionGetNumFields(section, &Nf)); |
There was a problem hiding this comment.
PetscDS ds; is declared but never used in UW_DMPlexComputeBdIntegral(). If the build treats warnings as errors (common in CI), this can fail compilation. Remove the unused variable, or use DMGetDS()/PetscDSGetNumFields() to obtain Nf instead of leaving ds unused.
DMPlexComputeBdIntegral (unlike DMPlexComputeIntegralFEM) does not perform an MPI reduction — it returns only the local process contribution. Add MPIU_Allreduce in UW_DMPlexComputeBdIntegral to sum across ranks. Found by testing with mpirun -np 5..8 on AnnulusInternalBoundary. Note: a pre-existing PETSc internal-facet orientation issue can cause small errors (~1.5%) on internal boundaries at certain partition counts (e.g. np=6 with the default annulus mesh). External boundaries are unaffected. This is upstream of our wrapper. Underworld development team with AI support from Claude Code
DMPlexComputeBdIntegral iterates over ALL label stratum points including ghost facets, so shared internal boundary facets get integrated on both the owning rank and the ghost rank. External boundaries are unaffected because external facets have only one supporting cell (no ghosting). The fix creates a temporary DMLabel containing only owned (non-ghost) boundary points by checking against the PetscSF leaf set, then passes this filtered label to DMPlexComputeBdIntegral. Also restructures tests to use lazy initialization for BoxInternalBoundary (which has a pre-existing MPI bug) so it doesn't block other tests under mpirun. Verified at np=1,5,6,7,8 — all produce identical results. Underworld development team with AI support from Claude Code
DMPlexComputeBdResidual_Internal and DMPlexComputeBdIntegral do not exclude ghost facets (SF leaves) from the facet IS, unlike the volume residual code which checks the ghost label. For internal boundaries at partition junctions, this causes shared facets to be integrated on multiple ranks, producing O(1) errors after MPI summation. The patch filters ghost facets using ISDifference, following the same pattern used by DMPlexComputeBdFaceGeomFVM for FVM face flux. It applies cleanly to PETSc v3.18.0 through v3.24.5. Includes: - petsc-custom/patches/plexfem-ghost-facet-fix.patch - build-petsc.sh integration (auto-applies after clone) - MR description for upstream PETSc submission Upstream MR branch pushed to gitlab.com/lmoresi/petsc (fix/plexfem-ghost-facet-boundary-residual) Underworld development team with AI support from Claude Code
28a06bb to
4f76375
Compare
PETSc changed DMPlexComputeBdIntegral from void (*func)(...) to void (**funcs)(...) in v3.22.0. CI uses PETSc 3.21.5 (pinned in environment.yaml), so the wrapper must handle both signatures. Underworld development team with AI support from Claude Code
…or bug) Underworld development team with AI support from Claude Code
|
This is a long-standing request from @jcgraciosa and @gthyagi to add surface integral calculations to uw3. There is a PETSC "bug" (unanticipated use-case) that does not handle internal surfaces correctly - we have seen this in calculating kernels and other analytical benchmarks (@gthyagi) and there is a potential fix for this that is currently added to the build script as a patch when compiling PETSc. This is also pushed to the PETSc gitlab as a merge-request, so we might get this fixed and released eventually. |
Summary
uw.maths.BdIntegralclass for computing integrals over labeled mesh boundaries (closes Add surface and line integral support (PETSc-backed) in UW for fault-based models #47)DMPlexComputeBdResidual_InternalandDMPlexComputeBdIntegral— a long-standing upstream bug affecting internal boundary natural BCs in parallelBdIntegral API
Uses PETSc's
DMPlexComputeBdIntegralvia a new C wrapper (UW_DMPlexComputeBdIntegral) inpetsc_compat.h. Follows the same pattern as the existingIntegralclass.PETSc ghost facet fix
DMPlexComputeBdResidual_InternalandDMPlexComputeBdIntegralconstructfacetISas all facets at depthdim-1but do not exclude ghost facets (SF leaves). Unlike the volume residual code (which checks the ghost label), the boundary residual path has no ghost check. For internal boundaries at partition junctions, shared facets are integrated on multiple ranks, causing double-counting.The patch filters ghost facets using
ISDifferenceafter obtainingfacetIS, following the same pattern used byDMPlexComputeBdFaceGeomFVMin PETSc's own FVM code. It applies cleanly to PETSc v3.18.0 through v3.24.5.petsc-custom/patches/plexfem-ghost-facet-fix.patchbuild-petsc.shduring custom PETSc buildsgitlab.com/lmoresi/petsc(pending GitLab availability)Test plan
test_0502_boundary_integrals.py) pass serialtest_0501_integrals.py) pass at np=4 (no regression)BoxInternalBoundaryhas a pre-existingUnboundLocalErrorbug in parallel (not this PR)Files changed
src/underworld3/cython/petsc_compat.hUW_DMPlexComputeBdIntegralC wrappersrc/underworld3/cython/petsc_extras.pxisrc/underworld3/cython/petsc_maths.pyxBdIntegralclasssrc/underworld3/maths/__init__.pyBdIntegraltests/test_0502_boundary_integrals.pypetsc-custom/patches/plexfem-ghost-facet-fix.patchpetsc-custom/build-petsc.shCLAUDE.mdUnderworld development team with AI support from Claude Code