Skip to content

Commit 2b18d9f

Browse files
authored
Merge pull request #1 from s-mayani/update_manual
Update manual
2 parents 5b7c162 + 02af47b commit 2b18d9f

7 files changed

Lines changed: 102 additions & 49 deletions

File tree

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,6 @@
33
/docs/
44
AGENTS.md~
55
.AGENTS.md.~undo-tree~
6+
.DS_Store
7+
**/.DS_Store
8+

figures/.DS_Store

6 KB
Binary file not shown.

figures/ippl-structure.jpg

-114 KB
Binary file not shown.

figures/ippl-structure.png

103 KB
Loading

sections/core-concepts/index.qmd

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ classDiagram
1717
}
1818
class UniformCartesian {
1919
+spacing
20+
}
21+
class Mesh {
22+
+gridsize
2023
+origin
2124
}
2225
class FieldLayout {
@@ -38,6 +41,7 @@ classDiagram
3841
FieldLayout --> NDIndex
3942
Field --> FieldLayout
4043
Field --> UniformCartesian
44+
UniformCartesian --> Mesh
4145
ParticleBase --> FieldLayout
4246
```
4347

@@ -142,7 +146,7 @@ flowchart LR
142146
Index1["Index y"]
143147
Index2["Index z"]
144148
Domain["NDIndex<Dim>"]
145-
Mesh["UniformCartesian<T, Dim>"]
149+
Mesh["UniformCartesian<T, Dim> (Mesh)"]
146150
Layout["FieldLayout<Dim>"]
147151
Field["Field<T, Dim, Mesh, Centering>"]
148152
@@ -157,8 +161,10 @@ flowchart LR
157161

158162
## `ParameterList`
159163

160-
Solver and backend options are commonly passed through `ippl::ParameterList`. The allowed value types are `double`, `float`, `bool`, `std::string`, `unsigned int`, `int`, and nested `ParameterList`.
161164

165+
Solvers take many options, such as type of algorithm used, etc. These also include options for the FFT provided by heffte. These are commonly passed through `ippl::ParameterList`.
166+
167+
The ParameterList is a collection of options, identified with a key (`std::string`) and given a value. The allowed value types in the ParameterList are `double`, `float`, `bool`, `std::string`, `unsigned int`, `int`. The following is an example of a ParameterList with some heffte options:
162168
```cpp
163169
ippl::ParameterList params;
164170
params.add("use_pencils", true);
@@ -167,5 +173,5 @@ params.add("use_gpu_aware", true);
167173

168174
bool gpuAware = params.get<bool>("use_gpu_aware");
169175
```
170-
171-
`add` rejects duplicate keys. `get` without a default throws if the key is missing, while `get(key, defaultValue)` returns the default for absent keys.
176+
To add options to the ParameterList, we use `add(key, value)`. `add` rejects duplicate keys. The value of a key can be updated using `update(key, value)`.
177+
To get the value of an option in the ParameterList, we can use the `get<T>(key)` function. `get(key)` without a default throws if the key is missing, while `get(key, defaultValue)` returns the default for absent keys.

sections/fields/index.qmd

Lines changed: 84 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,12 @@ Fields are distributed mesh data structures. They combine value type, dimension,
66

77
| Class | Role |
88
|---|---|
9-
| `BareField` | Core data container over Kokkos views. |
10-
| `Field` | User-facing field object with mesh and layout semantics. |
11-
| `FieldLayout` | Decomposition of a global index domain over MPI ranks. |
9+
| `BareField` | Core data container, with a Kokkos view to store the field data, and layout information. |
10+
| `Field` | User-facing field object with mesh and boundary condition information. |
11+
| `FieldLayout` | Decomposition of a global domain over MPI ranks. |
1212
| `SubFieldLayout` | Layout support for subdomains. |
1313
| `BConds` and `BcTypes` | Boundary condition containers and boundary condition types. |
14-
| `HaloCells` | Guard cell and halo exchange support. |
14+
| `HaloCells` | Ghost cell and halo exchange support. |
1515

1616
## Field anatomy
1717

@@ -52,11 +52,17 @@ classDiagram
5252
+getMeshVolume()
5353
+getVertexPosition()
5454
}
55+
class Mesh {
56+
+getOrigin()
57+
+getGridsize()
58+
}
59+
5560
5661
Field --|> BareField
5762
Field --> BConds
5863
Field --> FieldLayout
5964
Field --> UniformCartesian
65+
UniformCartesian --> Mesh
6066
```
6167

6268
## Typical field workflow
@@ -69,15 +75,36 @@ classDiagram
6975

7076
## Minimal field setup
7177

78+
The following sets up two fields: `rho`, a scalar field for the charge density, and `efield`, a vector field for the electric field.
79+
7280
```cpp
7381
constexpr unsigned Dim = 3;
7482

7583
using Mesh_t = ippl::UniformCartesian<double, Dim>;
84+
using Layout_t = ippl::FieldLayout<Dim>;
7685
using Centering_t = Mesh_t::DefaultCentering;
7786
using ScalarField = ippl::Field<double, Dim, Mesh_t, Centering_t>;
7887
using Vector_t = ippl::Vector<double, Dim>;
7988
using VectorField = ippl::Field<Vector_t, Dim, Mesh_t, Centering_t>;
8089

90+
// Initialize a mesh of a unit box [0,1]^3, with 4 points.
91+
int points = 4;
92+
double unit_box = 1.0;
93+
94+
ippl::Index I(pt);
95+
ippl::NDIndex<Dim> domain(I, I, I);
96+
97+
double hx = unit_box / points;
98+
Vector_t spacing = {hx, hx, hx};
99+
Vector_t origin = {0.0, 0.0, 0.0};
100+
Mesh_t mesh(domain, spacing, origin);
101+
102+
// specify that we parallelize in all dimensions
103+
std::array<bool, 3> isParallel;
104+
isParallel.fill(true);
105+
Layout_t layout(MPI_COMM_WORLD, domain, isParallel);
106+
107+
// Initialize fields
81108
ScalarField rho;
82109
VectorField efield;
83110

@@ -99,33 +126,47 @@ The `Field` template inherits from `BareField` and adds a mesh pointer plus fiel
99126
| `view_type` | Kokkos view type used for field storage. |
100127
| `BConds_t` | Boundary-condition container type for the field. |
101128
102-
The constructor form `ScalarField rho(mesh, layout);` is also used in tests. The default ghost depth is one cell unless a different `nghost` value is supplied to the constructor or `initialize`.
129+
The constructor form `ScalarField field(mesh, layout);` is also used in tests. The default ghost depth is one cell unless a different `nghost` value is supplied to the constructor or `initialize`.
103130
104131
## Accessing data in kernels
105132
106-
The Gaussian Poisson test shows the standard local-to-global mapping inside a field kernel:
107-
133+
Field data can be accessed within a Kokkos kernel. For the most common parallel pattern, `Kokkos::parallel_for`, there exists an `ippl::parallel_for` to loop over fields in a dimension independent manner. As an example, the following loops over a field and assigns values:
108134
```cpp
109-
auto view = rho.getView();
110-
const int nghost = rho.getNghost();
111-
const auto& ldom = layout.getLocalNDIndex();
112-
113-
Kokkos::parallel_for(
114-
"Assign rho field", rho.getFieldRangePolicy(),
115-
KOKKOS_LAMBDA(const int i, const int j, const int k) {
116-
const int ig = i + ldom[0].first() - nghost;
117-
const int jg = j + ldom[1].first() - nghost;
118-
const int kg = k + ldom[2].first() - nghost;
119-
120-
const double x = (ig + 0.5) * hr[0] + origin[0];
121-
const double y = (jg + 0.5) * hr[1] + origin[1];
122-
const double z = (kg + 0.5) * hr[2] + origin[2];
123-
124-
view(i, j, k) = gaussian(x, y, z);
135+
auto view = field.getView();
136+
const int nghost = field.getNghost();
137+
138+
using index_array_type = typename ippl::RangePolicy<Dim>::index_array_type;
139+
ippl::parallel_for(
140+
"Assign field values", field.getFieldRangePolicy(),
141+
KOKKOS_LAMBDA(const index_array_type& args) {
142+
// ippl::apply accesses the view at the given indices and obtains a
143+
// reference; see src/Expression/IpplOperations.h
144+
ippl::apply(view, args) = ...;
125145
});
126146
```
127147

128-
This pattern is worth documenting carefully because it makes three contracts explicit: local indices include ghost cells, `FieldLayout` provides the owned global offset, and cell-centered coordinates use the `0.5` shift.
148+
When doing operations which require the physical position of a point in the global domain (x, y, z), we need to map from the local index of the Field data in the Kokkos View (local to each MPI rank), to the global position in the full domain.
149+
150+
The standard way to do this is using the domain information in the layout and the mesh spacing. For example, when assigning field values according to a function, such as a sinusoidal:
151+
```cpp
152+
auto view = field.getView();
153+
const int nghost = field.getNghost();
154+
155+
const auto& ldom = layout.getLocalNDIndex();
156+
const auto hr = mesh.getMeshSpacing();
157+
const auto origin = mesh.getOrigin();
158+
159+
using index_array_type = typename ippl::RangePolicy<Dim>::index_array_type;
160+
ippl::parallel_for(
161+
"Assign field values", field.getFieldRangePolicy(),
162+
KOKKOS_LAMBDA(const index_array_type& args) {
163+
ippl::Vector<int, Dim> global_indices = args + ldom.first() - nghost;
164+
ippl::Vector<double, Dim> global_position = (global_indices + 0.5) * hr + origin;
165+
ippl::apply(view, args) = sin(global_position);
166+
});
167+
```
168+
The layout (`FieldLayout`) contains information about the offset of the sub-domain owned by this MPI rank with respect to the global origin, given by `ldom.first()`. The local indexing also contains the ghost cells, hence why it needs to be subtracted to obtain a global index.
169+
The `+ 0.5` in the global position computation is needed due to cell centering.
129170
130171
## Layout and neighbors
131172
@@ -157,6 +198,7 @@ Current field boundary conditions are stored in `BConds<Field, Dim>`, an array-l
157198
| `3` | upper face in dimension 1 |
158199
| `2 * d` | lower face in dimension `d` |
159200
| `2 * d + 1` | upper face in dimension `d` |
201+
The dimensions go from 0 up to Dim-1.
160202

161203
Supported field boundary-condition classes are:
162204

@@ -180,7 +222,7 @@ for (size_t face = 0; face < 2 * Dim; ++face) {
180222
bc[face] = std::make_shared<ippl::PeriodicFace<ScalarField>>(face);
181223
}
182224

183-
rho.setFieldBC(bc);
225+
field.setFieldBC(bc);
184226
```
185227

186228
Mixed boundary conditions are also possible. `test/field/TestFieldBC.cpp` uses periodic faces in the x direction, no boundary condition on the lower y face, a constant value on the upper y face, zero on the lower z face, and extrapolation on the upper z face:
@@ -195,7 +237,7 @@ bc[3] = std::make_shared<ippl::ConstantFace<ScalarField>>(3, 7.0);
195237
bc[4] = std::make_shared<ippl::ZeroFace<ScalarField>>(4);
196238
bc[5] = std::make_shared<ippl::ExtrapolateFace<ScalarField>>(5, 0.0, 1.0);
197239

198-
rho.setFieldBC(bc);
240+
field.setFieldBC(bc);
199241
```
200242

201243
`setFieldBC` stores the container and calls `findBCNeighbors` on the field. Differential operators such as `grad`, `div`, `laplace`, `curl`, and `hess` call `fillHalo()` and apply the field boundary conditions before building the expression object.
@@ -205,10 +247,10 @@ rho.setFieldBC(bc);
205247
Ghost cells provide local data for stencil operations at subdomain boundaries. A field has a ghost depth `nghost`, available through `getNghost()`. Halo exchange is invoked explicitly with:
206248

207249
```cpp
208-
rho.fillHalo();
250+
field.fillHalo();
209251
```
210252

211-
Field operations that require neighbor values call halo filling internally. For custom kernels, users must decide whether the operation reads only owned cells or also reads ghost cells. If the kernel reads stencil neighbors across rank boundaries, call `fillHalo()` and apply boundary conditions first.
253+
Field operations that require neighbor values call `fillHalo` internally. For custom kernels, users must decide whether the operation needs only owned cells of the MPI rank or also ghost cells. If the kernel needs neighbors across rank boundaries, e.g. for stencil operations, call `fillHalo()` and apply boundary conditions first.
212254

213255
## Expressions and reductions
214256

@@ -217,56 +259,55 @@ Fields support scalar assignment, field expressions, math functions, and reducti
217259
| Operation | Example |
218260
|---|---|
219261
| scalar assignment | `rho = 1.0;` |
220-
| deep copy | `auto copy = rho.deepCopy();` |
262+
| deep copy | `auto rho_copy = rho.deepCopy();` |
221263
| arithmetic | `rho = rho - exact;` |
222264
| sum | `double q = rho.sum();` |
223265
| norm | `double err = norm(rho) / norm(exact);` |
224266
| volume integral | `rho.getVolumeIntegral();` |
225267
| volume average | `rho.getVolumeAverage();` |
226-
| gradient | `efield = grad(rho);` |
268+
| gradient | `efield = grad(phi);` |
227269
| divergence | `rho = div(efield);` |
228270
| curl | `result = curl(vectorField);` |
229271
| Hessian | `matrixField = hess(rho);` |
230272
| Laplacian | `lap = laplace(rho);` |
231273

232-
The differential operators use mesh spacing from `UniformCartesian`. For example, `laplace` builds coefficients from `1 / h_d^2` in each dimension, while `grad` and `div` use centered differences scaled by `0.5 / h_d`.
274+
The differential operators use the spacing obtained from the `UniformCartesian` mesh. For example, `laplace` has a pre-factor of `1 / h[d]^2` in each dimension d.
233275

234276
## Laplacian example
235277

236-
`test/field/TestLaplace.cpp` is the compact example for differential field operators. It builds a periodic scalar field on `[-1, 1]^3`,
278+
`test/field/TestLaplace.cpp` is the compact example for differential field operators. It builds a 3-dimensinal periodic scalar field on `[-1, 1]^3`,
237279

238-
\[
280+
$$
239281
u(x,y,z) = \sin(\pi x)\sin(\pi y)\sin(\pi z),
240-
\]
282+
$$
241283

242-
then compares `laplace(field)` with
284+
then compares `laplace(field)` with the exact form
243285

244-
\[
286+
$$
245287
\nabla^2 u = -3\pi^2 \sin(\pi x)\sin(\pi y)\sin(\pi z).
246-
\]
288+
$$
247289

248290
The relevant operator call is deliberately simple:
249291

250292
```cpp
251293
Lap = laplace(field);
294+
295+
// Error computation
252296
Lap = Lap - Lap_exact;
253297
Lap = pow(Lap, 2);
254-
255298
double error = sqrt(Lap.sum()) / sqrt(Lap_exact.sum());
256299
```
257300

258-
Run shape:
301+
To run the test, we pass the number of grid-points (indicating the mesh spacing), and the number of times we want to repeat the operation:
259302

260303
```bash
261304
srun ./TestLaplace <points-per-dimension> <iterations>
262305
```
263306

264-
This example is a good manual candidate because it touches domain construction, mesh spacing, periodic boundary conditions, local-to-global coordinate conversion, expression assignment, reductions, timing, and explicit `ippl::fence()` calls inside a benchmark loop.
265-
266307
## Sub-layouts
267308

268-
The test suite contains parallel coverage for both full layouts and sub-layouts. The user-level contract is the same: a field is tied to a mesh and a layout, while the layout determines local domains, neighbor relationships, and halo ranges. Sub-layout documentation should be expanded once the `SubFieldLayout` examples are promoted into the manual.
309+
TO-DO
269310

270311
## Documentation tasks
271312

272-
Remaining migration work from the legacy LaTeX manual: older `where` examples, a compact table of mathematical functions supported by expressions, and a side-by-side update of legacy constructor forms to current constructor forms.
313+
Remaining migration work from the legacy LaTeX manual: older `where` examples, a compact table of mathematical functions supported by expressions, and a side-by-side update of legacy constructor forms to current constructor forms.

sections/overview/index.qmd

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
IPPL provides reusable components for scientific simulations that need distributed fields, particles, interpolation, FFTs, and solvers. The library is designed around templates for dimension, precision, and execution space, so the same user code can target serial, OpenMP, CUDA, or HIP builds when the backend stack is available.
44

5-
![](../../figures/ippl-structure.jpg){fig-alt="Legacy IPPL structure diagram" width="70%"}
5+
![](../../figures/ippl-structure.png){fig-alt="IPPL structure diagram" width="85%"}
66

77
## Main capabilities
88

@@ -25,19 +25,22 @@ flowchart TB
2525
Manager["Manager layer"]
2626
Particle["Particle containers"]
2727
Field["Distributed fields"]
28+
FieldSolver["Field solvers"]
2829
Mesh["Mesh, index, region"]
2930
Interp["Interpolation"]
30-
Solver["FFT, Poisson, Maxwell, FEM"]
31+
Solver["Poisson, Maxwell"]
3132
Comm["MPI communication"]
3233
Kokkos["Kokkos execution and memory spaces"]
3334
Heffte["HeFFTe FFT backend"]
3435
3536
User --> Manager
3637
Manager --> Particle
3738
Manager --> Field
39+
Manager --> FieldSolver
3840
Particle --> Interp
3941
Interp --> Field
4042
Field --> Mesh
43+
FieldSolver --> Solver
4144
Solver --> Field
4245
Solver --> Heffte
4346
Particle --> Comm

0 commit comments

Comments
 (0)