docs: swfft + CPE-DFTI build & usage HOWTO

Author: liutao

HOWTO_swfft_dfti.md

@@ -0,0 +1,65 @@
+# swfft + CPE-DFTI on Sunway — build & usage
+
+Branch `feat/swfft-dfti` (off `develop`). Three commits:
+
+1. `feat(pw/fft): add swfft (Sunway CPE 2DECOMP&FFT) backend` — per-process box 3D FFT backend (cherry-picked).
+2. `feat(pw/fft): CPE-DFTI acceleration of local 1D/3D FFTs (swFFT xMath)` — **the winner**.
+3. `feat(pw/fft): multi-process distributed swfft (Step C/D, Alltoallv)` — for completeness (slower than #2).
+
+All code is guarded by `__SWFFT` / `__SWDFTI` (+ `__MPI`). With them OFF the build is **byte-identical to develop** (verified: `fft_cpu.cpp`, `pw_basis_k.cpp` compile clean with `USE_SW=OFF`, and CMake reconfigures clean).
+
+## Performance summary (4GaAs, ecut60, 54³, np=1 unless noted)
+| route | veff_pw (FFT op) | total | note |
+|---|---|---|---|
+| baseline FFTW box | 73.2 s | 114 s | — |
+| **sticks + CPE-DFTI** (recommended) | **15–42 s** | **32–80 s** | 1.76× FFT @np1; np2/4/6 = 55/32/24 s |
+| distributed swfft (Step C/D) | 42 s @np4 | 59 s @np4 | scales but ~1.8× slower than sticks |
+
+## Build on the Sunway machine (x86 dev box just stores the source)
+
+1. **Isolate the xMath swfft symbols** (mandatory — else the bundled `fftw_*` hijack ABACUS's FFTW and the density FFT gives wrong physics, E_Hartree=0 / −4079 eV):
+   ```bash
+   SW_MATH=/usr/sw/yyzlib/xMath-SACA          # libswfft.a lives here (ISCAS swFFT, DFTI+CPE)
+   swnm $SW_MATH/libswfft.a \
+     | grep -E ' [TDBW] (d?fftw_|fftwf_)' \
+     | awk '{print $3, "swfftpriv_"$3}' > fftw_iso.map
+   swobjcopy --redefine-syms=fftw_iso.map \
+     $SW_MATH/libswfft.a \
+     source/source_base/module_fft/libswfft_xmath_iso.a
+   ```
+   Use `swobjcopy`/`swnm` (sw_64 format), NOT the host `objcopy`/`nm`.
+
+2. **Build the shim + 2DECOMP libs** (for the box & multi-process swfft backends):
+   - `swfft_shim.f90` → `libswfftshim.a` (already in `source/source_base/module_fft/`).
+   - Point CMakeLists at your `libmswfft.a` / `libswfft.a` (2DECOMP) install paths
+     (the `list(APPEND math_libs … start-group …)` block — currently the Sunway paths).
+
+3. **Configure & build**:
+   ```bash
+   cmake -S . -B build -DUSE_SW=ON          # -> defines __SWFFT, __SWDFTI (USE_SWDFTI default ON)
+   cmake --build build -j
+   ```
+   `-DUSE_SWDFTI=OFF` keeps swfft backend but drops the DFTI acceleration.
+
+4. **Run** CPE binaries via direct `bsub` (NOT `bash job.sh`):
+   ```bash
+   bsub -b -q q_swhnu -n <np> -cgsp 64 -share_size 4096 -host_stack 128 -o run.out ./abacus
+   ```
+
+## Runtime toggles (env vars)
+| var | effect |
+|---|---|
+| `ABACUS_WFC_STICKS=1` | keep wfc on native sparse **sticks + DFTI** (the fast route). **Recommended.** |
+| `ABACUS_NO_DFTI=1`     | disable the 1D-z/x CPE DFTI in FFT_CPU (fall back to FFTW) |
+| `ABACUS_BOX_NODFTI=1`  | disable box-3D DFTI in FFT_SWFFT (fall back to 2DECOMP swfft) |
+| (none / default)       | wfc on swfft box engine; box uses DFTI; sticks (density) use DFTI |
+
+## Recommended production config
+`USE_SW=ON, USE_SWDFTI=ON`, run with `ABACUS_WFC_STICKS=1` → wfc on sticks+DFTI (fastest, ~1.8× over distributed swfft, 1.76× over baseline box on the FFT operator).
+
+## The parked WIP
+The previous `cpu3d-experiment` uncommitted work (FFTW-3D box experiment) is safe in git stash:
+```bash
+git stash list           # stash@{0}: ... before swfft-dfti port
+git checkout cpu3d-experiment && git stash pop   # to restore it
+```