Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
5305435
Base structure for scientific defaults
franzpoeschel Jan 12, 2026
ecbe88d
Start adding defaults
franzpoeschel Jan 12, 2026
3e1cceb
debugging output
franzpoeschel Jan 12, 2026
4f6d7c8
Fixes
franzpoeschel Jan 12, 2026
dbf7050
BaseRecord --> unitDimension
franzpoeschel Jan 12, 2026
48cbfc5
nunja hehe
franzpoeschel Jan 12, 2026
56e7bc7
Fixes
franzpoeschel Jan 12, 2026
9195a91
Further fixes
franzpoeschel Jan 13, 2026
75faf5f
further fixes
franzpoeschel Jan 13, 2026
8631be3
tests temporarily passing :D
franzpoeschel Jan 13, 2026
853a5bc
Cleanup
franzpoeschel Jan 13, 2026
e190825
UnitDimension for position/positionOffset
franzpoeschel Jan 13, 2026
e92040c
WIP: RecordComponent stuff
franzpoeschel Jan 13, 2026
0ac271c
Fix coretests
franzpoeschel Jan 14, 2026
bd6b373
Clean up Iteration finalizing logic
franzpoeschel Jan 14, 2026
6a73475
Add defaults for Iteration class
franzpoeschel Jan 14, 2026
90c8e52
Fix variadic template template parameters
franzpoeschel Jan 15, 2026
59b98dd
Remove flush-time check for populized components
franzpoeschel Jan 16, 2026
3cd7a21
Make the default specification more flexible
franzpoeschel Jan 16, 2026
5b8125e
WIP
franzpoeschel Jan 20, 2026
c64220a
continue
franzpoeschel Jan 20, 2026
f2cd931
Split this into internal and public header
franzpoeschel Jan 21, 2026
4cf0d8f
Distinguish writing from reading
franzpoeschel Jan 21, 2026
245996d
Tests now working again
franzpoeschel Jan 21, 2026
25da6e0
Mostly move Mesh reading to ScientificDefaults
franzpoeschel Jan 21, 2026
bccbdbe
Fix type conversions between char and string
franzpoeschel Jan 21, 2026
99d5aa6
Somewhat working Mesh reading
franzpoeschel Jan 21, 2026
4650cfe
Fix little parent call bug
franzpoeschel Jan 22, 2026
83b8e2e
Mention expected datatypes in attribute reading error messages
franzpoeschel Jan 22, 2026
a5e1fc9
Move Iteration reading to ScientificDefaults class
franzpoeschel Jan 22, 2026
f2d8ca3
Introduce genericsetter, use for unitDimension
franzpoeschel Jan 22, 2026
2cfed86
Record / BaseRecord Reading
franzpoeschel Jan 22, 2026
c532fcd
Readers for everything now except version-dependent stuff
franzpoeschel Jan 22, 2026
ef80f24
Add gridUnitDimension in ScientificDefaults
franzpoeschel Jan 23, 2026
da9cfdd
Do not use templates for defaults setter
franzpoeschel Jan 26, 2026
adc659a
Wew NewAttributeReader
franzpoeschel Jan 27, 2026
6592640
Fixes
franzpoeschel Jan 27, 2026
3c27092
Fixes
franzpoeschel Jan 27, 2026
19adbb4
Defaults for patchRecord
franzpoeschel Jan 27, 2026
069ca8b
Adapt PatchRecordComponent
franzpoeschel Jan 27, 2026
761d5aa
Fixes
franzpoeschel Jan 27, 2026
035337a
Cleanup
franzpoeschel Jan 28, 2026
d8f9fa3
Cleanup
franzpoeschel Jan 28, 2026
2750513
Avoid overcomplex attribute type conversions
franzpoeschel Jan 28, 2026
29cf817
reduce binary size a bit
franzpoeschel Jan 28, 2026
0550bcb
Try further reducing amount of lambdas...
franzpoeschel Jan 28, 2026
16c4e59
Annotate MB sizes
franzpoeschel Jan 28, 2026
1274201
Reduce implementation sizes for RequireScalar / RequireVector
franzpoeschel Jan 29, 2026
9348900
vibe-coded suggestions for size optimization
franzpoeschel Jan 29, 2026
35516da
Avoid using lambdas for require_type
franzpoeschel Jan 29, 2026
07b2a8e
Change code order
franzpoeschel Jan 29, 2026
444b6f4
Replace if constexpr with SFINAE
franzpoeschel Jan 30, 2026
d4b85e6
Fix nvhpc/msvc builds
franzpoeschel Jan 30, 2026
3cf834b
Add memory selection task to backend
franzpoeschel Aug 7, 2025
bf64de6
Add future helper
franzpoeschel Aug 7, 2025
1ed34aa
Fixes for UniquePtr.hpp
franzpoeschel Aug 7, 2025
7830511
Main implementation of LoadStoreChunk.hpp
franzpoeschel Aug 7, 2025
94deb27
Fix for determineDatatype
franzpoeschel Aug 7, 2025
8c015ae
Adapt RecordComponent implementation based on new API
franzpoeschel Aug 7, 2025
404be13
Testing
franzpoeschel Aug 7, 2025
6c99100
Enable support for const unique pointers?
franzpoeschel Aug 7, 2025
8dfc4b8
Use a better trick to cheat clang-tidy
franzpoeschel Aug 7, 2025
0a8b0f5
clang-tidy fixes
franzpoeschel Aug 8, 2025
8b07c3a
Fix linker error
franzpoeschel Jan 8, 2026
ad6739c
Reimplement unique_ptr-based span API
franzpoeschel Jan 8, 2026
5bcc5ee
Fix parallel tests
franzpoeschel Jan 8, 2026
7f242ab
Type signature fix
franzpoeschel Jan 8, 2026
80075f9
Destructor for ~DeferredComputation
franzpoeschel Jan 9, 2026
23dae9e
doxygen fix
franzpoeschel Feb 2, 2026
212fda7
Merge branch 'defaults-upon-close' into HEAD
franzpoeschel Feb 3, 2026
bb3eba6
Extract GlobalParameters struct
franzpoeschel Jan 9, 2026
7ff5123
Better initialization of global params
franzpoeschel Jan 9, 2026
274a9f2
Add forgotten internal header
franzpoeschel Jan 12, 2026
692d6d3
First attempt at immediate flushing
franzpoeschel Jan 12, 2026
e84e07f
Fix coretests
franzpoeschel Jan 14, 2026
c08bef6
wip: try splitting m_work in two
franzpoeschel Jan 14, 2026
3f49f2f
push_chunk: distinguish immediate from deferred flushes
franzpoeschel Jan 14, 2026
153a267
Avoid flush_immediately for Span operations
franzpoeschel Jan 14, 2026
208a56f
Fix dirty handling
franzpoeschel Jan 14, 2026
ab44966
Bug fix in RecordComponent::flush
franzpoeschel Feb 2, 2026
6b7e4e8
tmp: deactivate some tests
franzpoeschel Feb 2, 2026
715e738
Properly distinguish flush modi in tests
franzpoeschel Feb 2, 2026
dd95ffc
CI fixes
franzpoeschel Feb 2, 2026
c725c7b
CI fixes
franzpoeschel Feb 2, 2026
84d0f30
CI fixes
franzpoeschel Feb 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,7 @@ set(CORE_SOURCE
src/Format.cpp
src/Iteration.cpp
src/IterationEncoding.cpp
src/LoadStoreChunk.cpp
src/Mesh.cpp
src/ParticlePatches.cpp
src/ParticleSpecies.cpp
Expand All @@ -411,6 +412,7 @@ set(CORE_SOURCE
src/version.cpp
src/auxiliary/Date.cpp
src/auxiliary/Filesystem.cpp
src/auxiliary/Future.cpp
src/auxiliary/JSON.cpp
src/auxiliary/JSONMatcher.cpp
src/auxiliary/Memory.cpp
Expand All @@ -426,6 +428,7 @@ set(CORE_SOURCE
src/backend/PatchRecord.cpp
src/backend/PatchRecordComponent.cpp
src/backend/Writable.cpp
src/backend/ScientificDefaults.cpp
src/auxiliary/OneDimensionalBlockSlicer.cpp
src/helper/list_series.cpp
src/snapshots/ContainerImpls.cpp
Expand Down
6 changes: 6 additions & 0 deletions include/openPMD/Dataset.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ namespace openPMD
using Extent = std::vector<std::uint64_t>;
using Offset = std::vector<std::uint64_t>;

struct MemorySelection
{
Offset offset;
Extent extent;
};

class Dataset
{
friend class RecordComponent;
Expand Down
3 changes: 2 additions & 1 deletion include/openPMD/Datatype.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,8 @@ template <typename T>
inline constexpr Datatype determineDatatype(T &&val)
{
(void)val; // don't need this, it only has a name for Doxygen
using T_stripped = std::remove_cv_t<std::remove_reference_t<T>>;
using T_stripped =
std::remove_extent_t<std::remove_cv_t<std::remove_reference_t<T>>>;
if constexpr (auxiliary::IsPointer_v<T_stripped>)
{
return determineDatatype<auxiliary::IsPointer_t<T_stripped>>();
Expand Down
2 changes: 2 additions & 0 deletions include/openPMD/IO/ADIOS/ADIOS2File.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
*/
#pragma once

#include "openPMD/Dataset.hpp"
#include "openPMD/IO/ADIOS/ADIOS2Auxiliary.hpp"
#include "openPMD/IO/ADIOS/ADIOS2PreloadAttributes.hpp"
#include "openPMD/IO/ADIOS/ADIOS2PreloadVariables.hpp"
Expand Down Expand Up @@ -112,6 +113,7 @@ struct BufferedUniquePtrPut
std::string name;
Offset offset;
Extent extent;
std::optional<MemorySelection> memorySelection;
UniquePtrWithLambda<void> data;
Datatype dtype = Datatype::UNDEFINED;

Expand Down
28 changes: 20 additions & 8 deletions include/openPMD/IO/ADIOS/ADIOS2IOHandler.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
*/
#pragma once

#include "openPMD/Dataset.hpp"
#include "openPMD/Error.hpp"
#include "openPMD/IO/ADIOS/ADIOS2Auxiliary.hpp"
#include "openPMD/IO/ADIOS/ADIOS2FilePosition.hpp"
Expand All @@ -30,6 +31,7 @@
#include "openPMD/IO/AbstractIOHandler.hpp"
#include "openPMD/IO/AbstractIOHandlerImpl.hpp"
#include "openPMD/IO/AbstractIOHandlerImplCommon.hpp"
#include "openPMD/IO/AbstractIOHandler_internal.hpp"
#include "openPMD/IO/FlushParametersInternal.hpp"
#include "openPMD/IO/IOTask.hpp"
#include "openPMD/IO/InvalidatableFile.hpp"
Expand Down Expand Up @@ -509,6 +511,7 @@ class ADIOS2IOHandlerImpl
adios2::Variable<T> verifyDataset(
Offset const &offset,
Extent const &extent,
std::optional<MemorySelection> const &memorySelection,
adios2::IO &IO,
adios2::Engine &engine,
std::string const &varName,
Expand Down Expand Up @@ -622,13 +625,26 @@ class ADIOS2IOHandlerImpl
var.SetSelection(
{adios2::Dims(offset.begin(), offset.end()),
adios2::Dims(extent.begin(), extent.end())});

if (memorySelection.has_value())
{
var.SetMemorySelection(
{adios2::Dims(
memorySelection->offset.begin(),
memorySelection->offset.end()),
adios2::Dims(
memorySelection->extent.begin(),
memorySelection->extent.end())});
}

return var;
}

struct
{
bool noGroupBased = false;
bool blosc2bp5 = false;
bool memorySelection = false;
} printedWarningsAlready;
}; // ADIOS2IOHandlerImpl

Expand Down Expand Up @@ -942,7 +958,7 @@ class ADIOS2IOHandler : public AbstractIOHandler
try
{
auto params = internal::defaultParsedFlushParams;
this->flush(params);
this->flush_impl(params);
}
catch (std::exception const &ex)
{
Expand All @@ -962,9 +978,7 @@ class ADIOS2IOHandler : public AbstractIOHandler
#if openPMD_HAVE_MPI

ADIOS2IOHandler(
std::optional<std::unique_ptr<AbstractIOHandler>> initialize_from,
std::string path,
Access,
internal::AbstractIOHandlerInitFrom &&initialize_from,
MPI_Comm,
json::TracingJSON options,
std::string engineType,
Expand All @@ -973,9 +987,7 @@ class ADIOS2IOHandler : public AbstractIOHandler
#endif

ADIOS2IOHandler(
std::optional<std::unique_ptr<AbstractIOHandler>> initialize_from,
std::string path,
Access,
internal::AbstractIOHandlerInitFrom &&initialize_from,
json::TracingJSON options,
std::string engineType,
std::string specifiedExtension);
Expand All @@ -990,6 +1002,6 @@ class ADIOS2IOHandler : public AbstractIOHandler
return true;
}

std::future<void> flush(internal::ParsedFlushParams &) override;
std::future<void> flush_impl(internal::ParsedFlushParams &) override;
}; // ADIOS2IOHandler
} // namespace openPMD
23 changes: 23 additions & 0 deletions include/openPMD/IO/ADIOS/macros.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,29 @@
#define openPMD_HAVE_ADIOS2_BP5 0
#endif

namespace openPMD
{
namespace detail
{
template <typename Variable, typename SFINAE = void>
struct CanTheMemorySelectionBeReset
{
static constexpr bool value = false;
};

template <typename Variable>
struct CanTheMemorySelectionBeReset<
Variable,
decltype(std::declval<Variable>().SetMemorySelection())>
{
static constexpr bool value = true;
};
} // namespace detail

constexpr bool CanTheMemorySelectionBeReset =
detail::CanTheMemorySelectionBeReset<adios2::Variable<int>>::value;
} // namespace openPMD

#else

#define openPMD_HAS_ADIOS_2_8 0
Expand Down
Loading
Loading