Skip to content

Revert "Increase maximum read-only mmap()s used from 1000 to 4096 on 64-bit systems"#52

Open
laanwj wants to merge 1 commit intobitcoin-core:bitcoin-forkfrom
laanwj:2025-05-revert-mmap-increase
Open

Revert "Increase maximum read-only mmap()s used from 1000 to 4096 on 64-bit systems"#52
laanwj wants to merge 1 commit intobitcoin-core:bitcoin-forkfrom
laanwj:2025-05-revert-mmap-increase

Conversation

@laanwj
Copy link
Copy Markdown
Member

@laanwj laanwj commented May 8, 2025

After bitcoin/bitcoin#30039, the number of ldb files created is 16 times smaller. 1000 files with the new default of 32MB is 32GB of database.

If we need more (for good performance), we can increase the default file size again.

This patch seems unnecessary now.

This reverts commit 92ae82c:

By default LevelDB will only mmap() up to 1000 ldb files for reading and then fall back
to using file desciptors.

The typical linux system has a 'vm.max_map_count = 65530', so mapping only 1000 files
seems arbitarily small. Increase this value to another arbitrarily small value, 4096.

…64-bit systems"

After bitcoin/bitcoin#30039, the number of ldb files created is 16 times
smaller. 1000 files with the new default of 32MB is 32GB of database.

If we need more, we can increase the default file size again.

This patch seems unnecessary now.

This reverts commit 92ae82c.
Copy link
Copy Markdown

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, the change was triggered by one of the synchronization efforts between our fork an the upstream repo google#1265

This revert basically leaves us closer to the original LevelDB implementation - where both env_posix.cc and env_windows.cc have this value, see: https://github.com/search?q=repo%3Agoogle%2Fleveldb%20kDefaultMmapLimit&type=code

ACK fd8f696

Comment thread util/env_posix.cc
// Up to 4096 mmap regions for 64-bit binaries; none for 32-bit.
constexpr const int kDefaultMmapLimit = (sizeof(void*) >= 8) ? 4096 : 0;
// Up to 1000 mmap regions for 64-bit binaries; none for 32-bit.
constexpr const int kDefaultMmapLimit = (sizeof(void*) >= 8) ? 1000 : 0;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, this matches upstream and the current the env_windows.cc version

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented May 9, 2025

Yes, the motivation here is to limit the amount of divergence from upstream leveldb (especially patches they are extremely unlikely to merge), though this is a harmless, trivial change, so could also keep it i don't feel strongly about it. #50 is worse.

@davidgumberg
Copy link
Copy Markdown

ACK fd8f696

I'd be in favor of just bumping the max file size if/when we start approaching the mmap limit again.

@darosior
Copy link
Copy Markdown
Member

darosior commented Sep 9, 2025

Concept ACK. Since bitcoin/bitcoin#30039 the maximum size of the cache will be 4096 * 32MiB = 131GiB. With this PR the maximum size of the cache will be 32GiB, which is still about twice the on-disk size of the UTxO set.

Copy link
Copy Markdown
Member

@darosior darosior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK fd8f696

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants