Skip to content

Commit 87ea5b8

Browse files
authored
add disable prealloc (#23209)
### **User description** ## What type of PR is this? - [ ] API-change - [x] BUG - [ ] Improvement - [ ] Documentation - [ ] Feature - [ ] Test and CI - [ ] Code Refactoring ## Which issue(s) this PR fixes: issue # ## What this PR does / why we need it: ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add `LogDBDisablePrealloc` configuration option to disable file preallocation - Propagate the setting to the underlying dragonboat LogDB configuration - Update dragonboat dependency to version with prealloc support - Support filesystems like NFS that don't support fallocate system call ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Config struct"] -- "LogDBDisablePrealloc field" --> B["getNodeHostConfig"] B -- "assigns to logdb.DisablePrealloc" --> C["dragonboat LogDB"] D["go.mod dependency"] -- "updated to v4.0.0-20251203123148" --> C ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>config.go</strong><dd><code>Add LogDBDisablePrealloc configuration field</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary> <hr> pkg/logservice/config.go <ul><li>Added <code>LogDBDisablePrealloc</code> boolean field to Config struct<br> <li> Field is mapped to TOML configuration key <code>logdb-disable-prealloc</code><br> <li> Includes documentation explaining the purpose for NFS and similar <br>filesystems</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/23209/files#diff-840aed48ffce69ec32afc8354a2223c3029b0dd64f3eef440283608154470c1b">+3/-0</a>&nbsp; &nbsp; &nbsp; </td> </tr> <tr> <td> <details> <summary><strong>store.go</strong><dd><code>Propagate prealloc disable setting to LogDB</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary> <hr> pkg/logservice/store.go <ul><li>Propagates <code>LogDBDisablePrealloc</code> config value to dragonboat's logdb <br>configuration<br> <li> Assigns <code>cfg.LogDBDisablePrealloc</code> to <code>logdb.DisablePrealloc</code> in <br>getNodeHostConfig function</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/23209/files#diff-c0fcef3144fd52dba31ca97e6755c28d2bba11b6837fa1585fb17199b7a12e72">+1/-0</a>&nbsp; &nbsp; &nbsp; </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>go.mod</strong><dd><code>Update dragonboat dependency version</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary> <hr> go.mod <ul><li>Updated dragonboat/v4 dependency from <br>v4.0.0-20241019050137-1c6138e9cf8b to <br>v4.0.0-20251203123148-9dcde946363f<br> <li> New version includes support for DisablePrealloc configuration option</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/23209/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6">+1/-1</a>&nbsp; &nbsp; &nbsp; </td> </tr> <tr> <td> <details> <summary><strong>go.sum</strong><dd><code>Update dragonboat dependency checksums</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary> <hr> go.sum <ul><li>Updated hash entries for dragonboat/v4 dependency to match new version<br> <li> Reflects the updated dependency version in go.sum checksums</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/23209/files#diff-3295df7234525439d778f1b282d146a4f1ff6b415248aaac074e8042d9f42d63">+2/-2</a>&nbsp; &nbsp; &nbsp; </td> </tr> </table></td></tr></tr></tbody></table> </details> ___
1 parent 8d3e34b commit 87ea5b8

File tree

4 files changed

+7
-3
lines changed

4 files changed

+7
-3
lines changed

go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,7 +247,7 @@ replace github.com/hashicorp/memberlist => github.com/matrixorigin/memberlist v0
247247
replace (
248248
github.com/elastic/gosigar v0.14.2 => github.com/matrixorigin/gosigar v0.14.3-0.20241204071856-40aab500bfac
249249
github.com/fagongzi/goetty/v2 v2.0.3-0.20230628075727-26c9a2fd5fb8 => github.com/matrixorigin/goetty/v2 v2.0.0-20240611082008-a4de209fff3d
250-
github.com/lni/dragonboat/v4 v4.0.0-20220815145555-6f622e8bcbef => github.com/matrixorigin/dragonboat/v4 v4.0.0-20241019050137-1c6138e9cf8b
250+
github.com/lni/dragonboat/v4 v4.0.0-20220815145555-6f622e8bcbef => github.com/matrixorigin/dragonboat/v4 v4.0.0-20251203123148-9dcde946363f
251251
github.com/lni/goutils v1.3.1-0.20220604063047-388d67b4dbc4 => github.com/matrixorigin/goutils v1.3.1-0.20220604063047-388d67b4dbc4
252252
github.com/lni/vfs v0.2.1-0.20220616104132-8852fd867376 => github.com/matrixorigin/vfs v0.2.1-0.20220616104132-8852fd867376
253253
)

go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -533,8 +533,8 @@ github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0V
533533
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
534534
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
535535
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
536-
github.com/matrixorigin/dragonboat/v4 v4.0.0-20241019050137-1c6138e9cf8b h1:DQMmYVMdZrmbJWu8KYZhtFEe+3FWKkYx0LKAiEL4ycY=
537-
github.com/matrixorigin/dragonboat/v4 v4.0.0-20241019050137-1c6138e9cf8b/go.mod h1:bR6ZGoUwApH4/P4+AezNGT0xehljg+j+Z/Q2Fz+Y4a0=
536+
github.com/matrixorigin/dragonboat/v4 v4.0.0-20251203123148-9dcde946363f h1:3iDaudrS54/39kxYV/7zn55+KbY7/qT9R7A++iD8d40=
537+
github.com/matrixorigin/dragonboat/v4 v4.0.0-20251203123148-9dcde946363f/go.mod h1:bR6ZGoUwApH4/P4+AezNGT0xehljg+j+Z/Q2Fz+Y4a0=
538538
github.com/matrixorigin/goetty/v2 v2.0.0-20240611082008-a4de209fff3d h1:wSlkJlWXZ3if1sH8Bc/lUUOWhTw91lgYGSFOHqy1tcw=
539539
github.com/matrixorigin/goetty/v2 v2.0.0-20240611082008-a4de209fff3d/go.mod h1:OwIBpVwRW1HjF/Jhc2Av3UvG2NygMg+bdqGxZaqwhU0=
540540
github.com/matrixorigin/gosigar v0.14.3-0.20241204071856-40aab500bfac h1:KRPOwOcSZRfWT7w8mr7BmFLoFB75ljgqjkg47xkN/Pc=

pkg/logservice/config.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,9 @@ type Config struct {
116116
LogDBBufferSize uint64 `toml:"logdb-buffer-size"`
117117
// LogDBMaxLogFileSize is the max size of the log db file.
118118
LogDBMaxLogFileSize uint64 `toml:"logdb-max-log-file-size"`
119+
// LogDBDisablePrealloc disables file preallocation using fallocate. This is useful
120+
// when the filesystem (e.g., NFS) does not support fallocate system call.
121+
LogDBDisablePrealloc bool `toml:"logdb-disable-prealloc"`
119122
// GossipAddress is the address used for accepting gossip communication.
120123
// It is deprecated and will be removed.
121124
GossipAddress string `toml:"gossip-address" user_setting:"advanced"`

pkg/logservice/store.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ func getNodeHostConfig(cfg Config, fs fileservice.FileService) config.NodeHostCo
7979
logdbFactory := (config.LogDBFactory)(nil)
8080
logdbFactory = tan.Factory
8181
logdb.MaxLogFileSize = cfg.LogDBMaxLogFileSize
82+
logdb.DisablePrealloc = cfg.LogDBDisablePrealloc
8283
if cfg.UseTeeLogDB {
8384
logutil.Warn("using tee based logdb backed by pebble and tan, for testing purposes only")
8485
logdbFactory = tee.TanPebbleLogDBFactory

0 commit comments

Comments
 (0)