Describe the problem
mma's ByteSize capacity derivation uses filesystem utilization (Capacity - Available) / Capacity to derive a logical capacity. However, this assumes that physical bytes and logical bytes scale proportionally, which is not accurate due to fixed disk overhead (reserved blocks, ballast, other files).
|
load[mmaprototype.ByteSize] = mmaprototype.LoadValue(desc.Capacity.LogicalBytes) |
|
// Available does not compensate for the ballast, so utilization will look |
|
// higher than actual. This is fine since the ballast is small (default is |
|
// 1% of capacity) and is for use in an emergency. |
|
byteSizeUtil := |
|
float64(desc.Capacity.Capacity-desc.Capacity.Available) / float64(desc.Capacity.Capacity) |
|
almostZeroUtil := byteSizeUtil < 0.01 |
|
if load[mmaprototype.ByteSize] != 0 && !almostZeroUtil { |
|
// Normal case. The store has some ranges, and is not almost empty. |
|
capacity[mmaprototype.ByteSize] = mmaprototype.LoadValue(float64(load[mmaprototype.ByteSize]) / byteSizeUtil) |
(Capacity - Available) includes:
- capacity.used (which includes ballast file (~1% of disk))
- reserved blocks (~5% on ext4)
- other files outside of the cockroachdb directory (core dumps, etc.)
These fixed overheads don't scale with LogicalBytes, causing:
- Underestimated capacity for small stores: when LogicalBytes is small, fixed overhead dominates, making cpacity appear much lower than it should be.
- Inconsistent derived capacity: derived capacity[ByteSize] varies based on fixed overhead, not just the actual data.
Jira issue: CRDB-59337
Epic CRDB-55052
Describe the problem
mma's ByteSize capacity derivation uses filesystem utilization (Capacity - Available) / Capacity to derive a logical capacity. However, this assumes that physical bytes and logical bytes scale proportionally, which is not accurate due to fixed disk overhead (reserved blocks, ballast, other files).
cockroach/pkg/kv/kvserver/mmaintegration/store_load_msg.go
Lines 114 to 123 in 494323e
(Capacity - Available) includes:
These fixed overheads don't scale with LogicalBytes, causing:
Jira issue: CRDB-59337
Epic CRDB-55052