Skip to content

branch-4.1: (tools) Add show_segment_data operation to meta_tool#62385

Merged
yiguolei merged 2 commits intoapache:branch-4.1from
gavinchou:gavin-pick-60608-branch-4.1
Apr 17, 2026
Merged

branch-4.1: (tools) Add show_segment_data operation to meta_tool#62385
yiguolei merged 2 commits intoapache:branch-4.1from
gavinchou:gavin-pick-60608-branch-4.1

Conversation

@gavinchou
Copy link
Copy Markdown
Contributor

Summary

Pick #60608 to branch-4.1

Add new operation show_segment_data to meta_tool for reading and displaying segment file column metadata and statistics.

Changes

  • Add get_field_type_string() to convert FieldType to readable string
  • Add get_encoding_string() to convert EncodingTypePB to readable string
  • Add get_compression_string() to convert CompressionTypePB to readable string
  • Add print_column_meta() to print column metadata recursively
  • Add show_segment_data() to display segment file info and statistics
  • Add common/cpp/private_member_accessor.hpp helper for accessing private members

Test

  • Compilation passed (BE meta_tool target)

Related Issue

close #60608

gavinchou and others added 2 commits April 11, 2026 14:35
Add new operation show_segment_data to meta_tool for reading and displaying
segment file column metadata and statistics.

- Add get_field_type_string() to convert FieldType to readable string
- Add get_encoding_string() to convert EncodingTypePB to readable string
- Add get_compression_string() to convert CompressionTypePB to readable string
- Add print_column_meta() to print column metadata recursively
- Add show_segment_data() to display segment file info and statistics

This allows users to inspect segment file structure without requiring
the full tablet schema.

```
meta_tool --operation=show_segment_data --file=020000000000004fd4412b500cb0833afcc5dd12e833c59b_0.dat

=== Segment File Info ===
File: 020000000000004fd4412b500cb0833afcc5dd12e833c59b_0.dat
Num Rows: 2
Num Columns: 16
Compression: LZ4F

=== column_id_0: type=VARCHAR, nullable=false, encoding=DICT_ENCODING ===
Dictionary Page: offset=56, size=53 bytes
Indexes: ORDINAL, ZONE_MAP
Data Values (2 of 2 rows, showing first 2):
  [0] '10023--1-id'
  [1] '10023--1-score'

=== column_id_1: type=VARCHAR, nullable=false, encoding=DICT_ENCODING ===
Dictionary Page: offset=164, size=25 bytes
Indexes: ORDINAL, ZONE_MAP
Data Values (2 of 2 rows, showing first 2):
  [0] '0'
  [1] '0'

...
```
- Update namespace from doris::vectorized to doris for IColumn, ColumnNullable, MutableColumnPtr, DataTypeFactory
- Remove duplicate vec/ includes (branch-4.1 uses core/ paths)
- Fix ACCESS_PRIVATE_FIELD macro types to match branch-4.1 ExecEnv members
- Fix variable name tracking -> tracking_memory
@gavinchou gavinchou requested a review from yiguolei as a code owner April 11, 2026 07:12
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@gavinchou
Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.96% (19874/37524)
Line Coverage 36.53% (187233/512517)
Region Coverage 32.84% (145346/442530)
Branch Coverage 33.98% (63693/187431)

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.30% (26925/36731)
Line Coverage 56.75% (289874/510800)
Region Coverage 54.26% (242236/446474)
Branch Coverage 55.91% (105082/187957)

@gavinchou gavinchou changed the title (tools) Add show_segment_data operation to meta_tool branch-4.1: (tools) Add show_segment_data operation to meta_tool Apr 12, 2026
@yiguolei yiguolei merged commit 3105e9d into apache:branch-4.1 Apr 17, 2026
26 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants