test: sqlness compat

[discord9]

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

run cargo sqlness compat --from=1.0.0-beta.1 --to=current --preserve-state to run compat test!
perhaps also need ci to test previous few versions automatically

Still need more feature cases, but the general framework is this, the case dir might be a little to verbose, but that's the easiest way to impl this, and the general idea of compat test from rfc, and also TEMPLATE sqlness interceptor is also needed
due to metasrv need to restart, must use etcd to run which in turn need docker to run test

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

• I have written the necessary rustdoc comments.
• I have added the necessary unit tests and integration tests.
• This PR requires documentation updates.
• API changes are backward compatible.
• Schema or data changes are backward compatible.

[gemini-code-assist]

Summary of Changes

Hello @discord9, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a robust compatibility testing framework within the sqlness test suite. It introduces a new compat command that orchestrates a three-phase testing process, allowing developers to verify backward compatibility of database features across different GreptimeDB versions. The framework leverages new version-aware interceptors to precisely control test execution, ensuring that changes in newer versions do not break functionality or data created by older versions. This is a foundational improvement for maintaining stability and reliability as the project evolves.

Highlights

• New Compatibility Testing Framework: Introduced a new compat subcommand to the sqlness test runner, enabling multi-version compatibility testing for GreptimeDB. This framework allows running tests across different versions of the database binary.
• Multi-Phase Test Execution: The compatibility tests are structured into three distinct phases: 1.feature (to create features/data on an older version), 2.verify (to verify compatibility on a newer version), and 3.cleanup (to remove test artifacts).
• Version Handling and Interceptors: Added a Version enum to parse and compare semantic versions and a special 'current' version. New SQLness interceptors (IGNORE_RESULT, SINCE, TILL) were implemented to control test execution based on version compatibility and result expectations.
• New Test Case Example: Included an example compatibility test case for creating a table with index.granularity and index.false_positive_rate options, demonstrating the usage of the new framework.

Changelog

• tests/compatibility/1.feature/distributed/common/granularity_and_false_positive_rate.result
  • Added a new SQLness result file for creating a table with specific index granularity and false positive rate.
• tests/compatibility/1.feature/distributed/common/granularity_and_false_positive_rate.sql
  • Added a new SQLness test file for creating a table with specific index granularity and false positive rate.
• tests/compatibility/1.feature/standalone/common
  • Added a reference to distributed common tests for standalone mode.
• tests/compatibility/2.verify/distributed/common/show-create-table.result
  • Added a new SQLness result file to verify table creation, expecting a 'Table not found' error.
• tests/compatibility/2.verify/distributed/common/show-create-table.sql
  • Added a new SQLness test file to verify table creation.
• tests/compatibility/2.verify/standalone/common
  • Added a reference to distributed common tests for standalone mode.
• tests/compatibility/3.cleanup/distributed/common/granularity_and_false_positive_rate.result
  • Added a new SQLness result file for dropping the test table.
• tests/compatibility/3.cleanup/distributed/common/granularity_and_false_positive_rate.sql
  • Added a new SQLness test file for dropping the test table.
• tests/compatibility/3.cleanup/standalone/common
  • Added a reference to distributed common tests for standalone mode.
• tests/runner/Cargo.toml
  • Added anyhow, semver, and snafu dependencies for error handling and version parsing.
• tests/runner/src/cmd.rs
  • Integrated the new compat module and CompatCommand into the main command structure.
• tests/runner/src/cmd/bare.rs
  • Imported and registered the ignore_result interceptor factory.
• tests/runner/src/cmd/compat.rs
  • Added a new file defining the CompatCommand struct and its run method, which orchestrates compatibility tests.
• tests/runner/src/compatibility_runner.rs
  • Added a new file defining CompatibilityRunner to manage the multi-phase compatibility testing process, including version handling and interceptor registration.
• tests/runner/src/env/compat.rs
  • Added a new file defining compat::Env which wraps bare::Env for compatibility testing.
• tests/runner/src/env/mod.rs
  • Renamed tests/runner/src/env.rs to tests/runner/src/env/mod.rs and added the compat module declaration.
• tests/runner/src/interceptors/ignore_result.rs
  • Added a new file defining IgnoreResultInterceptor and its factory, which allows tests to ignore result matching for specific queries.
• tests/runner/src/interceptors/mod.rs
  • Added a new file to declare ignore_result, since, and till interceptor modules.
• tests/runner/src/interceptors/since.rs
  • Added a new file defining SinceInterceptor and its factory, which skips tests if the target version is less than a specified version.
• tests/runner/src/interceptors/till.rs
  • Added a new file defining TillInterceptor and its factory, which skips tests if the target version is greater than a specified version.
• tests/runner/src/main.rs
  • Imported new modules (compatibility_runner, interceptors, version) and updated the main command dispatch to include CompatCommand.
• tests/runner/src/version.rs
  • Added a new file defining the Version enum and its parsing/comparison logic for semantic and 'current' versions.

Activity

• The author has implemented the core framework for compatibility testing, including the compat subcommand and its associated logic.
• The PR description indicates that the general framework is in place, but more feature cases are needed, suggesting ongoing development or future work.
• The author provided an example command to run the compatibility tests: cargo sqlness compat --from=1.0.0-beta.1 --to=current --preserve-state.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive compatibility testing framework, which is a great addition for ensuring stability across different versions of GreptimeDB. The implementation closely follows the design outlined in the RFC, with a clear three-phase approach (feature, verify, cleanup) and new sqlness interceptors (SINCE, TILL, IGNORE_RESULT) to support version-aware testing.

The code is well-structured, with new responsibilities cleanly separated into new modules like cmd/compat.rs, compatibility_runner.rs, version.rs, and the interceptors directory.

My review focuses on improving error handling and correctness in a few areas:

• Refactoring the compat command's run function to use idiomatic Rust error handling with Result and the ? operator.
• Improving the version comparison logic to more robustly handle potentially unparsable versions.
• Enhancing user-facing error messages in the new interceptors.

These changes should improve the robustness and maintainability of the new testing framework. Overall, this is a solid piece of work.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aefc28f23a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5990ad874

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".