Skip to content

refactor(compression): Unify compression APIs with libarchive backend#13788

Merged
HTRamsey merged 1 commit intomavlink:masterfrom
HTRamsey:refactor/unified-compression-api
Jan 9, 2026
Merged

refactor(compression): Unify compression APIs with libarchive backend#13788
HTRamsey merged 1 commit intomavlink:masterfrom
HTRamsey:refactor/unified-compression-api

Conversation

@HTRamsey
Copy link
Collaborator

Summary

Replace fragmented compression utilities (QGCLZMA, QGCZlib, QGCZip) with a unified QGCCompression interface backed by libarchive.

Changes

New Unified API

  • QGCCompression namespace with format detection and unified operations
  • QGClibarchive for ZIP archive operations via libarchive
  • Support for GZIP, XZ, ZSTD, ZIP, TAR formats with auto-detection

Compression Library Stack (via CPM)

Library Version Purpose
zlib-ng 2.3.2 Optimized zlib replacement
xz/liblzma 5.8.2 LZMA/XZ compression
zstd 1.5.7 Zstandard compression
lz4 1.10.0 Fast compression
bzip2 1.1.0 Bzip2 compression
libarchive 3.8.4 Archive handling

API Simplification

// Old API - multiple modules, manual format handling
if (fileName.endsWith(".lzma") || fileName.endsWith(".xz")) {
    QGCLZMA::inflateLZMAFile(fileName, outputFileName);
} else if (fileName.endsWith(".gz")) {
    QGCZlib::inflateGzipFile(fileName, outputFileName);
}

// New API - unified, auto-detecting
QString output = QGCCompression::decompressIfNeeded(fileName);

Removed

  • QGCLZMA module (replaced by QGCCompression)
  • QGCZlib module (replaced by QGCCompression)
  • QGCZip module (replaced by QGClibarchive)
  • Duplicate test files (consolidated into QGCCompressionTest)

Test Plan

  • QGCCompressionTest passes (10 tests)
    • Format detection (extension and magic bytes)
    • ZIP roundtrip with nested directories and empty files
    • ZIP extraction from Qt resources
    • GZIP and LZMA decompression
  • Build succeeds on Linux
  • Build verification on Windows/macOS/Android/iOS

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors QGroundControl's compression utilities by replacing three separate modules (QGCLZMA, QGCZlib, QGCZip) with a unified QGCCompression API backed by libarchive. The changes introduce automatic format detection, support for multiple compression formats (GZIP, XZ, ZSTD, ZIP, TAR variants), and consolidate the dependency management through CPM.

Key Changes

  • Introduced unified QGCCompression namespace with format detection and single API for all compression operations
  • Added QGClibarchive wrapper for ZIP archive operations using libarchive instead of Qt's private QZipWriter/QZipReader
  • Integrated 6 compression libraries (zlib-ng, xz/liblzma, zstd, lz4, bzip2, libarchive) via CPM with proper dependency configuration

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
src/Utilities/Compression/QGCCompression.h New unified compression API header with format detection and operations for single-file and archive compression
src/Utilities/Compression/QGCCompression.cc Implementation of unified compression API with libarchive-based decompression for GZIP, XZ, ZSTD formats
src/Utilities/Compression/QGClibarchive.h Header for libarchive-based ZIP archive operations
src/Utilities/Compression/QGClibarchive.cc Implementation of ZIP creation/extraction with Qt resource support and security checks
src/Utilities/Compression/CMakeLists.txt Build configuration for 6 compression libraries with CPM integration and cross-library compatibility setup
src/Utilities/Compression/QGCLZMA.{h,cc} Removed - replaced by QGCCompression
src/Utilities/Compression/QGCZlib.{h,cc} Removed - replaced by QGCCompression
src/Utilities/Compression/QGCZip.{h,cc} Removed - replaced by QGClibarchive
src/Vehicle/VehicleSetup/FirmwareUpgradeController.cc Updated to use QGCCompression::decompressFile() instead of QGCZlib::inflateGzipFile()
src/Vehicle/ComponentInformation/ComponentInformationTranslation.cc Simplified decompression logic using QGCCompression::decompressIfNeeded()
src/Vehicle/ComponentInformation/ComponentInformationManager.cc Simplified decompression logic using QGCCompression::decompressIfNeeded()
src/Camera/VehicleCameraControl.cc Replaced manual LZMA decompression with QGCCompression::decompressIfNeeded()
test/Utilities/Compression/QGCCompressionTest.{h,cc} New comprehensive test suite with 10 tests for format detection, ZIP operations, and single-file decompression
test/Utilities/Compression/DecompressionTest.{h,cc} Removed - consolidated into QGCCompressionTest
test/Utilities/Compression/CMakeLists.txt Updated to reference new test files
test/UnitTestList.cc Updated test registration from DecompressionTest to QGCCompressionTest
test/CMakeLists.txt Updated test name reference

Comment on lines 26 to 35
void QGCCompressionTest::cleanup()
{
delete _testDataDir;
_testDataDir = nullptr;

delete _tempOutputDir;
_tempOutputDir = nullptr;

UnitTest::cleanup();
}
Copy link

Copilot AI Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test creates temporary directories but doesn't verify that cleanup happened correctly. The QTemporaryDir destructor should handle cleanup, but if tests fail midway, it's good practice to verify the directories are cleaned up. Additionally, the test doesn't check that the temporary directories are actually deleted after cleanup() is called.

Copilot uses AI. Check for mistakes.
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from dc869a3 to 3f327b5 Compare December 20, 2025 14:50
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 3f327b5 to 444b68f Compare December 20, 2025 15:16
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 444b68f to 8bd1dc4 Compare December 20, 2025 16:11
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 8bd1dc4 to bc485bf Compare December 20, 2025 16:19
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from bc485bf to b7c9f30 Compare December 20, 2025 16:47
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from b7c9f30 to 3c837c7 Compare December 20, 2025 17:17
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 3c837c7 to 2303cf1 Compare December 20, 2025 17:24
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 2303cf1 to 2de2213 Compare December 20, 2025 19:26
@github-actions github-actions bot added size/XL and removed size/XL labels Dec 20, 2025
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch 3 times, most recently from f3b6757 to 7613cc7 Compare January 8, 2026 18:36
@github-actions github-actions bot added the github_actions Pull requests that update GitHub Actions code label Jan 8, 2026
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch 2 times, most recently from 47ab336 to 8052de9 Compare January 8, 2026 19:42
@HTRamsey HTRamsey requested a review from Copilot January 8, 2026 19:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 102 out of 109 changed files in this pull request and generated no new comments.

@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch 3 times, most recently from 8eb4057 to 699413c Compare January 8, 2026 23:26
…cture

Complete refactor of QGC compression system using libarchive as the single
backend for all archive operations (ZIP, TAR, GZIP, etc.).

Key changes:
- Unified QGCCompression API with decompression-only support
- QGCCompressionJob for async operations with progress tracking
- QGClibarchive backend replacing multiple format-specific handlers
- QGCArchiveModel for QML-friendly archive browsing
- QGCArchiveWatcher for monitoring archive directories

Enhanced file download infrastructure:
- QGCFileDownload with progress, retry logic, and range requests
- QGCCachedFileDownload with ETag/Last-Modified validation
- Concurrent download management
- Removed http_parser dependency (inline HTTP status handling)

Includes comprehensive test coverage for compression and downloads.
@HTRamsey HTRamsey force-pushed the refactor/unified-compression-api branch from 699413c to 05cf552 Compare January 9, 2026 01:52
@HTRamsey HTRamsey merged commit 4c161cb into mavlink:master Jan 9, 2026
16 of 17 checks passed
@HTRamsey HTRamsey deleted the refactor/unified-compression-api branch January 9, 2026 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CMake github_actions Pull requests that update GitHub Actions code size/XL Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant