feat: Backup and Restore for model and radio settings#134
feat: Backup and Restore for model and radio settings#134pfeerick merged 41 commits intoEdgeTX:mainfrom
Conversation
- Add support for individual .yml files in restore - Add export format selection (.etx, .zip, individual files) - Add labels.yml inclusion option - Improve collision detection and handling - Auto-reset UI after restore completes
pfeerick
force-pushed
the
radio-backup
branch
from
8663995 to
e02a0d7
Compare
|
Can you check what is going on with your dependencies... it seems the reason none of the CI taks were successful is because you are pulling from github rather than npm. i.e. see last commit in #135 - I was thinking at first it was a repo permissions issue, but locking |
Pin @electron/node-gyp to npm version 10.2.0-electron.2 to avoid CI failures caused by pulling from GitHub directly. This resolves dependency installation issues in CI environments.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #134 +/- ##
==========================================
+ Coverage 34.39% 34.95% +0.56%
==========================================
Files 102 115 +13
Lines 3175 4288 +1113
Branches 776 1033 +257
==========================================
+ Hits 1092 1499 +407
- Misses 1935 2585 +650
- Partials 148 204 +56 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Define DownloadProgress type locally instead of importing from ky - Add type resolution for @types/react and @types/react-dom to avoid conflicts - Fix WebDFUSettings interface with proper null handling for interfaceName - Wrap children in Fragment in ExecuationOverlay to fix children prop error - Add explicit ReactNode casts in Layout and FlashExecution components
- Fix code formatting with prettier in Layout.tsx and FlashExecution.tsx - Wrap Playwright test hooks in test.describe() blocks - Move beforeEach/afterEach inside proper test suite contexts - Fixes compatibility-notice, firmware-selection, and flash-firmware e2e tests
- Remove usage of private ._extendTest() API which causes Playwright test failures - Inline platform fixtures directly into baseTest using public .extend() - Fixes 'Playwright Test did not expect test.describe() to be called here' error - Resolves compatibility with current Playwright version
…Test export chain - Export 'test' from browserTest.ts and electronTest.ts - Import both as aliases in pageTest.ts and conditionally export - Simplify baseTest to extend platformTest directly - This preserves the Playwright test type and resolves 'did not expect test.describe()' errors
Restores the original test fixture chain using ._extendTest() API and proper TestType typing in pageTest.ts. This matches the working configuration from main branch.
Resolves version mismatch between playwright@1.58.0 and @playwright/test@1.29.1 that caused 'test.describe() called in unexpected context' errors when running yarn e2e:web.
- Add backup.spec.ts with 17 tests covering: - Query: localBackup, sdcardModelsWithNames, checkModelCollisions, availableModelSlots - Mutation: registerLocalBackup, restoreBackupToSdcard, createBackupFromSdcard, downloadIndividualModels - Fix Uint8Array conversion for Blob in jsdom environment - Revert e2e spec files to main branch version
|
I think it's already passed the tests, I don't know why codecov hasn't run again. To be honest, this technology stack is very new to me; I left the project almost a year ago and picked it up again a month ago. I'm also not very familiar with this CI. If there's anything else I can do... |
- Add BackupScreen.spec.tsx with 4 tests for tab navigation - Add DiffViewerModal.spec.tsx with 12 tests for diff viewer functionality - Add CollisionModal.spec.tsx with 15 tests for collision handling - Add BackupFileSummary.spec.tsx with 8 tests for file summary display - Add 4 error handling tests to backup.spec.ts for GraphQL resolvers Total: 43 new tests, bringing total test count to 212. All tests passing with no lint errors.
- Add BackupCreateFlow.spec.tsx with 13 tests covering rendering, UI elements, layout, and interactions - Add BackupRestoreFlow.spec.tsx with 19 tests covering upload area, collision detection, modals, and props - Add BackupUploader.spec.tsx with 18 tests covering GraphQL queries/mutations, callbacks, and state management Total: 50 new tests added (262 tests passing overall) Increases test coverage for radio-backup feature
…ility - Cast createElement mock implementation as 'typeof document.createElement' - Fixes TypeScript errors with overloaded createElement signatures - Resolves build failures in BackupCreateFlow and BackupRestoreFlow tests
It runs on every commit pushed, but edits in place. The "error" from it simply means there is code that does not have corresponding tests... not ideal, but since I this is a stack I'm also not very familiar with either, unfortunately something I'm not going to pay too much attention to. My primary interest is "does it work" ;) And I saw all green ticks except for the skipped jobs (which should now not be skipped because of #135 - preview can only be triggered by code in this repo, not a fork due to github security/permissions), so I have high hopes :) Use the "View deployment" button below to see the live preview site (or see the deployment comments in #135). |
So I shouldn't have been making half and half changes!
|
Ok, I think I fixed one or two issues that was in the code, and hopefully didn't break anything in the process. It seems to be working ok for both B&W and colorlcd still. Given some of the tests, I was concerned it would only work with colorlcd, it backed up and restored just fine. I added src/shared/firmware-constants.ts and we might have to think that through a little more, as it may cap things out at 60 models... maybe two MAX_MODEL constants, one for B&W and one for color, and it determines which to use based on the presence of I also noticed a bug in the Also, for the collision detection... what should we say it does when enabled... as presently the user only know what it does when disabled ;) Note: I am avoiding the Also, future housekeeping todo, should ignore MacOS |
- Changed the prompt from "When disabled, you'll be asked how to handle models that already exist on your SD card" to "When enabled, you'll be asked how to handle models that already exist on your SD card" across multiple locale files (be, cs, da, de, es, fr, it, ru, sv, uk, zh). feat(backup): include labels.yml in .etx backups for Companion compatibility - Updated the BackupCreateFlow component to always include labels when the radio has them for .etx format. - Added a tooltip to indicate that labels.yml is always included in .etx backups. refactor(backup): improve collision detection and model slot handling - Replaced overwriteExisting state with collisionDetection in BackupRestoreFlow. - Updated logic to check for collisions based on the new collisionDetection state. - Enhanced model slot naming based on radio type (B&W vs ColourLCD) using new utility functions. test(backup): add tests for model slots and RADIO/radio.yml handling - Added tests to ensure correct model slot availability based on existing models and labels.yml presence. - Included tests for the inclusion and restoration of RADIO/radio.yml in backups. refactor(backupStore): streamline model file validation and handling - Introduced isValidModelFile function to filter out invalid model files (macOS resource forks and labels). - Updated backup and restore functions to utilize the new validation logic. refactor(firmware-constants): enhance model slot management - Added getMaxModels and modelSlotName functions to manage model slots based on radio type. - Deprecated MAX_MODELS in favor of using specific constants for B&W and ColourLCD radios. fix(zipParser): ensure valid model files are extracted from ZIP - Updated the extractYamlFromZip function to filter out invalid model files using isValidModelFile.
|
Significant improvements have been made to the Backup & Restore system, starting with full dual support for B&W and ColourLCD radios. The radio type is now auto-detected by checking for the presence of labels.yml inside /MODELS, and slot filenames are generated accordingly. B&W radios use zero-padded, 0-indexed filenames (model00.yml–model59.yml), while ColourLCD radios use 1-indexed filenames (model1.yml–model99.yml). To reflect this, MAX_MODELS_BW = 60 and MAX_MODELS_COLOR = 99 were introduced along with a getMaxModels() helper. The original MAX_MODELS = 60 constant remains as a deprecated alias for backward compatibility, or maybe delete it. For .etx backups, labels.yml is now always included when present, as it is required for Companion compatibility. The checkbox is displayed as checked and disabled with a tooltip explanation. RADIO/radio.yml is also included and restored when available, and gracefully skipped on B&W radios. For .zip and individual exports, labels.yml remains user-controlled. The collision detection toggle logic has been corrected. When enabled (default), users are prompted to rename, overwrite, or cancel if conflicts are detected. When disabled, models are overwritten silently. The description was updated to match the actual behavior. Filtering was added for macOS resource fork files (._*) across model listing, backup creation, restore, collision checks, and ZIP parsing, preventing these metadata files from being treated as valid models. Finally, a production Electron crash was fixed. Dynamic imports of fflate and yaml in the renderer caused Webpack to create separate chunks that could not be loaded from app.asar, resulting in a chunk loading error. All dynamic imports in BackupRestoreFlow.tsx were replaced with static imports, ensuring they are bundled into the main chunk and work correctly in production. BackupCreateFlow was unaffected since compression there runs in the Node.js backend via GraphQL rather than in the renderer. |
…on and text file support - Added support for selecting an SD Card to enable restore in English and Chinese locales. - Updated BackupRestoreFlow to improve collision detection logic and handle radio settings appropriately. - Enhanced CollisionModal to differentiate between model and radio file collisions, providing better user feedback. - Modified BackupUploadArea to allow .txt files and improve restore button behavior based on SD Card selection. - Updated backup tests to cover scenarios involving radio.yml and text files, ensuring proper collision detection and restoration behavior. - Refactored firmware-constants to remove deprecated MAX_MODELS constant and added utility for identifying model text files.
|
Teh last commit fixes the loss of RADIO/radio.yml when restoring from a .etx backup and adds proper support for MODELS/.txt files. Previously, restoring from an .etx generated a temporary ZIP containing only parsed model .yml files, which caused radio.yml to be discarded. Now the frontend reuses the original backup ID, ensuring the complete archive (including radio.yml) is restored and used for collision detection. On the backend, radio.yml now respects the overwriteExisting option, so an existing file on the SD card is preserved if overwrite is disabled. If both the backup and the SD contain a radio.yml, it is included in the collision modal as a separate entry (since it cannot be renamed, only overwritten or skipped). Additionally, this commit adds full backup and restore support for MODELS/.txt files, such as model notes created by Companion. A new isModelTextFile() helper identifies valid .txt files, they are now included when creating backups, and they are restored to the SD card while respecting the overwriteExisting behavior, consistent with model files. A deprecated constant is removed, and some aesthetic adjustments are made. |
- Extract labels.yml from backup ZIP during restore instead of ignoring it - Preserve label assignments, bitmap paths and lastopen timestamps from backup - Merge backup Labels categories into existing ones on target SD card - Create labels.yml even when target SD card has none (previously skipped) - Fall back to empty stubs only when backup has no labels.yml
# Conflicts: # locales/be/flashing.json # locales/cs/flashing.json # locales/da/flashing.json # locales/de/flashing.json # locales/es/flashing.json # locales/fr/flashing.json # locales/it/flashing.json # locales/ru/flashing.json # locales/sv/flashing.json # locales/uk/flashing.json # locales/zh/flashing.json # src/shared/dfu/index.ts # yarn.lock
react-ga was removed from the upstream repo (tracking.tsx deleted). Replace exception() call with console.error to fix CI failures: - TS2307: Cannot find module 'react-ga' - no-unsafe-call ESLint error on exception() typed as any
|
Fix labels.yml not restored from .etx/.zip backups When restoring a backup, labels.yml was silently ignored because the entry filter (isValidModelFile) explicitly excludes it, the same filter used for model files. Unlike RADIO/radio.yml and .txt files, there was no separate step to extract it from the ZIP. Additionally, even when the destination SD card already had a labels.yml, restored models were written with empty stubs (labels: "", bitmap: ""), discarding all label assignments from the backup. Changes in restoreBackupToDirectory: Reads MODELS/labels.yml from the backup ZIP (same pattern already used for RADIO/radio.yml) |
|
EdgeTX team is happy to announce that this submission is the winning submission of EdgeTX 6th developer contest. Héctor Narbona Márquez (GitHub and EdgeTX Discord user The state-of-the-art radio and the receiver will be shipped directly from the RadioMaster factory to Héctor. We extend our heartfelt congratulations to Héctor Narbona Márquez for this well-deserved win! |
pfeerick
changed the title
3 participants
The Radio Backup feature provides a comprehensive two-way backup solution for EdgeTX radio models with dual workflows: Restore Backup (upload and restore models to SD card) and Create Backup (export models from SD card). The restore flow allows users to upload multiple file formats (.etx, .zip, or individual .yml files), mix models from different sources, and preview/select which models to restore. The create flow enables exporting selected models in three formats: single .etx file (EdgeTX standard), .zip archive, or individual .yml files, with optional labels.yml inclusion. Both workflows leverage native File System Access API for direct SD card access in browsers and Electron.
The standout feature is the intelligent collision detection system that automatically identifies models that already exist on the SD card and provides three resolution strategies: overwrite all existing models, manually rename conflicting models, or use auto-rename to assign available slots (model01-model60). Users can view side-by-side diffs with line-by-line highlighting to compare backup versions against existing models before deciding. The system automatically manages labels.yml synchronization, updating it with restored models and filtering it when creating backups of selected models.
The implementation includes a GraphQL backend API with queries for collision checking and model retrieval, mutations for backup registration and restoration, and an in-memory LRU cache storing up to 4 recent backups. The UI features drag-and-drop upload, real-time progress tracking, comprehensive error handling, file validation (100MB limit), and full internationalization support across 12 languages. All operations are non-blocking with visual feedback, making it a production-ready, enterprise-grade backup solution.