Best Practices for Storing Superset Dashboards in Git

[ampil]

Hi all,

I'm exploring the best practices for storing Superset dashboards in Git for version control purposes.

I haven’t come across many articles or discussions on this topic, so I wanted to reach out to the community for guidance.

Currently, my team and I follow this approach:

• Export newly developed dashboards as a zip archive.
• We have a 'Superset' folder in our Git repo with subfolders for charts, dashboards, databases, and datasets.
• We then place the contents of the zip (YAML files) into the corresponding directories and remove the ID numbers from the filenames. (for example dashboard Legal_Information_40.yaml becomes Legal_Information.yaml)
• Finally, we commit and push the changes.

As a result, each folder has a bunch of YAML-files. We don't store zip-archives.

Does this workflow align with best practices, or is there a more efficient way to handle version control for Superset dashboards? Any feedback, suggestions, or resources would be greatly appreciated!

Thanks in advance!

[dosubot[bot]]

Hey @ampil! 👋 I'm here to help you with any questions or issues you have. I can assist with bugs, answer your questions, and even guide you on becoming a contributor. Let's tackle this together!

The best practices for storing Superset dashboards in Git for version control include:

1. Exporting Dashboards as YAML Files: Use the ExportDashboardsCommand to export dashboards to YAML files, which are more suitable for version control compared to binary formats like zip archives.

2. Organizing Files into Subfolders: Organize the exported YAML files into logical subfolders (e.g., dashboards, charts, datasets, databases) for better manageability.

3. Removing ID Numbers from Filenames: Customize the filenames to exclude ID numbers if desired, using a consistent naming convention.

Here is a more efficient way to handle this process:

1. Automate the Export Process: Use the ExportDashboardsCommand to automate the export of dashboards to YAML files.
2. Use a Consistent Naming Convention: Customize the get_filename function to exclude ID numbers from filenames if desired.
3. Organize Files in a Logical Directory Structure: Ensure that the exported YAML files are organized into appropriate subfolders.

Example code snippet to export dashboards:

from superset.commands.dashboard.export import ExportDashboardsCommand

# Export a specific dashboard by its ID
dashboard_id = 1  # Replace with the actual dashboard ID
export_command = ExportDashboardsCommand([dashboard_id])
for file_name, file_content_callable in export_command.run():
    file_content = file_content_callable()
    with open(file_name, 'w') as file:
        file.write(file_content)

This approach leverages the existing ExportDashboardsCommand to efficiently export dashboards as YAML files, which can then be version-controlled in Git ^[1].

To continue talking to Dosu, mention @dosu.

[ampil]

Our team explored two competing approaches for organizing dashboard configuration files.

Approach 1: Centralized Directory Structure

In this approach, all dashboard YAML files were stored in a common folder hierarchy, as follows:

superset/
│
├── charts/
│   ├── chart_name_1.yaml
│   ├── chart_name_2.yaml
│   ├── chart_name_3.yaml
│   └── etc.
│
├── dashboards/
│   ├── dashboard_name_1.yaml
│   ├── dashboard_name_2.yaml
│   ├── dashboard_name_3.yaml
│   └── etc.
│
├── databases/
│   ├── Oracle_db.yaml
│   ├── Postgre_db.yaml
│   ├── Postgre_Test_db.yaml
│   └── etc.
│
└── datasets/
    ├── Oracle_db/
    │   ├── dataset_1.yaml
    │   ├── dataset_2.yaml
    │   ├── dataset_3.yaml
    │   └── etc.
    ├── Postgre_db/
    │   ├── dataset_4.yaml
    │   ├── dataset_5.yaml
    │   ├── dataset_6.yaml
    │   └── etc.
    └── etc.

Approach 2: Individual Dashboard Folders

This approach involved creating a separate folder for each dashboard, organizing data as follows:

superset/
├── dashboard_name_1/
│   ├── charts/
│   │   ├── chart_name_1.yaml
│   │   └── chart_name_2.yaml
│   ├── dashboards/
│   │   └── dashboard_name_1.yaml
│   ├── databases/
│   │   └── Oracle_db.yaml
│   └── datasets/
│       └── Oracle_db/
│           └── dataset_1.yaml
│
└── dashboard_name_2/
   ├── charts/
   │   └── chart_name_3.yaml
   ├── dashboards/
   │   └── dashboard_name_2.yaml
   ├── databases/
   │   └── Postgre_db.yaml
   └── datasets/
       └── Postgre_db/
           └── dataset_4.yaml

Decision

We opted for the second approach. It offers several significant advantages over the first:

• Enhanced Findability: Each dashboard's data is easily accessible.
• Simplified Import: We can quickly zip folder contents and import reports back into Superset. The first approach would require developing a script to look up UUIDs in each YAML file.
• Clearer Code Review: When reviewing code in Git, it's much simpler to understand file relationships to a specific dashboard.

Hope it helps other Superset fellows :)

[spens]

For your use case, is it possible to have dataset_4.yaml used in dashboard_name_3 and dashboard_name_4? How do you handle that? Do you just store a duplicate copy of the dataset in the repo under each dashboard?

[ampil]

Yes, that's possible. Actually, we have that a lot - our filtering datasets are shared between many dashboards. For example, one dataset containing dates is utilized in 10+ dashboards.

The tradeoff is between:
(a) keeping it simple for humans (the team).
Here we isolate each dashboard data while monitoring shared datasets. In case of changes in one dataset, all associated yaml files in all related dashboards must be changed.

(b) optimizing it for computers.
This means avoiding duplicates, optimizing storage space, and ensuring that our YAML files are always up to date. On the other hand, we need additional tools to create zip files for each dashboard and to pay closer attention to shared datasets during the review stage.

Since our inputs are:

• 'custom' datasets that are distinguished between each dashboard and
• 'simple' shared datasets that are not likely to change (like a column with dates, numbers, text like "Yes" and "No") ,

We decided to stick with option (a) until a better solution is found or the workload changes.

[meyyazhagan]

@ampil Thanks for the highlighting the approaches. I was trying to implementation the similar approach of your approach 2. Wondering to ask to get some reference with sample yaml for dashboard/dataset/charts? Will you be help me with some lights or some reference link to have a look? Much appreciated for your response.

[ampil]

Hi @meyyazhagan, those yaml files are simply files from the .zip archive you get while exporting the dashboard, nothing more. That's why I don't have any references or links - just use what you already have.

[meyyazhagan]

Many thanks for the note @ampil

Export/Import approach works fine on same environment. if I try to export from one env and import them on other, it fails.

Hence wondering to see an options to implement the superset as code eventually it works for all environment by using environment specific changes over pipeline.

Error log:

2025-07-02 08:40:34,555:INFO:superset.commands.dashboard.importers.dispatcher:Command failed validation
There was an error when importing the dashboards(s), please check the exception traceback in the log
Traceback (most recent call last):
File "/app/superset/cli/importexport.py", line 158, in import_dashboards
ImportDashboardsCommand(contents, overwrite=True).run()
File "/app/superset/commands/dashboard/importers/dispatcher.py", line 64, in run
raise exc
File "/app/superset/commands/dashboard/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/commands/importers/v1/init.py", line 71, in run
self.validate()
File "/app/superset/commands/importers/v1/init.py", line 109, in validate
raise CommandInvalidError(
superset.commands.exceptions.CommandInvalidError: Error importing dashboard
2025-07-02 08:40:34,555:ERROR:superset.cli.importexport:There was an error when importing the dashboards(s), please check the exception traceback in the log
Traceback (most recent call last):
File "/app/superset/cli/importexport.py", line 158, in import_dashboards
ImportDashboardsCommand(contents, overwrite=True).run()
File "/app/superset/commands/dashboard/importers/dispatcher.py", line 64, in run
raise exc
File "/app/superset/commands/dashboard/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/commands/importers/v1/init.py", line 71, in run
self.validate()
File "/app/superset/commands/importers/v1/init.py", line 109, in validate
raise CommandInvalidError(
superset.commands.exceptions.CommandInvalidError: Error importing dashboard

[ampil]

@meyyazhagan , yes that's a usual error. I don't have an automatic solution yet, my daily routine is to change datasets' source related with the dashboard. Then I export the dashboard from one environment and import it to another environment. If it works correctly (meaning no errors during import), I go back to my previous environment and revert connections back.

Idea 1: you may disable database connection check during the dashboard import process in the python code. You may check /app/superset/cli/importexport.py and play with the lines related with import error handling
https://github.com/apache/superset/blob/master/superset/cli/importexport.py#L308

Idea 2: think this PR #33250 might help. It is incorporated into Superset 5.0.0. Not tried that yet.

[gaeljw]

I'm dumping automatically the result of Superset export ZIP into a Git repository for traceability/history/indexation and came upon this thread.

I'm facing a weird behaviour where some IDs change in the dashboard exports even though nothing changed in the dashboards themselves. Is this expected? This makes the "history" feature less interesting due to commits with "fake changes".

Here's an example of such a "fake changes/diff" that I observe in a dashboard yaml export:

    parents:
    - ROOT_ID
    type: TABS
-  CHART-VAEM8ECF:
+  CHART-FU08RIS4:
    children: []
    id: CHART-VAEM8ECF
    id: CHART-FU08RIS4
    meta:
      chartId: 1347
      height: 50
      ...
      uuid: 9dfdd1dc-5808-4c33-bb7d-f46712891c58
      width: 4
    type: CHART
-  CHART-OYGMTK86:
+  CHART-4J8783D4:
    children: []
-    id: CHART-OYGMTK86
+    id: CHART-4J8783D4
    meta:
      chartId: 1333
      height: 50