STAC Hackathon

🌏 Welcome to the DEA Notebooks ODC-STAC Hackathon!

Thanks for joining our two-day hackathon to help improve Digital Earth Australia’s (DEA) Notebooks!

🏆 What’s the goal of this hackathon?

The primary goal of the hackathon is to transition DEA Notebooks to use odc-stac. This will be the first full repository refresh of DEA Notebooks since 2019. A more detailed list of hackathon goals is outlined below.

• Convert DEA Notebooks repo to use odc-stac.load rather than datacube.load​
• Develop skills and knowledge on odc-stac (and STAC more generally) ​
• Further test, refine, and document alternate access patterns for DEA (e.g. CodeSpaces, Colab)
• Develop communications and KH articles on the update to DEA Notebooks
• Address backlog DEA Notebooks tasks (‘Issues’)
• Add/update documentation and guides to the DEA Notebooks wiki on accessing DEA data via sandbox, GitHub Codespaces, Google Colab and local environments

🗓️ Hackathon Structure

Day 1

9am

• Introduction
• Assign yourself (or small team) to tasks in the Hackathon project board
• Get started!

Day 2

9am

• Open discussion on any issues encountered on Day 1

3.30pm

• Showcase of work
• Discuss next steps and timeline for approval and deployment of updated DEA-Notebooks
• Publication of JOSS paper timeline?

✅ Generalised Notebook Conversion Checklist

• Notebook updated to follow updated DEA Notebooks template format
• For STAC loading:
  • odc-stac load() and configure_s3_access imported
  • pystac_client Client imported
  • Environment setup includes catalog, stac_client and configure_s3_access()
  • Cell that searched a collection for items with pystac-client
  • Cell that loads discovered items into an xarray with odc-stac
• Notebook no longer requires datacube
• Notebook no longer does sys.path.insert(1, '../Tools/')

🔧 Helpful Resources

Planning tools

• Hackathon project board: See what needs doing, and what others are working on
• Prioritised notebook list: Shows which notebooks are highest priority for updating (but feel free to pick any!)

Getting set up

• Set up instructions: How to get your environment running - see below
• develop_stac branch: The branch we're working in - make sure you're on it!
• DEA_notebooks_template: The latest notebook layout and style we are standardising on

Example notebooks already coverted to STAC and new notebooks template

• Introduction to loading data with STAC
• Introduction to DEA Surface Reflectance (Landsat, Collection 3)
• Using "load_ard" to load and cloud mask Landsat and Sentinel-2
• Generating satellite imagery animations
• Monitoring coastal erosion along Australia's coastline

Other resources

• Knowledge Hub article on STAC
• DEAfrica odc-stac conversion repository
• FrontierSI odc-stac example notebooks
• odc-stac docs page
• pystac-client docs page
• pystac docs page
• STAC spec
• Introduction to STAC (with the Brazilian DataCube)

⚙️ Set up instructions

DEA Sandbox

1. Start up an "unstable" Sandbox server
2. Navigate to where you have cloned your repo, e.g.:

   cd dev/dea-notebooks
   

3. Make sure you have the latest copy of the repo:

   git pull
   

4. Check out develop_stac branch:

   git checkout --track origin/develop_stac
   

5. Create a copy of develop_stac to do your work:

   git checkout -b my_new_branch develop_stac
   

6. Install latest local version of DEA Tools:

   pip install -e .
   

Important

Installing DEA Tools with pip install -e . is important to make sure you can access any functions we update during the hack.

7. Update your notebook to use STAC and the new DEA Notebooks template
8. When ready, commit and push your changes:

   git commit -am 'I made some changes woooo'
   git push
   

9. Raise a pull request in the DEA Notebooks repository, setting the "base branch" as develop_stac:

Github CodeSpaces

Instead of using the sandbox environments, you can also launch a GitHub CodeSpace cloud instance by clicking the badge below. If you are accessing DEA notebooks from outside the Geoscience Australia organisation, you should use this option.

Tip

Allow CodeSpaces a few minutes to bootup and for all python libraries to install.

Warning

When you open a Codespace, the compute time is billed to your own GitHub account (using your free monthly allowance or any paid plan you may have). Free users get 120 compute hours per month. The compute size set up for this repo is 4 cores, 16GB RAM, 15GB of storage. On a free account, this means you have 30 free hours of usage per month. You can change these settings before launching the codespaces, but keep in mind this will affect the total number of hours you can use for free.

Opening Codespaces using VS Code (Desktop)

If you work within vs code, you can open codespaces within your IDE rather than in a web UI. To do this, you must:

• have vs code installed
• install the codespaces extension from the vs code extensions tab:

• Set the default editor to VS Code (Desktop) by:

• In the upper-right corner of any page on GitHub, click your profile picture, then click Settings
• In the "Code, planning, and automation" section of the sidebar, click Codespaces
• Under "Editor preference", select the option you want

You can also find instructions for setting up codespaces to open in vs code here.