Skip to content

miketsukerman/bsp-registry-tools

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

43 Commits
.github/workflows
.github/workflows
 
 
bsp
bsp
 
 
examples
examples
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
pyproject.toml
pyproject.toml
 
 

Repository files navigation

bsp-registry-tools

Python tools to build, fetch, and work with Yocto-based BSPs using the KAS build system.

Overview

bsp-registry-tools provides a command-line interface and Python API for managing Advantech Board Support Packages (BSPs). It uses YAML-based registry files to define BSP configurations, build environments, and Docker containers, making reproducible Yocto builds straightforward.

Key Features

  • 📋 BSP registry management via YAML configuration files
  • 🌐 Automatic remote registry fetching — clone/update a remote registry with no manual setup
  • 🐳 Docker container support for reproducible build environments
  • 🔧 KAS integration for Yocto-based builds (kas, kas-container)
  • 🖥️ Interactive shell access to build environments
  • 🔄 Environment variable expansion ($ENV{VAR} syntax)
  • 📤 Configuration export for sharing and archiving build configs
  • ✅ Comprehensive validation of configurations before building

Installation

From PyPI

pip install bsp-registry-tools

From Source

git clone https://github.com/Advantech-EECC/bsp-registry-tools.git
cd bsp-registry-tools
pip install .

Dependencies

Quick Start

Zero-Config Usage (Remote Registry)

If you have no local registry file, bsp automatically clones the default Advantech BSP registry into ~/.cache/bsp/registry and keeps it up-to-date on every run:

# First run: clones the registry, then lists BSPs
bsp list

# Subsequent runs: pulls latest changes, then lists BSPs
bsp list

# Skip the network update (useful offline or in CI)
bsp --no-update list

# Use a different remote or branch
bsp --remote https://github.com/my-org/bsp-registry.git --branch dev list

Manual Registry Usage

1. Create a BSP Registry File

Create a bsp-registry.yaml or bsp-registry.yml file (see examples/bsp-registry.yaml):

specification:
  version: "1.0"

environment:
  - name: "DL_DIR"
    value: "$ENV{HOME}/yocto-cache/downloads"
  - name: "SSTATE_DIR"
    value: "$ENV{HOME}/yocto-cache/sstate"

containers:
  - debian-bookworm:
      image: "bsp/registry/debian/kas:5.1"
      file: Dockerfile
      args:
        - name: "DISTRO"
          value: "debian-bookworm"
        - name: "KAS_VERSION"
          value: "5.1"

registry:
  bsp:
    - name: poky-qemuarm64-scarthgap
      description: "Poky QEMU ARM64 Scarthgap (Yocto 5.0 LTS)"
      build:
        path: build/qemu-arm64-scarthgap
        environment:
          container: "debian-bookworm"
        configuration:
          - kas/scarthgap.yaml
          - kas/qemu/qemuarm64.yaml

2. List Available BSPs

# With an explicit registry file
bsp --registry bsp-registry.yaml list

# Or simply if bsp-registry.yaml (or bsp-registry.yml) is in the current directory
bsp list
- poky-qemuarm64-scarthgap: Poky QEMU ARM64 Scarthgap (Yocto 5.0 LTS)

3. Build a BSP

bsp build poky-qemuarm64-scarthgap

4. Enter Interactive Shell

bsp shell poky-qemuarm64-scarthgap

CLI Reference

usage: bsp [-h] [--verbose] [--registry REGISTRY] [--no-color]
           [--remote REMOTE] [--branch BRANCH] [--update | --no-update]
           [--local]
           {build,list,containers,export,shell} ...

Advantech Board Support Package Registry

positional arguments:
  {build,list,containers,export,shell}
                        Command to execute
    build               Build an image for BSP
    list                List available BSPs
    containers          List available containers
    export              Export BSP configuration
    shell               Enter interactive shell for BSP

options:
  -h, --help            show this help message and exit
  --verbose, -v         Verbose output
  --registry REGISTRY, -r REGISTRY
                        BSP Registry file (local path; skips remote fetch)
  --no-color            Disable colored output
  --remote REMOTE       Remote registry git URL
                        (default: https://github.com/Advantech-EECC/bsp-registry.git)
  --branch BRANCH       Remote registry branch (default: main)
  --update              Update the cached registry clone before use (default)
  --no-update           Skip updating the cached registry clone
  --local               Force local registry lookup only (do not use remote)

Registry Resolution Priority

The tool determines which registry file to use in the following order:

  1. --registry <path> — explicit local file, remote fetch is skipped entirely.
  2. --local — use ./bsp-registry.yaml or ./bsp-registry.yml in the current directory; no network access.
  3. bsp-registry.yaml exists in the current directory — auto-detect (preferred extension).
  4. bsp-registry.yml exists in the current directory — auto-detect (alternate extension).
  5. Otherwise — clone/update the remote registry into ~/.cache/bsp/registry via RegistryFetcher.

Global Options

Option Description
--verbose, -v Enable verbose/debug output
--registry REGISTRY, -r REGISTRY Path to BSP registry file (local override)
--no-color Disable colored output
--remote REMOTE Remote registry git URL (default: Advantech BSP registry)
--branch BRANCH Remote registry branch (default: main)
--update / --no-update Update cached registry clone before use (default: update)
--local Force local lookup; never contact remote

Commands

list — List available BSPs

bsp list
bsp --registry my-registry.yaml list

containers — List available container definitions

bsp containers

build — Build a BSP image

bsp build <bsp_name> [--clean] [--checkout] [--path PATH]
Option Description
--clean Clean build directory before building
--checkout Validate configuration and checkout repos without building
--path PATH Override the output build directory path defined in the registry

Examples:

# Full build
bsp build poky-qemuarm64-scarthgap

# Checkout/validate only (fast, no build)
bsp build poky-qemuarm64-scarthgap --checkout

# Override the output build directory
bsp build poky-qemuarm64-scarthgap --path /mnt/fast-ssd/build

shell — Interactive shell in build environment

bsp shell <bsp_name> [--command COMMAND]
Option Description
--command COMMAND, -c COMMAND Execute a specific command instead of starting interactive shell

Examples:

# Interactive shell
bsp shell poky-qemuarm64-scarthgap

# Execute single command
bsp shell poky-qemuarm64-scarthgap --command "bitbake core-image-minimal"

export — Export BSP configuration

bsp export <bsp_name> [--output OUTPUT]
Option Description
--output OUTPUT, -o OUTPUT Output file path (default: stdout)

Examples:

# Print to stdout
bsp export poky-qemuarm64-scarthgap

# Save to file
bsp export poky-qemuarm64-scarthgap --output exported-config.yaml

Registry Configuration Reference

The BSP registry is a YAML file with the following top-level sections:

specification

specification:
  version: "1.0"

environment

Global environment variables applied to all builds. Supports $ENV{VAR_NAME} expansion to reference system environment variables.

environment:
  - name: "GITCONFIG_FILE"
    value: "$ENV{HOME}/.gitconfig"
  - name: "DL_DIR"
    value: "$ENV{HOME}/yocto-cache/downloads"
  - name: "SSTATE_DIR"
    value: "$ENV{HOME}/yocto-cache/sstate"

Supported variables:

Variable Description
DL_DIR Yocto downloads cache directory
SSTATE_DIR Yocto shared state cache directory
GITCONFIG_FILE Git configuration file path

containers

Docker container definitions for build environments:

containers:
  - ubuntu-22.04:
      image: "my-registry/ubuntu-22.04/kas:4.7"
      file: Dockerfile.ubuntu
      args:
        - name: "DISTRO"
          value: "ubuntu:22.04"
        - name: "KAS_VERSION"
          value: "4.7"
  - ubuntu-20.04:
      image: "my-registry/ubuntu-20.04/kas:4.7"
      file: Dockerfile.ubuntu
      args:
        - name: "DISTRO"
          value: "ubuntu:20.04"
  - isar-container:
      image: "my-registry/isar/kas:4.7"
      file: Dockerfile.isar
      args: []
      privileged: true    # Run container in privileged mode (required for ISAR builds)

Container fields:

Field Type Default Description
image string — Docker image name/tag
file string — Path to Dockerfile for building the image
args list [] Docker build arguments (name/value pairs)
privileged boolean false Run container with elevated privileges. Required for ISAR (Debian-based) builds. Adds --isar to kas-container, which enables the --privileged Docker flag.

registry.bsp

List of BSP definitions:

registry:
  bsp:
    - name: my-bsp-name           # Unique identifier
      description: "My BSP"       # Human-readable description
      os:                         # Optional OS information
        name: linux
        build_system: yocto
        version: "5.0"
      build:
        path: build/my-bsp        # Build output directory
        environment:
          container: "ubuntu-22.04"   # Reference to containers section
          # OR use direct Docker configuration:
          # docker:
          #   image: "my-image:latest"
        configuration:            # KAS configuration files (in order)
          - kas/scarthgap.yaml
          - kas/qemu/qemuarm64.yaml

KAS Configuration Files

KAS configuration files define Yocto layer repositories, machine settings, and build targets. See the examples/kas/ directory for reference configurations.

QEMU Example Configurations

The examples/ directory contains ready-to-use KAS configurations for QEMU targets:

File Description
examples/kas/scarthgap.yaml Yocto Scarthgap (5.0 LTS) base configuration
examples/kas/styhead.yaml Yocto Styhead (5.1) base configuration
examples/kas/qemu/qemuarm64.yaml QEMU ARM64 machine configuration
examples/kas/qemu/qemux86-64.yaml QEMU x86-64 machine configuration
examples/kas/qemu/qemuarm.yaml QEMU ARM (32-bit) machine configuration

KAS File Structure

header:
  version: 14
  includes:            # Optional: include other KAS files
    - base.yaml

distro: poky
machine: qemuarm64

target:
  - core-image-minimal

repos:
  poky:
    url: "https://git.yoctoproject.org/poky"
    commit: "abc123..."
    path: "layers/poky"
    layers:
      meta:
      meta-poky:

local_conf_header:
  my_config: |
    DISTRO_FEATURES += "x11"

Python API

You can also use bsp-registry-tools as a Python library:

from bsp import BspManager, EnvironmentManager, KasManager, RegistryFetcher

# Fetch registry from remote (clone on first call, pull on subsequent)
fetcher = RegistryFetcher()
registry_path = fetcher.fetch_registry(
    repo_url="https://github.com/Advantech-EECC/bsp-registry.git",
    branch="main",
    update=True,
)

# Load and manage BSP registry
manager = BspManager(str(registry_path))
manager.initialize()

# List BSPs programmatically
for bsp in manager.model.registry.bsp:
    print(f"{bsp.name}: {bsp.description}")

# Get a specific BSP
bsp = manager.get_bsp_by_name("poky-qemuarm64-scarthgap")

# Environment variable management with $ENV{} expansion
from bsp import EnvironmentVariable
env_vars = [
    EnvironmentVariable(name="DL_DIR", value="$ENV{HOME}/downloads"),
]
env_manager = EnvironmentManager(env_vars)
print(env_manager.get_value("DL_DIR"))  # Expanded path

# Use KasManager directly
kas = KasManager(
    kas_files=["kas/scarthgap.yaml", "kas/qemu/qemuarm64.yaml"],
    build_dir="build/my-bsp",
    use_container=False,
)
kas.validate_kas_files()

Development

Setup Development Environment

git clone https://github.com/Advantech-EECC/bsp-registry-tools.git
cd bsp-registry-tools
pip install -e ".[dev]"

Running Tests

# Run all tests
pytest

# Run with verbose output
pytest -v

# Run with coverage report
pytest --cov=bsp --cov-report=term-missing

# Run specific test class
pytest tests/test_bsp.py::TestEnvironmentManager -v

Project Structure

bsp-registry-tools/
├── bsp/
│   ├── __init__.py           # Public API exports
│   ├── cli.py                # CLI entry point
│   ├── bsp_manager.py        # Main BSP coordinator
│   ├── registry_fetcher.py   # Remote registry clone/update
│   ├── kas_manager.py        # KAS build system integration
│   ├── environment.py        # Environment variable management
│   ├── path_resolver.py      # Path utilities
│   ├── models.py             # Dataclass models
│   ├── utils.py              # YAML / Docker utilities
│   └── exceptions.py         # Custom exceptions
├── pyproject.toml            # Package configuration
├── README.md                 # This file
├── LICENSE                   # Apache 2.0 License
├── tests/
│   ├── conftest.py
│   ├── test_bsp_manager.py
│   ├── test_cli.py
│   ├── test_registry_fetcher.py
│   └── ...
├── examples/
│   ├── bsp-registry.yaml      # Sample BSP registry for QEMU targets
│   └── kas/
│       ├── scarthgap.yaml     # Yocto Scarthgap base config
│       ├── styhead.yaml       # Yocto Styhead base config
│       └── qemu/
│           ├── qemuarm64.yaml  # QEMU ARM64 machine config
│           ├── qemux86-64.yaml # QEMU x86-64 machine config
│           └── qemuarm.yaml    # QEMU ARM machine config
└── .github/
    └── workflows/
        ├── tests.yaml         # CI: run tests on push/PR
        └── publish.yaml       # CD: publish to PyPI on release

Publishing to PyPI

This repository uses GitHub Actions for automated publishing.

Setup

  1. Create PyPI and TestPyPI accounts and configure Trusted Publishers:

    • PyPI: Add GitHub Actions publisher for Advantech-EECC/bsp-registry-tools
    • TestPyPI: Same configuration on test.pypi.org

  2. Create GitHub Environments named pypi and testpypi in your repository settings.

Publish Workflow

Automatic (on GitHub Release):

  • Creating a non-prerelease GitHub Release automatically publishes to both TestPyPI and PyPI.
  • Creating a prerelease publishes to TestPyPI only.

Manual:

GitHub → Actions → "Publish to PyPI" → Run workflow → Select environment

Build Locally

pip install build
python -m build
# Artifacts are in dist/

Architecture

Classes

Class Description
BspManager Main coordinator for BSP operations
KasManager Handles KAS build system operations
EnvironmentManager Manages build environment variables with $ENV{} expansion
PathResolver Utility for path resolution and validation
RegistryFetcher Clones/updates a remote git-hosted BSP registry to a local cache

Data Classes

Class Description
RegistryRoot Root registry container
Registry Contains list of BSP definitions
BSP Single BSP definition
BuildSetup Build configuration (path, environment, KAS files)
BuildEnvironment Docker/container settings
Docker Docker image, build arg, and privileged mode configuration
EnvironmentVariable Name/value pair with $ENV{} expansion support

Exceptions

Exception Description
ScriptError Base exception for all script errors
ConfigurationError Configuration file issues
BuildError Build process failures
DockerError Docker operation failures
KasError KAS operation failures

License

This project is licensed under the Apache 2.0 License — see the LICENSE file for details.

Contributing

Contributions are welcome! Please open an issue or submit a pull request on GitHub.

About

Python tools to build, fetch, work with yocto based bsp's

Resources

Readme

License

Apache-2.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 100.0%