aerial-autonomy-stack

Aerial autonomy stack (AAS) is an all-in-one software stack to:

1. Develop multi-drone autonomy—with ROS2, PX4, and ArduPilot
2. Simulate faster-than-real-time perception and control—with YOLO and 3D LiDAR
3. Deploy in real drones—with JetPack, DeepStream, and NVIDIA Orin

For an example bill of materials, read BOM.md; for motivation, read RATIONALE.md

aerial-autonomy-stack-v3.mp4

Features (click to expand)

• PX4 and ArduPilot multi-vehicle simulation (quadrotors and VTOLs)
• ROS2 action-based autopilot interface (via XRCE-DDS or MAVROS)
• YOLO (with ONNX GPU Runtimes) and LiDAR Odometry (with KISS-ICP)
• 3D worlds for perception-based simulation
• Steppable Gymnasium environment and faster-than-real-time, multi-instance simulation
• Gazebo's wind effects and waves plugins
• Dockerized simulation based on Ubuntu with CUDA and cuDNN
• Dockerized deployment based on NVIDIA JetPack with DeepStream
• Windows 11 compatibility via WSL
• Multi-Jetson-in-the-loop (HITL) simulation to test NVIDIA- and ARM-based on-board compute
• Dual network to separate simulated sensors (SIM_SUBNET) and inter-vehicle comms (AIR_SUBNET)
• Zenoh inter-vehicle ROS2 bridge
• PX4 Offboard interface (e.g. CTBR/VehicleRatesSetpoint for agile, GNSS-denied flight)
• ArduPilot Guided interface (i.e. setpoint_velocity, setpoint_accel references)
• Logs analysis with flight_review (.ulg), MAVExplorer (.bin), and PlotJuggler (rosbag)

1. Installation

AAS is developed on Ubuntu 24.04 with nvidia-driver-580 using an i7-11 with 16GB RAM and RTX 3060

Read REQUIREMENTS_UBUNTU.md (or REQUIREMENTS_WSL.md for Windows 11) to install the requirements

sudo apt update && sudo apt install -y git xterm xfonts-base wget unzip
git clone https://github.com/JacopoPan/aerial-autonomy-stack.git && cd aerial-autonomy-stack/tools_and_docs/
./tests/check_requirements.sh                         # AAS requires nvidia-driver-580, docker, and nvidia-container-toolkit
./sim_build.sh                                        # The 1st build takes ~40' with good internet (`Ctrl + c` and restart if needed, cached stages will be preserved)

2. Simulation

On one terminal, start AAS:

cd aerial-autonomy-stack/tools_and_docs/
AUTOPILOT=px4 NUM_QUADS=1 NUM_VTOLS=1 WORLD=swiss_town HEADLESS=false RTF=3.0 ./sim_run.sh    # Start a simulation, check the script for more options (note: ArduPilot SITL checks take ~30-40s of simulated time before being ready to arm)

On another terminal, fly all drones:

for ID in {1..2}; do
  docker exec -d aircraft-container-inst0_$ID bash -c "source /opt/ros/humble/setup.bash &&
    source /aas/github_ws/install/setup.bash && source /aas/aircraft_ws/install/setup.bash &&
    ros2 run mission mission --conops yalla.yaml --ros-args -r __ns:=/Drone$ID -p use_sim_time:=true"
done

./sim_run.sh options:

- AUTOPILOT=px4, ardupilot
- HEADLESS/CAMERA/LIDAR=true, false
- NUM_QUADS/NUM_VTOLS=0, 1, ...
- WORLD=impalpable_greyness, apple_orchard, shibuya_crossing, swiss_town, waterworld
- RTF=1.0, 2.0, ... (real-time-factor, use 0.0 for "as fast as possible)
- INSTANCE=0, 1, ... (integer ID to run multiple parallel simulations)

WORLDs: (i) apple_orchard, a GIS world created using BlenderGIS / (ii) impalpable_greyness, an empty world with simple shapes / (iii) shibuya_crossing, a 3D world adapted from cgtrader / (iv) swiss_town, a photogrammetry world courtesy of Pix4D / pix4d.com / (v) waterworld, a dynamic world using the asv_wave_sim wave plugin

Tip

Edit sensor_config.yaml, then run sim_build.sh, to customize the sensor parameters

Use ROS2 drone and gimbal control primitives from CLI (click to expand)

# Takeoff action (quads and VTOLs)
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 40.0, vtol_transition_heading: 330.0, vtol_loiter_nord: 200.0, vtol_loiter_east: 100.0, vtol_loiter_alt: 120.0}'"

# Land (at home) action (quads and VTOLs)
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/land_action autopilot_interface_msgs/action/Land '{landing_altitude: 60.0, vtol_transition_heading: 60.0}'"

# Orbit action (quads and VTOLs)
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/orbit_action autopilot_interface_msgs/action/Orbit '{east: 500.0, north: 0.0, altitude: 150.0, radius: 200.0}'"

# Reposition service (quads only)
ros2 service call /Drone${DRONE_ID}/set_reposition autopilot_interface_msgs/srv/SetReposition '{east: 50.0, north: 100.0, altitude: 60.0}'

# Offboard action (PX4 quads and VTOLs offboard_setpoint_type: attitude = 0, rates = 1, trajectory = 2; ArduPilot quads offboard_setpoint_type: velocity = 3, acceleration = 4) 
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action autopilot_interface_msgs/action/Offboard '{offboard_setpoint_type: 1, max_duration_sec: 5.0}'"

# SetSpeed service (always limited by the autopilot params, for quads applies from the next command, not effective on ArduPilot VTOLs) 
ros2 service call /Drone${DRONE_ID}/set_speed autopilot_interface_msgs/srv/SetSpeed '{speed: 3.0}'

# Gimbal status and position control (in radians)
ros2 topic echo /gimbal_state
ros2 topic pub -1 /gimbal_pitch_cmd std_msgs/msg/Float64 "{data: 1.57}"

To analyze the flight logs in the Simulation's Xterm terminal:

/aas/simulation_resources/scripts/plot_logs.sh                                                # Analyze the flight logs at http://10.42.90.100:5006/browse or in MAVExplorer

To create a new mission, re-implement test_mission.yaml

Add or disable wind effects, in the Simulation's Xterm terminal (click to expand)

python3 /aas/simulation_resources/scripts/gz_wind.py --from_west 0.0 --from_south 3.0
python3 /aas/simulation_resources/scripts/gz_wind.py --stop_wind

Develop within live containers (click to expand)

Launching the sim_run.sh script with DEV=true, does not start the simulation and mounts folders [aircraft|ground|simulation]_resources, [aircraft|ground]_ws/src as volumes to more easily track, commit, push changes while building and testing them within the containers:

cd aerial-autonomy-stack/tools_and_docs/
DEV=true ./sim_run.sh                                                                       # Starts one simulation-image, one ground-image, and one aircraft-image where the *_resources/ and *_ws/src/ folders are mounted from the host

To build changes—made on the host—in the Ground or QUAD Xterm terminal:

cd /aas/aircraft_ws/                                                                        # Or cd /aas/ground_ws/
colcon build --symlink-install

To start the simulation, in the QUAD Xterm terminal:

tmuxinator start -p /aas/aircraft.yml.erb

In the Ground Xterm terminal:

tmuxinator start -p /aas/ground.yml.erb

In the Simulation Xterm terminal:

tmuxinator start -p /aas/simulation.yml.erb

To end the simulation, in each terminal detach Tmux with Ctrl + b, then d; kill all lingering processes with tmux kill-server && pkill -f gz

3. Jetson Deployment

AAS is tested on a Holybro Jetson Baseboard with Pixhawk 6X and NVIDIA Orin NX 16GB on an X650

Read SETUP_AVIONICS.md and BOM.md to setup the requirements on the Jetson and configure the Pixhawk

sudo apt update && sudo apt install -y git
git clone https://github.com/JacopoPan/aerial-autonomy-stack.git && cd aerial-autonomy-stack/tools_and_docs/
./deploy_build.sh                                     # Build for arm64, on Jetson Orin NX the first build takes ~50', including building onnxruntime-gpu with TensorRT support from source

On a Jetson Orin, start the aircraft-image:

cd aerial-autonomy-stack/tools_and_docs/
AUTOPILOT=px4 DRONE_ID=1 CAMERA=true LIDAR=false AIR_SUBNET=10.223 HEADLESS=true ./deploy_run.sh
# The 1st run of `./deploy_run.sh` requires ~10' to build the FP16 TensorRT cache

./deploy_run.sh options:

- DRONE_TYPE=quad, vtol
- AUTOPILOT=px4, ardupilot
- DRONE_ID=1, 2, ... (ROS_DOMAIN_ID of the drone, matching the MAV_SYS_ID/SYSID_THISMAV of the autpilot)
- HEADLESS/CAMERA/LIDAR=true, false

On a laptop, start the ground-image (QGC, Zenoh, SSH, and GStreamer):

cd aerial-autonomy-stack/tools_and_docs/
./sim_build.sh                                        # Build all images for amd64, including ground-image
GROUND=true NUM_QUADS=1 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.sh

HITL Simulation (click to expand)

Note: HITL simulation validates the Jetson compute and the inter-vehicle network. Use USB2.0 ASIX Ethernet adapters to add multiple network interfaces to the Jetson baseboards

Set up a LAN on an arbitrary SIM_SUBNET with netmask 255.255.0.0 (e.g. 172.30.x.x) between:

• One simulation computer, with IP [SIM_SUBNET].90.100
• One ground computer, with IP [SIM_SUBNET].90.101
• N Jetson Baseboards with IPs [SIM_SUBNET].90.1, ..., [SIM_SUBNET].90.N

Optionally, set up a second LAN :AIR_SUBNET with netmask 255.255.0.0 (e.g. 10.223.x.x) between:

• One ground computer, with IP [AIR_SUBNET].90.101
• N Jetson Baseboards with IPs [AIR_SUBNET].90.1, ..., [AIR_SUBNET].90.N

First, start all aircraft containers, one on each Jetson (e.g. via SSH):

# On the Jetson with IPs ending in 90.1
HITL=true DRONE_ID=1 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh                      # Add HEADLESS=false if a screen is connected to the Jetson

# On the Jetson with IPs ending in 90.2
HITL=true DRONE_ID=2 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh                      # Add HEADLESS=false if a screen is connected to the Jetson

Then, start the simulation:

# On the computer with IPs ending in 90.100
HITL=true NUM_QUADS=2 SIM_SUBNET=172.30 ./sim_run.sh

Finally, start QGC and the Zenoh bridge:

# On the computer with IPs ending in 90.101
HITL=true GROUND=true NUM_QUADS=2 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.sh

Note: running only the first 3 commands with GND_CONTAINER=false puts the Zenoh bridge on the SIM_SUBNET, removing the need for the optional AIR_SUBNET and the computer with IP ending in 90.101

Once done, detach Tmux (and remove the containers) with Ctrl + b, then d

4. Gymnasium Environment

Using a Python venv or a conda environment is optional but recommended (click to expand)

wget https://repo.anaconda.com/archive/Anaconda3-2025.12-2-Linux-x86_64.sh # Or a newer version in https://repo.anaconda.com/archive/
bash Anaconda3-2025.12-2-Linux-x86_64.sh              # Install; start a new terminal
conda config --set auto_activate_base false           # Turn off auto initialization of (base); start a new terminal
conda update --all -n base -c defaults                # Update to the latest conda version
conda create -n aas python=3.12                       # Latest Python version beyond "bugfix" status https://devguide.python.org/versions/

Install the aas-gym package (after completing the steps in "Installation"):

conda activate aas                                    # If using Anaconda
cd aerial-autonomy-stack/aas-gym/
pip3 install -e .

Examples:

conda activate aas                                    # If using Anaconda
cd aerial-autonomy-stack/tools_and_docs
python3 gym_run.py --mode step                        # Manually step AAS @1Hz
python3 gym_run.py --mode speedup                     # Speed-up test @50Hz
python3 gym_run.py --mode vectorenv-speedup           # Vectorized speed-up test @50Hz

Citation

@INPROCEEDINGS{panerati2026aas,
    author={Jacopo Panerati and Sina Sajjadi and Sina Soleymanpour and Varunkumar Mehta and Iraj Mantegh},
    booktitle={2026 International Conference on Unmanned Aircraft Systems (ICUAS)},
    title={{aerial-autonomy-stack}---a faster-than-real-time, autopilot-agnostic, {ROS2} framework to simulate and deploy perception-based drones},
    year={2026}}

Appendix: Architecture

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace'}}}%%
flowchart TB
    subgraph aas [" "]
        subgraph sim ["#nbsp;simulation#nbsp;container#nbsp;(amd64)"]
            sitl("[N x] PX4 || <br/> ArduPilot SITL"):::resource
            gz(Gazebo Sim):::resource
            subgraph models [Models]
                drones(aircraft_models/):::resource
                worlds(simulation_worlds/):::resource
            end

            drones --> gz
            worlds --> gz
            sitl <--> |"gz_bridge || ardupilot_gazebo"| gz
        end

        subgraph gnd ["#nbsp;ground#nbsp;container#nbsp;(amd64)"]
            mlrouter{{mavlink-router}}:::bridge
            ground_system[/ground_system\]:::algo
            qgc(QGroundControl):::resource
            zenoh_gnd{{zenoh-bridge}}:::bridge

            ground_system --> |"/tracks"| zenoh_gnd
            mlrouter <--> qgc
            mlrouter --> ground_system
        end

        subgraph air ["[N#nbsp;x]#nbsp;aircraft#nbsp;container(s)#nbsp;(amd64,#nbsp;arm64)"]
            subgraph perception [Perception]
                yolo_py[/yolo_py/]:::algo
                kiss_icp[/kiss_icp/]:::algo
            end
            subgraph control [Control]
                offboard_control(offboard_control):::algo
                autopilot_interface(autopilot_interface):::algo
                mission(mission):::algo
            end
            ap_link{{"uxrce_dds <br/> || MAVROS"}}:::bridge
            subgraph swarm [Swarm]
                state_sharing[/state_sharing\]:::algo
            end
            zenoh_air{{zenoh-bridge}}:::bridge

            kiss_icp -.-> |"/TBD"| ap_link
            ap_link <--> autopilot_interface
            ap_link --> state_sharing
            yolo_py --> |"/detections"| offboard_control
            offboard_control --> |"/reference"| autopilot_interface
            mission --> |"ros2 action/srv"| autopilot_interface
            zenoh_air <--> |"/state_drone_n"| state_sharing
        end

        repo(((aerial#nbsp;autonomy#nbsp;stack)))
    end

    repo ~~~ gz
    gz --> |"gz_gst_bridge <br/> [SIM_SUBNET]"| yolo_py
    gz --> |"/lidar_points <br/> [SIM_SUBNET]"| kiss_icp
    sitl <--> |"UDP <br/> [SIM_SUBNET]"| ap_link
    sitl <--> |"MAVLink <br/> [SIM_SUBNET]"| mlrouter 
    zenoh_gnd <-.-> |"TCP <br/> [AIR_SUBNET]"| zenoh_air

    classDef bridge fill:#ffebd6,stroke:#f5a623,stroke-width:2px;
    classDef algo fill:#e1f5fe,stroke:#0277bd,stroke-width:2px;
    classDef resource fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
    classDef blueStyle  fill:#e1f0ff,stroke:#666,stroke-width:2px;
    classDef whiteStyle fill:#f9f9f9,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5;
    classDef greyStyle  fill:#eeeeee,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5;

    class aas,repo blueStyle;
    class air,gnd,sim whiteStyle;
    class perception,control,models,swarm greyStyle;
    linkStyle 14,15,16,17 stroke:teal,stroke-width:3px;
    linkStyle 18 stroke:blue,stroke-width:4px;

Loading

Repository structure (click to expand)

aerial-autonomy-stack
│
├── aas-gym
│   └── src
│       └── aas_gym
│           └── aas_env.py                            # aerial-autonomy-stack as a Gymnasium environment
│
├── aircraft
│   ├── aircraft_ws
│   │   └── src
│   │       ├── autopilot_interface                   # Ardupilot/PX4 high-level actions (Takeoff, Orbit, Offboard, Land)
│   │       ├── mission                               # Orchestrator of the actions in `autopilot_interface`
│   │       ├── offboard_control                      # Low-level references for the Offboard action in `autopilot_interface`
│   │       ├── state_sharing                         # Publisher of the `/state_sharing_drone_N` topic broadcasted by Zenoh
│   │       └── yolo_py                               # GStreamer video acquisition and publisher of YOLO bounding boxes
│   │
│   └── aircraft.yml.erb                              # Aircraft docker tmux entrypoint
│
├── ground
│   ├── ground_ws
│   │   └── src
│   │       └── ground_system                         # Publisher of topic `/tracks` broadcasted by Zenoh
│   │
│   └── ground.yml.erb                                # Ground docker tmux entrypoint
│
├── simulation
│   ├── simulation_resources
│   │   ├── aircraft_models
│   │   │   ├── alti_transition_quad                  # ArduPilot VTOL model
│   │   │   ├── iris_with_ardupilot                   # ArduPilot quad model
│   │   │   ├── sensor_camera                         # Camera model
│   │   │   ├── sensor_gimbal                         # 3D gimbal used with sensor_camera
│   │   │   ├── sensor_lidar                          # LiDAR model
│   │   │   ├── standard_vtol                         # PX4 VTOL model
│   │   │   ├── x500                                  # PX4 quad model
│   │   │   └── sensor_config.yaml                    # Intrinsics and extrinsics for all sensor and vehicle models
│   │   └── simulation_worlds
│   │       ├── apple_orchard.sdf
│   │       ├── impalpable_greyness.sdf
│   │       ├── shibuya_crossing.sdf
│   │       ├── swiss_town.sdf
│   │       └── waterworld.sdf
│   │
│   └── simulation.yml.erb                            # Simulation docker tmux entrypoint
│
└── tools_and_docs
    ├── docker
    │   ├── aircraft.dockerfile                       # Docker image for aircraft simulation and deployment
    │   ├── ground.dockerfile                         # Docker image for ground system simulation and deployment
    │   └── simulation.dockerfile                     # Docker image for SITL and HITL simulation
    │
    ├── deploy_build.sh                               # Build `aircraft.dockerfile` for arm64/Orin
    ├── deploy_run.sh                                 # Start the aircraft docker on arm64/Orin or the ground docker on amd64 (deploy or HITL)
    │
    ├── gym_run.py                                    # Examples for the Gymnasium aas-gym package
    │
    ├── sim_build.sh                                  # Build all dockerfiles for amd64/simulation
    └── sim_run.sh                                    # Start the simulation (SITL or HITL)

Dependencies management (click to expand)

• Host OS: Ubuntu 22.04/24.04/26.04 (LTS, ESM 4/2036)
• Jetpack: 6.2.1 (rev. 1) [L4T 36.4.4, Ubuntu 22-based]
  • TODO: test on JP 6.2.2 [L4T 36.5.0, Ubuntu 22-based]
• nvidia-driver-580
  • NOTE: nvidia-driver-590 does not support the presets in Ubuntu 22's GStreamer 1.20 and it requires updating the amd64 base images to Ubuntu 24 or compiling GStreamer 1.24 from source
  • AAS sticks with nvidia-driver-580 and Ubuntu 22 amd64 base images for parity with the L4T 36.x, Ubuntu 22-based arm64 base image
  • TODO: test nvidia-driver-595
• Docker Engine v29
• NVIDIA Container Toolkit 1.19
• amd64 base image: cuda:12.9.1-cudnn-runtime-ubuntu22.04
  • NOTE: onnxruntime-gpu 1.23 does not support CUDA 13
• arm64/Jetson base image: l4t-jetpack:r36.4.0
• DeepStream 7.1
  • NOTE: latest DeepStream supported on Orin Series
• ROS2 Humble (LTS, EOL 5/2027)
• Gazebo Sim Harmonic (LTS, EOL 9/2028)
• PX4 1.16.2
• ArduPilot 4.6.3
• YOLO26
• ONNX Runtime 1.23.2
  • NOTE: updating to 1.24 from wheel requires switching to Python >3.11/Ubuntu 24

Transitive constraints:

Jetson Orin only supports JetPack up to version 6
  -> JetPack 6 is based on L4T 36 (Ubuntu 22)
    -> Ubuntu 22's system Python is version 3.10
      -> the last available ONNX Runtime wheel for Python 3.10 is version 1.23.2
        -> ONNX Runtime 1.23.2 does not support CUDA 13
    -> Ubuntu 22's GStreamer apt package is version 1.20
      -> GStreamer 1.20's `nvh264enc preset` are no longer supported beyond `nvidia-driver-580`

External repositories:

• PX4/PX4-Autopilot tag/branch: v1.16.2
• PX4/px4_msgs tag/branch: release/1.16
• PX4/flight_review tag/branch: main
• ArduPilot/ardupilot tag/branch: Copter-4.6.3
• ArduPilot/ardupilot_gazebo tag/branch: main
• srmainwaring/asv_wave_sim tag/branch: master
• mavlink/c_library_v2 tag/branch: master
• mavlink-router/mavlink-router tag/branch: master
• eProsima/Micro-XRCE-DDS-Agent tag/branch: master
• PRBonn/kiss-icp tag/branch: main
• microsoft/onnxruntime tag/branch: v1.23.2
• Livox-SDK/Livox-SDK2 tag/branch: master
• Livox-SDK/livox_ros_driver2 tag/branch: master

---

You've done a man's job, sir. I guess you're through, huh?