Skip to content

Commit 235bfb9

Browse files
nickvinesclaude
nickvines
and
claude
authored
Add Verkada CI workflows with FIPS support (#15)
* Enable FIPS for OpenSSL builds - Add enable-fips flag to OpenSSL 3.5 configure (non-musl only) - Preserve fips.so when removing shared libraries - Copy FIPS modules and config to Python installation - Skip fips.so/fips.dylib in distribution validation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Verkada CI workflows for 3 target builds - Add vlinux.yml: Build aarch64 + x86_64 Linux targets on ubuntu-latest - Add vmacos.yml: Build aarch64 macOS target on macos-latest - Add vrelease.yml: Manual release workflow - Disable upstream linux/macos/windows workflow triggers (use workflow_dispatch only) - Reduce release.rs to only 3 targets: aarch64-apple-darwin, aarch64-unknown-linux-gnu, x86_64-unknown-linux-gnu Targets only build pgo+lto and freethreaded+pgo+lto (3.13+) variants. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Use namespace-profile-linux-arm for aarch64 builds - Update build job to use namespace-profile-linux-arm for aarch64-unknown-linux-gnu - Add crate-build matrix to build pythonbuild on both x86_64 and aarch64 runners - Add aarch64 Docker images (build.debian9, gcc.debian9) on namespace runner - Download correct pythonbuild artifact based on target architecture This fixes the 'No space left on device' errors by using native aarch64 builds instead of cross-compilation on x86_64 runners. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> * Switch to large caching namespace runners - Use namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching for x86_64 - Use namespace-profile-ubuntu-22-04-amd64-arm-large-caching for aarch64 These larger runners with caching should provide better performance. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> * Add debug output for Docker image loading - Show files in build directory - Add file existence checks before decompressing/loading - Show loaded Docker images after loading This will help diagnose why Docker images aren't being found. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> * Fix Docker image ID mismatch with containerd snapshotter Docker Buildx with containerd snapshotter returns a different image ID (config digest) than what docker load actually assigns (manifest digest). Solution: Capture the actual loaded image ID from docker load output and update the ID files so pythonbuild/docker.py can find the images. This fixes the ImageNotFound error that was causing builds to fail. Root cause identified by Opus agent analysis. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> * Use namespace macOS runner for vmacos workflow Switch to namespace-profile-mac-small-tahoe for macOS builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent e613457 commit 235bfb9

File tree

10 files changed

+577
-256
lines changed

10 files changed

+577
-256
lines changed

‎.github/workflows/linux.yml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
name: linux
22

33
on:
4-
push:
5-
branches: [main]
6-
pull_request:
4+
# Disabled - use vlinux.yml instead
5+
# push:
6+
# branches: [main]
7+
# pull_request:
8+
workflow_dispatch:
79

810
concurrency:
911
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}

‎.github/workflows/macos.yml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
name: macos
22

33
on:
4-
push:
5-
branches: [main]
6-
pull_request:
4+
# Disabled - use vmacos.yml instead
5+
# push:
6+
# branches: [main]
7+
# pull_request:
8+
workflow_dispatch:
79

810
concurrency:
911
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}

‎.github/workflows/vlinux.yml‎

Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
name: vlinux
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
concurrency:
9+
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
10+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
11+
12+
env:
13+
FORCE_COLOR: 1
14+
15+
permissions: {}
16+
17+
jobs:
18+
crate-build:
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
include:
23+
- runner: namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching
24+
arch: x86_64
25+
artifact_name: crate-linux-x86_64
26+
- runner: namespace-profile-ubuntu-22-04-amd64-arm-large-caching
27+
arch: aarch64
28+
artifact_name: crate-linux-aarch64
29+
runs-on: ${{ matrix.runner }}
30+
name: crate / ${{ matrix.arch }}
31+
steps:
32+
- name: Install System Dependencies
33+
run: |
34+
sudo apt update
35+
sudo apt install -y --no-install-recommends libssl-dev pkg-config
36+
37+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
38+
with:
39+
persist-credentials: false
40+
41+
- name: Emit rustc version
42+
run: |
43+
rustc --version > .rustc-version
44+
45+
- uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
46+
with:
47+
path: |
48+
~/.cargo/registry
49+
~/.cargo/git
50+
target
51+
key: ${{ runner.os }}-${{ matrix.arch }}-pythonbuild-${{ hashFiles('Cargo.lock', '.rustc-version') }}
52+
53+
- name: Build
54+
run: |
55+
cargo build --release
56+
57+
- name: Upload pythonbuild Executable
58+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
59+
with:
60+
name: ${{ matrix.artifact_name }}
61+
path: target/release/pythonbuild
62+
63+
image:
64+
strategy:
65+
fail-fast: false
66+
matrix:
67+
include:
68+
- name: build
69+
arch: x86_64
70+
runner: namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching
71+
- name: build.cross
72+
arch: x86_64
73+
runner: namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching
74+
- name: gcc
75+
arch: x86_64
76+
runner: namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching
77+
- name: build.debian9
78+
arch: aarch64
79+
runner: namespace-profile-ubuntu-22-04-amd64-arm-large-caching
80+
- name: gcc.debian9
81+
arch: aarch64
82+
runner: namespace-profile-ubuntu-22-04-amd64-arm-large-caching
83+
name: image / ${{ matrix.arch }} / ${{ matrix.name }}
84+
runs-on: ${{ matrix.runner }}
85+
permissions:
86+
packages: write
87+
steps:
88+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
89+
with:
90+
persist-credentials: false
91+
92+
- name: Set up uv
93+
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
94+
with:
95+
enable-cache: false
96+
97+
- name: Set up Docker Buildx
98+
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
99+
100+
- name: Login to GitHub Container Registry
101+
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
102+
with:
103+
registry: ghcr.io
104+
username: ${{ github.actor }}
105+
password: ${{ secrets.GITHUB_TOKEN }}
106+
107+
- name: Generate Dockerfiles
108+
run: |
109+
./build.py --make-target empty
110+
repo_name=$(echo "${GITHUB_REPOSITORY,,}" | sed 's|\.|_|g')
111+
git_ref_name=$(echo "${GITHUB_REF_NAME,,}" | sed 's|[^a-z0-9_-]|_|g')
112+
echo "REPO_NAME=${repo_name}" >> "${GITHUB_ENV}"
113+
echo "GIT_REF_NAME=${git_ref_name}" >> "${GITHUB_ENV}"
114+
115+
- name: Build Image
116+
id: build-image
117+
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
118+
env:
119+
SOURCE_DATE_EPOCH: 0
120+
DOCKER_BUILD_SUMMARY: false
121+
DOCKER_BUILD_RECORD_UPLOAD: false
122+
with:
123+
context: .
124+
file: build/${{ matrix.name }}.Dockerfile
125+
labels: org.opencontainers.image.source=https://github.com/${{ env.REPO_NAME }}
126+
cache-from: |
127+
type=registry,ref=ghcr.io/${{ env.REPO_NAME }}:${{ matrix.name }}-linux_${{ matrix.arch }}-${{ env.GIT_REF_NAME }}
128+
type=registry,ref=ghcr.io/${{ env.REPO_NAME }}:${{ matrix.name }}-linux_${{ matrix.arch }}-main
129+
type=registry,ref=ghcr.io/astral-sh/python-build-standalone:${{ matrix.name }}-linux_${{ matrix.arch }}-main
130+
cache-to: |
131+
type=registry,ref=ghcr.io/${{ env.REPO_NAME }}:${{ matrix.name }}-linux_${{ matrix.arch }}-${{ env.GIT_REF_NAME }},ignore-error=true
132+
outputs: |
133+
type=docker,dest=build/image-${{ matrix.name }}.linux_${{ matrix.arch }}.tar
134+
135+
- name: Compress Image
136+
run: |
137+
echo ${STEPS_BUILD_IMAGE_OUTPUTS_IMAGEID} > build/image-${MATRIX_NAME}.linux_${MATRIX_ARCH}
138+
zstd -v -T0 -6 --rm build/image-*.tar
139+
touch -t 197001010000 build/image-*
140+
env:
141+
STEPS_BUILD_IMAGE_OUTPUTS_IMAGEID: ${{ steps.build-image.outputs.imageid }}
142+
MATRIX_NAME: ${{ matrix.name }}
143+
MATRIX_ARCH: ${{ matrix.arch }}
144+
145+
- name: Upload Docker Image
146+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
147+
with:
148+
name: image-${{ matrix.name }}-linux_${{ matrix.arch }}
149+
path: build/image-*
150+
compression-level: '0'
151+
152+
build:
153+
needs:
154+
- crate-build
155+
- image
156+
permissions:
157+
id-token: write
158+
attestations: write
159+
runs-on: ${{ matrix.target_triple == 'aarch64-unknown-linux-gnu' && 'namespace-profile-ubuntu-22-04-amd64-arm-large-caching' || 'namespace-profile-ubuntu-22-04-amd64-x86-64-large-caching' }}
160+
strategy:
161+
fail-fast: false
162+
matrix:
163+
target_triple:
164+
- aarch64-unknown-linux-gnu
165+
- x86_64-unknown-linux-gnu
166+
python:
167+
- "3.10"
168+
- "3.11"
169+
- "3.12"
170+
- "3.13"
171+
- "3.14"
172+
build_options:
173+
- pgo+lto
174+
include:
175+
# Freethreaded builds for Python 3.13+
176+
- target_triple: aarch64-unknown-linux-gnu
177+
python: "3.13"
178+
build_options: freethreaded+pgo+lto
179+
- target_triple: aarch64-unknown-linux-gnu
180+
python: "3.14"
181+
build_options: freethreaded+pgo+lto
182+
- target_triple: x86_64-unknown-linux-gnu
183+
python: "3.13"
184+
build_options: freethreaded+pgo+lto
185+
- target_triple: x86_64-unknown-linux-gnu
186+
python: "3.14"
187+
build_options: freethreaded+pgo+lto
188+
name: ${{ matrix.target_triple }} / ${{ matrix.python }} / ${{ matrix.build_options }}
189+
steps:
190+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
191+
with:
192+
fetch-depth: 0
193+
persist-credentials: false
194+
195+
- name: Set up uv
196+
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
197+
with:
198+
enable-cache: false
199+
200+
- name: Download pythonbuild
201+
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
202+
with:
203+
name: ${{ matrix.target_triple == 'aarch64-unknown-linux-gnu' && 'crate-linux-aarch64' || 'crate-linux-x86_64' }}
204+
path: build
205+
206+
- name: Download images
207+
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
208+
with:
209+
pattern: image-*
210+
path: build
211+
merge-multiple: true
212+
213+
- name: Cache downloads
214+
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
215+
with:
216+
path: build/downloads
217+
key: ${{ matrix.target_triple }}-${{ hashFiles('pythonbuild/downloads.py')}}
218+
restore-keys: |
219+
${{ matrix.target_triple }}-${{ hashFiles('pythonbuild/downloads.py')}}
220+
${{ matrix.target_triple }}-
221+
222+
- name: Load Docker Images
223+
run: |
224+
echo "Files in build directory:"
225+
ls -lah build/
226+
227+
echo "Looking for image archives..."
228+
ls -lah build/image-* || echo "No image files found"
229+
230+
for f in build/image-*.tar.zst; do
231+
if [ -f "$f" ]; then
232+
echo "decompressing $f"
233+
zstd -d --rm ${f}
234+
fi
235+
done
236+
237+
for f in build/image-*.tar; do
238+
if [ -f "$f" ]; then
239+
echo "loading $f"
240+
# Capture the loaded image ID from docker load output
241+
LOADED_ID=$(docker load --input $f 2>&1 | grep "Loaded image ID:" | awk '{print $4}')
242+
echo "Loaded image ID: $LOADED_ID"
243+
244+
# Update the ID file with the actual loaded ID
245+
ID_FILE="${f%.tar}"
246+
if [ -n "$LOADED_ID" ]; then
247+
echo "$LOADED_ID" > "$ID_FILE"
248+
echo "Updated $ID_FILE with $LOADED_ID"
249+
fi
250+
fi
251+
done
252+
253+
echo "Loaded Docker images:"
254+
docker images
255+
256+
- name: Build
257+
run: |
258+
# Do empty target so all generated files are touched.
259+
./build.py --make-target empty
260+
261+
# Touch mtimes of all images so they are newer than autogenerated files above.
262+
touch build/image-*
263+
264+
./build.py --target-triple ${MATRIX_TARGET_TRIPLE} --python cpython-${MATRIX_PYTHON} --options ${MATRIX_BUILD_OPTIONS}
265+
env:
266+
MATRIX_TARGET_TRIPLE: ${{ matrix.target_triple }}
267+
MATRIX_PYTHON: ${{ matrix.python }}
268+
MATRIX_BUILD_OPTIONS: ${{ matrix.build_options }}
269+
270+
- name: Generate attestations
271+
uses: actions/attest-build-provenance@977bb373ede98d70efdf65b84cb5f73e068dcc2a # v3.0.0
272+
if: ${{ github.ref == 'refs/heads/main' }}
273+
with:
274+
subject-path: dist/*
275+
276+
- name: Upload Distribution
277+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
278+
with:
279+
name: cpython-${{ matrix.python }}-${{ matrix.target_triple }}-${{ matrix.build_options }}
280+
path: dist/*
281+
282+
- name: Validate Distribution
283+
run: |
284+
chmod +x build/pythonbuild
285+
286+
# Only run validation with --run for x86_64 (native architecture)
287+
if [ "${MATRIX_TARGET_TRIPLE}" == "x86_64-unknown-linux-gnu" ]; then
288+
EXTRA_ARGS="--run"
289+
fi
290+
291+
build/pythonbuild validate-distribution ${EXTRA_ARGS} dist/*.tar.zst
292+
env:
293+
MATRIX_TARGET_TRIPLE: ${{ matrix.target_triple }}

0 commit comments

Comments
 (0)