Skip to content

Commit 74fdf4f

Browse files
gopalldbclaude
gopalldb
and
claude
authored
Use Databricks protected runner group for CI actions (#1369)
## Summary - Reverts #1350 (migration to GitHub-hosted runners) — GitHub-hosted runner IPs are blocked by the Databricks org IP allow list - Adds Maven dependency caching infrastructure for forked PRs to work with JFrog Artifactory proxy ## Problem After migrating to JFrog Artifactory as a Maven registry proxy (supply chain security), forked PRs cannot authenticate to JFrog because GitHub Actions restricts OIDC token minting for fork workflows. This breaks all forked PR CI (unit tests, integration tests, formatting, coverage). ## Solution: Cache-based dependency resolution for forks ### New files - `.github/actions/setup-maven/action.yml` — Reusable composite action that detects forked PRs and either authenticates to JFrog (same-repo) or restores the dependency cache (fork). Eliminates duplicated OIDC + cache blocks across workflows. - `.github/workflows/warmMavenCache.yml` — Privileged workflow that resolves all dependencies via JFrog and saves the cache. Triggers: push to main (on pom.xml changes), daily schedule (prevents 7-day cache eviction), manual dispatch with optional PR number to warm cache from a fork pom.xml (sparse checkout of only pom.xml files — no source code from fork). ### Modified workflows (use composite action) - `prCheck.yml` (formatting, unit tests, packaging) — 3 jobs now use fork detection + composite action - `prIntegrationTests.yml` — uses composite action - `coverageReport.yml` — uses composite action ### How it works 1. Main branch and same-repo PRs: unchanged — OIDC to JFrog as before 2. Cache warmer saves dependencies with key `{os}-maven-deps-{hash(pom.xml)}` — each unique pom.xml gets its own cache entry (content-addressable), so concurrent PRs with different dependency versions coexist 3. Forked PRs: restore cache from default branch (GitHub allows this per cache sharing rules), no JFrog credentials 4. If a forked PR changes pom.xml and the new dependency is not in cache, CI fails with a clear error. A maintainer triggers the cache warmer with the PR number to warm the cache, then the contributor re-runs CI. 5. Stale cache entries (from closed/merged PRs) are auto-evicted by GitHub after 7 days of no access ## Test plan - [ ] CI checks pass on this PR using Databricks runners - [ ] After merge: manually trigger "Warm Maven Dependency Cache" workflow on main - [ ] Verify cache entries appear via Actions > Caches - [ ] Create a test forked PR and verify CI restores cache and builds pass - [ ] Verify forked PR with pom.xml change fails clearly, then passes after cache warm NO_CHANGELOG=true --------- Signed-off-by: Gopal Lal <gopal.lal@databricks.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent addeeb7 commit 74fdf4f

24 files changed

+991
-82
lines changed

‎.github/actions/setup-maven/action.yml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: 'Setup Maven with JFrog or Cache'
2+
description: |
3+
Configures Maven for builds. For same-repo PRs and main branch pushes,
4+
authenticates to JFrog Artifactory via OIDC token exchange. For forked PRs,
5+
restores the Maven dependency cache from the default branch (no credentials).
6+
7+
inputs:
8+
is-fork:
9+
description: 'Whether this is a forked PR (true/false). Caller should compute this.'
10+
required: true
11+
12+
runs:
13+
using: 'composite'
14+
steps:
15+
# --- Forked PR path: restore cache only, no JFrog credentials ---
16+
- name: Restore Maven cache (fork)
17+
if: inputs.is-fork == 'true'
18+
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
19+
with:
20+
path: ~/.m2/repository
21+
key: ${{ runner.os }}-maven-deps-${{ hashFiles('**/pom.xml') }}
22+
restore-keys: ${{ runner.os }}-maven-deps-
23+
24+
- name: Configure Maven without credentials (fork)
25+
if: inputs.is-fork == 'true'
26+
shell: bash
27+
run: |
28+
mkdir -p ~/.m2
29+
cat > ~/.m2/settings.xml << 'SETTINGS_EOF'
30+
<settings>
31+
<mirrors>
32+
<mirror>
33+
<id>jfrog-central</id>
34+
<mirrorOf>*</mirrorOf>
35+
<url>https://databricks.jfrog.io/artifactory/db-maven/</url>
36+
</mirror>
37+
</mirrors>
38+
<!-- No <servers> block: forked PRs use cached dependencies only.
39+
Any cache miss will produce a clear HTTP 401 error indicating
40+
the dependency is not in cache. Ask a maintainer to run the
41+
"Warm Maven Dependency Cache" workflow with your PR number. -->
42+
</settings>
43+
SETTINGS_EOF
44+
45+
echo "Maven configured for forked PR (cache-only, no JFrog credentials)"
46+
47+
# --- Same-repo path: full JFrog OIDC authentication ---
48+
- name: Get JFrog OIDC token
49+
if: inputs.is-fork == 'false'
50+
shell: bash
51+
run: |
52+
set -euo pipefail
53+
54+
# Get GitHub OIDC ID token
55+
ID_TOKEN=$(curl -sLS \
56+
-H "User-Agent: actions/oidc-client" \
57+
-H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
58+
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=jfrog-github" | jq .value | tr -d '"')
59+
echo "::add-mask::${ID_TOKEN}"
60+
61+
# Exchange for JFrog access token
62+
ACCESS_TOKEN=$(curl -sLS -XPOST -H "Content-Type: application/json" \
63+
"https://databricks.jfrog.io/access/api/v1/oidc/token" \
64+
-d "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"${ID_TOKEN}\", \"provider_name\": \"github-actions\"}" | jq .access_token | tr -d '"')
65+
echo "::add-mask::${ACCESS_TOKEN}"
66+
67+
if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
68+
echo "FAIL: Could not extract JFrog access token"
69+
exit 1
70+
fi
71+
72+
echo "JFROG_ACCESS_TOKEN=${ACCESS_TOKEN}" >> "$GITHUB_ENV"
73+
74+
echo "JFrog OIDC token obtained successfully"
75+
76+
- name: Configure Maven with JFrog credentials
77+
if: inputs.is-fork == 'false'
78+
shell: bash
79+
run: |
80+
set -euo pipefail
81+
82+
mkdir -p ~/.m2
83+
cat > ~/.m2/settings.xml << EOF
84+
<settings>
85+
<mirrors>
86+
<mirror>
87+
<id>jfrog-central</id>
88+
<mirrorOf>*</mirrorOf>
89+
<url>https://databricks.jfrog.io/artifactory/db-maven/</url>
90+
</mirror>
91+
</mirrors>
92+
<servers>
93+
<server>
94+
<id>jfrog-central</id>
95+
<username>gha-service-account</username>
96+
<password>${JFROG_ACCESS_TOKEN}</password>
97+
</server>
98+
</servers>
99+
</settings>
100+
EOF
101+
102+
echo "Maven configured to use JFrog registry"
103+
104+
- name: Cache Maven packages
105+
if: inputs.is-fork == 'false'
106+
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
107+
with:
108+
path: ~/.m2
109+
key: ${{ runner.os }}-maven-deps-${{ hashFiles('**/pom.xml') }}
110+
restore-keys: ${{ runner.os }}-maven-deps-

‎.github/workflows/bugCatcher.yml‎

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,15 @@ on:
66
- cron: '0 0 * * 1'
77

88
permissions:
9+
id-token: write
910
contents: read
1011

1112
jobs:
1213
build-and-test:
1314
name: Build and Run Integration Tests
14-
runs-on: ubuntu-latest
15+
runs-on:
16+
group: databricks-protected-runner-group
17+
labels: linux-ubuntu-latest
1518
environment: azure-prod
1619
steps:
1720
- name: Checkout code
@@ -23,6 +26,58 @@ jobs:
2326
java-version: 21
2427
distribution: 'adopt'
2528

29+
- name: Get JFrog OIDC token
30+
run: |
31+
set -euo pipefail
32+
33+
# Get GitHub OIDC ID token
34+
ID_TOKEN=$(curl -sLS \
35+
-H "User-Agent: actions/oidc-client" \
36+
-H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
37+
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=jfrog-github" | jq .value | tr -d '"')
38+
echo "::add-mask::${ID_TOKEN}"
39+
40+
# Exchange for JFrog access token
41+
ACCESS_TOKEN=$(curl -sLS -XPOST -H "Content-Type: application/json" \
42+
"https://databricks.jfrog.io/access/api/v1/oidc/token" \
43+
-d "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"${ID_TOKEN}\", \"provider_name\": \"github-actions\"}" | jq .access_token | tr -d '"')
44+
echo "::add-mask::${ACCESS_TOKEN}"
45+
46+
if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
47+
echo "FAIL: Could not extract JFrog access token"
48+
exit 1
49+
fi
50+
51+
echo "JFROG_ACCESS_TOKEN=${ACCESS_TOKEN}" >> "$GITHUB_ENV"
52+
53+
echo "JFrog OIDC token obtained successfully"
54+
55+
- name: Configure maven
56+
run: |
57+
set -euo pipefail
58+
59+
mkdir -p ~/.m2
60+
cat > ~/.m2/settings.xml << EOF
61+
<settings>
62+
<mirrors>
63+
<mirror>
64+
<id>jfrog-central</id>
65+
<mirrorOf>*</mirrorOf>
66+
<url>https://databricks.jfrog.io/artifactory/db-maven/</url>
67+
</mirror>
68+
</mirrors>
69+
<servers>
70+
<server>
71+
<id>jfrog-central</id>
72+
<username>gha-service-account</username>
73+
<password>${JFROG_ACCESS_TOKEN}</password>
74+
</server>
75+
</servers>
76+
</settings>
77+
EOF
78+
79+
echo "Maven configured to use JFrog registry"
80+
2681
- name: Cache Maven packages
2782
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
2883
with:

‎.github/workflows/checkNextChangelog.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,9 @@ permissions:
99

1010
jobs:
1111
check-next-changelog:
12-
runs-on: ubuntu-latest
12+
runs-on:
13+
group: databricks-protected-runner-group
14+
labels: linux-ubuntu-latest
1315

1416
steps:
1517
- name: Checkout base branch (main)

‎.github/workflows/claude-code-review.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ jobs:
1212
(github.event.comment.author_association == 'OWNER' ||
1313
github.event.comment.author_association == 'MEMBER' ||
1414
github.event.comment.author_association == 'COLLABORATOR')
15-
runs-on: ubuntu-latest
15+
runs-on:
16+
group: databricks-protected-runner-group
17+
labels: linux-ubuntu-latest
1618
permissions:
1719
contents: read
1820
pull-requests: write

‎.github/workflows/claude.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@ jobs:
2121
(github.event.review.author_association == 'OWNER' || github.event.review.author_association == 'MEMBER' || github.event.review.author_association == 'COLLABORATOR')) ||
2222
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')) &&
2323
(github.event.issue.author_association == 'OWNER' || github.event.issue.author_association == 'MEMBER' || github.event.issue.author_association == 'COLLABORATOR'))
24-
runs-on: ubuntu-latest
24+
runs-on:
25+
group: databricks-protected-runner-group
26+
labels: linux-ubuntu-latest
2527
permissions:
2628
contents: read
2729
pull-requests: read

‎.github/workflows/closeStale.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ permissions:
1010

1111
jobs:
1212
stale:
13-
runs-on: ubuntu-latest
13+
runs-on:
14+
group: databricks-protected-runner-group
15+
labels: linux-ubuntu-latest
1416
steps:
1517
# Release v8 (https://github.com/actions/stale/tree/v8.0.0)
1618
- uses: actions/stale@1160a2240286f5da8ec72b1c0816ce2481aabf84 # v8

‎.github/workflows/concurrencyExecutionTests.yml‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,15 @@ on:
1515
default: 'databricks/databricks-jdbc'
1616

1717
permissions:
18+
id-token: write
1819
contents: read
1920

2021
jobs:
2122
concurrency-tests:
2223
name: Run Concurrency Execution Tests
23-
runs-on: ubuntu-latest
24+
runs-on:
25+
group: databricks-protected-runner-group
26+
labels: linux-ubuntu-latest
2427
environment: azure-prod
2528
strategy:
2629
fail-fast: false
@@ -41,12 +44,12 @@ jobs:
4144
java-version: ${{ matrix.java-version }}
4245
distribution: 'adopt'
4346

44-
- name: Cache Maven packages
45-
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
47+
# This workflow only runs on push to main or manual dispatch — never on forked PRs.
48+
# is-fork is always false here, but we use the composite action for consistency.
49+
- name: Setup Maven
50+
uses: ./.github/actions/setup-maven
4651
with:
47-
path: ~/.m2
48-
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
49-
restore-keys: ${{ runner.os }}-m2
52+
is-fork: 'false'
5053

5154
- name: Run Concurrency Execution Tests
5255
run: mvn -pl jdbc-core -B test -Dtest=com.databricks.jdbc.integration.e2e.ConcurrentExecutionTests -DargLine="-ea"

‎.github/workflows/coverageReport.yml‎

Lines changed: 35 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,52 @@
11
name: Code Coverage
22

33
permissions:
4+
id-token: write
45
contents: read
56

67
on:
78
pull_request:
89

910
jobs:
1011
coverage:
11-
runs-on: ubuntu-latest
12-
12+
runs-on:
13+
group: databricks-protected-runner-group
14+
labels: linux-ubuntu-latest
15+
1316
steps:
1417
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
1518
with:
1619
fetch-depth: 0 # Needed for coverage comparison
1720
ref: ${{ github.event.pull_request.head.ref || github.ref_name }}
1821
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
19-
22+
2023
- name: Set up JDK
2124
uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
2225
with:
2326
java-version: '21'
2427
distribution: 'adopt'
25-
26-
- name: Cache dependencies
27-
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
28+
29+
- name: Check if fork
30+
id: fork-check
31+
shell: bash
32+
run: |
33+
if [ "${{ github.event.pull_request.head.repo.full_name }}" != "" ] && \
34+
[ "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]; then
35+
echo "is_fork=true" >> $GITHUB_OUTPUT
36+
echo "This is a forked PR — will use cached dependencies"
37+
else
38+
echo "is_fork=false" >> $GITHUB_OUTPUT
39+
echo "This is a same-repo PR — will use JFrog OIDC"
40+
fi
41+
42+
- name: Setup Maven
43+
uses: ./.github/actions/setup-maven
2844
with:
29-
path: ~/.m2
30-
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
31-
restore-keys: |
32-
${{ runner.os }}-m2-
33-
45+
is-fork: ${{ steps.fork-check.outputs.is_fork }}
46+
3447
- name: Run tests with coverage
3548
run: mvn -pl jdbc-core clean test -Dgroups='!Jvm17PlusAndArrowToNioReflectionDisabled' jacoco:report
36-
49+
3750
- name: Check for coverage override
3851
id: override
3952
env:
@@ -49,7 +62,7 @@ jobs:
4962
echo "override=false" >> $GITHUB_OUTPUT
5063
echo "No coverage override found"
5164
fi
52-
65+
5366
- name: Check coverage percentage
5467
if: steps.override.outputs.override == 'false'
5568
run: |
@@ -58,22 +71,22 @@ jobs:
5871
echo "ERROR: Coverage file not found at $COVERAGE_FILE"
5972
exit 1
6073
fi
61-
74+
6275
# Install xmllint if not available
6376
if ! command -v xmllint &> /dev/null; then
6477
sudo apt-get update && sudo apt-get install -y libxml2-utils
6578
fi
66-
79+
6780
COVERED=$(xmllint --xpath "string(//report/counter[@type='INSTRUCTION']/@covered)" "$COVERAGE_FILE")
6881
MISSED=$(xmllint --xpath "string(//report/counter[@type='INSTRUCTION']/@missed)" "$COVERAGE_FILE")
6982
TOTAL=$((COVERED + MISSED))
70-
83+
7184
# Use Python for floating-point math
7285
PERCENTAGE=$(python3 -c "covered=${COVERED}; total=${TOTAL}; print(round((covered/total)*100, 2))")
73-
86+
7487
echo "Branch Coverage: $PERCENTAGE%"
7588
echo "Required Coverage: 85%"
76-
89+
7790
# Use Python to compare the coverage with 85
7891
python3 -c "import sys; sys.exit(0 if float('$PERCENTAGE') >= 85 else 1)"
7992
if [ $? -eq 1 ]; then
@@ -82,15 +95,15 @@ jobs:
8295
else
8396
echo "SUCCESS: Coverage is $PERCENTAGE%, which meets the required 85%"
8497
fi
85-
98+
8699
- name: Coverage enforcement summary
87100
env:
88101
OVERRIDE: ${{ steps.override.outputs.override }}
89102
OVERRIDE_REASON: ${{ steps.override.outputs.reason }}
90103
run: |
91104
if [ "$OVERRIDE" == "true" ]; then
92-
echo "⚠️ Coverage checks bypassed: $OVERRIDE_REASON"
105+
echo "Coverage checks bypassed: $OVERRIDE_REASON"
93106
echo "Please ensure this override is justified and temporary"
94107
else
95-
echo "✅ Coverage checks enforced - minimum 85% required"
96-
fi
108+
echo "Coverage checks enforced - minimum 85% required"
109+
fi

‎.github/workflows/dco-check.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ permissions:
1010

1111
jobs:
1212
dco-check:
13-
runs-on: ubuntu-latest
13+
runs-on:
14+
group: databricks-protected-runner-group
15+
labels: linux-ubuntu-latest
1416
name: Check DCO Sign-off
1517
steps:
1618
- name: Checkout

0 commit comments

Comments
 (0)