-
Notifications
You must be signed in to change notification settings - Fork 26
Add GitLab Runner integration #711
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
pkg/integrations/gitlab/glrunner.go
Outdated
| ) | ||
|
|
||
| // Download URL | ||
| const runnerBaseURL = "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-%s-%s" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here I would prefer fixed version...you can pick it from a var and move it through the Makefile so we can update when new version is released.
Cause using latest we can no be sure which version we are shipping
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I added the following changes:
- Default version: 18.8.0 (variable in glrunner.go)
- Downloads from: https://gitlab-runner-downloads.s3.amazonaws.com/v18.8.0/binaries/...
- Build overrides: Makefile injects the version via ldflags at compile time (similar to how GITHUB_RUNNER parameter works)
- Auto-updates: Renovate bot will create PRs when new GitLab Runner versions are released (similar to how it works in renovate configuration)
56534ac to
e44f12a
Compare
| cirrusPWLabels string = "it-cirrus-pw-labels" | ||
| cirrusPWLabelsDesc string = "additional labels to use on the persistent worker (--it-cirrus-pw-labels key1=value1,key2=value2)" | ||
|
|
||
| glRunnerToken string = "glrunner-token" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think labels should be needed, as we need to control them to distribute jobs from gitlab as we do currently
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing it out! i just added a new flag --glrunner-tags in order to allow adding tags to the gitlab runner instance.
adrianriobo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| return err | ||
|
|
||
| // Generate userdata - if GitLab runner args present, create runner via Pulumi first | ||
| var userDataB64 pulumi.StringPtrInput |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this here, the logic should be moved to the fedora helper... you can do the check there and add the content
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I’ve moved the logic into the Fedora helper.
- Added
GenerateUserdatain the Fedora helper to handle the GitLab runner check and userdata generation - The action now just calls this helper instead of containing conditional logic
This should keep the action layer thin and centralize Fedora-specific logic in one place.
| userDataB64, err := rhelCloudConfig.CloudConfig() | ||
| if err != nil { | ||
| return err | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here same as fedora no?
| } | ||
| userDataB64, err := cloudConfigWindowsServer.Userdata(ctx, &amiUserDefault, password, keyResources) | ||
| if err != nil { | ||
| return err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here same as others
|
|
||
| if r.isRequestOperation { | ||
| return macSetup.Request( | ||
| // Check if GitLab runner args exist |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here the same
| userDataB64, err = r.cloudConfigAsUserData.CloudConfig() | ||
| if err != nil { | ||
| return fmt.Errorf("error creating RHEL Server on Azure: %v", err) | ||
| // Check if this is RHEL cloud config and GitLab runner args exist |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here same
Migrate GitLab Runner integration from registration token workflow
to authentication token workflow using Pulumi's GitLab provider.
This aligns with GitLab 16.0+ recommended practices and enables
programmatic runner creation.
Changes:
- --glrunner-token now expects GitLab PAT (api/create_runner scope)
- New flags: --glrunner-project-id and --glrunner-group-id
- Removed flags: --glrunner-executor, --glrunner-tags, --glrunner-scope
Features:
- Support for both project runners and group runners
- Automatic runner creation via Pulumi during stack deployment
- Runner configuration managed as code (tags, executor, access level)
- Auto-detection of runner type based on ID provided
Implementation:
- Created pkg/integrations/gitlab/pulumi.go for Pulumi GitLab provider
- Updated GitLabRunnerArgs with GitLabPAT, ProjectID, GroupID, AuthToken
- Added SetAuthToken() for dynamic token injection during deployment
- Implemented UserdataWithGitLabToken() for async token handling
- Updated AWS Fedora provider to create runner before userdata generation
- Modified CLI params to support new flag structure
Usage:
./mapt aws fedora create \
--glrunner-token <gitlab-pat> \
--glrunner-group-id 12345 \
--glrunner-url https://gitlab.com
Signed-off-by: Jose Angel Morena <[email protected]>
Add GitLab Runner integration to mapt
This PR adds native GitLab Runner support to mapt, allowing automatic runner registration on provisioned hosts and automatic creation of the gitlab runner on the gitlab server.
At the moment, the integration is limited to infrastructure-based platforms and does not support container-based deployments.
Features
New CLI flags:
--glrunner-token--glrunner-urlAutomatic, non-interactive GitLab Runner registration
Shell executor support (only executor supported for now)
Runner name automatically generated from the mapt run ID
Implementation details
Introduced
pkg/integrations/gitlab/with platform-specific bootstrap snippetsAdded GitLab runner flags to CLI parameter handling
Integrated GitLab runner setup into cloud-config for:
Updated context manager to initialize the GitLab integration
Extended all host provisioning commands to support GitLab runner parameters
Technical notes
Runner binaries are downloaded from GitLab’s official distribution bucket
Registration is performed non-interactively using an authentication token
Service installation is platform-specific:
Runner configuration such as tags, executor settings, and scope are intentionally managed from the GitLab UI rather than via CLI
Example usage
./out/mapt aws fedora create \ --glrunner-token ${GITLAB_TOKEN} \ --glrunner-group-id ${GITLAB_GROUP_ID} \ --glrunner-url https://gitlab.com \ --glrunner-tags linux,aws,fedora \ --project-name test-gitlab-runner \ --backed-url file:///$PWD/state \ --conn-details-output $PWD/outputsSupported platforms
AWS
Azure
Fixes #500