Add opt-in ReadyToRun bootstrap for startup perf testing#13302
Add opt-in ReadyToRun bootstrap for startup perf testing#13302JanProvaznik wants to merge 2 commits intodotnet:mainfrom
Conversation
Add OverlayReadyToRunBootstrap target that crossgen2-compiles MSBuild DLLs and overlays them into the bootstrap SDK folder. Usage: build.cmd -c Release /p:PublishReadyToRun=true The target runs after the normal BootstrapNetCore, does a RID-specific restore, then publishes with --no-build to skip recompilation and only run crossgen2 on the already-built DLLs. This adds ~10s to the build (pure crossgen2 time, no double compilation). The RID is auto-detected via NETCoreSdkRuntimeIdentifier. 16 MSBuild-specific DLLs are R2R'd (Microsoft.Build.dll, Framework, Tasks, Utilities, MSBuild.dll, StringTools, plus System.* dependencies). SDK-owned DLLs are already R2R'd by the .NET SDK itself. Without /p:PublishReadyToRun=true, build behavior is unchanged. Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds an opt-in bootstrap step to produce a ReadyToRun (R2R) overlay of MSBuild assemblies for startup perf testing, and documents how to use it.
Changes:
- Add
OverlayReadyToRunBootstraptarget that runs RID-specific restore +dotnet publish --no-buildwithPublishReadyToRun=true, then copies published DLLs into the bootstrap SDK. - Document the new
/p:PublishReadyToRun=truebootstrap flow and suggested benchmarking invocation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| eng/BootStrapMsBuild.targets | Introduces the opt-in R2R overlay target executed after BootstrapNetCore. |
| documentation/wiki/Bootstrap.md | Documents the R2R bootstrap option and benchmarking guidance. |
| </PropertyGroup> | ||
|
|
||
| <Message Text="Publishing MSBuild with ReadyToRun for $(NETCoreSdkRuntimeIdentifier)..." Importance="High" /> | ||
|
|
||
| <!-- First, restore with the target RID so the assets file has the net10.0/rid target. --> | ||
| <Exec Command=""$(DotNetTool)" restore "$(RepoRoot)src\MSBuild\MSBuild.csproj" -p:TargetFramework=$(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier)" /> | ||
|
|
||
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | ||
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | ||
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | ||
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> |
There was a problem hiding this comment.
$(NETCoreSdkRuntimeIdentifier) is referenced for the RID, but it is not defined anywhere in this repo (only used here). If it’s empty, both dotnet restore -r and dotnet publish -r will fail due to a missing argument. Consider introducing an internal property (e.g., _R2RRid) that falls back to a known MSBuild/SDK-provided RID property (such as $(RuntimeIdentifier) when set) or computes one from OS/arch, and fail with a clear message if a RID can’t be determined.
| </PropertyGroup> | |
| <Message Text="Publishing MSBuild with ReadyToRun for $(NETCoreSdkRuntimeIdentifier)..." Importance="High" /> | |
| <!-- First, restore with the target RID so the assets file has the net10.0/rid target. --> | |
| <Exec Command=""$(DotNetTool)" restore "$(RepoRoot)src\MSBuild\MSBuild.csproj" -p:TargetFramework=$(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier)" /> | |
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | |
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | |
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | |
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> | |
| <!-- Resolve the RID for ReadyToRun publishing. | |
| Prefer an explicitly provided NETCoreSdkRuntimeIdentifier, otherwise fall back to RuntimeIdentifier. --> | |
| <_R2RRid Condition="'$(_R2RRid)' == '' and '$(NETCoreSdkRuntimeIdentifier)' != ''">$(NETCoreSdkRuntimeIdentifier)</_R2RRid> | |
| <_R2RRid Condition="'$(_R2RRid)' == '' and '$(RuntimeIdentifier)' != ''">$(RuntimeIdentifier)</_R2RRid> | |
| </PropertyGroup> | |
| <!-- Fail fast with a clear message if we cannot determine a runtime identifier. --> | |
| <Error Condition="'$(_R2RRid)' == ''" | |
| Text="OverlayReadyToRunBootstrap requires a runtime identifier. Set 'RuntimeIdentifier' or 'NETCoreSdkRuntimeIdentifier' when invoking this build." /> | |
| <Message Text="Publishing MSBuild with ReadyToRun for $(_R2RRid)..." Importance="High" /> | |
| <!-- First, restore with the target RID so the assets file has the net10.0/rid target. --> | |
| <Exec Command=""$(DotNetTool)" restore "$(RepoRoot)src\MSBuild\MSBuild.csproj" -p:TargetFramework=$(LatestDotNetCoreForMSBuild) -r $(_R2RRid)" /> | |
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | |
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | |
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | |
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(_R2RRid) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> |
| <PropertyGroup> | ||
| <_R2RPublishDir>$(ArtifactsBinDir)MSBuild.ReadyToRun\</_R2RPublishDir> | ||
| </PropertyGroup> | ||
|
|
||
| <Message Text="Publishing MSBuild with ReadyToRun for $(NETCoreSdkRuntimeIdentifier)..." Importance="High" /> | ||
|
|
||
| <!-- First, restore with the target RID so the assets file has the net10.0/rid target. --> | ||
| <Exec Command=""$(DotNetTool)" restore "$(RepoRoot)src\MSBuild\MSBuild.csproj" -p:TargetFramework=$(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier)" /> | ||
|
|
||
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | ||
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | ||
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | ||
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> | ||
|
|
||
| <ItemGroup> | ||
| <_R2RPublishedDlls Include="$(_R2RPublishDir)*.dll" /> | ||
| </ItemGroup> |
There was a problem hiding this comment.
The publish directory is reused ($(_R2RPublishDir)), and _R2RPublishedDlls is a wildcard over *.dll. If a previous run left extra DLLs in that folder, they’ll be copied into the bootstrap even if the current publish no longer produces them. Deleting/cleaning $(_R2RPublishDir) (and recreating it) before publishing would avoid stale overlays.
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | ||
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | ||
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | ||
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> |
There was a problem hiding this comment.
This target runs an explicit dotnet restore and then dotnet publish without --no-restore, so publish will typically restore again. Adding --no-restore to the publish invocation would avoid the redundant restore and make the behavior depend solely on the preceding restore step.
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> | |
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-restore --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> |
| </ItemGroup> | ||
|
|
||
| <Copy SourceFiles="@(_R2RPublishedDlls)" | ||
| DestinationFolder="$(InstallDir)sdk\$(BootstrapSdkVersion)\" /> |
There was a problem hiding this comment.
The overlay Copy doesn’t set SkipUnchangedFiles="true", which can cause unnecessary file writes into the bootstrap SDK folder on repeated runs. Setting SkipUnchangedFiles would reduce IO overhead and improve incremental perf for this opt-in step.
| DestinationFolder="$(InstallDir)sdk\$(BootstrapSdkVersion)\" /> | |
| DestinationFolder="$(InstallDir)sdk\$(BootstrapSdkVersion)\" | |
| SkipUnchangedFiles="true" /> |
| build.cmd -c Release /p:PublishReadyToRun=true | ||
| ``` | ||
|
|
||
| This runs the normal build, then crossgen2-compiles the 16 MSBuild-specific DLLs (Microsoft.Build.dll, Framework, Tasks, Utilities, etc.) and overlays them into the bootstrap. The R2R binaries are ~2× larger but reduce JIT work at startup. The RID is auto-detected from the current platform. |
There was a problem hiding this comment.
Docs state the RID is “auto-detected from the current platform”, but the implementation relies on $(NETCoreSdkRuntimeIdentifier) being set. Since that property isn’t defined in this repo, either clarify in the docs that the RID property must be provided, or update the build logic to compute/fallback the RID so the statement is accurate.
| This runs the normal build, then crossgen2-compiles the 16 MSBuild-specific DLLs (Microsoft.Build.dll, Framework, Tasks, Utilities, etc.) and overlays them into the bootstrap. The R2R binaries are ~2× larger but reduce JIT work at startup. The RID is auto-detected from the current platform. | |
| This runs the normal build, then crossgen2-compiles the 16 MSBuild-specific DLLs (Microsoft.Build.dll, Framework, Tasks, Utilities, etc.) and overlays them into the bootstrap. The R2R binaries are ~2× larger but reduce JIT work at startup. The RID used for R2R is taken from the `NETCoreSdkRuntimeIdentifier` property; if that property is not set, you must supply an explicit RID (for example, via `/p:RuntimeIdentifier=<rid>`). |
| <!-- Publish with no-build to skip recompilation and only run crossgen2 on already-built DLLs. | ||
| AppendRuntimeIdentifierToOutputPath=false lets publish find the existing non-RID build output. | ||
| BuildProjectReferences=false avoids rebuilding dependency projects. --> | ||
| <Exec Command=""$(DotNetTool)" publish "$(RepoRoot)src\MSBuild\MSBuild.csproj" -c $(Configuration) -f $(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier) --no-build -p:PublishReadyToRun=true -p:ErrorOnDuplicatePublishOutputFiles=false -p:AppendRuntimeIdentifierToOutputPath=false -p:BuildProjectReferences=false -p:PublishDir="$(_R2RPublishDir)"" /> |
There was a problem hiding this comment.
If I'm not wrong, this is a bin-clash. One of the less interesting one but still. The base intermediate and intermediate output paths clash during restore. Restore runs again with the additional set of inputs. This results in the project.assets.json file getting overwritten. If you don't need to re-restore, consider passing in the ´--no-restore` flag. FWIW our skill would detect that.
There was a problem hiding this comment.
Why are we invoking dotnet directly here at all instead of using normal MSBuild Task invocations? Invoking builds of any kinda via exec should be considered a smell.
|
My first thought on seeing this is "can we defer to using the VMR for this?" Especially since I wouldn't want drift between how-we-invoke-crossgen and how-the-official-SDK-build-does-it to cause us confusion. |
| <!-- First, restore with the target RID so the assets file has the net10.0/rid target. --> | ||
| <Exec Command=""$(DotNetTool)" restore "$(RepoRoot)src\MSBuild\MSBuild.csproj" -p:TargetFramework=$(LatestDotNetCoreForMSBuild) -r $(NETCoreSdkRuntimeIdentifier)" /> |
There was a problem hiding this comment.
if we have RuntimeIdentifers set for all of the RIDs we want to target we don't have to do separate restores - as long as we also have PublishReadyToRun set at Restore-time. For the actual builds we can toggle PublishReadyToRun however we need, the idea is just that the Restore becomes as 'maximal' as possible.
SDK official builds do weird invocations of crossgen anyway - they have an SDK-layout specific Task + Targets that call the crossgen DLL on specific lists of assemblies: https://github.com/dotnet/sdk/blob/3e0e00aab9ba5b3d2de2e58edb1dad2a31eb5f8d/src/Layout/redist/targets/Crossgen.targets#L153-L187 that's sufficiently nonstandard that I'm not sure we can really use it? |
That's kinda what I'm saying--if we can't simulate the actual way the product is constructed, are we getting the benefits out of doing it differently here? |
Summary
Adds an opt-in \OverlayReadyToRunBootstrap\ target that crossgen2-compiles MSBuild DLLs and overlays them into the bootstrap SDK folder. This is useful to get a bootstrap with closer performance characteristics to the real product
Usage: \\�uild.cmd -c Release /p:PublishReadyToRun=true\\
What it does
Performance
The ~10s overhead is pure crossgen2 time. MSBuild DLLs grow ~2× (e.g. Microsoft.Build.dll: 3.2MB → 7.2MB) confirming AOT compilation.
Scope
Testing