Skip to content

Commit ae8bb9c

Browse files
authored
Update new-repo.md
1 parent 75ac9b3 commit ae8bb9c

File tree

1 file changed

+6
-188
lines changed

1 file changed

+6
-188
lines changed

Documentation/sourcebuild-in-repos/new-repo.md

Lines changed: 6 additions & 188 deletions
Original file line numberDiff line numberDiff line change
@@ -5,90 +5,11 @@ build.
55

66
1. [Source Build Configuration](#source-build-configuration)
77
1. [Setup CI](#setup-ci)
8-
1. [Incorporate the new repo into the source build dependency
9-
tree](#incorporate-the-new-repo-into-the-source-build-dependency-tree)
108
1. [Source build repos and the VMR](#source-build-repos-and-the-vmr)
119
1. [Validate](#validate)
12-
1. [Additional resources](#additional-resources)
1310

1411
## Source Build Configuration
1512

16-
Configuring source build involves setting up files in `eng/` that determine the
17-
behavior of source build in the repo.
18-
19-
These changes are all needed before source build will work:
20-
21-
* [`eng/DotNetBuild.props`](#engdotnetbuildprops) - Basic properties, such as
22-
repo name.
23-
* [`eng/SourceBuildPrebuiltBaseline.xml`](#engsourcebuildprebuiltbaselinexml) -
24-
List of allowed prebuilts (approval required).
25-
* [`eng/Version.Details.xml`](#engversiondetailsxml) - Already exists, but
26-
modifications are needed to pull dependencies from upstream [intermediate
27-
nupkgs](../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs).
28-
29-
See the following sections for details:
30-
31-
### `eng/DotNetBuild.props`
32-
33-
```xml
34-
<Project>
35-
36-
<PropertyGroup>
37-
<GitHubRepositoryName>this-repo</GitHubRepositoryName>
38-
<SourceBuildManagedOnly>true</SourceBuildManagedOnly>
39-
</PropertyGroup>
40-
41-
</Project>
42-
```
43-
44-
This file contains basic configuration used to restore the correct dependencies
45-
(managed-only or not) and produce the right name for the repo's [intermediate
46-
nupkg].
47-
48-
* `this-repo` should be the same as the repo's name on GitHub, without the
49-
GitHub organization name.
50-
* `SourceBuildManagedOnly` defaults to false if omitted. `true` means that the
51-
repo doesn't produce any RID-specific artifacts like
52-
`Microsoft.NETCore.App.Runtime.linux-x64`, only managed code.
53-
54-
These two properties determine the name of the [intermediate nupkg]:
55-
`Microsoft.SourceBuild.Intermediate.$(GitHubRepositoryName)[.$(RidSuffix)]`.
56-
57-
It's possible more configuration will be required for specific repos.
58-
`eng/DotNetBuild.props`, similar to `eng/Build.props`, is a place to add extra
59-
MSBuild code that can change the way source build behaves.
60-
61-
### `eng/SourceBuildPrebuiltBaseline.xml`
62-
63-
```xml
64-
<!-- When altering this file or making other Source Build related changes, include @dotnet/source-build as a reviewer. -->
65-
<!-- See aka.ms/dotnet/prebuilts for guidance on what pre-builts are and how to eliminate them. -->
66-
67-
<UsageData>
68-
<IgnorePatterns>
69-
<UsagePattern IdentityGlob="Microsoft.SourceBuild.Intermediate.*/*" />
70-
</IgnorePatterns>
71-
</UsageData>
72-
```
73-
74-
This defines which prebuilt NuGet packages are allowed to be used during the
75-
build. Initially, only the source build intermediate packages are allowed. The
76-
`*/*` glob means "any intermediate package, any version". All other prebuilts
77-
require approval from the source build team.
78-
79-
When a PR introduces an unexpected prebuilt, PR validation will fail and let us
80-
resolve the source-buildability issue before the PR gets merged.
81-
82-
### CODEOWNERS
83-
84-
Add the source build team as the
85-
[CODEOWNER](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners)
86-
of the source build infrastructure.
87-
88-
``` text
89-
/eng/SourceBuild* @dotnet/source-build
90-
```
91-
9213
### Trying it out locally
9314

9415
If a repo passes build options through to the common arcade build script, a
@@ -98,97 +19,30 @@ source build can be triggered via the following command.
9819
./build.sh -sb
9920
```
10021

101-
If a repo does not pass through build options through to the common arcade build
102-
script, then you must specify the `/p:ArcadeBuildFromSource=true` property as
103-
follows.
104-
105-
```bash
106-
./build.sh -c Release --restore --build --pack /p:ArcadeBuildFromSource=true -bl
107-
```
108-
10922
> Note: [source build is not supported on
11023
Windows](https://github.com/dotnet/source-build/issues/1190), only Linux and
11124
macOS.
11225

113-
After running the build, source build artifacts will be in
114-
`artifacts/sb`, and the [intermediate nupkg] will be something like
115-
`artifacts/packages/*/Microsoft.SourceBuild.Intermediate.*.nupkg`.
116-
117-
The MSBuild binlog will be placed somewhere like:
118-
`artifacts/log/Release/Build.binlog`. However, this "outer" binlog doesn't contain
119-
the meat of the build: the "inner" build runs inside an `Exec` task. The inner
120-
binlog will be written to somewhere like:
121-
`artifacts/sb/src/artifacts/log/Release/source-inner-build.binlog`.
122-
12326
### Excluding components
12427

12528
It is not always necessary or correct to include all repo components in source
12629
build. Components should be excluded if they are not required for the platform
12730
being source-built. Including them expands the source build graph and may not
12831
be possible because of licensing. Examples include tests, Windows components
12932
(remember source build currently only supports Linux and macOS), etc. To exlcude
130-
these components use the `DotNetBuildFromSource` msbuild property to
33+
these components use the `DotNetBuildSourceOnly` msbuild property to
13134
conditionally exclude.
13235

13336
```code
134-
Condition="'$(DotNetBuildFromSource)' != 'true'"
37+
Condition="'$(DotNetBuildSourceOnly)' != 'true'"
13538
```
13639

13740
See the [Unified Build Controls](https://github.com/dotnet/arcade/blob/main/Documentation/UnifiedBuild/Unified-Build-Controls.md)
13841
for more options on how to exclude components from source build.
13942

140-
### `eng/Version.Details.xml`
141-
142-
```xml
143-
...
144-
<Dependency Name="Microsoft.NETCore.App.Runtime.win-x64" Version="6.0.0-alpha.1.20468.7"
145-
CoherentParentDependency="Microsoft.NET.Sdk">
146-
<Uri>https://github.com/dotnet/runtime</Uri>
147-
<Sha>a820ca1c4f9cb5892331e2624d3999c39161fe2a</Sha>
148-
<SourceBuild RepoName="runtime" />
149-
</Dependency>
150-
...
151-
<Dependency Name="Microsoft.SourceBuild.Intermediate.source-build-reference-packages"
152-
Version="5.0.0-alpha.1.20473.1">
153-
<Uri>https://github.com/dotnet/source-build-reference-packages</Uri>
154-
<Sha>def2e2c6dc5064319250e2868a041a3dc07f9579</Sha>
155-
<SourceBuild RepoName="source-build-reference-packages"
156-
ManagedOnly="true" />
157-
</Dependency>
158-
...
159-
```
160-
161-
Dependency changes may include adding `SourceBuild` to existing dependency
162-
elements, and adding a new `source-build-reference-packages` element.
163-
164-
`SourceBuild` causes the source build targets in the Arcade SDK to download
165-
[intermediate nupkg]s from the upstream repo's official build, rather than using
166-
prebuilt binaries to fulfill the dependencies. Note that `RepoName` is used to
167-
calculate the ID of the [intermediate
168-
nupkg](../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs):
169-
the `Dependency` `Name` is ignored by source build.
170-
171-
Building with the source-built versions of your dependencies also means that any
172-
upstream repos will have been built in a source build context, including things
173-
projects that are excluded from the source build. This can help you find issues
174-
where your source build depends on an upstream component that isn't actually
175-
built in source build.
176-
177-
`ManagedOnly` determines whether a RID suffix is necessary on the [intermediate
178-
nupkg](../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs)
179-
ID. For example, running source build on `dotnet/installer` with `linux-x64`
180-
with the above example configuration will restore:
181-
182-
* `Microsoft.SourceBuild.Intermediate.runtime.linux-x64`
183-
* `.linux-x64` because `ManagedOnly` is not `true`.
184-
* `Microsoft.SourceBuild.Intermediate.source-build-reference-packages`
185-
* Ends with the `RepoName` without a suffix because `ManagedOnly="true"`.
186-
18743
## Setup CI
18844

189-
Source build needs to run during official builds to create source build
190-
[intermediate nupkg]s for the downstream repos that will consume them. Source
191-
build should also run in PR validation builds, to prevent regressions.
45+
Source build should run in PR validation builds, to prevent regressions.
19246

19347
Source build CI can be activated with a single flag in the ordinary Arcade SDK
19448
jobs template for easy consumption. If a repo can't simply use the Arcade SDK
@@ -297,46 +151,19 @@ the most granular template, and may be useful if some repo-specific preamble or
297151
cleanup steps are required, or if the repo already has job matrix templates and
298152
this just happens to fit in nicely.
299153

300-
### Official build publishing
301-
302-
Publishing [intermediate nupkg]s in the official build is handled by the
303-
standard Arcade publish infrastructure, which should already be set up in the
304-
repo. The source build steps handle uploading the [intermediate nupkg] to the
305-
pipeline in the standard way that Arcade will detect and publish.
306-
307-
[intermediate nupkg]:
308-
https://github.com/dotnet/source-build/blob/main/Documentation/planning/arcade-powered-source-build/intermediate-nupkg.md
309-
310-
## Incorporate the new repo into the source build dependency tree
311-
312-
Once your repo can be source-built, it is time to register it into the source
313-
build dependency tree. The graph of the product is defined by the
314-
`eng/Version.Details.xml` files. This dependency graph starts at
315-
[dotnet/sdk](https://github.com/dotnet/sdk/blob/main/eng/Version.Details.xml).
316-
The dependendecies of repos declared in these files are walked and the first
317-
copy/commit of each repo found in the dependency graph is used.
318-
319-
Therefore, when adding new repositories, a dependency must be created within the
320-
`eng/Version.Details.xml` graph. To do this, **go to the repo which depends on
321-
the new repo and add a [new source build dependency](#engversiondetailsxml) to
322-
the new source build repo**.
323-
324154
## Source build repos and the VMR
325155

326156
Another effect of adding a new source build repository is that its sources will
327157
be synchronized into the [Virtual Monolithic Repository of
328158
.NET](https://github.com/dotnet/dotnet). The VMR is then where the official
329-
source build happens from. The sources are synchronized once the associated
330-
commit/package flows into `dotnet/sdk`.
159+
source build happens from.
331160

332161
In order for the sources of the new repo to by synchronized into the VMR, the
333162
repo needs to be registered in the [`source-mappings.json`
334163
file](https://github.com/dotnet/dotnet/blob/main/src/source-mappings.json) which
335164
tells the tooling where from and which sources should be synchronized. Please
336-
open a PR in [`dotnet/sdk`](https://github.com/dotnet/sdk) and add
337-
your repository into `src/VirtualMonoRepo/source-mappings.json`. The name must
338-
match the name declared in the `SourceBuild` tag in `Version.Details.xml`
339-
created in the previous step.
165+
open a PR in [`dotnet/sdk`](https://github.com/dotnet/dotnet) and add
166+
your repository into `src/source-mappings.json`.
340167

341168
### Cloaking (filtering) the repository sources
342169

@@ -360,12 +187,3 @@ VMR will be built. This will validate that no prebuilts were added to the system
360187
and everything is functioning correctly. Please notify
361188
[@source-build](https://github.com/orgs/dotnet/teams/source-build)
362189
to be on the lookout for the new repo and they will validate as necessary.
363-
364-
## Additional resources
365-
366-
* For more details about how the build executes, see [Arcade's build
367-
tools](https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.Arcade.Sdk/tools/SourceBuild).
368-
* The source code for the build tasks that run for prebuilt validation and
369-
intermediate nupkg dependency reading are maintained in
370-
[Arcade](https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.SourceBuild)
371-
as well.

0 commit comments

Comments
 (0)