Skip to content

Commit 4410dbf

Browse files
benrr101benrr101
authored
AKV Official Pipeline (#3150)
* In Solution: Move source projects under source folder, remove add-ons folder * Add engineering files to solution * Add new variables files for AKV (and v2) pipelines * ESRP signing step (and associated variables) * Renaming esrp steps * Adding steps for publishing AKV symbols * Adding nuget packaging steps * Make symbols publishing more generic * Add task for roslyn analyzers * Adding task for building AKV project * Finishing up wiring the tasks to jobs to pipeline * Fix errors, round 1 * Fix typo 1 * Fix errors in job file * Forgot a colon * Renaming SET step to make sure it follows convention * Apparently script needs to go first * Reworking the entire code signing and publish steps to use parameters instead of variables. * Fix parameter reference * Fix parameter reference 2 * Adding onebranch variables to pipeline * Fixing definition of common variables * Let's try again without the ob variables file * Uhhhh... * UH. * WTF. * Another attempt. * I'm confused. * Idk, let's try it how the docs say to do it. * Remove common variables group * bring back the onebranch variables? * Rework the AKV build target ... why the heck does the build.proj totally mess everything up * Fix typo with key path * Let's try this without the multiple lines? * What's wrong with YAML strings. * Fix mistake? * Maybe it doesn't like the extra spaces... * Debug the parameters! * Rename variables? * Ugh is this another one that doesn't support multi-line? * So apparently extra indenting means yaml preserves the dang newline. * More debug... * "Artifact" folder doesn't exist yet. Duh. * * Cleanup build.proj targets a tiny bit * Attempt to make roslyn work without indentations * Make sure project reference parameters are case-correct * Fix property definition in build.proj :shrug_man: * Fixing build.proj ... there's still some complexities in here that are breaking things * Copy the dll/pdb for ApiScan * Why even have array types if we can't use them * Rename file * Reinstating the tree command, fixing some path variables. * Fixing source path for apiscan extract * Man just when I think I get it, I don't get it. * Not sure why it matters if there is a slash at the end or not. But let's try it without one. * I guess variable templates can't depend on variables from other templates? * There's no type safety anywhere else, but there is when I want to print an array... * 😑 * Fix template name for symbols publishing step * Fix values name * Yeah, I don't know what's broken here. This is a random guess. * Parameter to symbols publishing step * Artifacts path != Build output artifacts folder * Remove \" * Language injection would've helped me with that * Needed some quote marks * Fix issue with network isolation * Rename buildPreview to runSdlTasks, cleanup a couple to-dos * Whoops. * Couple tweaks * Remove signing from roslyn analyzers * Removing one more to-do * I doubt case sensitivity matters here, but juuuuust in case, we can try it. * Disable nuget auditing on CI builds, remove KS2 from feature flags * Can I turn it off at all? * ContinuousIntegrationBuild is set later in the file. Use the base field * Only report to TSA on official builds * Convert to json * Ok whatever * I love it when examples don't actually work * What in the heck * THIS IS SO DUMB * Claude save me * Escape sequence fix * Fixing the replacement to be good * WORDS WORDS WORDS * Standardizing the quotes
1 parent e65635b commit 4410dbf

15 files changed

+966
-2
lines changed

‎build.proj

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,39 @@
201201
<RemoveDir Directories='$([System.IO.Directory]::GetDirectories(".",".nuget", SearchOption.AllDirectories))' />
202202
</Target>
203203

204+
<!-- AKV Targets ========================================================= -->
205+
<Target Name="BuildAkv">
206+
<!-- @TODO: TestTargetOS for restore poisons the project.assets.json file... We should remove it. -->
207+
<!-- @TODO: TestTargetOS makes this far more complicated than it needs to be. We should remove it. -->
208+
<!-- @TODO: RemoveProperties shouldn't be necessary -->
209+
<Message Text=">>> Restoring AKV project" />
210+
<MSBuild Projects="@(AKVProvider)" Targets="Restore"/>
211+
212+
<PropertyGroup>
213+
<BuildAkvProperties>$(CI);TestTargetOS=$(TestOS)netfx;Platform=AnyCPU;$(ProjectProperties);$(NugetPackProperties)</BuildAkvProperties>
214+
</PropertyGroup>
215+
<Message Text=">>> Building AKV project for netfx [$(BuildAkvProperties)]" />
216+
<MSBuild Projects="@(AKVProvider)" Properties="$(BuildAkvProperties);" />
217+
218+
<PropertyGroup>
219+
<BuildAkvProperties>$(CI);TestTargetOS=$(TestOS)netcoreapp;$(ProjectProperties);Platform=AnyCPU;OSGroup=Unix;</BuildAkvProperties>
220+
</PropertyGroup>
221+
<Message Text=">>> Building AKV project for netcore/unix [$(BuildAkvProperties)]" />
222+
<MSBuild Projects="@(AKVProvider)" Properties="$(BuildAkvProperties)" RemoveProperties="TargetsWindows;TargetsUnix;" />
223+
224+
<PropertyGroup>
225+
<BuildAkvProperties>$(CI);TestTargetOS=$(TestOS)netcoreapp;$(ProjectProperties);Platform=AnyCPU;OSGroup=Windows_NT</BuildAkvProperties>
226+
</PropertyGroup>
227+
<Message Text=">>> Building AKV project for netcore/windows [$(BuildAkvProperties)]" />
228+
<MSBuild Projects="@(AKVProvider)" Properties="$(BuildAkvProperties);" RemoveProperties="TargetsWindows;TargetsUnix;" />
229+
230+
<PropertyGroup>
231+
<BuildAkvProperties>$(CI);TestTargetOS=$(TestOS)netcoreapp;$(ProjectProperties);Platform=AnyCPU;OSGroup=AnyOS;</BuildAkvProperties>
232+
</PropertyGroup>
233+
<Message Text=">>> Building AKV project for netcore/anyos [$(BuildAkvProperties)]" />
234+
<MSBuild Projects="@(AKVProvider)" Properties="$(BuildAkvProperties)" RemoveProperties="TargetsWindows;TargetsUnix;" />
235+
</Target>
236+
204237
<Target Name="BuildAKVNetFx" Condition="'$(IsEnabledWindows)' == 'true'">
205238
<MSBuild Projects="@(AKVProvider)" Targets="restore" Properties="TestTargetOS=$(TestOS)netfx" />
206239
<Message Text=">>> Building AKVNetFx [$(CI);TestTargetOS=$(TestOS)netfx;Platform=AnyCPU;$(TestProjectProperties)] ..." Condition="!$(ReferenceType.Contains('Package'))"/>

‎eng/pipelines/akv-official-pipeline.yml

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
#################################################################################
2+
# Licensed to the .NET Foundation under one or more agreements. #
3+
# The .NET Foundation licenses this file to you under the MIT license. #
4+
# See the LICENSE file in the project root for more information. #
5+
#################################################################################
6+
7+
name: $(Year:YY)$(DayOfYear)$(Rev:.r)
8+
9+
# @TODO: Add triggers and schedules
10+
11+
parameters:
12+
- name: oneBranchType
13+
displayName: 'OneBranch template'
14+
type: 'string'
15+
values:
16+
- 'Official'
17+
- 'NonOfficial'
18+
default: 'Official'
19+
20+
- name: buildConfiguration
21+
displayName: 'Build configuration'
22+
type: 'string'
23+
values:
24+
- 'Release'
25+
- 'Debug'
26+
default: 'Release'
27+
28+
- name: publishSymbols
29+
displayName: 'Publish symbols'
30+
type: 'boolean'
31+
default: false
32+
33+
- name: runSdlTasks
34+
displayName: 'Run SDL Tasks'
35+
type: 'boolean'
36+
default: true
37+
38+
variables:
39+
- template: /eng/pipelines/variables/common-variables.yml@self
40+
- template: /eng/pipelines/variables/onebranch-variables.yml@self
41+
- template: /eng/pipelines/variables/akv-official-variables.yml@self
42+
43+
resources:
44+
repositories:
45+
- repository: templates
46+
type: 'git'
47+
name: 'OneBranch.Pipelines/GovernedTemplates'
48+
ref: 'refs/heads/main'
49+
50+
extends:
51+
template: 'v2/OneBranch.${{ parameters.oneBranchType }}.CrossPlat.yml@templates'
52+
53+
parameters:
54+
featureFlags:
55+
WindowsHostVersion:
56+
Version: '2022'
57+
58+
globalSdl:
59+
# See https://aka.ms/obpipelines/sdl
60+
61+
apiscan:
62+
enabled: ${{ parameters.runSdlTasks }}
63+
softwareFolder: '${{ variables.apiScanDllPath }}'
64+
softwareName: 'Microsoft.Data.SqlClient' # Note: This name is registered with ApiScan
65+
softwareVersionNum: '${{ variables.assemblyFileVersion }}'
66+
symbolsFolder: '${{ variables.apiScanPdbPath }}'
67+
68+
armory:
69+
enabled: ${{ parameters.runSdlTasks }}
70+
break: true
71+
72+
asyncSdl:
73+
# If this should be enabled, move supported tools under this item,
74+
# see https://aka.ms/obpipelines/asyncsdl
75+
enabled: false
76+
77+
binskim:
78+
enabled: ${{ parameters.runSdlTasks }}
79+
break: true
80+
81+
codeinspector:
82+
enabled: ${{ parameters.runSdlTasks }}
83+
logLevel: Error
84+
85+
codeql:
86+
enabled: ${{ parameters.runSdlTasks }}
87+
sourceRoot: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient/add-ons/AzureKeyVaultProvider'
88+
# Note, this can only be done if project doesn't depend on other projects. In
89+
# package reference mode, this is true, but if we ever enable project reference
90+
# builds, this will have to be removed.
91+
92+
credscan:
93+
enabled: ${{ parameters.runSdlTasks }}
94+
suppressionsFile: '$(REPO_ROOT)/.config/CredScanSuppressions.json'
95+
96+
eslint:
97+
enabled: false
98+
99+
policheck:
100+
enabled: ${{ parameters.runSdlTasks }}
101+
break: true
102+
exclusionFile: '$(REPO_ROOT)/.config/PolicheckExclusions.xml'
103+
104+
roslyn:
105+
enabled: ${{ parameters.runSdlTasks }}
106+
break: true
107+
# Requires RoslynAnalyzers task to be added after build task
108+
109+
publishLogs:
110+
enabled: ${{ parameters.runSdlTasks }}
111+
112+
sbom:
113+
enabled: ${{ parameters.runSdlTasks }}
114+
packageName: 'Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider'
115+
packageVersion: ${{ variables.nugetPackageVersion }}
116+
117+
tsa:
118+
# OneBranch publishes all sdl results to TSA. If TSA is disabled all SDL tools will
119+
# be forced into 'break' build mode.
120+
enabled: ${{ eq(parameters.oneBranchType, 'Official') }}
121+
configFile: '$(REPO_ROOT)/.config/tsaoptions.json'
122+
123+
stages:
124+
- stage: BuildAkv
125+
displayName: 'Build AKV'
126+
jobs:
127+
- template: /eng/pipelines/jobs/build-akv-official-job.yml@self
128+
parameters:
129+
apiScanDllPath: '${{ variables.apiScanDllPath }}'
130+
apiScanPdbPath: '${{ variables.apiScanPdbPath }}'
131+
assemblyFileVersion: '${{ variables.assemblyFileVersion }}'
132+
buildConfiguration: '${{ parameters.buildConfiguration }}'
133+
nugetPackageVersion: '${{ variables.nugetPackageVersion }}'
134+
mdsPackageVersion: '${{ variables.mdsPackageVersion }}'
135+
publishSymbols: '${{ parameters.publishSymbols }}'
136+
signingAppRegistrationClientId: '$(SigningAppRegistrationClientId)'
137+
signingAppRegistrationTenantId: '$(SigningAppRegistrationTenantId)'
138+
signingAkvName: '$(SigningAkvName)'
139+
signingAuthCertName: '$(SigningAuthCertName)'
140+
signingConnectedServiceName: '$(SigningConnectedServiceName)'
141+
signingSignCertName: '$(SigningSignCertName)'
142+
symbolsAzureSubscription: '$(SymbolsAzureSubscription)'
143+
symbolsPublishProjectName: '$(SymbolsPublishProjectName)'
144+
symbolsPublishServer: '$(SymbolsPublishServer)'
145+
symbolsPublishTokenUri: '$(SymbolsPublishTokenUri)'
146+
symbolsUploadAccount: '$(SymbolsUploadAccount)'

‎eng/pipelines/jobs/build-akv-official-job.yml

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
#################################################################################
2+
# Licensed to the .NET Foundation under one or more agreements. #
3+
# The .NET Foundation licenses this file to you under the MIT license. #
4+
# See the LICENSE file in the project root for more information. #
5+
#################################################################################
6+
7+
parameters:
8+
- name: apiScanDllPath
9+
type: string
10+
11+
- name: apiScanPdbPath
12+
type: string
13+
14+
- name: assemblyFileVersion
15+
type: string
16+
17+
- name: buildConfiguration
18+
type: string
19+
20+
- name: nugetPackageVersion
21+
type: string
22+
23+
- name: mdsPackageVersion
24+
type: string
25+
26+
- name: publishSymbols
27+
type: boolean
28+
29+
- name: signingAppRegistrationClientId
30+
type: string
31+
32+
- name: signingAppRegistrationTenantId
33+
type: string
34+
35+
- name: signingAkvName
36+
type: string
37+
38+
- name: signingAuthCertName
39+
type: string
40+
41+
- name: signingConnectedServiceName
42+
type: string
43+
44+
- name: signingSignCertName
45+
type: string
46+
47+
- name: symbolsAzureSubscription
48+
type: string
49+
50+
- name: symbolsPublishProjectName
51+
type: string
52+
53+
- name: symbolsPublishServer
54+
type: string
55+
56+
- name: symbolsPublishTokenUri
57+
type: string
58+
59+
- name: symbolsUploadAccount
60+
type: string
61+
62+
# @TODO: This should be determined from build output, or at a higher level
63+
- # Note: not intended to be passed in, is only used for copying files for ApiScan.
64+
# This is only defined as a parameter since ADO pipelines do not support array variables.
65+
name: targetFrameworks
66+
type: object
67+
default:
68+
- net462
69+
- net8.0
70+
- net9.0
71+
72+
jobs:
73+
- job: buildSignedAkvPackage
74+
displayName: 'Build Signed AKV Package'
75+
pool:
76+
type: windows
77+
78+
variables:
79+
ob_outputDirectory: '$(ARTIFACT_PATH)'
80+
81+
steps:
82+
- template: ../steps/script-output-environment-variables-step.yml@self
83+
84+
- powershell: |
85+
$jsonParams = '${{ convertToJson(parameters) }}' -replace '\\', '\\'
86+
$jsonParams | ConvertFrom-Json | Format-List
87+
displayName: 'Output Job Parameters'
88+
89+
- template: ../steps/compound-build-akv-step.yml@self
90+
parameters:
91+
assemblyFileVersion: '${{ parameters.assemblyFileVersion }}'
92+
buildConfiguration: '${{ parameters.buildConfiguration }}'
93+
mdsPackageVersion: '${{ parameters.mdsPackageVersion }}'
94+
95+
- ${{ each targetFramework in parameters.targetFrameworks }}:
96+
- template: ../steps/compound-extract-akv-apiscan-files-step.yml
97+
parameters:
98+
buildConfiguration: '${{ parameters.buildConfiguration }}'
99+
dllPath: '${{ parameters.apiScanDllPath }}'
100+
pdbPath: '${{ parameters.apiScanPdbPath }}'
101+
referenceType: Package
102+
targetFramework: '${{ targetFramework }}'
103+
104+
- template: ../steps/roslyn-analyzers-akv-step.yml@self
105+
parameters:
106+
buildConfiguration: '${{ parameters.buildConfiguration }}'
107+
mdsPackageVersion: '${{ parameters.mdsPackageVersion }}'
108+
109+
- template: ../steps/compound-esrp-code-signing-step.yml@self
110+
parameters:
111+
akvName: '${{ parameters.signingAkvName }}'
112+
appRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}'
113+
appRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}'
114+
artifactType: 'dll'
115+
authCertName: '${{ parameters.signingAuthCertName }}'
116+
connectedServiceName: '${{ parameters.signingConnectedServiceName }}'
117+
signingCertName: '${{ parameters.signingSignCertName }}'
118+
119+
- template: ../steps/compound-nuget-pack-step.yml@self
120+
parameters:
121+
buildConfiguration: '${{ parameters.buildConfiguration }}'
122+
generateSymbolsPackage: true # Always generate symbols, even if they are not published
123+
packageVersion: '${{ parameters.nugetPackageVersion }}'
124+
nuspecPath: '$(REPO_ROOT)/tools/specs/add-ons/$(PACKAGE_NAME).nuspec'
125+
outputDirectory: '$(ARTIFACT_PATH)'
126+
referenceType: 'Package'
127+
128+
- template: ../steps/compound-esrp-code-signing-step.yml@self
129+
parameters:
130+
akvName: '${{ parameters.signingAkvName }}'
131+
appRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}'
132+
appRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}'
133+
artifactType: 'pkg'
134+
authCertName: '${{ parameters.signingAuthCertName }}'
135+
connectedServiceName: '${{ parameters.signingConnectedServiceName }}'
136+
signingCertName: '${{ parameters.signingSignCertName }}'
137+
138+
- ${{ if parameters.publishSymbols }}:
139+
- template: ../steps/compound-publish-symbols-step.yml@self
140+
parameters:
141+
artifactName: 'akv_symbols_$(System.TeamProject)_$(Build.Repository.Name)_$(Build.SourceBranchName)_${{ parameters.nugetPackageVersion }}_$(System.TimelineId)'
142+
azureSubscription: '${{ parameters.symbolsAzureSubscription }}'
143+
publishProjectName: '${{ parameters.symbolsPublishProjectName }}'
144+
packageName: '$(PACKAGE_NAME)'
145+
publishServer: '${{ parameters.symbolsPublishServer }}'
146+
publishToInternal: true
147+
publishToPublic: true
148+
publishTokenUri: '${{ parameters.symbolsPublishTokenUri }}'
149+
referenceType: 'Package'
150+
searchPattern: |
151+
Windows_NT/${{ parameters.buildConfiguration }}.AnyCPU/AzureKeyVaultProvider/**/$(PACKAGE_NAME).pdb
152+
AnyOS/${{ parameters.buildConfiguration }}.AnyCPU/AzureKeyVaultProvider/**/$(PACKAGE_NAME).pdb
153+
uploadAccount: '${{ parameters.symbolsUploadAccount }}'
154+
version: '${{ parameters.nugetPackageVersion }}'

‎eng/pipelines/steps/compound-build-akv-step.yml

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#################################################################################
2+
# Licensed to the .NET Foundation under one or more agreements. #
3+
# The .NET Foundation licenses this file to you under the MIT license. #
4+
# See the LICENSE file in the project root for more information. #
5+
#################################################################################
6+
7+
# @TODO: This can probably be made generic and pass in the command lines for msbuild
8+
# BUT, they should be kept separate by now as we rebuild build.proj in parallel, we won't
9+
# affect >1 project at a time.
10+
# @TODO: NugetPackageVersion should not be used for MDS package version
11+
12+
parameters:
13+
- name: assemblyFileVersion
14+
type: string
15+
16+
- name: buildConfiguration
17+
type: string
18+
19+
- name: mdsPackageVersion
20+
type: string
21+
22+
steps:
23+
- task: DownloadSecureFile@1
24+
displayName: 'Download Signing Key'
25+
inputs:
26+
retryCount: 5
27+
secureFile: 'netfxKeypair.snk'
28+
29+
- task: UseDotNet@2
30+
displayName: 'Install .NET 9.x SDK'
31+
inputs:
32+
packageType: 'sdk'
33+
version: '9.x'
34+
35+
- task: UseDotNet@2
36+
displayName: 'Install .NET 8.x Runtime'
37+
inputs:
38+
packageType: 'runtime'
39+
version: '8.x'
40+
41+
- task: MSBuild@1
42+
displayName: 'Build.proj - BuildAkv'
43+
inputs:
44+
solution: '$(REPO_ROOT)/build.proj'
45+
configuration: '${{ parameters.buildConfiguration }}'
46+
msbuildArguments: >-
47+
-t:BuildAkv
48+
-p:AssemblyFileVersion=${{ parameters.assemblyFileVersion }}
49+
-p:NugetPackageVersion=${{ parameters.mdsPackageVersion }}
50+
-p:ReferenceType=Package
51+
-p:SigningKeyPath=$(Agent.TempDirectory)/netfxKeypair.snk
52+
53+
- script: tree /a /f $(BUILD_OUTPUT)
54+
displayName: Output Build Output Tree

0 commit comments

Comments
 (0)