- 
                Notifications
    You must be signed in to change notification settings 
- Fork 1.8k
feat: custard integration #5205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Draft
      
      
            davidcavazos
  wants to merge
  17
  commits into
  main
  
    
      
        
          
  
    
      Choose a base branch
      
     
    
      
        
      
      
        
          
          
        
        
          
            
              
              
              
  
           
        
        
          
            
              
              
           
        
       
     
  
        
          
            
          
            
          
        
       
    
      
from
custard
  
      
      
   
  
    
  
  
  
 
  
      
    base: main
Could not load branches
            
              
  
    Branch not found: {{ refName }}
  
            
                
      Loading
              
            Could not load tags
            
            
              Nothing to show
            
              
  
            
                
      Loading
              
            Are you sure you want to change the base?
            Some commits from the old base branch may be removed from the timeline,
            and old review comments may become outdated.
          
          
  
     Draft
                    Changes from 1 commit
      Commits
    
    
            Show all changes
          
          
            17 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      6764545
              
                custard integration
              
              
                 0520593
              
                add debugging info
              
              
                 fe159ab
              
                fix build command
              
              
                 d314392
              
                install custard directly
              
              
                 72c56a5
              
                remove version tag
              
              
                 bf4f624
              
                pass specific path
              
              
                 6c09ffb
              
                go get first
              
              
                 7c652a8
              
                build custard on separate directory
              
              
                 b0b807a
              
                build custard, clean, then get diffs
              
              
                 10f734b
              
                test without -e
              
              
                 70b6234
              
                fix affected
              
              
                 6279cee
              
                pass the config file as input
              
              
                 e14bdda
              
                better step names
              
              
                 890c8c1
              
                do not deserialize on reusable output
              
              
                 29ff4cb
              
                fix more serialization stuff
              
              
                 46283f3
              
                more serialization fixes
              
              
                 fd8abcb
              
                decrease serialization payloads
              
              
                 File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /* | ||
| Copyright 2025 Google LLC | ||
|  | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|  | ||
| https://www.apache.org/licenses/LICENSE-2.0 | ||
|  | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|  | ||
| { | ||
| "package-file": [ "go.mod" ], | ||
| "ci-setup-filename": "ci-setup.json", | ||
|  | ||
| // If these change, please update the .github/config/README.md too! | ||
| "ci-setup-defaults": { | ||
| "env": { }, | ||
| "secrets": { }, | ||
| "go-version-earliest": 1.23, | ||
| "go-version-latest": 1.24, | ||
| "timeout-minutes": 10 | ||
| }, | ||
|  | ||
| "ignore": [ | ||
| ".github/blunderbuss.yaml", | ||
| ".github/CODEOWNERS", | ||
| // TODO: do not ignore .github/config once everything is in prod | ||
| ".github/config/", // prevent changes to exclusions from running all tests | ||
| ".github/flakybot.yaml", | ||
| ".github/header-checker-lint.yaml", | ||
| ".github/ISSUE_TEMPLATE/", | ||
| ".github/PULL_REQUEST_TEMPLATE.md", | ||
| ".github/renovate.json", | ||
| ".github/scripts/", | ||
| ".github/snippet-bot.yml", | ||
| ".gitignore", | ||
| "badfiles_test.go", | ||
| "cloud-samples-tools", // checked out by GH action in ci-*.yml | ||
| "CODE_OF_CONDUCT.md", | ||
| "CONTRIBUTING.md", | ||
| "LICENSE", | ||
| "Makefile", | ||
| "README.md", | ||
| "regiontag_test.go", | ||
| "SECURITY.md", | ||
| "Taskfile.yaml" | ||
| ], | ||
|  | ||
| "exclude-packages": [ | ||
| ] | ||
| } | ||
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # Copyright 2024 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|  | ||
| name: Custard setup | ||
| description: Sets up the testing environment. | ||
|  | ||
| inputs: | ||
| path: | ||
| description: Path of the package to run. | ||
| required: true | ||
| affected: | ||
| description: The job for affected packages. | ||
| required: true | ||
| project_id: | ||
| description: The Google Cloud project ID. | ||
| required: true | ||
| workload_identity_provider: | ||
| description: The Google Cloud workload identity provider. | ||
| required: true | ||
| service_account: | ||
| description: The Google Cloud service account to use for credentials. | ||
| required: true | ||
|  | ||
| outputs: | ||
| ci-setup: | ||
| description: The CI setup configuration for the given path. | ||
| value: ${{ inputs.affected.outputs.ci-setups[inputs.path] }} | ||
|  | ||
| runs: | ||
| using: composite | ||
| steps: | ||
| - uses: google-github-actions/auth@6fc4af4b145ae7821d527454aa9bd537d1f2dc5f # v2 | ||
| id: auth | ||
| with: | ||
| project_id: ${{ inputs.project_id }} | ||
| workload_identity_provider: ${{ inputs.workload_identity_provider }} | ||
| service_account: ${{ inputs.service_account }} | ||
| access_token_lifetime: 600s # 10 minutes | ||
| token_format: id_token | ||
| id_token_audience: https://action.test/ # service must have this custom audience | ||
| id_token_include_email: true | ||
| - name: Export environment variables | ||
| uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 | ||
| id: vars | ||
| with: | ||
| script: | | ||
| const { default: setupVars } = await import('${{ github.workspace }}/.github/scripts/setup-vars.js'); | ||
| return await setupVars({ | ||
| core, | ||
| projectId: '${{ inputs.project_id }}', | ||
| setup: ${{ toJson(inputs.affected.outputs.ci-setups[inputs.path]) }}, | ||
| serviceAccount: '${{ inputs.service_account }}', | ||
| idToken: '${{ steps.auth.outputs.id_token }}', | ||
| }) | ||
| - uses: google-github-actions/get-secretmanager-secrets@e5bb06c2ca53b244f978d33348d18317a7f263ce # v2 | ||
| if: ${{ fromJson(steps.vars.outputs.result).secrets }} | ||
| with: | ||
| secrets: ${{ fromJson(steps.vars.outputs.result).secrets }} | ||
| export_to_environment: true | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| /* | ||
| Copyright 2025 Google LLC | ||
|  | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|  | ||
| https://www.apache.org/licenses/LICENSE-2.0 | ||
|  | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|  | ||
| import fs from "node:fs"; | ||
| import path from "node:path"; | ||
| import setupVars from "../setup-vars.js"; | ||
|  | ||
| const project_id = process.env.PROJECT_ID; | ||
| if (!project_id) { | ||
| console.error( | ||
| "Please set the PROJECT_ID environment variable to your Google Cloud project." | ||
| ); | ||
| process.exit(1); | ||
| } | ||
|  | ||
| const core = { | ||
| exportVariable: (_key, _value) => null, | ||
| }; | ||
|  | ||
| const setupFile = process.argv[2]; | ||
| if (!setupFile) { | ||
| console.error("Please provide the path to a setup file."); | ||
| process.exit(1); | ||
| } | ||
| const data = fs.readFileSync(path.join("..", "..", setupFile), "utf8"); | ||
| const setup = JSON.parse(data); | ||
|  | ||
| setupVars({ project_id, core, setup }); | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "name": "custard", | ||
| "version": "1.0.0", | ||
| "type": "module", | ||
| "license": "Apache-2.0", | ||
| "private": true, | ||
| "scripts": { | ||
| "vars": "node cmd/vars.js", | ||
| "test": "mocha -p -j 2 **/*.test.js" | ||
| }, | ||
| "devDependencies": { | ||
| "mocha": "^11.1.0" | ||
| } | ||
| } | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| /* | ||
| Copyright 2025 Google LLC | ||
|  | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|  | ||
| https://www.apache.org/licenses/LICENSE-2.0 | ||
|  | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|  | ||
| export default function setupVars( | ||
| { projectId, core, setup, serviceAccount, idToken }, | ||
| runId = null | ||
| ) { | ||
| // Define automatic variables plus custom variables. | ||
| const vars = { | ||
| PROJECT_ID: projectId, | ||
| RUN_ID: runId || uniqueId(), | ||
| SERVICE_ACCOUNT: serviceAccount, | ||
| ...(setup.env || {}), | ||
| }; | ||
|  | ||
| // Apply variable interpolation. | ||
| const env = Object.fromEntries( | ||
| Object.keys(vars).map((key) => [key, substituteVars(vars[key], vars)]) | ||
| ); | ||
|  | ||
| // Export environment variables. | ||
| console.log("env:"); | ||
| for (const key in env) { | ||
| const value = env[key]; | ||
| console.log(` ${key}: ${value}`); | ||
| core.exportVariable(key, value); | ||
| } | ||
|  | ||
| // Show exported secrets, for logging purposes. | ||
| // TODO: We might want to fetch the secrets here and export them directly. | ||
| // https://cloud.google.com/secret-manager/docs/create-secret-quickstart#secretmanager-quickstart-nodejs | ||
| console.log("secrets:"); | ||
| for (const key in setup.secrets || {}) { | ||
| // This is the Google Cloud Secret Manager secret ID. | ||
| // NOT the secret value, so it's ok to show. | ||
| console.log(` ${key}: ${setup.secrets[key]}`); | ||
| } | ||
|  | ||
| // Set global secret for the Service Account identity token | ||
| // Use in place of 'gcloud auth print-identity-token' or auth.getIdTokenClient | ||
| // usage: curl -H 'Bearer: $ID_TOKEN' https:// | ||
| core.exportVariable("ID_TOKEN", idToken); | ||
| core.setSecret(idToken); | ||
| // For logging, show the source of the ID_TOKEN | ||
| console.log(` ID_TOKEN: steps.auth.outputs.id_token (from GitHub Action)`); | ||
|  | ||
| // Return env and secrets to use for further steps. | ||
| return { | ||
| env: env, | ||
| // Transform secrets into the format needed for the GHA secret manager step. | ||
| secrets: Object.keys(setup.secrets || {}) | ||
| .map((key) => `${key}:${setup.secrets[key]}`) | ||
| .join("\n"), | ||
| }; | ||
| } | ||
|  | ||
| export function substituteVars(value, env) { | ||
| for (const key in env) { | ||
| let re = new RegExp(`\\$(${key}\\b|\\{\\s*${key}\\s*\\})`, "g"); | ||
| value = value.replaceAll(re, env[key]); | ||
| } | ||
| return value; | ||
| } | ||
|  | ||
| export function uniqueId(length = 6) { | ||
| const min = 2 ** 32; | ||
| const max = 2 ** 64; | ||
| return Math.floor(Math.random() * max + min) | ||
| .toString(36) | ||
| .slice(0, length); | ||
| } | 
      
      Oops, something went wrong.
        
    
  
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Uh oh!
There was an error while loading. Please reload this page.