Skip to content

Conversation

@MattShirley
Copy link
Collaborator

@MattShirley MattShirley commented Sep 23, 2025

Backend work for supporting custom TopCP images from whitelisted Docker registry repositories: ssl-hep/ServiceX_frontend#658

Allowed images can be defined using the codeGen.topcp.allowedImages value, where the value should be a list of strings. Each string in the list will be tested to see if it is a valid prefix of the custom image. If it is, it will be allowed. The default value will allow any tags for the sslhep/servicex_science_image_topcp repository.

codeGen:
  topcp:
    allowedImages:
      - "sslhep/servicex_science_image_topcp:"

Sample files that work with a custom TopCP image but not with the default/base image:

# reco.yaml
CommonServices:
  runSystematics: False

PileupReweighting: {}

EventCleaning: {}

Trigger:
    triggerChainsPerYear:
        '2015':
            - 'HLT_2mu6_bBmumuxv2'
            - 'HLT_2mu4_bBmumuxv2'
        '2016':
            - 'HLT_2mu6_bBmumux_BsmumuPhi_delayed'
            - 'HLT_2mu6_bBmumux_BsmumuPhi_delayed_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_2mu6_bBmumux_BsmumuPhi_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_mu10_mu6_bBmumux_BsmumuPhi_delayed'
            - 'HLT_2mu6_bBmumuxv2'
            - 'HLT_2mu6_bBmumuxv2_delayed'
        '2017':
            - 'HLT_mu6_mu4_bBmumux_BsmumuPhi_L1BPH-2M9-MU6MU4_BPH-0DR15-MU6MU4'
            - 'HLT_2mu6_bBmumux_BsmumuPhi_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_2mu6_bJpsimumu_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_2mu10_bBmumuxv2'
        '2018':
            - 'HLT_mu6_mu4_bBmumux_BsmumuPhi_L1BPH-2M9-MU6MU4_BPH-0DR15-MU6MU4'
            - 'HLT_2mu6_bBmumux_BsmumuPhi_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_2mu6_bJpsimumu_L1BPH-2M9-2MU6_BPH-2DR15-2MU6'
            - 'HLT_2mu10_bBmumuxv2'
    noFilter: True
    noGlobalTriggerEff: True

Muons:
  - containerName: AnaMuons
    WorkingPoint:
      - selectionName: loose
        quality: Loose
        isolation: NonIso
      - selectionName: medium
        quality: Medium
        isolation: NonIso

BPhyVertex:
  - vertexName: 'BPHY28BsKKMuMuCandidates'
    decorations: []


BPHY28:
  - vertexName: 'BPHY28BsKKMuMuCandidates'
    inputMuons: 'AnaMuons'
    decorateTrackParameters: True
    decoratePixelHits: True
    decorateDedx: True

# After configuring each container, many variables will be saved automatically.
Output:
  treeName: 'reco'
  vars: []
  metVars: []
  containers:
      # Format should follow: '<suffix>:<output container>'
#      vtx_: 'BPHY28BsKKMuMuCandidates'
      mu_: 'AnaMuons'
      '': 'EventInfo'
  commands: []

AddConfigBlocks:
  - modulePath: 'TopCPToolkit.BPHY28Config'
    functionName: 'BPHY28Config'
    algName: 'BPHY28'
    pos: 'Output'
  - modulePath: 'TopCPToolkit.BPhyVertexConfig'
    functionName: 'BPhyVertexConfig'
    algName: 'BPhyVertex'
    pos: 'Output'
# spec.yaml
General:
  OutFilesetName: "fileset_ntuple_v1_af"
  OutputDirectory: "."

Sample:
  ## data18
  - Name: data18_periodD
    Dataset: !Rucio data18_13TeV:DAOD_BPHY28.44180527._000001.pool.root.1
    Query: !TopCP |
      reco=reco.yaml,
      docker_registry=gitlab.cern.ch,
      docker_image=sslhep/servicex_science_image_topcp:25.2.50-v2.18.0_v0.2

@MattShirley MattShirley linked an issue Sep 23, 2025 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Oct 28, 2025

Codecov Report

❌ Patch coverage is 94.44444% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.88%. Comparing base (e68e6a4) to head (91343d2).

Files with missing lines Patch % Lines
...t/servicex/TopCP_code_generator/query_translate.py 66.66% 0 Missing and 1 partial ⚠️
...pp/servicex_app/resources/transformation/submit.py 96.77% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1163      +/-   ##
===========================================
+ Coverage    85.83%   85.88%   +0.04%     
===========================================
  Files           94       94              
  Lines         3191     3223      +32     
  Branches       359      368       +9     
===========================================
+ Hits          2739     2768      +29     
- Misses         378      380       +2     
- Partials        74       75       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MattShirley MattShirley marked this pull request as ready for review October 31, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support custom TopCP images

2 participants