Skip to content

Update branch from main #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jun 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1988a90
worflow for package publish + auto version bump commit on release
ArturU043 Apr 18, 2025
805fa35
Moving workflow into correct subdir
ArturU043 Apr 18, 2025
c87f48c
Merge pull request #4 from ssl-hep/fille_peek_dev
ArturU043 Apr 23, 2025
595cb39
Fix if statement in pypi workflow
ArturU043 Apr 23, 2025
3b43ff6
fix job name pypi.yml
ArturU043 Apr 23, 2025
e2448ee
fix job dependencies syntax
ArturU043 Apr 23, 2025
2a4be4a
Update version to 1.0.0a4
github-actions[bot] Apr 23, 2025
d59102e
Remove if statement on publish
ArturU043 Apr 23, 2025
5f0ed28
black format __init__.py
ArturU043 Apr 23, 2025
e21c9bb
Update version to 1.0.0a5
github-actions[bot] Apr 23, 2025
2502493
change bot version bump for __init__.py to solve black format issue
ArturU043 Apr 23, 2025
9a8cd7d
fix auto version bump by bot
ArturU043 Apr 23, 2025
f5b300b
Print info for release name conflict in publish workflow
ArturU043 Apr 23, 2025
492fcee
Correct version bump re-writting for bot
ArturU043 Apr 23, 2025
e449049
Update version to 1.0.0.a8
github-actions[bot] Apr 23, 2025
a43e208
Update version to 1.0.1
github-actions[bot] Apr 23, 2025
e721f34
#6 Replace file_endswith to check file format in `to_awk`
ArturU043 Apr 24, 2025
1663529
Fix typo in elif
ArturU043 Apr 24, 2025
97bd57c
Update README.md
ArturU043 Apr 24, 2025
142e4b4
Update version to 1.0.2
github-actions[bot] Apr 24, 2025
ff826de
Adding modules for doc config and materialization
ArturU043 Apr 28, 2025
a538220
docs built + file introspection
ArturU043 May 2, 2025
e6c90e5
read the docs config + docs pip requirements
ArturU043 May 2, 2025
6bf056b
Merge pull request #11 from ssl-hep/docs-dev
ArturU043 May 2, 2025
5232031
Update README.md for doc url
ArturU043 May 2, 2025
c666e03
#8 Raise value errors for empty inputs in `to_awk`. Added related tests
ArturU043 May 5, 2025
fdf347c
Change notes on get-structure
ArturU043 May 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions .github/workflows/pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: Publish package to PyPi

on:
release:
types: [released, prereleased]

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install uv
uses: astral-sh/setup-uv@v5

- name: Install python-build and twine
run: |
uv pip install --system build twine
uv pip list --system

- name: Set env
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV

- name: Update Version
run: |
sed -i 's/^version\s*=\s*".*"/version = "${{ env.RELEASE_VERSION }}"/' pyproject.toml
sed -i '/__version__/ s/__version__\s*=.*/__version__ = "${{ env.RELEASE_VERSION }}"/' servicex_analysis_utils/__init__.py


- name: Build a sdist and wheel
run: |
python -m build .

- name: Check dist
run: twine check --strict dist/*

- name: List contents of sdist
run: python -m tarfile --list dist/servicex_analysis_utils-*.tar.gz

- name: List contents of wheel
run: python -m zipfile --list dist/servicex_analysis_utils-*.whl

- name: Upload an artifact of dist
uses: actions/upload-artifact@v4
with:
name: dist-gha-artifact
path: dist

publish:
needs: build
runs-on: ubuntu-latest

permissions:
id-token: write # required to push with trusted-pypi-token

steps:
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Download artifact
uses: actions/download-artifact@v4
with:
name: dist-gha-artifact
path: dist

- name: Publish distribution
uses: pypa/gh-action-pypi-publish@v1.12.3
with:
print-hash: true

commit-version-bump:
needs:
- build
- publish
runs-on: ubuntu-latest
permissions:
contents: write # required to push with GITHUB_TOKEN

steps:
- name: Checkout main branch
uses: actions/checkout@v4
with:
ref: main

- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Set env
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV

- name: Update version to commit
run: |
sed -i 's/^version\s*=\s*".*"/version = "${{ env.RELEASE_VERSION }}"/' pyproject.toml
sed -i '/__version__/ s/__version__\s*=.*/__version__ = "${{ env.RELEASE_VERSION }}"/' servicex_analysis_utils/__init__.py


- name: Commit version change
run: |
git config --local user.name "github-actions[bot]"
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git add ./pyproject.toml servicex_analysis_utils/__init__.py

# Check if there are changes to commit
if ! git diff --cached --quiet; then
echo "Changes detected, committing..."
git commit -m "Update version to ${{ env.RELEASE_VERSION }}" --no-verify
else
echo "No changes to commit"
fi

- name: Push commit to main
# GITHUB_TOKEN is automatically provided
run: |
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
git push origin main
33 changes: 33 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Read the Docs configuration file for Sphinx projects
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the OS, Python version and other tools you might need
build:
os: ubuntu-22.04
tools:
python: "3.12"

# Build documentation in the "docs/" directory with Sphinx
sphinx:
configuration: docs/conf.py
# # You can configure Sphinx to use a different builder, for instance use the dirhtml builder for simpler URLs
builder: html
# Fail on all warnings to avoid broken references
# fail_on_warning: true

# Optionally build your docs in additional formats such as PDF and ePub
formats:
- pdf

# Optional but recommended, declare the Python requirements required
# to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- method: pip
path: .
extra_requirements:
- docs
28 changes: 25 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# ServiceX analysis utils
This repository provides analysis tools to be used with the [ServiceX Client](https://github.com/ssl-hep/ServiceX_frontend/tree/master)
This repository hosts tools that depend on the [ServiceX Client](https://github.com/ssl-hep/ServiceX_frontend/tree/master). These tools facilitate `ServiceX`'s general usage and offer specific features that replace parts of an analyser workflow.

### To install
```
pip install servicex-analysis-utils
```
#### Requirements
This package depends requires a `servicex.yml` configuration file granting access to one endpoint with a deployed ServiceX backend.

##### This package contains the to_awk() function:
### This package contains:
#### `to_awk()`:
```
Load an awkward array from the deliver() output with uproot or uproot.dask.

Expand All @@ -22,5 +25,24 @@ Returns:
dict: keys are sample names and values are awkward arrays, uproot generator objects or dask-awkward arrays.
```


#### `get-structure()`:

```
Creates and sends the ServiceX request from user inputed datasets to retrieve file stucture.
Calls print_structure_from_str() to dump the structure in a user-friendly format

Parameters:
datasets (dict,str,[str]): The datasets from which to print the file structures.
A name can be given as the key of each dataset in a dictionary
kwargs : Arguments to be propagated to print_structure_from_str, e.g filter_branch
```

Function can be called from the command line, e.g:

```
$ servicex-get-structure "mc23_13TeV:some-dataset-rucio-id" --filter_branch "truth"
```

## Documentation
The different functions are documented in TBD
The different functions are documented in [readthedcos](https://servicex-analysis-utils.readthedocs.io/en/latest/file_introspecting.html)
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
Binary file added docs/_build/doctrees/environment.pickle
Binary file not shown.
Binary file added docs/_build/doctrees/file_introspecting.doctree
Binary file not shown.
Binary file added docs/_build/doctrees/index.doctree
Binary file not shown.
Binary file added docs/_build/doctrees/installation.doctree
Binary file not shown.
Binary file added docs/_build/doctrees/materialization.doctree
Binary file not shown.
4 changes: 4 additions & 0 deletions docs/_build/html/.buildinfo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: ca3626de684797951ef03443c45ac66f
tags: 645f666f9bcd5a90fca523b33c5a78b7
Loading