Skip to content

Commit 92e2188

Browse files
committed
Convert to using paths for everything - kill off file_path.
1 parent 7f29550 commit 92e2188

File tree

3 files changed

+8
-8
lines changed

3 files changed

+8
-8
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Create an async callback method that `yield`s file info dictionaries. For exampl
1818
async def my_callback(did_name: str, info: Dict[str, Any]):
1919
for i in range(0, 10):
2020
yield {
21-
'file_path': [f"root://atlas-experiment.cern.ch/dataset1/file{i}.root"],
21+
'paths': [f"root://atlas-experiment.cern.ch/dataset1/file{i}.root"],
2222
'adler32': b183712731,
2323
'file_size': 0,
2424
'file_events': 0,
@@ -33,7 +33,7 @@ The arguments to the method are straight forward:
3333

3434
Yield the results as you find them - ServiceX will actually start processing the files before your DID lookup is finished if you do this. The fields you need to pass back to the library are as follows:
3535

36-
* `file_path`: An ordered list of URIs that a transformer in ServiceX can access to get at the file. Often these are either `root://` or `http://` schema URI's. When accessing the file, URIs will be tried in ordered listed.
36+
* `paths`: An ordered list of URIs that a transformer in ServiceX can access to get at the file. Often these are either `root://` or `http://` schema URI's. When accessing the file, URIs will be tried in ordered listed.
3737
* `adler32`: A CRC number for the file. This CRC is calculated in a special way by rucio and is not used. Leave as 0 if you do not know it.
3838
* `file_size`: Number of bytes of the file. Used to calculate statistics. Leave as zero if you do not know it (or it is expensive to look up).
3939
* `file_events`: Number of events in the file. Used to calculate statistics. Leave as zero if you do not know it (or it is expensive to look up).
@@ -118,7 +118,7 @@ In the end, all DID finders for ServiceX will run under Kubernetes. ServiceX com
118118

119119
for i in range(0, 10):
120120
yield {
121-
'file_path': [f"root://atlas-experiment.cern.ch/dataset1/file{i}.root"]
121+
'paths': [f"root://atlas-experiment.cern.ch/dataset1/file{i}.root"]
122122
'adler32': b183712731,
123123
'file_size': 0,
124124
'file_events': 0,

src/servicex_did_finder_lib/servicex_adaptor.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ def put_file_add(self, file_info):
7272
try:
7373
mesg = {
7474
"timestamp": datetime.now().isoformat(),
75-
"paths": [self._prefix_file(fp) for fp in file_info['file_path']],
75+
"paths": [self._prefix_file(fp) for fp in file_info['paths']],
7676
'adler32': file_info['adler32'],
7777
'file_size': file_info['file_size'],
7878
'file_events': file_info['file_events']

tests/servicex_did_finder_lib/test_communication.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ async def my_callback(did_name: str, info: Dict[str, Any]):
102102
nonlocal seen_name
103103
seen_name = did_name
104104
yield {
105-
'file_path': ["fork/it/over"],
105+
'paths': ["fork/it/over"],
106106
'adler32': 'no clue',
107107
'file_size': 22323,
108108
'file_events': 0,
@@ -172,7 +172,7 @@ async def my_callback(did_name: str, info: Dict[str, Any]) \
172172
nonlocal called
173173
called = True
174174
yield {
175-
'file_path': ["fork/it/over"],
175+
'paths': ["fork/it/over"],
176176
'adler32': 'no clue',
177177
'file_size': 22323,
178178
'file_events': 0,
@@ -226,13 +226,13 @@ async def test_run_file_fetch_loop(SXAdaptor, mocker):
226226
async def my_user_callback(did, info):
227227
return_values = [
228228
{
229-
'file_path': ['/tmp/foo'],
229+
'paths': ['/tmp/foo'],
230230
'adler32': '13e4f',
231231
'file_size': 1024,
232232
'file_events': 128
233233
},
234234
{
235-
'file_path': ['/tmp/bar'],
235+
'paths': ['/tmp/bar'],
236236
'adler32': 'f33d',
237237
'file_size': 2046,
238238
'file_events': 64

0 commit comments

Comments
 (0)