Skip to content

feat: expose label to create scores API #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

desaxce
Copy link
Contributor

@desaxce desaxce commented Oct 10, 2024

Changes:

  • expose label to create_scores API

To test:

  • create an experiment from Python:
experiment = dataset.create_experiment(
    name="Next 1",
    params=[{ "type": "Mon expérience", "top_k": 2 }]
)

for index in range(2):
    scores = [{ 
        "name": "Sentiment",
        "type": "AI",
        "label": f"happy-{index}", # HERE IS THE LINE TO ADD COMPARED TO CLASSIC EXPERIMENT
        "value": 55
    }]

    experiment_item = {
        "datasetItemId": # Put the ID of a dataset item
        "scores": scores,
        "input": { "question": "kesako?" },
        "output": { "contexts": ["rien"] }
    }
    
    experiment.log(experiment_item)

If you have a platform version greater or equal to 0.0.627-beta, you should see the label instead of the score value in the experiment details:
image

Copy link
Contributor

@clementsirieix clementsirieix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI label is not defined in ScoreDict but it should.
Also it does not seem to be part of any signatures so it's hard to tell if this is intended.

@desaxce
Copy link
Contributor Author

desaxce commented Oct 11, 2024

FYI label is not defined in ScoreDict but it should. Also it does not seem to be part of any signatures so it's hard to tell if this is intended.

Added. label is not very discoverable indeed. DatasetExperiment.log will now have it, but it's hidden behind DatasetExperimentItemDict.scores. We need to find a way that logs an experiment item in an easier way: having to pass scores in the experiment item dict is cumbersome.

Copy link
Contributor

@clementsirieix clementsirieix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@desaxce desaxce merged commit ba22ec3 into main Oct 11, 2024
2 checks passed
@desaxce desaxce deleted the hugues/eng-1922-show-value-instead-of-index-in-experiments-page branch October 11, 2024 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants