Documentation updates #15

normandy7 · 2024-08-15T16:01:53Z

Add README
Add/modify docstrings
Edit messages
Point docs URLs to docs-beta
Bring in changes from Some changes to the API: param names, more specialized functions #29, fix: Release package name #28 and Various fixes after playing around with some scenarios #25

README.md

kgodlewski · 2024-08-29T08:26:37Z

README.md

+| `creation_time`  | `datetime`, optional | `None` | Custom creation time of the run. |
+| `from_run_id`    | `str`, optional  | `None` | If forking off an existing run, ID of the run to fork from. |
+| `from_step`      | `int`, optional  | `None` | If forking off an existing run, step number to fork from. |
+| `max_queue_size` | `int`, optional  | `None` | Maximum number of operations allowed in the queue. |


README.md

Co-authored-by: Krzysztof Godlewski <krzysiek@dajerade.pl>

README.md

src/neptune_scale/__init__.py

Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>

README.md

Raalsky · 2024-09-02T06:37:29Z

README.md

+)
+
+run.log(
+    metrics={"Metric1": metric1_value, "Metric2": metric2_value},


How about actual values instead? It doesn't look to be very useful right now.

Raalsky · 2024-09-02T06:37:51Z

README.md

+from neptune_scale import Run
+
+run = Run(
+    family="RunFamilyName",


Shouldn't we mention project and API token? (via env variables for instance)

Raalsky · 2024-09-02T06:42:22Z

README.md

+| `mode`           | `Literal`, `"async"` or `"disabled"` | `"async"` | Mode of operation. If set to `"disabled"`, the run doesn't log any metadata. |
+| `as_experiment`  | `str`, optional  | `None` | Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. |
+| `creation_time`  | `datetime`, optional | `None` | Custom creation time of the run. |
+| `from_run_id`    | `str`, optional  | `None` | If forking off an existing run, ID of the run to fork from. |


Maybe we should mention that both from_step and from_run_id are mandatory if user would like to use forking.

Raalsky · 2024-09-02T06:43:54Z

README.md

+
+| Name          | Type                                               | Default | Description                                                               |
+|---------------|----------------------------------------------------|---------|---------------------------------------------------------------------------|
+| `step`        | `Union[float, int]`, optional                      | `None`  | Index of the log entry. Must be increasing. If not specified, the `log()` call increments the step starting from the highest already logged value. **Tip:** Using float rather than int values can be useful, for example, when logging substeps in a batch. |


⚠️ Will not be incremented; And we should mention that this cannot be lower than from_step Run was forked from another.

Then how does this work when omitting the step? Or are we missing that this is connected to metrics (FloatSeries) only?

README.md

Raalsky · 2024-09-02T07:08:16Z

README.md

+| `timestamp`   | `datetime`, optional                               | `None`  | Time of logging the metadata. |
+| `fields`      | `Dict[str, Union[float, bool, int, str, datetime, list, set]]`, optional  | `None` | Dictionary of configs or other values to log. Independent of the step value. Available types: float, integer, Boolean, string, and datetime. To log multiple values at once, pass multiple dictionaries. |
+| `metrics`     | `Dict[str, float]`, optional                       | `None`  | Dictionary of metrics to log. Each metric value is associated with a step. To log multiple metrics at once, pass multiple dictionaries. |
+| `add_tags`    | `Dict[str, Union[List[str], Set[str]]]`, optional  | `None`  | Dictionary of tags to add to the run, as a list of strings. Independent of the step value. |


Independent of the step value.

I'm not sure about it in terms of forking, let's do not mention such phrase for now.

Raalsky · 2024-09-02T07:08:56Z

README.md

+...
+```
+
+**Note:** Calling `log()` without specifying the step still increments the index. To correlate logged values, make sure to send all metadata related to a step in a single `log()` call, or specify the step explicitly.


@normandy7 It's fine to call log() with a specific step value more than once:

This:

run.log(step=1, metrics={"loss": 0.08}) run.log(step=1, metrics={"acc": 0.86})

is equivalent to:

run.log(step=1, metrics={"loss": 0.08, "acc": 0.86})

Yes, but I'm not sure it helps clarify which part of the note is false. Maybe the problem is that I didn't include a mention of metrics?

Isn't it true that if the step is omitted when calling log_metrics(), the highest index found among logged FloatSeries fields is used as the reference step?

README.md

kgodlewski · 2024-09-02T10:40:41Z

README.md

+    run.wait_for_processing()
+    run.log(fields={"scores/some_score": some_score_value})  # called once submitted data has been processed
+```
+


Error handling

In case an unrecoverable error is encountered, you can terminate the failed run in your error callback.
Note that this will effectively disable processing in-flight operations, as well as logging new data. However,
the training process won't be interrupted.

def my_error_callback(exc): run.terminate() run = Run(..., on_error_callback=my_error_callback)

README.md

Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>

SiddhantSadangi · 2024-09-03T14:33:08Z

README.md

+> [!NOTE]
+> This package only works with the `3.0` version of neptune.ai called Neptune Scale, which is in beta.
+>
+> It's supported on Linux and MacOS.


Update to this.
It can be used on Windows if the run is initialized inside the if __name__ == "__main__": guard.

Read more here: https://docs.python.org/3/library/multiprocessing.html#multiprocessing-programming > Safe importing of main module

cc: @Raalsky

It's even not specific to Windows, it's common for all of the operating systems and a common pitfall of Python itself.

I was able to use it without the guard in Linux 🤔

Weird, on Mac it doesn't work without the guard.

I conclude that we don't need any OS-specific notes for now.

To cover our bases, I'd prefer if we add a debugging/Help section of sorts and inform users to initialize the run inside the if __name__ == "__main__": guard if they get the below error:

RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

SiddhantSadangi · 2024-09-03T15:12:55Z

README.md

+)
+
+run.log(
+    metrics={"Metric1": metric1_value, "Metric2": metric2_value},


Suggested change

metrics={"Metric1": metric1_value, "Metric2": metric2_value},

metrics={"acc": 0.98, "loss": 0.2},

SiddhantSadangi · 2024-09-03T15:16:32Z

README.md

+
+run.log(
+    metrics={"Metric1": metric1_value, "Metric2": metric2_value},
+    fields={"Field1": field1_value}


Suggested change

fields={"Field1": field1_value}

fields={"params/lr": 0.01, "params/optimizer": "adam"}

SiddhantSadangi · 2024-09-03T15:17:53Z

README.md

+| `api_token`      | `str`, optional  | `None`  | Your Neptune API token or a service account's API token. If `None`, the value of the `NEPTUNE_API_TOKEN` environment variable is used. To keep your token secure, don't place it in source code. Instead, save it as an environment variable. |
+| `resume`         | `bool`, optional | `False` | If `False` (default), creates a new run. To continue an existing run, set to `True` and pass the ID of an existing run to the `run_id` argument. To fork a run, use `from_run_id` and `from_step` instead. |
+| `mode`           | `Literal`, `"async"` or `"disabled"` | `"async"` | Mode of operation. If set to `"disabled"`, the run doesn't log any metadata. |
+| `as_experiment`  | `str`, optional  | `None` | Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. |


Suggested change

| `as_experiment` | `str`, optional | `None` | Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. |

| `as_experiment` | `str`, optional | `None` | Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. Max length: 730 bytes |

SiddhantSadangi · 2024-09-03T15:18:59Z

README.md

+
+| Name             | Type             | Default | Description                                                               |
+|------------------|------------------|---------|---------------------------------------------------------------------------|
+| `family`         | `str`            | -       | Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 characters. |


Suggested change

| `family` | `str` | - | Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 characters. |

| `family` | `str` | - | Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 bytes. |

SiddhantSadangi · 2024-09-03T15:21:45Z

README.md

+| Name             | Type             | Default | Description                                                               |
+|------------------|------------------|---------|---------------------------------------------------------------------------|
+| `family`         | `str`            | -       | Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 characters. |
+| `run_id`         | `str`            | -       | Identifier of the run. Must be unique within the project. Max length: 128 characters. |


Suggested change

| `run_id` | `str` | - | Identifier of the run. Must be unique within the project. Max length: 128 characters. |

| `run_id` | `str` | - | Identifier of the run. Must be unique within the project. Max length: 128 bytes. |

normandy7 added 2 commits August 15, 2024 15:56

list methods in class docstring

890a8c8

tweak docstring for Run.__init__

4a5d79e

Base automatically changed from dev/minimal-flow to main August 21, 2024 11:45

normandy7 added 7 commits August 28, 2024 08:59

temp precommit workaround

339c088

merge main

f356e8e

tweak log() docstring

5fe6fb7

revert precommit workaround

583b6e4

create README

a6a484e

fix run_id default in API reference

c757b70

fix from_stepd default in API reference

7fc2269

normandy7 marked this pull request as ready for review August 28, 2024 11:56

normandy7 requested review from Raalsky, dzwiedziu, szaganek and kgodlewski August 28, 2024 11:57

kgodlewski reviewed Aug 29, 2024

View reviewed changes

normandy7 and others added 3 commits August 29, 2024 09:14

Apply suggestions from code review

6cca3d5

Co-authored-by: Krzysztof Godlewski <krzysiek@dajerade.pl>

add link to experiments documentation

34fbe71

address review comments

1ba43c8

szaganek reviewed Aug 29, 2024

View reviewed changes

normandy7 and others added 2 commits August 29, 2024 12:46

Apply suggestions from code review

7e16fe0

Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>

Update README.md

bcaefcf

Raalsky reviewed Sep 2, 2024

View reviewed changes

SiddhantSadangi reviewed Sep 2, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

kgodlewski reviewed Sep 2, 2024

View reviewed changes

normandy7 commented Sep 3, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

SiddhantSadangi requested changes Sep 3, 2024

View reviewed changes

README.md Show resolved Hide resolved

normandy7 commented Sep 3, 2024

View reviewed changes

README.md Show resolved Hide resolved

normandy7 commented Sep 3, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

Apply suggestions from code review

180e962

Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>

SiddhantSadangi requested changes Sep 3, 2024

View reviewed changes

Raalsky mentioned this pull request Sep 3, 2024

Create proper README #32

Merged

normandy7 closed this Sep 4, 2024

normandy7 deleted the sabine/docs branch May 26, 2025 13:40

	\| `max_queue_size` \| `int`, optional \| `None` \| Maximum number of operations allowed in the queue. \|
	\| `max_queue_size` \| `int`, optional \| `None` \| Maximum number of operations queued for processing. 1M by default. You should raise this value if you see the `on_queue_full_callback` being called. \|

	metrics={"Metric1": metric1_value, "Metric2": metric2_value},
	metrics={"acc": 0.98, "loss": 0.2},

	fields={"Field1": field1_value}
	fields={"params/lr": 0.01, "params/optimizer": "adam"}

	\| `as_experiment` \| `str`, optional \| `None` \| Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. \|
	\| `as_experiment` \| `str`, optional \| `None` \| Name of the experiment to associate the run with. Learn more about [experiments](https://docs-beta.neptune.ai/experiments) in the Neptune documentation. Max length: 730 bytes \|

	\| `family` \| `str` \| - \| Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 characters. \|
	\| `family` \| `str` \| - \| Identifies related runs. All runs of the same lineage must have the same `family` value. That is, forking is only possible within the same family. Max length: 128 bytes. \|

	\| `run_id` \| `str` \| - \| Identifier of the run. Must be unique within the project. Max length: 128 characters. \|
	\| `run_id` \| `str` \| - \| Identifier of the run. Must be unique within the project. Max length: 128 bytes. \|

Documentation updates #15

Documentation updates #15

Uh oh!

Conversation

normandy7 commented Aug 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Error handling

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

normandy7 commented Aug 15, 2024 •

edited

Loading