Update to use only summary bars for uploads when in notebooks #3243
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Hey @hoytak , thanks for working on that. I left a few comments as I think we should still display per-file progress in google colabs (I tried it before and it works correctly)
(x-posting internal convo)
| the summary version is shown in notebooks and guis and the detailed file progress is shown in consoles. | ||
| """ | ||
|
|
||
| def __init__(self, n_lines: int = 10, description_width: int = 30, per_file_progress=None): |
There was a problem hiding this comment.
| def __init__(self, n_lines: int = 10, description_width: int = 30, per_file_progress=None): | |
| def __init__(self, n_lines: int = 10, description_width: int = 30): |
Let's remove the argument here since it's not used anywhere. Given that XetProgressReporter is used only internally, we don't need flexibility
| If per_file_progress is True, then per-file progress is shown in a scrolling list. Otherwise, | ||
| only the summary bars showing file processing progress and data upload are shown. By default, | ||
| the summary version is shown in notebooks and guis and the detailed file progress is shown in consoles. |
There was a problem hiding this comment.
| If per_file_progress is True, then per-file progress is shown in a scrolling list. Otherwise, | |
| only the summary bars showing file processing progress and data upload are shown. By default, | |
| the summary version is shown in notebooks and guis and the detailed file progress is shown in consoles. | |
| Shows summary progress bars when running in notebooks or GUIs, and detailed per-file progress in console environments. |
Rephrase for when per_file_progress is not an option anymore
| @classmethod | ||
| def in_console(cls) -> bool: | ||
| """Returns true if running in a standard console environment and false if running in a notebook or gui.""" | ||
|
|
||
| # Returns true if the current display method is the one in the standard tqdm class, or false if it's been | ||
| # overwritten by the gui, notebook, keras, etc. subclassing it. | ||
| return cls.display is std_tqdm.display | ||
|
|
There was a problem hiding this comment.
I'd remove this method and use a combination of is_notebook and is_google_colab.
- if is_google_colab => per-file progress (as colab is quite good with this)
- else if is_notebook => summary only
- else => per-file progress
(you can import them like this from huggingface_hub.utils import is_google_colab)
| if per_file_progress is None: | ||
| self.per_file_progress = tqdm.in_console() | ||
| else: | ||
| self.per_file_progress = per_file_progress |
There was a problem hiding this comment.
| if per_file_progress is None: | |
| self.per_file_progress = tqdm.in_console() | |
| else: | |
| self.per_file_progress = per_file_progress | |
| self.per_file_progress = is_google_colab() or not is_notebook() |
(given the above)
Currently, the progress of uploading files to a Xet enabled repo does both per-file and summary reporting of the progress. However, the per-file progress may not be communicated to gui or notebook environments in a useful way as different bars are repurposed for different files to display a scrolling list of in-progress files with completed files removed. This PR disables the scrolling per-file reporting for gui and notebook environments, showing only the two summary bars.