[FR] don't insert header blocks

[pank]

Many thanks for providing this extension! It provides a great solution for using git-friendly .py files, while still allowing colleagues to edit the files via the vs code notebook interface.

Maybe I'm missing something (not familiar with vs code), but I can't figure out how to tell the extension not to insert headers. Maybe it's hardcoded?
I don't want the headers in the .py files as they are duplicating the global settings (specifying that files use the percent format).
The header block isn't inserted when I use the jupytext directly only when invoking it via the extension.

Would it be possible to make the inserting of the header configurable? (Assuming I have not missed something obvious, in which case I apologize)

[caenrigen]

Hi @pank

I am glad you found the extension useful!

I don't want the headers in the .py files

I understand that this can be desired. I personally never use it like that.

Currently I am not sure if my extension can support it. Could elaborate on your exact setup? i.e.:

• what does your jupytext config looks like
• where is the config located
• what exact jupytext ... command do you invoke manually?

And regarding your desired feature:

• what is the ideal workflow you would like to achieve using this extension?
• what jupytext ... commands would you expect it to invoke to achieve your desired outcome?
• based on what condition (e.g. save an .ipynb file that is opened in the notebook editor) would these jupytext ... commands be executed?

Note that one main goal of this extension is to keep paired files in sync, from your input, so far, I'm not sure if your goal is to have a manual "press a button to run jupytext".

---

Maybe I'm missing something (not familiar with vs code), but I can't figure out how to tell the extension not to insert headers. Maybe it's hardcoded?

The extension itself does not handle the headers directly, most things are handled by jupytext. The extension is mostly a wrapper with the primary goal of keeping paired files in sync. However, the way the extension invokes jupytext might not support your use case currently:

• When you request the extension to pair some file/notebook via Jupytext, it first executes jupytext --set-formats ....
• Afterwards, on every save/open/close file (according to the extension settings) the extension uses Jupytext to read the paired formats from the file (from the metadata in the headers) to check if for that file the extension should invoke a jupytext --sync ... command .

Note that when the headers are missing in a file it becomes tricky to decide if this extension should execute anything or not. I had to fix a bug recently to avoid interfering when a file has no headers: #2

---

I am open to Pull Requests to better support Jupytext workflows

[pank]

Hi,
Sorry for the slow reply. I wanted to explore the jupytext cli a bit better before replying.

My usage of jupytext

• In a subfolder, we have files that generate a bunch of figures. The people who update these, tend to be new programmers and like to use notebooks. But notebooks don't give nice diffs and can't be easily used from standard python programs (e.g. imported as modules).
• The idea is therefore to use your extension to allow people to edit .py files as notebooks in vs code, but still have plaintext in the git repo.

Desired approach

• People can use the VS Code notebook interface to edit the .py via your extension.
• No headeres / metadata should be inserted to keep the files "clean"
• The py:percent format is set globally for the project (see below) and is non-negotiable
  • (caveat: people have to have this is mind if editing the .py files rather than the notebook files but oh well).

Config

I store the jupytext config in a pyproject.toml file in the (git) project root:

$ grep -A 2 tool.jupytext pyproject.toml
[tool.jupytext]
formats = "ipynb,py:percent"

(I'm honestly not sure this is necessary).

"Manual" test of jupytext

I wanted to see if I can use jupytext to create the desired approach above.
Note, I rarely use notebooks and jupytext so I could easily be missing something. If that's the case, I apologize for my ignorance!

Create notebook from py:percent file:

$ cat test.py
# %% [markdown]
# ### welcome to the `test` module
# this file is used to test `jupytext` syncing
#
# %%
d = {"a": [0, 1, 2], "b": ["a", "b", "c"]}
d

$ jupytext --sync test.py
[jupytext] Reading test.py in format py
[jupytext] Updating test.ipynb
[jupytext] Updating the timestamp of test.py

$ cat test.py
# %% [markdown]
# ### welcome to the `test` module
# this file is used to test `jupytext` syncing
#
# %%
d = {"a": [0, 1, 2], "b": ["a", "b", "c"]}
d

No header has been inserted.

Edit in VS code

# The file was edited via the VS Code notebook interface
$ jupytext --sync test.py
[jupytext] Reading test.py in format py
[jupytext] Loading test.ipynb
[jupytext] Unchanged test.ipynb  # note: don't understand why it says unchanged here
[jupytext] Updating test.py
#  test if it makes a difference using as arg test.ipynb
$ jupytext --sync test.ipynb
[jupytext] Reading test.ipynb in format ipynb
[jupytext] Loading test.py
[jupytext] Unchanged test.ipynb
[jupytext] Unchanged test.py
$ cat test.py
# %% [markdown]
# ### welcome to the `test` module
# this file is used to test `jupytext` syncing.
# Edit from *VS Code*.
# %%
d = {"a": [0, 1, 2], "b": ["a", "b", "c"]}
d

Still no header

Where I think the header comes from

I think the header with metadata is inserted because jupytext individual pairing is always assumed (and I don't think that's a necessary assumption).

If I set the format on the individual file, I get the header:

$ jupytext --set-formats ipynb,py:percent test.py
[jupytext] Reading test.py in format py
[jupytext] Updating notebook metadata with '{"jupytext": {"formats": "ipynb,py:percent"}}'
[jupytext] Loading test.ipynb
[jupytext] Updating test.ipynb
[jupytext] Updating test.py

$ cat test.py
# ---
# jupyter:
#   jupytext:
#     cell_metadata_filter: -all
#     formats: ipynb,py:percent
#     text_representation:
#       extension: .py
#       format_name: percent
#       format_version: '1.3'
#       jupytext_version: 1.17.1
# ---

# %% [markdown]
# ### welcome to the `test` module
# this file is used to test `jupytext` syncing.
# Edit from *VS Code*.
# %%
d = {"a": [0, 1, 2], "b": ["a", "b", "c"]}
d

Simple solution

I'm not familiar with typescript(?), but I guess headers are inserted when runJupytextSetFormats is called, in particular via pair and then setFormats when pairing from VS Code.

So I would want to make the execution of that line optional and rely on the global jupytext configs instead.

Would this be OK with you?
Would you like me to provide a patch?
If so, is it correctly inferred that VS code settings are stored in package.json? (not really a vs code user :)

[caenrigen]

Thanks for the detailed input

(caveat: people have to have this is mind if editing the .py files rather than the notebook files but oh well).

One of the main points of my extension is to keep the .py and .ipynb in sync every time either of the paired files are saved.
Keeping the headers in the .py files would solve this issue of people forgetting to sync them.

---

[jupytext] Unchanged test.ipynb # note: don't understand why it says unchanged here

that is expected since you edited the notebook (the .ipynb file), and on the line above jupytext loaded it and then it updated the .py file that was out of sync [jupytext] Updating test.py

---

I think the header with metadata is inserted because jupytext individual pairing is always assumed (and I don't think that's a necessary assumption).

You are right that header comes from the --set-formats command being issued. I was not fully aware of the details of how jupytext handled the "no headers" mode.

(and I don't think that's a necessary assumption)

Here is the main question that must be figured out in order to support your use case: how does my extension know if a given .py file the user opened in VS Code is:

1. [use case of other people] simply some python module that should never be touched (i.e. the percent format # %% should never be applied); OR
2. [your use case] a .py file that matches your use cases and should be synced via jupytext --sync

Note that currently jupytext --sync is executed every time a .py/.ipynb file is saved, BUT only if the respective file contains in its metadata header the jupytext formats. This was implemented in this extension by design such that jupytext --sync does run automatically when it is not supposed to (e.g. "plain/clean" .py python modules).

The question can be broadened to simply ask: how to handle both of these 2 use cases correctly?

To support your use case, the most simple and straightforward way I can think of, without interfering with the current workflow, is to add an extra Command/Icon that allows to manually invoke jupytext --sync on the currently opened file(.py)/notebook(.ipynb), which would run jupytext --sync regardless of anything else, a "Sync via Jupytext" command. In VS Code is always possible to assign a keyboard shortcut to such a command.

Let me know if that would work well enough for you

---

I am wondering if jupytext within JupyterLab handles these 2 use cases or if it forces all .py files to obey by the settings in the project.toml. Might be worth for you check on that, perhaps there is functionality in jupytext that we could borrow. I am trying to delegate to jupytext as much as possible within my VS Code extension.

[pank]

One of the main points of my extension is to keep the .py and .ipynb in sync every time either of the paired files are saved.
Keeping the headers in the .py files would solve this issue of people forgetting to sync them.

Sorry, I wasn't clear. I was referring to "manually" inserting cell delimiters, # %%, when editing .py files, otherwise they look wrong when opened as notebooks.

that is expected since you edited the notebook (the .ipynb file), and on the line above jupytext loaded it and then it updated the .py file that was out of sync [jupytext] Updating test.py

Thanks!

1. [use case of other people] simply some python module that should never be touched (i.e. the percent format # %% should never be applied); OR

Aside: I ran it from the terminal to understand how jupytext itself worked; to have simple examples.
In real world use, people would always go via your extension in my use case, when wanting to use notebooks. So it would always be run on specific files, not general files. But maybe I'm misunderstanding.

Note that currently jupytext --sync is executed every time a .py/.ipynb file is saved, BUT only if the respective file contains in its metadata header the jupytext formats.

But why rely on that rather than e.g. matching filenames in .jupytext-sync-ipynb and the "main" folder. Isn't that how jupytext does it? Both approaches sound flaky, but to me, matching filenames sounds less fraught.
(Aside: the ipynb file contains a metadata block with a jupytext section that might be useful for identifying files).

To support your use case, the most simple and straightforward way I can think of, without interfering with the current workflow, is to add an extra Command/Icon that allows to manually invoke jupytext --sync on the currently opened file(.py)/notebook(.ipynb), which would run jupytext --sync regardless of anything else, a "Sync via Jupytext" command. In VS Code is always possible to assign a keyboard shortcut to such a command.

I don't think this would work. I think new programmers (whom I want to use it with) would forget to click tbh...

Aside: I have found out that jupytext filters can be used to strip the headers, e.g. jupytext --opt notebook_metadata_filter="-all" test.py. Unfortunately, they do not work with --set-format, but I could probably use it as a git pre-commit filter. I would still rather that the extension wouldn't rely on the header.

[pank]

Actually, if I set notebook_metadata_filter and cell_metadata_filter to -all in pyproject.toml or jupytext.toml I don't get the headers when using VC Code extension.

$ grep -A 3 tool.jupytext pyproject.toml
[tool.jupytext]
formats = "ipynb,py:percent"
notebook_metadata_filter="-all"
cell_metadata_filter="-all"

Having opened test.py as a notebook via right click -> open as paired notebook..., syncing from the .ipynb file back to the .py file works as expected, but syncing from the .py file to the notebook does not work, presumably because it only does so if the header is there. Edit: I.e.

• test 0: make change in synced ipynb file and save: it shows up in the .py file.
• test 1. make change in the .py file and save: the changes aren't synced to the ipynb file, but I would expect that they were.

[caenrigen]

You seem to have figured it out by now hehe, but here is my reply to your previous message

---

In real world use, people would always go via your extension in my use case, when wanting to use notebooks. So it would always be run on specific files, not general files. But maybe I'm misunderstanding.

Currently the extension executes too independent steps triggered by independent events:

1. When you click the "Pair via Jupytext"/"Open as notebook via Jupytext", it runs jupytext --set-formats. This is essential for novice users and likely the most common use case of people not having any jupytext configuration files setup.
2. When you save ANY file triggered by, e.g., pressing on the keyboard ctrl+s (or cmd+s on macOS), it decides if the jupytext --sync should be run and then run it if the criteria is met. The decision is made based on the file extension + calling (from typescript) python code from the jupytext python modules which extracts if there are any paired formats from the jupytext metadata within the contents of the file, aka "was this file paired via Jupytext before? if yes then we should sync it now because the user saved the file". If the code does not do this, then this issue happens.

Your jupytext settings are applied whenever jupytext --sync is executed, but my extension has the extra responsibility of deciding if the jupytext --sync should be executed when a file is saved. And this issue proves that my extension cannot blindly run the jupytext --sync command whenever you save ANY .ipynb/.py files. Please read that issue to understand some implication of having a "global" jupytext config file and running jupytext --sync on every .py/.ipynb file.

---

But why rely on that rather than e.g. matching filenames in .jupytext-sync-ipynb and the "main" folder.

.jupytext-sync-ipynb dir is an optional default, I personally don't use it because it has the inconvenience of the current working directory not matching the .py file.

Isn't that how jupytext does it?

idk, I am trying to avoid spaghetti code in my extension and delegate as much as possible to Jupytext.

Both approaches sound flaky, but to me, matching filenames sounds less fraught.

Having 2 files with the same name does not mean these files are necessarily intended to be jupytext-synced. That by itself is a weak criteria. It might be the case 100% of the time in your project directory but certainly not guaranteed in other projects and other people's machines.

Aside: I have found out that jupytext filters can be used to strip the headers, e.g. jupytext --opt notebook_metadata_filter="-all" test.py. Unfortunately, they do not work with --set-format

Can you check if these setting are picked up if you put them in your jupytext config file instead? and then run jupytext --set-format? It could solve your problem well enough if you keep the jupytext metadata inside the notebook file only and jupytext --set-format does not add into the .py file. And later the .py file would be kept in sync if my extension can still understand that the notebook has been paired with a .py file.

(Aside: the ipynb file contains a metadata block with a jupytext section that might be useful for identifying files).

But how does that block end up in there in the first place?

---

As a side note I came across this https://code.visualstudio.com/updates/v1_83#_finalized-notebook-code-action-api setting in VS Code that you might be able to adapt such that you run jupytext --sync when you save a notebook.

[caenrigen]

presumably because it only does so if the header is there.

correct

[caenrigen]

test 1. make change in the .py file and save: the changes aren't synced to the ipynb file, but I would expect that they were.

The only way I see to make this sync possible without the metadata being in the .py file is to add a configuration option to my extension that allows jupytext --sync to run always ignoring the missing metadata headers (for supported file extensions ofc). With the default behaviour being the current implementation, so opt-in from the user being required. The nice thing is that this can be configured per workspace (or even workspace dir).

Implementing this should not be too hard, I don't have bandwidth to implement it, but PR is welcome.
My extension already has many examples of VS Code settings (defined in package.json).

[pank]

1. When you click the "Pair via Jupytext"/"Open as notebook via Jupytext", it runs jupytext --set-formats. This is essential for novice users and likely the most common use case of people not having any jupytext configuration files setup.
2. When you save ANY file triggered by, e.g., pressing on the keyboard ctrl+s (or cmd+s on macOS), it decides if the jupytext --sync should be run and then run it if the criteria is met. The decision is made based on the file extension + calling (from typescript) python code from the jupytext python modules which extracts if there are any paired formats from the jupytext metadata within the contents of the file, aka "was this file paired via Jupytext before? if yes then we should sync it now because the user saved the file". If the code does not do this, then Modifies .py files with no jupytext metadata #2 happens.

I think the question that is to be answered is "does a paired notebook exists?" but you are asking "does the file have a header?". It happens that under the default config, there's a high probability (maybe even 1) of the answers coinciding.

Having 2 files with the same name does not mean these files are necessarily intended to be jupytext-synced. That by itself is a weak criteria.

I agree in principal, but not in the case of jupytext. Relying on "paired paths", like jupytext itself, is more reliable than the above imho. You can temporarily detach the link between $path0/$name.py and $path1/$name.ipynb with jupytext -o argument, but it's not stored in the metadata in either representation (example below).

Unfortunately, jupytext --paired-paths $file isn't reliable cf. jupytext #928 (and my own experimentation) [it works OK-ish from python].

The only way I see to make this sync possible without the metadata being in the .py file is to add a configuration option to my extension that allows jupytext --sync to run always ignoring the missing metadata headers (for supported file extensions ofc).

I honestly think that paired path is the most reliable way. I.e. if the paired path of $name.py exists, then sync. That is, imho, the closest you can get to answering "does a paired notebook exists". Then you'd be relying on jupytext checks for whether $name.ipynb is a valid counterpart [Unfortunately, there does not seem to be at least a cli interface for making this check without issuing e.g. --sync].

Anyway, I will think a bit more about this and see how easy it is to get this done. I might look at the vim implementation and see how they solve it.

Thanks a lot for all the clarifications and the discussion!

Example illustrating how "hard-coded" $name equivalence is

$ ls 
# export to difference name
$ jupytext --set-format  "ipynb,py:percent" test.py -o foo.ipynb
[jupytext] Reading test.py in format py
[jupytext] Updating notebook metadata with '{"jupytext": {"formats": "ipynb,py:percent"}}'
[jupytext] Loading test.ipynb
[jupytext] Writing foo.ipynb

$ cat test.py  # note, no reference to foo
# ---
# jupyter:
#   jupytext:
#     cell_metadata_filter: -all
#     formats: ipynb,py:percent
#     text_representation:
#       extension: .py
#       format_name: percent
#       format_version: '1.3'
#       jupytext_version: 1.17.1
# ---

# %%
d = {"a": [1, 2, 3]}

$ ls
foo.ipynb  test.py

$ jupytext --sync foo.ipynb  # forgotten about test.py
[jupytext] Reading foo.ipynb in format ipynb
[jupytext] Unchanged foo.ipynb
[jupytext] Updating foo.py

$ jupytext --sync test.py  # forgotten about foo.ipynb
[jupytext] Reading test.py in format py
[jupytext] Updating test.ipynb
[jupytext] Updating the timestamp of test.py

$ ls
foo.ipynb  test.ipynb  test.py

$ jupytext --paired-paths test.py
test.ipynb

[caenrigen]

I think the question that is to be answered is "does a paired notebook exists?" but you are asking "does the file have a header?".

Both of these questions are opinionated implementation details. The true question my extension has to answer is "should jupytext --sync run when the user saves the current document/notebook?"

Anyway, I will think a bit more about this and see how easy it is to get this done. I might look at the vim implementation and see how they solve it.

I would appreciate if you could verify what is the behaviour in JupyterLab (latest JupyterLab and latest jupytext) when you have a jupytext config file in the working dir and you save a plain .py file. Is the .py file forced to use the percent format even though you did not explicitly pair it nor open it as notebook?

---

In principle it should be possible to cover all cases, but I have limited bandwidth to deal with all the pesky edge cases that make the code base more complex. It would likely require to implement proper tests, etc..

---

Unfortunately, jupytext --paired-paths $file isn't reliable cf. mwouts/jupytext#928 (and my own experimentation) [it works OK-ish from python].

jupytext --paired-paths seems at least partially broken from my own tests.

how come does it work from python? does it actually work or not always?

---

You can temporarily detach the link between $path0/$name.py and $path1/$name.ipynb with jupytext -o argument, but it's not stored in the metadata in either representation (example below).

I did not understand much of what you meant here. Supporting sync between files with distant base names (e.g. name1.py and name2.ipynb) seems out of scope of this thread. jupytext -o seems to be a convenience argument for quick exports or something similar.

---

I actually found out that the bug in issue #2 is still happening... I will have to look at it some other time, not sure it might be an issue with the latest jupytext v1.17.2 or it simply slipped through due to lack of testing different configuration setups.

But this means that your use case is currently kind of supported, though unintentionally because I would say it should NOT happen automatically to convert a plain .py file into a py:percent when you save a .py file in VS Code that has never been explicitly paired before. But that is something up to discussion in #2.

That said, I think the main problem that prevented you from getting it working, then way you wanted it, is that you need to use the same formats in both your jupytext's config file, e.g.:

# pyproject.toml
[tool.jupytext]
formats = ".ipynb-jupytext-sync//ipynb,py:percent"
notebook_metadata_filter = "-all"
cell_metadata_filter = "-all"

AND in the Jupytext Sync extension configuration (inside VS Code) such that when my extension runs jupytext --set-formats (in case your users run the "Pair via Jupytext" or "Open as paired notebook via Jupytext" commands). Though with that juptyext config, pairing seems unnecessary because juptyext seems to report (via python code that my extension runs) that any .py file in the project dir has the formats specified in the config. That kind of makes sense...

Mind these Jupytext Sync setting in VS Code:

With this setup the headers were not inserted in the .py file from my testing.

That said, if you want to preserve this flow, you might want to fix the version of jupytext and of my extension to an explicit version. There are several subtleties and unintended behaviours involved that might break in future versions.

PS in case you did not spot it already it can be useful to keep track of the logs my extension's outputs (as show in the screenshot above), it prints what jupytext command output.

[caenrigen]

Quick update, I realised I was not using the latest version of the extension in my IDE when I tested the above. So if you want to achieve the behaviour I described, use Jupytext Sync 1.1.3 (instead of the current 1.1.4).

[td-anne]

1. When you click the "Pair via Jupytext"/"Open as notebook via Jupytext", it runs jupytext --set-formats. This is essential for novice users and likely the most common use case of people not having any jupytext configuration files setup.
2. When you save ANY file triggered by, e.g., pressing on the keyboard ctrl+s (or cmd+s on macOS), it decides if the jupytext --sync should be run and then run it if the criteria is met. The decision is made based on the file extension + calling (from typescript) python code from the jupytext python modules which extracts if there are any paired formats from the jupytext metadata within the contents of the file, aka "was this file paired via Jupytext before? if yes then we should sync it now because the user saved the file". If the code does not do this, then Modifies .py files with no jupytext metadata #2 happens.

I think the question that is to be answered is "does a paired notebook exists?" but you are asking "does the file have a header?". It happens that under the default config, there's a high probability (maybe even 1) of the answers coinciding.

In my setting, when someone checks the project out from git, there are no .ipynb files at all. There are .py files. Some of these .py files should produce a .ipynb file when jupytext --sync is run, but others should not (command-line scripts, importable modules). I distinguish between these cases by having the jupytext header in the files that should be accompanied by a .ipynb file. Any file that has no jupytext header should never be touched by jupytext; this is the default behaviour of the command-line jupytext program.

How do you propose to distinguish between .py files that should be accompanied by .ipynb files (and jupytext --sync should generate these notebooks) and .py files that should not be accompanied by .ipynb files without a header in the .py file?

I will point out that a .py file that is intended to be used as a notebook should have # %% section markers, while files not intended to be notebooks frequently don't (but might). So notebook-friendly .py files already have some special strings in them; why is the header particularly objectionable?

[pank]

In out setup, it's at the programmer's discretion when to use a notebook,
within a certain folder (e.g. $root/figures). So we distinguish via folders.
helper functions are typically organized in packages, that are imported, so we don't have e.g. a helper_functions.py in that folder.

Only .pys are committed.

The pct markers # %% are meaningful, even if the program is not consumed through a notebook so I don't mind them.
(Aside: Jupytext supports a number markers for blocks, but I don't know if it makes sense to use anything but the percent format in vsc).

The headers are ugly and unnecessary and could lead to noisy git commits (e.g. some colleagues will commit any change regardless, as they simply click git sync in vsc :/; if I'm lucky the commit messages are meaningful).

AFAIK (but I could easily be wrong), jupytext --sync does not look for headers. It is operated as jupytext --sync $file. If $file is missing it raises a ValueError. If $file and $target look too different it raises am exception and stops (I don't remember which exception). But I admit, my knowledge on this is sketchy.

So my view, and repeating myself, the most jupytexty approach to differentiating between a vanilla py and a "text notebook" is too look for the existence of $file's $target (like paired_paths) and then rely on jupytext's checks for likeness.

But Victor has different views and I respect that, especially as I have offered no code to implement my view :)

[td-anne]

In my case, when I'm not relying on VSCode, I simply run jupytext --sync analysis/*.py. This goes through all the .py files, and for the ones that have headers, ensures that the corresponding .ipynb exists (creating it if necessary), and is in sync (copying content from the newer to the older). The .py files that do not have headers are left untouched.

Your proposed scheme of looking for the existence of the .ipynb file will not work in my situation, since the .ipynb files do not exist in git and therefore do not exist in new clones of the project. (The .ipynb file is in the same directory and has the same base name as the .py file.)

As I understand it, we have several rules under discussion for whether jupytext --sync should synchronise the contents of files:

1. Synchronise .py/.ipynb/whatever files if and only if they contain jupytext metadata (the header, in .py files, which also contains information about the formats to use).
2. Synchronise all .py files (drawing any configuration/details of the synchronisation from global config files), possibly restricted to those sitting in certain directories (presumably also configured in the same global config files).
3. Synchronise .py files if and only if the .ipynb already exists (drawing details from the above configuration files).

The first of these is what jupytext --sync *.py already does, with no configuration. It works well with version control systems that do not keep the .ipynb (a primary goal of jupytext). It makes it easy to customise the formats a given file is kept in by setting options in the file itself (using jupytext --set-formats); this includes keeping files that do not require other formats in the same directory as files that do. It does result in some machine-generated comments in the .py files; these are invisible when interacting with the .ipynb file. This setup also works automatically with JupyterLab.

I don't have experience with either of the other two rules, to know how they are supported. In the scheme you describe, where the .ipynb is never committed, if someone checks out a clean clone of the repository, how does jupytext know to create the paired .ipynb files that do not exist yet? Are you relying on the existence of a parent directory (because the .ipynb files are not created in the same directory as the .py files)? Or do they need to be explicitly listed in some configuration file?

As I see it, the goal of this VSCode extension is essentially to avoid needing to remember to run jupytext --sync before doing version control operations, by automatically running it when relevant files are saved to disk. It can also replicate jupytext --set-formats (the pairing commands).

What the first of these operations needs to know is whether or not to run jupytext --sync when you save changes to a file. Would it be effective to simply run a sync any time a file of an appropriate type was saved? With my (no) configuration, jupytext --sync file.py does nothing if the header does not exist in that file, and synchronises the file if the header does exist - exactly what I want to happen. With your configuration, is it safe to run jupytext --sync on all files when they are edited?

The second, pairing, operation, would need to know how you kept track of which files were paired; but I don't see as much need for the pairing operation, since it's relatively rare compared to editing files.