Juju scp: no such file or directory

[sinclert-canonical]

This issue describe a problem with the Jubilant Juju.scp function (wrapping juju's scp command), where it cannot locate local files.

How to reproduce

import os
import pathlib
import tempfile

import jubilant
from jubilant import Juju
from jubilant.statustypes import UnitStatus

MYSQL_APP_NAME = "mysql"


def _get_juju_keys() -> tuple[str, str]:
    """Get Juju public and private keys."""
    config_dir = pathlib.Path("~/.local/share/juju")
    if juju_data := os.getenv("JUJU_DATA"):
        config_dir = pathlib.Path(juju_data)

    return (
        str(config_dir.expanduser().resolve() / "ssh" / "juju_id_rsa.pub"),
        str(config_dir.expanduser().resolve() / "ssh" / "juju_id_rsa"),
    )


def get_app_units(juju: Juju, app_name: str) -> dict[str, UnitStatus]:
    """Get the units for the given application."""
    model_status = juju.status()
    app_status = model_status.apps[app_name]
    return app_status.units


def scp_unit_file(juju: Juju, source: str, target: str) -> None:
    """Copy a file from source to target."""
    juju_keys = _get_juju_keys()
    juju.scp(source, target, host_key_checks=False, scp_options=["-B", "-i", juju_keys[1]])


if __name__ == "__main__":
    juju = Juju(model="testing")
    
    juju.deploy(
        MYSQL_APP_NAME,
        app=MYSQL_APP_NAME,
        base="ubuntu@22.04",
        config={"profile": "testing"},
        num_units=1,
    )
    juju.wait(
        ready=lambda status: all((
            jubilant.all_agents_idle(status),
            jubilant.all_active(status, MYSQL_APP_NAME),
        )),
        timeout=20 * 60,
    )
    
    for unit_name in get_app_units(juju, MYSQL_APP_NAME):
        with tempfile.NamedTemporaryFile(mode="w") as temp_file:
            temp_file.write("data")
            temp_file.flush()
            scp_unit_file(
                juju=juju,
                source=f"{temp_file.name}",
                target=f"{unit_name}:/tmp/file",
            )

Traceback:

juju scp --model testing --no-host-key-checks -- -B -i /root/.local/share/juju/ssh/juju_id_rsa /tmp/tmp12d6m2pr mysql/0:/tmp/file
ERROR exit status 255 (scp: stat local "/tmp/tmp12d6m2pr": No such file or directory)

Work-around

- def scp_unit_file(juju: Juju, source: str, target: str) -> None:
-     """Copy a file from source to target."""
-     juju_keys = _get_juju_keys()
-     juju.scp(source, target, host_key_checks=False, scp_options=["-B", "-i", juju_keys[1]])

+ import subprocess
+
+ def get_unit_ip(juju: Juju, app_name: str, unit_name: str) -> str:
+     """Get the application unit IP."""
+     model_status = juju.status()
+     app_status = model_status.apps[app_name]
+     for name, status in app_status.units.items():
+         if name == unit_name:
+             return status.public_address
+
+     raise Exception("No application unit found")
+
+
+ def scp_file_to_unit(juju: Juju, app_name: str, unit_name: str, source_path: str) -> None:
+     """Copy a file from a given unit."""
+     juju_keys = _get_juju_keys()
+     unit_address = get_unit_ip(juju, app_name, unit_name)
+     unit_username = "ubuntu"
+
+     subprocess.check_output([
+         "scp",
+         "-B",
+         "-o",
+         "StrictHostKeyChecking=no",
+         "-i",
+         juju_keys[1],
+         f"{source_path}",
+         f"{unit_username}@{unit_address}:/tmp/file",
+     ])

Versions

• Jubilant: 1.4.0 (latest).
• Python: 3.10
• Juju: 3.6.9 (latest)
• LXD: 5.21.4 (latest)
• Ubuntu: 24.04 (latest LTS).

[benhoyt]

I wonder if this is happening because you've installed juju from the Snap, so it can only access Snap-related directories. We have the same issue in Jubilant when using NamedTemporaryFile, so we specify a directory, like this:

with tempfile.NamedTemporaryFile('w+', dir=self._temp_dir) as file:
    ...

Where _temp_dir is a private method, defined as follows:

    @functools.cached_property
    def _juju_is_snap(self) -> bool:
        which = shutil.which(self.cli_binary)
        return which is not None and '/snap/' in which

    @functools.cached_property
    def _temp_dir(self) -> str:
        if self._juju_is_snap:
            # If Juju is running as a snap, we can't use /tmp, so put temp files here instead.
            temp_dir = os.path.expanduser('~/snap/juju/common')
            os.makedirs(temp_dir, exist_ok=True)
            return temp_dir
        else:
            return tempfile.gettempdir()

I wouldn't recommend using Juju._temp_dir directly in real code as it's private and we make no guarantees it won't change, but you could try using the same or similar approach and see if that works for you.

[sinclert-canonical]

Hey Ben, thanks for the quick reply!

I wonder if this is happening because you've installed juju from the Snap, so it can only access snap-related directories

That may be the case, as the referenced Jubilant scp call did not work on any snap-install Juju envs (I tried locally and on CI).

I wouldn't recommend using Juju._temp_dir directly in real code as it's private and we make no guarantees

I agree with this statement: we would like to avoid relying on Jubilant private methods, as that is a horrible practice. IMHO, it is better to use the subprocess alternative I described as a work-around, than re-defining our own temp-dir helper function. On this topic, I think there are two action items to make this situation better:

1. To explain this behavior on the Jubilant scp method doc-string (and any other method that may fail depending on user-provider paths). The fact that snap-installed Juju may be limited, directories wise, is commonly overlooked / forgotten.
2. Ideally, the Charm Tech team can assess the quality / maintainability of Jubilant scp required _temp_dir method, so it can eventually be exposed publicly, avoiding users to re-define, over and over, the same helper function for their tests. If you are open to it, I would argue that there is another convenient method that is required for the Jubilant scp function to work: one to get Juju SSH private key, similar to this one from the ops_test framework.

WDYT?

[benhoyt]

I'm open to that. We'll need to figure out the exact API, or whether we just make Juju._temp_dir public -- I've had a go at that in #205.

[sinclert-canonical]

🗞️ Small update on this topic:

Even if handling the temporal directory transparently to Jubilant users (whenever JuJu is installed as a snap) would be a welcome change, there is a simpler work-around to the one I originally described in the issue:

+ from pathlib import Path

for unit_name in get_app_units(juju, MYSQL_APP_NAME):
-     with tempfile.NamedTemporaryFile(mode="w") as temp_file:
+     with tempfile.NamedTemporaryFile(mode="w", dir=Path.home()) as temp_file:
        temp_file.write("data")
        temp_file.flush()
-       scp_unit_file(
-           juju=juju,
-           source=f"{temp_file.name}",
-           target=f"{unit_name}:/tmp/file",
-        )
+       juju.scp(
+           f"{temp_file.name}",
+           f"{unit_name}:/tmp/file",
+       )