Pass file descriptors in std::comand::Command

[axelkar]

Proposal

Problem statement

I want to pass any number of file descriptors into child processes, without manual unsafe, and using only the standard library.

Motivating examples or use cases

systemd's sd_listen_fds: receives FDs usually right after the stdin, stdout and stderr FDs. Gets the range of FDs from SD_LISTEN_FDS_START and LISTEN_FDS. Notes: LISTEN_PID has to be known after the fork, but before exec. The FD numbers have to be known in advance for getting the range for the environment variables.

Object capability security (CloudABI and WASI. see also the cap_std ecosystem): Passing FDs with accessible files and directories.

Solution sketch

Note lifetime parameters: #t-libs-api/api-changes > Extra FDs in CommandExt.

Lifetimes will also have to be added to pre_exec.

impl<'fds> std::os::fd::CommandExt for std::command::Command<'fds> {
  // TODO: No loop is probably better?
  fn pass_fds(&mut self, fds: &'fds [(BorrowedFd<'fds>, std::os::fd::RawFd)]) {
    // SAFETY: TODO
    unsafe {
      self.pre_exec(|| {
        for (old_fd, new_fd) in fds {
          libc::dup2(old_fd.as_raw_fd(), new_fd);
          libc::fnctl(new_fd, ???); // Unset O_CLOEXEC
        }
      })
    }
  }
}

Alternatives

Passing FDs can also be done with Unix sockets, but it's a different API for both sides, and is not guaranteed to be instant/atomic. It's fully safe and possible to do with current APIs.

An extension trait is also possible to implement with OwnedFd (may have to dup for multiple command spawns with the same file description) outside of the standard library. Note that it requires unsafe code.

Links and related work

• cap_std_ext::cmdext::CapStdExtCommandExt::take_fd_n

• Similar thing on Windows (see comments): Ability to stop child process from Inheriting Handles #264

  Seems to also support not passing handles which aren't included (Including stdio. Analogous to closing non-CLOEXEC fds?). Maybe not in the scope here?

• [Feature request] Pass file descriptors in std::comand::Command rust#144191

• Python's subprocess.Popen's close_fds=True and pass_fds

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[the8472]

As discussed in the linked threads, protocols like sd_listen_fd and the jobserver protocol are soundness footguns. I don't think we should encourage this pattern in std until we can offer a robust approach or alternatives.

And ideally a similar-enough protocol for handles windows too so people don't have to juggle two different approaches.

Passing FDs can also be done with Unix sockets, but it's a different API for both sides, and is not guaranteed to be instant/atomic. It's fully safe and possible to do with current APIs.

Could you elaborate what the problems with that are? It should only cost maybe 1-2 extra context switches to use a unix socket to pass up to 250 fds in a single message to the child.

[the8472]

// Unset O_CLOEXEC

Another problem with this is that they keep getting inherited to child processes like that unless the child explicitly sets that flag again. Passing via socket also avoids this passive leakage because a child has to explicitly extract the descriptors via CMSG otherwise they will be consigned to oblivion.

[axelkar]

Passing FDs can also be done with Unix sockets, but it's a different API for both sides, and is not guaranteed to be instant/atomic. It's fully safe and possible to do with current APIs.

Could you elaborate what the problems with that are? It should only cost maybe 1-2 extra context switches to use a unix socket to pass up to 250 fds in a single message to the child.

For it to work, you have to do the following:

1. Connect to the socket to receive the FDs. This requires the Unix socket to be at a known path in the current filesystem namespace, and that the child process has permission to access it. This has to be a blocking operation, because the "FD server" has to accept the connection.
2. Communicate with the "FD server" and receive the FDs using a bespoke protocol. Blocking operation.

For my use cases, the first step is the worst issue here. It means that the "FD server" has to give ambient authority for all child processes of its connectees to access its socket.

It's also incompatible with what sd_listen_fd and the jobserver protocol currently do.