unpackPhase breaks with proot

[milahu]

nix-portable works on my nixos machine
but fails on a debian-based shared server

i have converted some nix expressions
from fetchFromGitHub to fetchurl
in my nur-packages
to fix failing unpackPhase branches
for example milahu/nur-packages@b91e204

pushd $(mktemp -d)
git clone https://github.com/milahu/nur-packages
cd nur-packages
git checkout 0a32ec362221753afe4e02f755af28c75981e177
nix-build . -A python3.pkgs.torf

Running phase: unpackPhase
unpacking source archive /nix/store/i29chfwff7bmigq47jjq5qwy0n9bspnr-source
cp: setting permissions for 'source': No such file or directory
do not know how to unpack source archive /nix/store/i29chfwff7bmigq47jjq5qwy0n9bspnr-source

/nix/store/i29chfwff7bmigq47jjq5qwy0n9bspnr-source
is the cloned git source

i tried to use

rev = "dc647cfd631699e968a3f99af5670c1b4312b4f3"; 

or

tag = "v${version}";

but no success

with preUnpack = "set -x; pwd";

Running phase: unpackPhase
+++++ pwd
/tmp/nix-build-libtorrent-rasterbar-2.0.11.drv-0
...
+++ destination=source
+++ '[' -e source ']'
+++ cp -r --preserve=mode,timestamps --reflink=auto -- /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source source
cp: setting permissions for 'source': No such file or directory
+++ return 1

with

  preUnpack = ''                                   
    set -x; pwd; ls;                                 
    cp -r --preserve=mode,timestamps \
      --reflink=auto -- $src source || true       
    stat $src source; du -sb $src source                                   
    exit 1                                        
  '';

Running phase: unpackPhase                        
+++++ pwd                                         
/tmp/nix-build-libtorrent-rasterbar-2.0.11.drv-0  
+++++ ls                                          
env-vars
+++++ cp -r --preserve=mode,timestamps --reflink=auto -- /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source source                                    
cp: setting permissions for 'source': No such file or directory                                     
+++++ true                                        
+++++ stat /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source source                                 
  File: /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source                                            
  Size: 4096            Blocks: 8          IO Block: 4096   directory                               
Device: 65,209  Inode: 32337935    Links: 14      
Access: (0555/dr-xr-xr-x)  Uid: ( 2320/  milahu)   
 Gid: ( 2320/  milahu)                            
Access: 2025-04-13 12:34:49.000000000 +0000       
Modify: 1970-01-01 00:00:01.000000000 +0000       
Change: 2025-04-13 12:35:08.703574879 +0000        
 Birth: 2025-04-13 12:35:01.972559948 +0000
  File: source                                    
  Size: 40              Blocks: 0          IO Block: 4096   directory                               
Device: 0,46    Inode: 56412437    Links: 2       
Access: (0555/dr-xr-xr-x)  Uid: ( 2320/  milahu)  
 Gid: ( 2320/  milahu)                            
Access: 2025-04-13 13:57:44.415258603 +0000       
Modify: 2025-04-13 13:57:44.415258603 +0000       
Change: 2025-04-13 13:57:44.415258603 +0000
 Birth: 2025-04-13 13:57:44.415258603 +0000
+++++ du -sb /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source source                              
13396636        /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source                                  
0       source

strace

+++++ strace cp -r --preserve=mode,timestamps --reflink=auto -- /nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source source
execve("/nix/store/yh6qg1nsi5h2xblcr67030pz58fsaxx3-coreutils-9.6/bin/cp", ["cp", "-r", "--preserve=mode,timestamps", "--reflink=auto", "--", "/nix/store/jccwym60qp35f73m93byc"..., "source"], 0x7fffffffc350 /* 90 vars */) = 0
brk(NULL)                               = 0x56d000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffff7ff7000
access("/etc/ld-nix.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/glibc-hwcaps/x86-64-v2/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/glibc-hwcaps/x86-64-v2/", 0x7fffffffb3e0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=44416, ...}) = 0
mmap(NULL, 41008, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ffff7fec000
mmap(0x7ffff7fee000, 20480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ffff7fee000
mmap(0x7ffff7ff3000, 8192, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7ffff7ff3000
mmap(0x7ffff7ff5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8000) = 0x7ffff7ff5000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/pn0rk0wsmkjazbz380y8xzb99n8sr07g-attr-2.5.2/lib/glibc-hwcaps/x86-64-v2/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/pn0rk0wsmkjazbz380y8xzb99n8sr07g-attr-2.5.2/lib/glibc-hwcaps/x86-64-v2/", 0x7fffffffb3c0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/pn0rk0wsmkjazbz380y8xzb99n8sr07g-attr-2.5.2/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=30632, ...}) = 0
mmap(NULL, 28696, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ffff7fe4000
mmap(0x7ffff7fe6000, 12288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ffff7fe6000
mmap(0x7ffff7fe9000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7ffff7fe9000
mmap(0x7ffff7fea000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7ffff7fea000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/pn0rk0wsmkjazbz380y8xzb99n8sr07g-attr-2.5.2/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rlpanzl29s3gv3wgs5720q04rbgysl8a-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v2/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/rlpanzl29s3gv3wgs5720q04rbgysl8a-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v2/", 0x7fffffffb3a0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rlpanzl29s3gv3wgs5720q04rbgysl8a-gmp-with-cxx-6.3.0/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=746072, ...}) = 0
mmap(NULL, 696840, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ffff7f39000
mmap(0x7ffff7f4b000, 520192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12000) = 0x7ffff7f4b000
mmap(0x7ffff7fca000, 94208, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x91000) = 0x7ffff7fca000
mmap(0x7ffff7fe1000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa8000) = 0x7ffff7fe1000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/4izi23kn57b3cw47jddzq14wcv01b5y5-acl-2.3.2/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/pn0rk0wsmkjazbz380y8xzb99n8sr07g-attr-2.5.2/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rlpanzl29s3gv3wgs5720q04rbgysl8a-gmp-with-cxx-6.3.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/", 0x7fffffffb380, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\243\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
fstat(3, {st_mode=S_IFREG|0555, st_size=2388416, ...}) = 0
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
mmap(NULL, 2121176, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ffff7c00000
mmap(0x7ffff7c28000, 1527808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x28000) = 0x7ffff7c28000
mmap(0x7ffff7d9d000, 352256, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19d000) = 0x7ffff7d9d000
mmap(0x7ffff7df3000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f2000) = 0x7ffff7df3000
mmap(0x7ffff7df9000, 52696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ffff7df9000
close(3)                                = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffff7f37000
arch_prctl(ARCH_SET_FS, 0x7ffff7f38140) = 0
set_tid_address(0x7ffff7f38410)         = 986861
set_robust_list(0x7ffff7f38420, 24)     = 0
rseq(0x7ffff7f38a60, 0x20, 0, 0x53053053) = 0
mprotect(0x7ffff7df3000, 16384, PROT_READ) = 0
mprotect(0x7ffff7fe1000, 8192, PROT_READ) = 0
mprotect(0x7ffff7fea000, 4096, PROT_READ) = 0
mprotect(0x7ffff7ff5000, 4096, PROT_READ) = 0
mprotect(0x546000, 53248, PROT_READ)    = 0
mprotect(0x6f0000036000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
prctl(PR_SET_NAME, "cp")                = 0
prctl(PR_SET_MM, PR_SET_MM_ARG_START, 0x7fffffffc7b7, 0, 0) = -1 EPERM (Operation not permitted)
getrandom("\xcf\x58\x0a\xcd\x81\x21\xe0\xdc", 8, GRND_NONBLOCK) = 8
brk(NULL)                               = 0x56d000
brk(0x58e000)                           = 0x58e000
geteuid()                               = 2320
openat(AT_FDCWD, "source", O_RDONLY|O_PATH|O_DIRECTORY) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/jccwym60qp35f73m93bycl2rhf2lf0fc-source", {st_mode=S_IFDIR|0555, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
newfstatat(AT_FDCWD, "source", 0x7fffffffbd10, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
mkdirat(AT_FDCWD, "source", 0555)       = 0
newfstatat(AT_FDCWD, "source", {st_mode=S_IFDIR|0555, st_size=40, ...}, AT_SYMLINK_NOFOLLOW) = 0
fchmodat2(AT_FDCWD, "source", 040755, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
fcntl(1, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
write(2, "cp: ", 4cp: )                     = 4
write(2, "setting permissions for 'source'", 32setting permissions for 'source') = 32
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
lseek(0, 0, SEEK_CUR)                   = 0
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

cp: setting permissions for 'source': No such file or directory

this error also shows with cp -r -- $src source

this issue started
when i switched from bwrap to proot backend

the latest failing expression is

git checkout ce7de400e00ec1bdfb74db38626217c1d8a5c14b
nix-build . -A python3.pkgs.selenium

which is resistant to the fetchurl workaround

this works:
nix-portable on bwrap
nix-on-droid on proot (todo why?)

this breaks:
nix-portable on proot

fchmodat2(AT_FDCWD, "source", 040755, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
fcntl(1, F_GETFL)

→ upstream issue proot-me/proot#387

building proot from source
with proot-me/proot#393

spoiler:
that PR is not supposed to fix proot-me/proot#387

nix-shell -p gcc gnumake pkg-config git wget gnutar talloc

git clone https://github.com/remexre/proot --branch emulate-mknod
cd proot

if ! pkg-config --cflags talloc; then
wget https://www.samba.org/ftp/talloc/talloc-2.4.3.tar.gz
tar xf talloc-2.4.3.tar.gz
mv talloc-2.4.3 talloc
cd talloc
./configure --prefix=$PWD/usr --disable-python
make
make install
cd ..
export PKG_CONFIG_PATH=$PWD/talloc/usr/lib/pkgconfig/
fi

# follow README.rst
# TODO build static proot
make -C src loader.elf loader-m32.elf build.h
make -C src proot

ldd src/proot
mv ~/.nix-portable/bin/proot ~/.nix-portable/bin/proot.old
cp src/proot ~/.nix-portable/bin/proot

but i still get

fchmodat2(AT_FDCWD, "source", 040755, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)

minimal reproducer

nix-portable nix-build -E '
  with import <nixpkgs> { };
  stdenv.mkDerivation {
    name = "x";
    # `cp ${nix}/bin/nix .` works
    # `cp -r ${nix}/ .` fails with `cp: setting permissions for $dir: No such file or directory`
    buildCommand = "set -x; cp ${nix}/bin/nix .; cp -r ${nix}/ .; set +x; exit 1";
  }
'

calling proot directly works

export NP_LOCATION="$HOME"
src=/nix/store/2zws1lzvq1pww90d07x904bcxxfk06zf-nix-2.28.1 # must be dir
dst=$(mktemp -u /tmp/test-fchmodat2-proot.XXXXXXXX)
proot -b "$NP_LOCATION/.nix-portable/nix:/nix" -b /tmp:/tmp cp -r $src $dst
chmod -R +w $dst; rm -rf $dst

so this seems to be an issue with the nix build sandbox

nix-build --option sandbox false has no effect

fchmodat2 only fails with dfd = AT_FDCWD

test-fchmodat2.c

$ git clone --depth=1 https://github.com/milahu/nur-packages
$ cd nur-packages
$ nix-portable nix-build --option sandbox false . -A test-fchmodat2
this derivation will be built:
  /nix/store/hlafr3agwj29r1w8flcbcc4xk5hssfv0-test-fchmodat2-0.0.1.drv
building '/nix/store/hlafr3agwj29r1w8flcbcc4xk5hssfv0-test-fchmodat2-0.0.1.drv'...
+++ touch testfile1
+++ mkdir testdir2
+++ mkdir subdir
+++ touch subdir/testfile3
+++ mkdir subdir/testdir4
+++ chmod 0777 testfile1 testdir2 subdir/testfile3 subdir/testdir4
+++ stat -c '%a %n' testfile1 testdir2 subdir/testfile3 subdir/testdir4
777 testfile1
777 testdir2
777 subdir/testfile3
777 subdir/testdir4
+++ cp /nix/store/0wavvvkmzljsfdh87gl0z8lvvv373v1g-test-fchmodat2.c test-fchmodat2.c
+++ gcc -o test-fchmodat2 test-fchmodat2.c
+++ ./test-fchmodat2
test 1: path = 'testfile1', fchmodat2 res = -1
test 2: path = 'testdir2', fchmodat2 res = -1
test 3: path = 'subdir/testfile3', fchmodat2 res = 0
test 3: path = 'subdir/testdir4', fchmodat2 res = 0
+++ stat -c '%a %n' testfile1 testdir2 subdir/testfile3 subdir/testdir4
777 testfile1
777 testdir2
600 subdir/testfile3
600 subdir/testdir4

fchmodat2 source: linux/fs/open.c

i suspect this is related to
NixOS/nix#10501

i tried to use private linux headers like namei.h
so i can debug user_path_at etc
i tried to run configure to generate headers
and compile with
gcc -o test-fchmodat2 -Iinclude -Itools/include -Iarch/x86/include -Iarch/sh/include -Ibuild/include -Ibuild/arch/x86/include/generated test-fchmodat2.c
but then i get many compile errors...

maybe its easier to trace the fchmodat syscall?
i tried to follow https://devkernel.io/posts/ftrace_intro/
but the trace output is not helpful

maybe downgrade to nix before NixOS/nix@ba68045 (nix 2.22.0)
no, fchmodat also fails with nix 2.10.0

[milahu]

tried to apply workaround from
proot-me/proot#387 (comment)
but no luck

diff

--- a/bin/nix-portable.v012
+++ b/bin/nix-portable.v012
@@ -44,6 +44,7 @@
 store="$dir/nix/store"
 # create /nix/var/nix to prevent nix from falling back to chroot store.
 mkdir -p $dir/{bin,nix/var/nix,nix/store}
+mkdir -p $dir/tmp
 # sanitize the tmpbin directory
 rm -rf "$dir/tmpbin"
 # create a directory to hold executable symlinks for overriding
@@ -23889,6 +23890,7 @@
 
   toBind=""
   for p in $pathsTopLevel; do
+    if [ "$p" = "/tmp" ]; then continue; fi
     if [ -e "$p" ]; then
       real=$(realpath $p)
       if [ -e "$real" ]; then
@@ -24009,6 +24011,7 @@
     --bind $dir/emptyroot /\
     --dev-bind /dev /dev\
     --bind $dir/nix /nix\
+    --bind $dir/tmp /tmp\
     $binds"
     # --bind $dir/busybox/bin/busybox /bin/sh\
 else
@@ -24019,6 +24022,7 @@
     -r $dir/emptyroot\
     -b /dev:/dev\
     -b $dir/nix:/nix\
+    -b $dir/tmp:/tmp\
     $binds"
     # -b $dir/busybox/bin/busybox:/bin/sh\
 fi

now /nix and /tmp are on the same device
but cp still fails