Tracking Benchmarks

[wcampbell0x2a]

The priority of this project was having a library that could be called from rust code, with mutating. But performance is nice vs squashfs-tools. Warning, these are some simple benchmarks. I might be missing something important that nullifies these measurements.

our branch: cd0e143 vs squashfs-tools 4.6.1. These are done using hyperfine!

In most testing, we match the performance of squashfs-tools/unsquashfs:

  './target/release/unsquashfs -f -d /tmp/BHQSi -o 2252752 test-assets/test_openwrt_tplink_archera7v5/openwrt-22.03.2-ath79-generic-tplink_archer-a7-v5-squashfs-factory.bin' ran
    1.59 ± 0.24 times faster than '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BHErM      -f -o 2252752 -ignore-errors test-assets/test_openwrt_tplink_archera7v5/openwrt-22.03.2-ath79-generic-tplink_archer-a7-v5-squashfs-factory.bin'

  './target/release/unsquashfs -f -d /tmp/BHrtI -o 2883712 test-assets/test_openwrt_netgear_ex6100v2/openwrt-22.03.2-ipq40xx-generic-netgear_ex6100v2-squashfs-factory.img' ran
    1.68 ± 0.28 times faster than '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BH7dH      -f -o 2883712 -ignore-errors test-assets/test_openwrt_netgear_ex6100v2/openwrt-22.03.2-ipq40xx-generic-netgear_ex6100v2-squashfs-factory.img'

  '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BH6oO      -f -o 0 -ignore-errors test-assets/test_re815_xev160/870D97.squashfs' ran
    1.43 ± 0.09 times faster than './target/release/unsquashfs -f -d /tmp/BHkm0 -o 0 test-assets/test_re815_xev160/870D97.squashfs'

  './target/release/unsquashfs -f -d /tmp/BHcb4 -o 0 test-assets/test_tplink_ax1800/img-1571203182_vol-ubi_rootfs.ubifs' ran
    1.12 ± 0.06 times faster than '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BHFEg      -f -o 0 -ignore-errors test-assets/test_tplink_ax1800/img-1571203182_vol-ubi_rootfs.ubifs'

 '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BHVdj      -f -o 0 -ignore-errors test-assets/test_archlinux_iso_rootfs/airootfs.sfs' ran
    3.35 ± 0.22 times faster than './target/release/unsquashfs -f -d /tmp/BHv9T -o 0 test-assets/test_archlinux_iso_rootfs/airootfs.sfs'

 './target/release/unsquashfs -f -d /tmp/BH44S -o 0 test-assets/test_er605_v2_2/2611E3.squashfs' ran
    1.37 ± 0.12 times faster than '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BHQpX      -f -o 0 -ignore-errors test-assets/test_er605_v2_2/2611E3.squashfs'

 '/usr/bin/unsquashfs -quiet -no-progress -d /tmp/BHfap      -f -o 188392 -ignore-errors test-assets/test_appimage_plexamp/Plexamp-4.6.1.AppImage' ran
    2.78 ± 0.11 times faster than './target/release/unsquashfs -f -d /tmp/BH9MI -o 188392 test-assets/test_appimage_plexamp/Plexamp-4.6.1.AppImage'

PC details

lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
    CPU family:          6
    Model:               158
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            9
    CPU(s) scaling MHz:  98%
    CPU max MHz:         4500.0000
    CPU min MHz:         800.0000
    BogoMIPS:            8403.00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_t
                         sc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault i
                         nvpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida ar
                         at pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:        VT-x
Caches (sum of all):
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    1 MiB (4 instances)
  L3:                    8 MiB (1 instance)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-7
Vulnerabilities:
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Mitigation; IBRS
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Mitigation; TSX disabled

[wcampbell0x2a]

Memory usage using heaptrack

backhand

squashfs-tools

[qkaiser]

Did you look into why squashfs-tools is leaking so much memory ? It's like they do not deallocate any of their allocations.

[wcampbell0x2a]

I didn't, but you're right it does look like that