micropython-fnv1a32

FNV1a32 is a simple 32-bit hash function that is optimized for speed while maintaining a low collision rate.

This repo implements a micropython native module of the fnv1a32 hash function. To use a precompiled micropython native module, download the appropriate architecture/micropython-version from the release page. Requires MicroPython >1.22.0.

Usage

This library supplies a single function, fnv1a32, that can handle a variety of datatypes. The resulting hash is an integer object (not bytes!).

Hashing Data In-Memory

To hash bytes/bytearray/str in-memory:

from fnv1a32 import fnv1a32

fnv1a32_hash = fnv1a32(b"this is the data to be hashed")

To continue hashing, supply the previous hash into the next fnv1a32 invocation:

from fnv1a32 import fnv1a32

fnv1a32_hash = fnv1a32(b"this is the data to be hashed")
fnv1a32_hash = fnv1a32(b"more data", fnv1a32_hash)

Hashing File

To hash a file:

from fnv1a32 import fnv1a32

with open("foo.bin") as f:
    # Defaults to using 4096-byte chunks
    fnv1a32_hash = fnv1a32(f)

To read and hash bigger chunks at a time (uses more memory, may improve speed):

from fnv1a32 import fnv1a32

with open("foo.bin") as f:
    fnv1a32_hash = fnv1a32(f, buffer=16384)

A pre-allocated buffer may also be used:

from fnv1a32 import fnv1a32

buffer = bytearray(16384)
with open("foo.bin") as f:
    fnv1a32_hash = fnv1a32(f, buffer=buffer)

Unit Testing

To run the unittests, install Belay and run the following commands:

make clean
make

belay run micropython -m unittest tests/test_fnv1a32.py

Benchmark

The following were benchmarked on an rp2040 hashing 50KB of data in-memory.

Implementation	Bytes/s	Relative Speed
vanilla micropython	24,912	1.00x
@micropython.native	26,619	1.07x
@micropython.viper	2,438,786	97.90x
micropython native module	25,906,736	1040x

To run the benchmark, install Belay and run the following commands:

export MPY_DIR=../micropython  # Replace with your micropython directory.
make clean
ARCH=armv6m make  # Change the arch if running on different hardware.

belay install /dev/ttyUSB0 --with=dev
belay sync /dev/ttyUSB0 fnv1a32.mpy
belay run /dev/ttyUSB0 benchmark/fnv1a32_benchmark.py