The Acurast Benchmark suite is designed to evaluate the performance of processors running in the Acurast network. It provides a comprehensive set of tests to measure computational throughput, memory allocation and access efficiency, and storage read/write performance. The benchmarks include support for multithreaded execution to leverage modern multi-core architectures.
The CPU benchmarks evaluate the computational capabilities of the device. They include:
- Crypto: Measures encryption and decryption throughput using AES-256 and hashing throughput using SHA-256.
- Math: Tests matrix multiplication performance, with support for three algorithms:
- Divide and Conquer: A recursive algorithm used for single-threaded execution when SIMD is disabled or for multithreaded benchmarks in general.
- Iterative: A straightforward implementation used when SIMD optimization is enabled but the device lacks the required hardware capabilities.
- SIMD-Optimized: An optimized version of the iterative algorithm leveraging SIMD instructions, used when the device supports the required hardware capabilities.
- Sort: Benchmarks sorting algorithms, including single-threaded and multithreaded merge sort.
The CPU benchmark suite is available in two variations:
- Single-Core: Executes all tests using a single thread to evaluate the performance of a single core.
- Multi-Core: Executes all tests using multiple threads to leverage the full computational power of the device.
The RAM benchmarks assess memory allocation and access patterns:
- Allocation: Measures the time taken to allocate and initialize memory.
- Access: Evaluates sequential, random, and concurrent memory access patterns.
The storage benchmarks focus on file I/O performance:
- Access: Measures sequential and random read/write throughput for files on the storage medium.
- Crypto: Configure the duration of the test and the size of the data to encrypt and hash.
- Math: Specify the matrix size, whether to enable SIMD optimizations, and the duration of the test.
- Sort: Define the size of the dataset and the duration of the test.
- Crypto: The test encrypts and decrypts data using AES-256 and computes hashes using SHA-256.
- Math: Performs matrix multiplication using one of three algorithms:
- Divide and Conquer: Used for single-threaded execution when SIMD is disabled or for multithreaded benchmarks.
- Iterative: Used when SIMD is enabled but the device lacks the required hardware capabilities.
- SIMD-Optimized: Used when SIMD is enabled and the device supports the required hardware capabilities.
- Sort: Executes merge sort on a dataset, either single-threaded or multithreaded.
- Crypto: Reports the throughput in bytes per second for encryption and hashing.
- Math: Outputs the number of operations per second for matrix multiplication.
- Sort: Provides the number of sorted elements per second.
-
Score: The CPU score is calculated as the average of the throughput (
$TPS$ ) values from the Crypto, Math, and Sort benchmarks. Higher throughput results in a higher score.
- Allocation: Specify the number of iterations and the size of memory to allocate in each iteration.
- Access: Configure the number of iterations and the size of data for sequential, random, and concurrent access patterns.
- Allocation: Allocates and initializes memory repeatedly to measure allocation performance.
- Access: Reads and writes data in sequential, random, and concurrent patterns to evaluate memory access efficiency.
- Allocation: Reports the average time taken to allocate and initialize memory.
- Access: Provides average times for sequential, random, and concurrent access patterns.
- Total RAM: The total available RAM memory.
-
Score: The RAM score is calculated based on the inverse of the average times (
$T$ ) for Allocation and Access benchmarks. Higher efficiency results in a higher score.
- Access: Define the number of iterations and the size of data for sequential and random read/write operations.
- Access: Creates temporary files and performs sequential and random read/write operations. Files are removed after each iteration.
- Access: Reports the average time for sequential and random read/write operations.
- Available Storage: The available storage capacity.
-
Score: The Storage score is calculated as the average of the inversed average times (
$T$ ) for sequential and random read/write operations. Lower average times result in a higher score.
To use the Acurast Benchmark library in your Android project, add the following to your settings.gradle file to enable Jitpack:
dependencyResolutionManagement {
repositories {
maven { url = uri("https://jitpack.io") }
google()
mavenCentral()
}
}Then, add the dependency to your build.gradle file:
dependencies {
implementation("com.github.acurast:acurast-benchmark:<version>")
}Replace <version> with the latest version available on Jitpack.
-
Initialize the Library
Create an instance ofAcubench:import com.acurast.bench.Acubench val acubench = Acubench(context)
-
Run Benchmarks
Run the desired benchmarks using the provided methods:fun cpuSingleCore() { val report = acubench.cpu() println(""" CPU Single-Core :::: crypto ${report.cryptoTps} ops/s :::: math ${report.mathTps} ops/s :::: sort ${report.sortTps} ops/s ---- :::: score ${report.score} """.trimIndent()) } fun cpuMultiCore() { val report = acubench.cpuMultithread() println(""" CPU Multi-Core :::: crypto ${report.cryptoTps} ops/s :::: math ${report.mathTps} ops/s :::: sort ${report.sortTps} ops/s ---- :::: score ${report.score} """.trimIndent()) } fun ram() { val report = acubench.ram() println(""" RAM :::: total memory ${report.totalMemory / 1024f / 1024f} GB :::: alloc ${report.allocAvgTime} s :::: access (sequential) ${report.accessSequentialAvgTime} s :::: access (random) ${report.accessRandomAvgTime} s :::: access (concurrent) ${report.accessConcurrentAvgTime} s ---- :::: score ${report.score} """.trimIndent()) } fun storage() { val storage = acubench.storage(context) println(""" Storage :::: available ${report.availableStorage / 1024f / 1024f} GB :::: access (sequential) ${report.accessSequentialAvgTime} s :::: access (random) ${report.accessRandomAvgTime} s ---- :::: score ${report.score} """.trimIndent()) } cpuSingleCore() cpuMultiCore() ram() storage()
-
Clean Up
Release the resources when done:acubench.destroy()
The configuration structures allow you to customize the behavior of the benchmarks. Each structure provides fields to fine-tune the parameters for specific tests. If not specified, default values will be used.
-
CpuConfig: Configures the CPU benchmark.
data class CpuConfig( val cryptoDuration: Duration = 1.seconds, // Duration of the crypto test val cryptoDataSize: Long = 10 * 1024, // Size of data to encrypt and hash (in bytes) val mathDuration: Duration = 1.seconds, // Duration of the math test val mathDataSize: Long = 200, // Size of the matrix for multiplication val mathSimd: Boolean = true, // Enable SIMD optimizations val sortDuration: Duration = 1.seconds, // Duration of the sort test val sortDataSize: Long = 100_000 // Number of elements to sort )
-
RamConfig: Configures the RAM benchmark.
data class RamConfig( val allocIters: Long = 10, // Number of iterations for memory allocation val allocDataSize: Long = 64 * 1024 * 1024, // Size of memory to allocate per iteration (in bytes) val accessSequentialIters: Long = 10, // Number of iterations for sequential access val accessSequentialDataSize: Long = 64 * 1024, // Size of data for sequential access (in bytes) val accessRandomIters: Long = 10, // Number of iterations for random access val accessRandomDataSize: Long = 64 * 1024, // Size of data for random access (in bytes) val accessConcurrentIters: Long = 10, // Number of iterations for concurrent access val accessConcurrentDataSize: Long = 64 * 1024 // Size of data for concurrent access (in bytes) )
-
StorageConfig: Configures the Storage benchmark.
data class StorageConfig( val dir: File, // Directory to perform storage tests val accessSequentialIters: Long = 1, // Number of iterations for sequential access val accessSequentialDataSizeMB: Long = 50, // Size of data for sequential access (in MB) val accessRandomIters: Long = 1, // Number of iterations for random access val accessRandomDataSizeMB: Long = 50 // Size of data for random access (in MB) )
The report structures provide detailed results of the benchmarks.
-
CpuReport: Contains CPU benchmark results.
data class CpuReport( val cryptoTps: Double, // Throughput for crypto operations (ops/s) val mathTps: Double, // Throughput for matrix multiplication (ops/s) val sortTps: Double, // Throughput for sorting operations (ops/s) val score: Double // Overall CPU performance score )
-
RamReport: Contains RAM benchmark results.
data class RamReport( val totalMemory: Long, // Total available RAM (in bytes) val allocAvgTime: Double, // Average time for memory allocation (in seconds) val accessSequentialAvgTime: Double, // Average time for sequential memory access (in seconds) val accessRandomAvgTime: Double, // Average time for random memory access (in seconds) val accessConcurrentAvgTime: Double, // Average time for concurrent memory access (in seconds) val score: Double // Overall RAM performance score )
-
StorageReport: Contains Storage benchmark results.
data class StorageReport( val availableStorage: Long, // Available storage space (in bytes) val accessSequentialAvgTime: Double, // Average time for sequential storage access (in seconds) val accessRandomAvgTime: Double, // Average time for random storage access (in seconds) val score: Double // Overall Storage performance score )
To use the Acurast Benchmark library in your iOS project, add the following to your Package.swift file to fetch the library via Swift Package Manager (SPM):
dependencies: [
.package(url: "https://github.com/acurast/acurast-benchmark.git", .upToNextMajor(from: "<version>"))
]Replace <version> with the latest version available on GitHub Releases.
Then, add the library as a dependency to your target:
.target(
name: "TargetName",
dependencies: [
.product(name: "Acubench", package: "acurast-benchmark")
]
)-
Initialize the Library
Create an instance ofAcubench:import Acubench let acubench = Acubench()
-
Run Benchmarks
Run the desired benchmarks using the provided methods:func cpuSingleCore() throws { let report = try acubench.cpu() print( """ CPU Single-Core :::: crypto \(report.cryptoTPS.formatted()) ops/s :::: math \(report.mathTPS.formatted()) ops/s :::: sort \(report.sortTPS.formatted()) ops/s ---- :::: score \(report.score.formatted()) """) } func cpuMultiCore() throws { let report = try acubench.cpuMultithread() print( """ CPU Multi-Core :::: crypto \(report.cryptoTPS.formatted()) ops/s :::: math \(report.mathTPS.formatted()) ops/s :::: sort \(report.sortTPS.formatted()) ops/s ---- :::: score \(report.score.formatted()) """) } func ram() throws { let report = try acubench.ram() print( """ RAM :::: total memory \(report.totalMemory / 1024 / 1024) GB :::: alloc \(report.allocAvgTime.formatted()) s :::: access (sequential) \(report.accessSequentialAvgTime.formatted()) s :::: access (random) \(report.accessRandomAvgTime.formatted()) s :::: access (concurrent) \(report.accessConcurrentAvgTime.formatted()) s ---- :::: score \(report.score.formatted()) """) } func storage() throws { let report = try acubench.storage() print( """ Storage :::: available \(report.availableStorage / 1024 / 1024 / 1024) GB :::: access (sequential) \(report.accessSequentialAvgTime.formatted()) s :::: access (random) \(report.accessRandomAvgTime.formatted()) s ---- :::: score \(report.score.formatted()) """) } try cpuSingleCore() try cpuMultiCore() try ram() try storage()
The configuration structures allow you to customize the behavior of the benchmarks. Each structure provides fields to adjust the parameters for specific tests.
-
CPUConfig: Configures the CPU benchmark.
struct CPUConfig { let cryptoDuration: TimeInterval = 1.0 // Duration of the crypto test (in seconds) let cryptoDataSize: Int = 10 * 1024 // Size of data to encrypt and hash (in bytes) let mathDuration: TimeInterval = 1.0 // Duration of the math test (in seconds) let mathDataSize: Int = 200 // Size of the matrix for multiplication let mathSimd: Bool = true // Enable SIMD optimizations let sortDuration: TimeInterval = 1.0 // Duration of the sort test (in seconds) let sortDataSize: Int = 100_000 // Number of elements to sort }
-
RAMConfig: Configures the RAM benchmark.
struct RAMConfig { let allocIters: Int = 10 // Number of iterations for memory allocation let allocDataSize: Int = 64 * 1024 * 1024 // Size of memory to allocate per iteration (in bytes) let accessSequentialIters: Int = 10 // Number of iterations for sequential access let accessSequentialDataSize: Int = 64 * 1024 // Size of data for sequential access (in bytes) let accessRandomIters: Int = 10 // Number of iterations for random access let accessRandomDataSize: Int = 64 * 1024 // Size of data for random access (in bytes) let accessConcurrentIters: Int = 10 // Number of iterations for concurrent access let accessConcurrentDataSize: Int = 64 * 1024 // Size of data for concurrent access (in bytes) }
-
StorageConfig: Configures the Storage benchmark.
struct StorageConfig { let dir: URL // Directory to perform storage tests let accessSequentialIters: Int = 1 // Number of iterations for sequential access let accessSequentialDataSizeMB: Int = 50 // Size of data for sequential access (in MB) let accessRandomIters: Int = 1 // Number of iterations for random access let accessRandomDataSizeMB: Int = 50 // Size of data for random access (in MB) }
The report structures provide detailed results of the benchmarks.
-
CPUReport: Contains results for the CPU benchmark.
struct CPUReport { let cryptoTPS: Double // Throughput for crypto operations (ops/s) let mathTPS: Double // Throughput for matrix multiplication (ops/s) let sortTPS: Double // Throughput for sorting operations (ops/s) let score: Double // Overall CPU performance score }
-
RAMReport: Contains results for the RAM benchmark.
struct RAMReport { let totalMemory: Int64 // Total available RAM (in bytes) let allocAvgTime: Double // Average time for memory allocation (in seconds) let accessSequentialAvgTime: Double // Average time for sequential memory access (in seconds) let accessRandomAvgTime: Double // Average time for random memory access (in seconds) let accessConcurrentAvgTime: Double // Average time for concurrent memory access (in seconds) let score: Double // Overall RAM performance score }
-
StorageReport: Contains results for the Storage benchmark.
struct StorageReport { let availableStorage: Int64 // Available storage space (in bytes) let accessSequentialAvgTime: Double // Average time for sequential storage access (in seconds) let accessRandomAvgTime: Double // Average time for random storage access (in seconds) let score: Double // Overall Storage performance score }