Skip to content

kimchi: Improve memory usage and parallelism of expr.evaluations #3127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
1d31dd8
kimchi: benchmark expression framework
Fizzixnerd Apr 2, 2025
4d81a31
kimchi: fix typo
Fizzixnerd Apr 2, 2025
5c27623
kimchi: enable feature flags unconditionally
Fizzixnerd Apr 2, 2025
944c368
kimchi: refactor to use ToPrimitive
Fizzixnerd Apr 2, 2025
03a6594
kimchi: fix typo
Fizzixnerd Apr 2, 2025
cfa8d6c
kimchi: remove dyn and replace with impl
Fizzixnerd Apr 2, 2025
2afcd9f
kimchi: always compute evaluations over D8
Fizzixnerd Apr 2, 2025
935d29c
kimchi: Challenge -> Challenges
Fizzixnerd Apr 2, 2025
1c8f5e7
kimchi: add evaluations_iter
Fizzixnerd Apr 2, 2025
01dab68
kimchi: test and bench new evaluations_iter
Fizzixnerd Apr 2, 2025
180ac3f
kimchi: add par_collect to EvaluationsIter
Fizzixnerd Apr 2, 2025
1301abc
kimchi: cargo fmt
Fizzixnerd Apr 2, 2025
8173fe0
kimchi: improved interface to par_collect; clippy
Fizzixnerd Apr 2, 2025
3b53d40
kimchi: switch `evaluations` to new iterator version and resulting de…
Fizzixnerd Apr 2, 2025
444fd1d
kimchi: cargo fmt
Fizzixnerd Apr 2, 2025
9926ccc
kimchi: cargo +nightly fmt
Fizzixnerd Apr 2, 2025
a79271e
Remove useless tests
Fizzixnerd Apr 2, 2025
fa6dd49
kimchi: clippy appeasement
Fizzixnerd Apr 2, 2025
fe8d42c
kimchi: cargo fmt _again_
Fizzixnerd Apr 2, 2025
7a8aace
kimchi: clippy and fmt test_expr.rs
Fizzixnerd Apr 2, 2025
270ab89
update flake.lock
Fizzixnerd Apr 3, 2025
b4e61ae
kimchi: compute over different domains
Fizzixnerd Apr 3, 2025
e868d3b
kimchi: remove value_ and compute domain better
Fizzixnerd Apr 3, 2025
30303a4
kimchi: fix typo
Fizzixnerd Apr 3, 2025
12a6e23
kimchi: remove unneeded &mut
Fizzixnerd Apr 3, 2025
384faad
kimchi: cargo fmt
Fizzixnerd Apr 3, 2025
b2270df
kimchi: Fix rest of tests
Fizzixnerd Apr 7, 2025
4b12394
kimchi: cargo fmt
Fizzixnerd Apr 7, 2025
47f489f
Merge remote-tracking branch 'origin/master' into fizzixnerd/expr-bench
Fizzixnerd Apr 7, 2025
0fa4e82
WIP
Fizzixnerd Apr 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions kimchi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ harness = false
name = "amortization"
harness = false

[[bench]]
name = "expr"
harness = false

[features]
default = []
internal_tracing = ["internal-tracing/enabled"]
Expand Down
80 changes: 80 additions & 0 deletions kimchi/benches/expr.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
use criterion::{criterion_group, criterion_main, Criterion};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file, in addition to the change in kimchi/Cargo.toml, can be extracted from this PR, gathered in a single commit, and provide a good starting point for the changes that you try to introduce in this patch.
A reviewer could go after that on the next commits and gradually use the commands you recommend to use to confirm your claims.
In addition to that, it follows the engineering practices we try to enforce. For instance, this practice has been followed in this PR, and it ended up very useful to have atomic commits compiling and passing the whole CI when we had to revert commits.

It seems you started doing in the first commit, but there are comments that you're removing later. Would you mind trying to squash the different commits touching this file to get a clean first commit?

use kimchi::linearization::constraints_expr;
use std::{collections::HashMap, hint::black_box};

use ark_ff::UniformRand;
use ark_poly::{Evaluations, Radix2EvaluationDomain as D};
use kimchi::{
circuits::{
berkeley_columns::{BerkeleyChallenges, Environment, E},
domains::EvaluationDomains,
expr::Constants,
},
curve::KimchiCurve,
};
use mina_curves::pasta::{Fp, Vesta};
use rand::Rng;

fn create_random_evaluation(domain: D<Fp>, rng: &mut impl Rng) -> Evaluations<Fp, D<Fp>> {
let evals = (0..domain.size).map(|_| Fp::rand(rng)).collect::<Vec<_>>();
Evaluations::from_vec_and_domain(evals, domain)
}

fn benchmark_expr_evaluations(c: &mut Criterion) {
let domains = EvaluationDomains::<Fp>::create(1 << 16).unwrap();
let domain = domains.d8;
let mut rng = rand::thread_rng();
let randomized_witness = (0..15)
.map(|_| create_random_evaluation(domain, &mut rng))
.collect::<Vec<_>>()
.try_into()
.unwrap();
let randomized_coefficients = (0..15)
.map(|_| create_random_evaluation(domain, &mut rng))
.collect::<Vec<_>>()
.try_into()
.unwrap();
let randomized_vanishes_on_zero_knowledge_and_previous_rows =
create_random_evaluation(domain, &mut rng);
let randomized_z = create_random_evaluation(domain, &mut rng);
let randomized_l0_1 = Fp::rand(&mut rng);
let constants = Constants {
endo_coefficient: Fp::rand(&mut rng),
mds: &Vesta::sponge_params().mds,
zk_rows: 0,
};
let challenges = BerkeleyChallenges {
alpha: Fp::rand(&mut rng),
beta: Fp::rand(&mut rng),
gamma: Fp::rand(&mut rng),
joint_combiner: Fp::rand(&mut rng),
};

let env = Environment {
witness: &randomized_witness,
coefficient: &randomized_coefficients,
vanishes_on_zero_knowledge_and_previous_rows:
&randomized_vanishes_on_zero_knowledge_and_previous_rows,
z: &randomized_z,
index: HashMap::new(),
l0_1: randomized_l0_1,
constants,
challenges,
domain: domains,
lookup: None,
};

let expr: E<Fp> = constraints_expr(None, true).0;

c.bench_function("expr_evals_par", |b| {
b.iter(|| black_box(expr.clone()).evaluations(black_box(&env)));
});
}

criterion_group!(
name = evaluation_bench_seq;
config = Criterion::default().sample_size(10);
targets = benchmark_expr_evaluations
);

criterion_main!(evaluation_bench_seq);
2 changes: 1 addition & 1 deletion kimchi/src/circuits/constraints.rs
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ impl<F: PrimeField> ConstraintSystem<F> {
/// - `disable_gates_checks: false`,
///
/// How to use it:
/// 1. Create your instance of your builder for the constraint system using `crate(gates, sponge params)`
/// 1. Create your instance of your builder for the constraint system using `create(gates, sponge params)`
/// 2. Iterativelly invoke any desired number of steps: `public(), lookup(), runtime(), precomputations()``
/// 3. Finally call the `build()` method and unwrap the `Result` to obtain your `ConstraintSystem`
pub fn create(gates: Vec<CircuitGate<F>>) -> Builder<F> {
Expand Down
Loading
Loading