Replies: 2 comments 8 replies
-
|
What are the absolute times we're talking about here? A 50% increase in execution times on the order of nanoseconds is basically a meaningless statistic - we need to know what the baseline time and standard deviation for each run is. I wouldn't be surprised if async is too expensive to make the default - but I'm not convinced by what you've laid out here without further context to the numbers. A couple of other thoughts:
I don't know that we can draw any conclusions about our approach just yet. |
Beta Was this translation helpful? Give feedback.
-
In some sense I saw this scenario as underestimating the cost of the true solution, since most of the work is done processing
As mentioned above, I believe we current model an underestimate of this, since most of the work is spent in
Agreed - but honestly didn't spend too much time on it after seeing the scenario 2's results. But if I missed something then we should absolutely revisit/redesign it. |
Beta Was this translation helpful? Give feedback.
-
I was expecting us to be able to do something like the following: struct ExecuteContext<'a, H: Host> {
program: &'a MastForest,
kernel: &'a Kernel,
host: &'a mut H,
continuation_stack: &'a mut Vec<MastNodeId>,
reductions: u16
}
async fn execute_mast_node<H: Host>(&mut self, node_id: MastNodeId, context: &mut ExecutionContext<'_, H>) -> Result<(), ExecutionError> {
let mut next_node_id = node_id;
loop {
if let Some(new_reductions) = context.reductions.checked_sub(1) {
context.reductions = new_reductions;
self.clk += 1;
} else {
tokio::task::yield_now().await;
context.reductions = 500;
}
let node = context.program.get_node_by_id(next_node_id)
.ok_or(ExecutionError::MastNodeNotFoundInForest { node_id: next_node_id })?;
// process before decorators
// ..snip..
match node {
MastNode::Block(basic_block_node) => {
// inlined implementation of self.execute_basic_block_node()
}
MastNode::Join(join_node) => {
next_node_id = join_node.first();
context.continuation_stack.push(join_node.second());
continue;
}
MastNode::Split(split_node) => {
let condition = self.stack_get(0);
self.decrement_stack_size();
if condition == ONE {
next_node_id = split_node.on_true();
} else if condition == ZERO {
next_node_id = split_node.on_false();
} else {
let err_ctx = err_ctx!(program, split_node, self.source_manager.clone());
return Err(ExecutionError::not_binary_value_if(condition, &err_ctx))
}
self.clk += 1;
continue;
}
MastNode::Loop(loop_node) => {
let mut condition = self.stack_get(0);
self.decrement_stack_size();
let mut saved_continuation = core::mem::swap(&mut context.continuation_stack, Vec::with_capacity(256));
while condition == ONE {
assert_eq!(context.continuation_stack.len(), 0);
self.execute_mast_node(loop_node.body(), context).await?;
condition = self.stack_get(0);
self.decrement_stack_size();
if condition == ONE {
self.clk += 1;
}
}
core::mem::swap(&mut context.continuation_stack, saved_continuation);
self.clk += 1;
if condition != ZERO {
let err_ctx = err_ctx!(program, loop_node, self.source_manager.clone());
return Err(ExecutionError::not_binary_value_loop(condition, &err_ctx));
}
}
MastNode::Call(call_node) => {
let err_ctx = err_ctx!(program, call_node, self.source_manager.clone());
add_error_ctx_to_external_error(
self.execute_call_node(call_node, context).await?,
err_ctx,
)?
}
MastNode::Dyn(dyn_node) => {
let err_ctx = err_ctx!(program, dyn_node, self.source_manager.clone());
add_error_ctx_to_external_error(
self.execute_dyn_node(dyn_node, context).await?,
err_ctx,
)?
},
MastNode::External(external_node) => {
self.execute_external_node(external_node, context).await?
},
}
// process after decorators
// ..snip..
match context.continuation_stack.pop() {
Some(continue_with) => {
next_node_id = continue_with;
}
None => return Ok(()),
}
}
}In short, we reify an explicit continuation stack, and inline the implementations of the execute methods which we want to be able to dispatch without an await point, even if internally we may need to await at some point on some paths through it. As shown above, we would avoid awaits for
I think the time spent awaiting on every other node type ends up dwarfing that, but either way, my expectation would be that our approach would be more subtle here anyway, as outlined above. |
Beta Was this translation helpful? Give feedback.
-
|
My current thinking is that for now we implement async in the fast processor in the naive way:
The rationale for this is to get async functionality into the fast processor before the next release (which practically means that it should be done next week). And then we can improve on it in the follow-up releases. Specifically, we could:
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for putting these together! A few initial observations/comments:
|
Beta Was this translation helpful? Give feedback.
-
Updated the discussion with absolute timings
My thoughts as well. We should probably focus on that next. |
Beta Was this translation helpful? Give feedback.
-
|
Looking at the absolute timings, But even in that example - at 485k cycles, the difference in the worst case scenario (3) is 1ms; that's basically an irrelevant amount of time. Even if we multiply that by a factor of 1000x, that's the difference between a program of 485M cycles taking a full second longer to execute, which seems to me like it is basically noise. Unless the scaling is superlinear (i.e. the async overhead grows faster than the cycle count, while the sync version doesn't exhibit the same behavior), I just don't think the sync vs async performance is actually going to matter in practice. What are we expecting to see on average for program cycle counts? |
Beta Was this translation helpful? Give feedback.
-
We're expecting programs to be on the order of 64k cycles - so based on these numbers, the fast processor would execute them in ~175us (assuming current workloads, which as mentioned in the initial discussion, might not be representative of typical transactions). However, as of the latest merged PR, the
Hence, even though the cycle count of single transactions is low, in the context of (1) my intuition is that we really care about maximal performance. However you bring up a good point that we might be too fixated on fast processor performance here, and would in practice tolerate worst performance (if e.g. the parallel trace generation is really fast). @bobbinth would like to hear your thoughts. |
Beta Was this translation helpful? Give feedback.
-
|
Definitely agree that faster makes for good marketing, at least to the degree that we're outperforming other equivalent VMs, or matching the fastest of them - otherwise I suspect it does little to help, and we'd be better served by outstripping competition in other ways. I do wonder whether 64k cycles is actually too low of an estimate though. Some functionality, particularly related to hashing and signatures, will almost certainly blow that right out of the water - or at least that was my impression from conversations on the topic in the past. It's not so much that the programs themselves are very complicated, but that they just rely on primitives that are going to be costly in terms of cycles - but I really have no good intuition for what is reasonable to expect on average here, particularly with Rust in the picture. The use case for |
Beta Was this translation helpful? Give feedback.
-
|
The number of cycles will vary by the use case, but here is my rough rule of thumb:
Important note: we really need async only for transaction proving, for block/batch proving we know all the data upfront and so, should be able to run in a sync mode (if one is available).
The way we plan to address these is via "precompiles" - basically, such computations would have dedicated circuits proven outside of the VM and we'd recursively verify them inside the VM (with recursive proof verification taking less than 64K cycles). |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
The main outcome of the call last week was to benchmark the cost of having the fast processor be async. The idea was that if it turns out that async is cheap then we could have the fast processor's async implementation be first class (and optionally provide a sync API that calls the async, perhaps using
tokio::runtime::block_on.I ran the
blake3_1to1,blake3_2to1andsha256benchmarks for the following three scenarios:FastProcessor::execute()is madeasync; everything else remainssyncexecute_mast_node()and its first-level children were madeasyncMastNodeis done executing. This would be needed e.g. in cases where the VM runs in the same thread that updates the UI.Results
Baseline timings (on
next):- blake3_1to1: 1.73ms
- blake3_2to1: 25us
- sha256: 60us
Number of cycles for each benchmark:
- blake3_1to1: 485k cycles
- blake3_2to1: 4.8k cycles
- sha256: 17k cycles
The results are as followed. Each is presented as a performance diff compared with the latest
nextbranch. For example,+ 10%means that the scenario being benchmarked is 10% slower than the current (sync) implementation innext.Scenario 1
Scenario 2
Scenario 3
Conclusion
Clearly, the performance hit is important enough so that we cannot afford to use
asyncwhen maximal performance is required (and interactivity isn't required).Beta Was this translation helpful? Give feedback.
All reactions