Speed up Warning propagation

[GregoryTravis]

I recently made a change to Vector.Builder to always propagate warnings, which resulted in a measurable slowdown in some cases. (I then added a flag to disable warning propagation, for times when it is not needed.)

Since Vector.Builder is used everywhere, I spent some time on benchmarking and optimization, but concluded that it might be more useful to think about speed up Warning propagation in general.

Agenda:

• Do we think the warning system needs to be sped up?
• Do we think it's possible to speed it up?
• What are some ideas to do so?

I have only one idea for this:

Don’t attach warnings to values

• gather warnings during the evaluation of a node
• optionally attach values to the warnings (instead of warnings to the values)
• propagate a node’s warnings to downstream nodes (but do not propagate a value’s warnings to every downstream value)

Please add other ideas here.

[JaroslavTulach]

• Migrate WithWarnings to use EnsoHashMap to speed them up significantly #8682

[JaroslavTulach]

Another related issue that @hubertp reminded us about is

• HostObject interop with wrapping TruffleObject oracle/graal#4916

The issue wasn't originally created to speed things up. I have no estimates, how much performance we could gain. The original goal was to improve consistency when calling into hosted Java code.

[radeusgd]

• propagate a node’s warnings to downstream nodes (but do not propagate a value’s warnings to every downstream value)

I'm not sure if I understand the distinction here. How would the propagation to 'downstream nodes' work in any other way than by attaching to values?

[JaroslavTulach]

The current warnings system in Enso reminds me of value taint tracking as described for example in Towards efficient, multi-language dynamic taint analysis - I do understand the implication of value taint tracking, I have a clear mental model of it and there are scientific papers describing how it should work and how it can be implemented using Truffle. Except the above also nodeperf work may be interesting.

gather warnings during the evaluation of a node
optionally attach values to the warnings (instead of warnings to the values)
propagate a node’s warnings to downstream nodes
(but do not propagate a value’s warnings to every downstream value)

I have to admit, I don't have a mental model of how this shall work and why. Some ThreadLocal?

[GregoryTravis]

Meeting summary:

• The first thing to do is Migrate WithWarnings to use EnsoHashMap to speed them up significantly #8682, since it shouldn't be too hard and may speed things up enormously
• Additional things to consider
  • Implement the wrapping in GraalVM itself (HostObject interop with wrapping TruffleObject oracle/graal#4916)
  • Doing propagation at a node granularity rather than at all function applications, possibly following taint analysis

[JaroslavTulach]

Please note that

• WithWarnings uses EnsoHashMap to speed things up #10555

by @Akirathan has just been integrated. If the speedup isn't enough, then we need another brainstorming meeting.