Calling Action twice as expensive as method. Why?

[maxima120]

I am trying to compare performance of different ways to execute code.
Surprisingly using action is twice as expensive as using method. I dont understand why. Could someone please explain?

The rationale behind is that I need to find best way to propagate changes from a server using an API to WPF. This is high-throughput, near real-time app. The API in question is being built and I have some freedom to influence its design.

    internal class Program
    {
        const int runs = 10;
        static void Main(string[] args)
        {
            var bc = new BenchContainer();
            bc.Init();

            // warm up
            Run("NoMethodRunner", bc.NoMethodRunner, 1);
            Run("MethodRunner", bc.MethodRunner, 1);
            Run("ActionRunner", bc.ActionRunner, 1);
            Run("EventRunner", bc.EventRunner, 1);

            Console.WriteLine("------------------------------------------------------");

            Run("NoMethodRunner", bc.NoMethodRunner, runs);
            Run("MethodRunner", bc.MethodRunner, runs);
            Run("ActionRunner", bc.ActionRunner, runs);
            Run("EventRunner", bc.EventRunner, runs);
        }

        private static void Run(string title, Func<double> runner, int runs)
        {
            Stopwatch sw = new Stopwatch();
            for (int i = 0; i < runs; i++)
            {
                sw.Start();
                var res = runner();
                sw.Stop();
                Console.WriteLine(res);
            }
            sw.Stop();
            Console.WriteLine("{0}: {1}", title, sw.Elapsed);
        }
    }

    public class BenchContainer
    {
        const int steps = 10000000;
        public double[] data = new double[steps];

        public void Init()
        {
            var rnd = new Random((int)DateTime.Now.Ticks);
            for (int i = 0; i < steps; i++) { data[i] = rnd.NextDouble(); }

            Action = (d1, d2) => d1 + d2;
        }

        public Func<double, double, double> Action;

        public double NoMethodRunner()
        {
            double res = 0;

            for (int i = 0; i < data.Length - 1; i++)
            {
                res += data[i] + data[i + 1];
            }

            return res;
        }
        public double MethodRunner()
        {
            double res = 0;

            for (int i = 0; i < data.Length - 1; i++)
            {
                res += Method(data[i], data[i + 1]);
            }

            return res;
        }

        public double ActionRunner()
        {
            double res = 0;

            for (int i = 0; i < data.Length - 1; i++)
            {
                res += Action(data[i], data[i + 1]);
            }

            return res;
        }

        public double EventRunner()
        {
            eventRes = 0;

            Event += BenchContainer_Event;

            for (int i = 0; i < data.Length - 1; i++)
            {
                EventMethod(data[i], data[i + 1]);
            }

            Event -= BenchContainer_Event;

            return eventRes;
        }

        //

        double eventRes = 0;
        private void BenchContainer_Event(object sender, EventArgs e)
        {
            eventRes += (double)sender;
        }

        public double Method(double d1, double d2)
        {
            return d1 + d2;
        }

        public event EventHandler Event;

        public void EventMethod(double d1, double d2)
        {
            if(Event != null) { Event(d1 + d2, EventArgs.Empty); }
        }

    }

[gfoidl]

For benchmarks please use something like https://benchmarkdotnet.org that accounts for JIT-compilation, warm-up, etc. The results are much more precise than measuring the way you have shown.

In you setup the "method" is simple, and will very likely be inlined by the JIT. Whereas the "action" (actually a Func-delegate) won't be inlined (or at least not inlined in the same sense as the "method" will be). So the comparison is not really valid.

If the code in method is larger or if you put [MethodImpl(MethodImplOptions.NoInlining)] to the method, the numbers should become closer.

---

propagate changes from a server using an API to WPF

Did you think of using SignalR for such cases?

[maxima120]

I added NoInlining and it is the same:

when I said "propagate changes from a server using an API to WPF" - I mean my immediate concern is to propagate changes inside the same OS process / .NET app context - eg after update message received from the middleware inside the app layers

[hez2010]

It is expected that an indirect call is more expensive than a direct call.

[AndyAyersMS]

PGO (now enabled by default in 8 Preview 5) will diminish the cost of the ActionRunner case somewhat. But not quite as cheap as it ought to be -- it should end up comparable to the first two cases:

NoMethodRunner: 00:00:00.1478415
MethodRunner: 00:00:00.1346791
ActionRunner: 00:00:00.2021358
EventRunner: 00:00:00.6097093

For more realistic tests I would expect it to do pretty well.

@jakobbotsch can you look into why we don't clone the loop in ActionRunner for method GDV (and/or clone for bounds checks?). We end up with an OSR method so perhaps something is messing up loop recognition?

[jakobbotsch]

The delegate instance is accessed inside the loop as a class field that could conceivably be modified by the GDV fallback case, so I don't think there's much we could do there. Perhaps if we used deopt/reopt for the fallback case we could have a chance to get cases like this with a bit of work.

If the delegate is pulled into a local before the loop we do manage to loop clone.

[maxima120]

Thank you guys. I didn't expect devs to pick this one up.. Very grateful.

If the delegate is pulled into a local before the loop

You mean like public event EventHandler MyEvent;... var evt = MyEvent; evt(this, args); ?

[jakobbotsch]

Not quite, I mean if the benchmark is changed to

        public double ActionRunner()
        {
            double res = 0;

            var action = Action;
            for (int i = 0; i < data.Length - 1; i++)
            {
                res += action(data[i], data[i + 1]);
            }

            return res;
        }

In that case it should be on par with the NoMethodRunner and MethodRunner benchmarks -- the JIT is then able to check once before the loop that action is pointing to the (d1, d2) => d1 + d2 lambda and effectively generate the same loop as in the cases without a lambda.

That might not be too relevant for your actual use case, assuming the event/action is not invoked inside a loop.

[maxima120]

Nope. Didnt help.
I modified ActionRunner and it takes twice as longer anyway (1000 steps)

        public double ActionRunner()
        {
            double res = 0;
            var action = Action;
            for (int i = 0; i < data.Length - 1; i++)
            {
                res += action(data[i], data[i + 1]);
            }

            return res;
        }

NoMethodRunner: 00:00:09.6840595 (00:00:00.0096841)
MethodRunner  : 00:00:08.9670192 (00:00:00.0089670)
ActionRunner  : 00:00:19.3809619 (00:00:00.0193810)   

I also tried PGO - not sure if this is the correct way (ChatGPT suggested it):

dotnet build -c Release /p:UseProfileGuidedOptimization=true
dotnet run -c Release /p:UseProfileGuidedOptimization=true

[jakobbotsch]

Tiered PGO is only enabled in .NET 8 preview 5 by default. If you are on an older version then you need to enable it manually. You can follow these instructions: https://learn.microsoft.com/en-us/dotnet/core/runtime-config/compilation#profile-guided-optimization

Note that we do not support the loop cloning optimization discussed above in .NET 7, so you really need a .NET 8 preview to see the biggest possible improvement. But you should still see a good improvement in .NET 7 with PGO enabled.

[maxima120]

yes it helped. With tiered PGO it runs like this:

NoMethodRunner: 00:00:09.2565476 (00:00:00.0092565)
MethodRunner  : 00:00:09.1726117 (00:00:00.0091726)
ActionRunner  : 00:00:14.6210306 (00:00:00.0146210)  

and if swap steps and runs to make it steps = 1000 and runs = 10000000 which is more realistic scenario as I can have 1000 subscribers easily and run it 10000000 times in a minute - the results are even better:

NoMethodRunner: 00:00:07.3648234 (00:00:00.0000007)
MethodRunner  : 00:00:07.2732654 (00:00:00.0000007)
ActionRunner  : 00:00:09.6677880 (00:00:00.0000010)