Increase the default number of the autotuner generations in python bindings.

[ftynse]

TensorComprehensions/tensor_comprehensions/tc_unit.py

Line 192 in 4798154

self, pop_size=10, crossover_rate=80, mutation_rate=7, generations=2,

By default, the autotuner in python runs 2 generations with very small population size. This is largely insufficient, even "some check" tests run longer.

@prigoyal @nicolasvasilache @ttheodor

• bump up the defaults in tc_unit.py
• modify python unit tests to still use old settings to have faster CI testing

[thetheodor]

This doesn't make sense at all. It should definitely be bumped up.

[prigoyal]

Hi all, thanks for the discussion. What settings would you like to have for these options? concrete numbers would help.

cc @soumith @hmansell

[prigoyal]

assigning to @ftynse and @ttheodor for providing some concrete numbers on the settings. I can make the required changes then. Thank you :)

[ftynse]

I would recommend a benchmark-centered approach. We run tuning on scale once and analyze after how many generations the performance increases only at a small rate, and pick that value.

However, I am unable to run tuning on scale and neither is @ttheodor AFAIK.

[prigoyal]

that sounds reasonable. assigning to Nicolas as well for providing some stats. :)

[fsword73]

During the evaluation of SGEMM, MNK from 128 to 4096. The best performance is only 18% of Peakflops. Most SGEMM's performance is less than 5% PeakFlops.

It will be great with big polulation size and generations.

A demo autotuner of SGEMM with effective 80% Peak Flops will bring out real confidence to encourage developer to integrate TC as backend. Or Even demo tuning process of SGEMM will be execellent enough

[ftynse]

@fsword73

Performance of sgemm is not likely to increase significantly however long you run the TC autotuner.

We need to perform several other code transformations than we currently do to speed up gemm more. Note that gemm is the most hand-optimized kernel in history. For the vast majority of sizes, library versions do a good job on it. So if your library version runs fast, you are better off just using it. In fact, we are going to match library calls instead of generating code whenever we deem this more efficient. We believe it should not stop framework integration of TC. One does not have to express all layers in TC, especially not those where achieved performance is already good enough.

[prigoyal]

very exciting to hear about the library call matching. @ftynse , I was curious if we have heuristics or ideas around how to detect whether a kernel for some sizes will be more efficient in a library or TC should do codegen. Do we have thoughts on that? :) Thanks.

[nicolasvasilache]

@prigoyal it shouldn't be a surprise, we've been discussing this for a few months already :) Reg. implementation of the decision, a prereq is to have proper modeling and a roofline estimate. #198 is a first step in that direction.

[prigoyal]

thanks @nicolasvasilache, definitely not a surprise to me :) I am rather curious about the details/ideas there and if this is happening anytime soon. Thanks for pointer to #198. I will wait out for a roadmap to be available likely for more information :) thanks again.