Add Non-Sample Anti-Aliasing (and/or similar scalable anti-aliasing techniques)

[Destranix]

Aliasing is a problem resulting from high-frequency signals being discretized with too low sample count. In today's image rendering, for anti-aliasing methods are usually used that query more samples per image area (so higher frequencies can be correctly discretized).
Nevertheless, these methods do not scale well. They only work as long as no higher frequency appears in the signal and require larger sample counts than higher the frequencies are.

There exist different methods that don't have this drawback. For example, in earlier years of OpenGL, GL_POLYGON_SMOOTH allowed rendering edges of triangles fuzzy, which smoothed transitions and thereby reduced frequencies at triangle edges.
The feature, though, was poorly implemented and lacked hardware support. Also, not being customizable, it was rarely used.

Another method is to, instead of sampling single fragments in the scene and aggregating these samples per pixel, average all parts of primitives that are covered by a pixel frustum part to calculate the resulting value.
In modern graphics hardware, this can be done by conservative rasterization. Conservative rasterization allows for every pixel to retrieve all primitives that have parts mapped to that pixel.
Additionally, auxilliar data, like how much of the pixels area is covered by a primitive or which primitive has the lowest/highest depth, can be retrieved using built-in functionality.

As every (visible) fragment in a pixel has an effect on the resulting pixel data and the data is applied to the whole pixel, regardless of the actual position of the fragment within the pixel, frequencies are smoothed out and less or no aliasing occurs.
(Also, the algorithm can be extended beyond a single pixel, further reducing aliasing.)
Alternatively viewed, every primitive extends smoothly beyond their edges, similar to an SDF, so no hard transitions between a primitive's edge and other geometry appear.

Along with anti-aliasing, conservative rasterization also allows for easy computation of other effects, like, e.g. special transparency effects or blurring.
Modern anti-aliasing can also be combined with different kinds of super-sampling methods (like sampling every fragment multiple times or perform modern anti-aliasing per sample instead per pixel).

Examples and Sources:
https://www.cg.tuwien.ac.at/research/publications/2013/Auzinger_2013_NSAA/Auzinger_2013_NSAA-Paper.pdf
https://cwyman.org/papers/i3d15_ftizb.pdf
https://patents.google.com/patent/EP2854108A2/en

Other anti-aliasing approach, but contains much Related Work:
https://dl.acm.org/doi/pdf/10.1145/3452097

[mrjustaguy]

I've looked at the paper (first link) and there's some very obvious problems with it. Didn't go through the other stuff as I don't have a lot of time right now

1. seems to only affect Geometry, much like MSAA, which is only half the aliasing in modern games
2. seems to be way worse compared to MSAA with it's memory and performance costs while offering very niche improvements.

[Calinou]

Kinda right. Texture anti-aliasing must still be performed with other methods. There do exist good and usable approaches though (namely mipmapping).

Mipmapping only resolves a small portion of aliasing, since most of it is shader-induced now (i.e. specular aliasing, or transparency aliasing).

Mipmapping has also been used for a long time in 3D graphics (it was introduced at roughly the same time we got hardware-accelerated 3D, almost 30 years ago now).

Additionally, if needed, one can take several samples of a fragment (what might sound like simple supersampling, but is superior, as it can be varied per model and samples are guranteed to hit geometry).

Selective supersampling is a thing, but it's seldom used in practice as it's still pretty expensive. Evaluating where the aliasing happens (and where supersampling is needed) is a big problem on its own, especially when you have multiple lights in the scene as the cost is per-light.

[Destranix]

Mipmapping only resolves a small portion of aliasing, since most of it is shader-induced now (i.e. specular aliasing, or transparency aliasing).

True. That's a problem with MSAA too though.

The (heuristic) Non-sample Anti-Aliasing approach aims for eliminating geometry aliasing as much as possible. Other aliasing still must be faced by different methods (or other variants of the same method).

Selective supersampling is a thing, but it's seldom used in practice as it's still pretty expensive. Evaluating where the aliasing happens (and where supersampling is needed) is a big problem on its own, especially when you have multiple lights in the scene as the cost is per-light.

To perform better (in terms of alias reduction) than standard MSAA or SSAA, it's already an advantage to be able to include the model information into the selection of sample positions.
For example, instead of randomly taking x additional samples at random positions within the pixel, one can decide to take these additional samples at models which one knows have high frequency data.

Also it should be noted: Non-Sample Anti Aliasing without additional samples taken is kinda comparable with an SSAA with sample count equaling the count of fragments for a pixel. The main difference is, that in (classical) NSAA every sample is on a different fragment, while in (classical) SSAA the samples usually hit near geometry first and discard far geometry.
Note that with conservative rasterization this can also be achived, one can handle layers differently depending on their depth and special functionallity exists to get the depth of the nearest and farest fragment.
This means, NSAA produces good results in reducing aliasing along the whole depth, while SSAA produces semi-good results in that scenario. At the same time, when only needing anti-aliasing in the first (or specific) layer, conservative rasterization does not exclude the use of MSAA or SSAA.

In fact, as soon as you either need as much samples for geometry anti-aliasing, that you also could handle every fragment instead, or when you create a fragment shader invocation for every fragment anyway, NSAA is clearly superior.
In soem scenarios we definitely are on that point already. In others we will be some day in the future.

[mrjustaguy]

Honestly, I don't see much point in bothering with NSAA. It's results are largely similar enough to MSAA while it's performance is questionable, and is a fairly significant time investment to try to get it integrated, and i Doubt you'll see anybody volunteer to implement it.

[Destranix]

and is a fairly significant time investment to try to get it integrated

I though that too on first sight, but actually it's not.
It's like instead of doing supersamlping, you sample every fragment per pixel the same way and weight it using a kernel function.

At least a first demo version (not including special effects) should be accomplishable within a fews hours.

It's results are largely similar enough to MSAA while it's performance is questionable

I doubt that. But I still couldn't tested a modern version.

[Destranix]

I thought a bit more about how exactly to implement NSAA using conservative rasterization. My first draft:
1.) Depth-only or Depth-and-Coverage Pass, to retrieve first hit per sample
2.) Conservative rasterization Pass. Overestimate primitives to match used weighting kernel.
In fragment shader, sample weight either by barycentric coords of the fragments primitive or by weight value passed from the vertex shader. Weight at the primitive should be 1, when the primitive was not hit the weight should reduce from 1 to 0 smoothly.
Retrieve an on-primitive sample position which is used for shading. This could be the nearest position on the primitive to the sample ray. The position can be retrieved like the weight by barycentric coords or by a attribute passed from previous shaders.
For all fragments smaller than the first hit, blend values together. Fragments after first hit are blended with reduced weights to preserve perceived occlusion (but they should not be discarded, otherwise aliasing occurs when the sample position moves beyond the first-hit primitives edge). The first-hit-fragment can be blended in a special way if needed. The reduction of weights for objects behind the first hit should happen smoothly, otherwise aliasing might occure.

Several primitives overlapping each others can be handled by considering all fragments in a certain depth range "first-hit" in terms of this algorithm and handling them accordingly.
Extension support exists for testing if a pixel is fully covered by a fragment. This might allow to simplify rendering for those pixels.
The amount of pixel area covered by a primitive can currently not be considered, as there is no extension support for it. With support though the algorithm could be improved.
Any kind of depth sorting opens possibilities for algorithms respecting the order of fragments more than just in terms of "first-hit" sorting.