Skip to content

[FEA] Hopper group-gemm of mixed type #1614

@jcao-ai

Description

@jcao-ai

Is your feature request related to a problem? Please describe.
For MoE inference, mixed type group-gemm is helpful. But now cutlass seems only support mixed type matmul, and group-gemm of non-mixed type.

Describe the solution you'd like
group-gemm of fp16_int4, fp16_int8, e4m3_int4

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions