[Rust] Inference Server

[nikg4]

• Build an Oumi inference server in Rust w/ misc optimizations (data parallel?), and Oumi specific logic.
• Can include auth, rate limiting, quotas, billing, monitoring, tracing, etc for future Enterprise Platform i.e., control plane layer above pure inference/data plane.
• Built-in RAG support ?

Doc link: https://docs.google.com/document/d/1FcZYLK4ylSAogvnZNc0PMX_2PS-eyf4IP3qCjGuGBxI/edit?tab=t.0#bookmark=id.ppt46vrtgwd3