Supported Networking Solutions

• Mirror
• FishNet

Other Unity networking solutions can easily be implemented by composing an agnostic MonoBehaviour and implementing a minimal interface — please feel free to contribute additional network provider implementations. Please make PRs with any of your changes or improvements if you feel they would be helpful to the public. Please create issues for things as you see fit.

Table of Contents

• Installation
• Notice for Unity Versions 2021 and Older!!!
• Information
• Video Tutorial
• Tutorial
• Tips
• Thank Yous
• Network Providers
• Missing Things
  • Network Providers
  • Features
• Extending Functionality
  • Network Provider
  • VcAudioInput
  • VcAudioOutput
  • VcInputFilter
  • VcOutputFilter
• Things Using MetaVoiceChat
• License
• Contact and Support
• Examples
• Changelog

Installation

1. Go to "Releases" and download and install the latest Unity package.
2. See Tutorial for additional steps.

Notice for Unity versions 2021 and older!!!

• For older versions of Unity that do not include the System.Memory namespace, you must use the legacy branch of MetaVoiceChat.
• Unity made this change somewhere in the 2021 versions of Unity.
• If you are using 2021, try the main branch first, but if you get errors, use the legacy branch.
• If you are using 2020 or earlier, use the legacy branch.

Information

• Simple
  • Default Unity microphone
  • No user code required for basic usage and completely self-contained
  • No complicated cloud services required -- everything just works with your existing networking library
• Configurable
  • Exposed Opus settings (defaults in bold)
    • Application: VOIP, Audio, or Restricted Low-Delay
    • Complexity: 0-10 (0 is less complex, while 10 is more complex)
    • Frame size: 10 ms, 20 ms, or 40 ms
    • Signal: Voice, Music, or Auto
  • General settings
    • Jitter calculation time window size in seconds
    • Jitter mean time offset calculation window size in updates
    • Output min buffer size in fractional frames
  • Audio pipeline system
    • VcAudioInput settings
      • First input filter (nullable)
    • VcInputFilter settings
      • Next input filter (nullable)
    • VcAudioOutput settings
      • First output filter (nullable)
    • VcOutputFilter settings
      • Next output filter (nullable)
  • VcAudioSourceOutput settings
    • Audio source
    • Buffer frame lifetime in seconds
    • Buffer max negative latency in seconds
    • Pitch proportional gain in percent per second of latency error
    • Pitch max correction in percent
  • Default input is the Unity microphone
  • Default output is a Unity Audio Source
• Functional
  • Optional realtime noise suppression using Vatsal Ambastha's RNNoise4Unity (see tutorial for usage)
  • Automatic microphone reconnection configurable with VcMicAudioInput.SetSelectedDevice(string device)
  • VcMicAudioInput OnActiveDeviceChanged event
  • Functionality and reactive properties with events for
    • Speaking
    • Deafening yourself
    • Input muting yourself
    • Output muting others
  • Unity microphone wrapper
  • Circular audio clip
  • RMS jitter calculation utility within a time window and mean time offset window using a mean deviation method
  • Thread-safe fixed length array pool utility
  • Serializable reactive property utility
  • Frame stopwatch utility
  • Microphone devices listener utility
• Modular
  • Abstract VcAudioInput and VcAudioOutput classes
  • Abstract VcInputFilter and VcOutputFilter pipelines
• Testable
  • Echo mode to playback your own voice
  • Sine wave voice override mode
  • Max codec milliseconds before warning
    • These warnings are limited to once per frame by default
• Details
  • No memory garbage created at runtime using pooled data buffers
  • Constants
    • 48kHz sampling frequency
    • Wideband Opus bandwidth
    • SILK Opus mode
    • Single audio channel
    • 16-bit audio
    • 1 second input and output audio clip loop time
  • Average latency is ~200 ms with default settings
  • Dynamic buffer latency compensation using a latency error P-controller with RMS jitter, sender FPS, server/host FPS, and receiver FPS adjustments
• Opus features
  • Variable bitrate encoding
  • Opus Official Website

Video Tutorial

https://youtu.be/2fSqSAnRS5M

Tutorial

1. Ensure you have properly completed the installation steps.
2. Create a "Voice Chat" or similarly named game object as a child of your networked player game object or other networked game object that you would like to have networked audio on.
3. Add the "MetaVc" component to the "Voice Chat" game object.
4. Add your desired audio input, such as "VcMicAudioInput" to the game object.
5. Likewise, add your desired audio output, such as "VcAudioSourceOutput" to the game object.
6. Connect your audio input and output to the "MetaVc" fields.
7. Lastly, add your desired network provider, such as "MirrorNetProvider" or "FishNetNetProvider" to the game object.
8. It should look something like this now:
   • All of these defaults should be okay, however you may want to decrease the "Complexity" field for games with many concurrently connected users or lower-end devices.
   • Remember to use "Max Codec Milliseconds" to ensure that your complexity value is not too high for your game.
     • Note: The codec performs sluggishly in bursts, so set this value higher than your desired frame period and ensure warnings are not being spammed and then increase it back to the max to disable it when you are done testing.
   • Use "Allow Multiple Codec Warnings Per Frame" to ensure you get the total frame time used by the codec when there are warnings.
9. Create a "Voice Chat Output" or similarly named Audio Source game object as a child of the networked game object. If this is a player, it should be in the mouth area.
10. Configure the output Audio Source
    • "Output" = the voice chat audio mixer group (optional)
    • "Play On Awake" is set to false internally, so don't worry
    • "Loop" is set to true internally, so don't worry
    • "Spacial Blend" = 1 for 3D proximity chat and 0 for normal voice chat
    • "3D Sound Settings"
      • "Doppler Level" is set to 0 internally, so don't worry (It must be 0 because of how Unity implements this)
      • "Max Distance" = ~50 meters or whatever you think is best
      • Ensure the volume roll-off curve's last data point has a volume of zero.
    • Your general settings should look like this:
    • Your 3D sound settings should look like this:
11. Connect your voice chat output Audio Source to the "Audio Source" field under the "Vc Audio Source Output" component.
12. Connect the "Meta Vc" fields under your audio input and output to the "Meta Vc" sibling component.
13. At this point, the "Voice Chat" game object should look like this:
14. You are done!!! See Examples for advanced usage.
15. Optionally, you can install Vatsal Ambastha's RNNoise4Unity to use the noise suppression RnnoiseVcInputFilter.
16. Be sure to follow the scoped registry setup instructions for RNNoise4Unity.
17. Uncomment the #define ENABLE_RNNOISE_FOR_META_VOICE_CHAT in RnnoiseVcInputFilter.
18. Set up RnnoiseVcInputFilter like this:
19. Now you are even more done! See tips below.

Tips

• Change Project Settings/Audio/DSP Buffer Size from "Best performance" to "Best latency"
• Apply input and output filters to audio inputs and outputs by using the first filter fields
• Chain together input and output filters to form pipelines by using the next filter fields
• Disable and enable input and output filters by doing filterComponent.enabled = false/true or use the inspector tick box
• A frame size of 10 ms is useful for achieving lower latency on higher-end devices with high network send rates for all users. A frame size of 40 ms is useful for optimization on lower-end devices and networks, but this negatively impacts ear-to-ear latency and worsens the audio quality when packets are dropped. A frame size of 20 ms is a good balance.

Thank Yous

A massive thank you to Vatsal Ambastha and his projects UniVoice and UniMic that were heavily referenced when starting this project in late 2023. His RNNoise4Unity project enables optional realtime noise suppression (separate package, see tutorial for usage).

Another thank you to Concentus: Opus for Everyone for their native C# implementation of Opus that makes it extremely easy to add Opus to projects like this.

Network Providers

• Mirror
• Fish-Net

Missing Things

Missing Network Providers

• Netcode for GameObjects
• LiteNetLib
• LiteEntitySystem
• Dark Rift 2
• Unity WebRTC
• Photon Unity Networking 2
• PurrNet

Missing Features

• Configurable sampling rates
• Multithreading for Opus
• Compared to Dissonance Voice Chat
  • Audio preprocessing
    • Dynamic range compression
    • Automatic gain control
  • Audio postprocessing
    • Acoustic echo cancellation
    • Soft clipping
    • Soft channel fade
  • Opus forward error correction (FEC)
  • Multiple chat rooms (you can implement these yourself with VcOutputFilters)

Extending Functionality

How do I write a network provider implementation?

• Reference the Mirror network provider implementation
• Also see the FishNet network provider implementation for another perspective
• For network solutions that do not have define symbols (i.e. Netcode for GameObjects), just make the PR with your own define symbol #if over each file. I will put a nested Unity package with a #define META_VOICE_CHAT_FOR_NGO script or whatever in the release so users can just double click the package to install the provider.
• Just make a PR and I will check it out! Thanks!

How do I write a VcAudioInput?

• Ideas for you: transmit an audio file or in-game audio
• Extend VcAudioInput and call SendAndFilterFrame(int index, float[] samples) whenever you have a frame ready to send
• index should start at zero and never reset
• samples array should be null when not sending audio data but still sending empty heartbeats that maintain the timing
• Reference VcMicAudioInput

How do I write a VcAudioOutput?

• Ideas for you: save audio or do speech-to-text
• Extend VcAudioOutput and implement ReceiveFrame(int index, float[] samples, float targetLatency)
• Reference VcAudioSourceOutput

How do I write a VcInputFilter?

• Ideas for you: implement Push-To-Talk by setting samples to null when disabled
• Extend VcInputFilter and implement Filter(int index, ref float[] samples)
• Setting the samples array to null will stop the pipeline and signal that the samples should not be sent. The incoming samples array may be null.

How do I write a VcOutputFilter?

• Ideas for you: implement dynamic player mouth movement by listening to the RMS of incoming samples
• Extend VcOutputFilter and implement Filter(int index, float[] samples, float targetLatency)
• Directly modify the samples array to achieve the desired filter. The incoming samples array may be null.

Things Using MetaVoiceChat

• MirrorVR
• My game Tater Tussle

License

• This project is licensed under the MIT License
• You are technically permitted to sell this project exactly as it is under the MIT license, but please don't. Just share this project for free with others that would like to use it.
• Now go and make great games, whether for profit or free, using this! Please share links to your projects -- I would love to see what you all make!

Contact and Support

• Discord: metater
• Support in my Discord server: https://discord.gg/k4ZtGAA2Nt
  • Just post in #meta-voice-chat-support and ping me