-
one widely used codec for (realtime) audio appoications is the OPUS Codec
- most applications use this coded inside RTP = Real-Time Transport Protocoll, which then uses UDP = User Datagram Protocoll
-
many things (eg Discord) use audio frames of 20ms (they transmit audio every 20ms)
- why 20ms…????
-
my Audio Video Streaming Issues, Linux Audio and Video Performance Analyzing and Optimizing
How to deal with Packet Loss…
- do you replace the lost time with silence, Weißes Rauschen
- you can request to resend the lost data … NACK = Negative Acknoledgement
- produces latency…
- a thing called: Inband FEC
- it’s the default thing to do with OPUS Codec
- “you send a reduced quality version of the last packet inside your current package…”
- uses RTCP = RTP Control Protocol fraction loss
- i think you use the metric of how many packets get lost, to determine, how much “duplicate sending” you do
- a thing called RED for redundancy
- you just send the last n packets in every packet…
- or there is NetEq, which is a “jitter buffer and packet loss concealer” from google
- it basically repeats the last 20ms at a different Amplitude
- even better, you use generative AI = Artificial Intelligence to make up the missing data
- used in Microsoft Teams and Google’s WaveNetEq which is used in Google Duo
there is SDP = Session Description Protocol