If you've been following my social media for the past few years, or have read my Recording or Streaming on NVIDIA Turing/Ampere guides, I'm always chasing the next higher level of "perfection". And this time I was chasing the lowest possible latency on Twitch - and it appears that I have finally found it, after days of trying.

My search began as usual, with a manual investigation into the problem - I didn't have infinite resources to throw at the problem, and I didn't want to increase Twitch's hosting costs either. Therefore I was stuck with manual testing, which worked out well enough in the end for me, and didn't take much time either. I started my testing with the Key-Frame Interval, going down from 5 Seconds to 1 Second in 1 Second interval.

Oddly enough, I noticed that certain Intervals would end up "grouped together" into a higher latency. For example, 1 Second and 2 Seconds would end up being 3 Seconds of latency, while 3 Seconds and 4 Seconds would end up being 5 Seconds of latency, and 5 Seconds ended up being 7 Seconds of latency. Unable to make sense of it, I moved on to other settings.

Look-Ahead was the first one to recieve the axe, and it made no difference - 30 Frames or 0 Frames, both had the same latency. Similar to this was Zero-Latency and B-Frames, neither made a difference. But one option did, and it is what every RTMP+HLS based service recommends against: Adaptive I-Frames, or Scene Cut in x264 terms.

Both x264 and NVENC use this option to insert complete Key-Frames if they happen to be a better option, for example with a fade to black, or a complete cut to new content. Unfortunately this option results in a measurable and visible increase in latency between a 250 and 500ms - half a Second gone to unfortunate Key-Frame placement.

This brought the latency ladder down to ~2.4 Seconds for a Key-Frame Interval of 1 and 2 Seconds, ~4.2 Seconds for an Interval of 3 and 4, and ~5.6 Seconds for an Interval of 5. Careful observers may have just noticed the same thing I did, which is that the latency appears to climb in ~1.5 Second intervals - and that's what I tested next.

I set up the Key-Frame Interval to 1.5 Seconds, and then I watched the Video Stats...

I finally had it. Almost a perfectly flat 1 Second latency for the stream itself, though with all the Rendering, Encoding and Muxing buffering, it still ended up being roughly 2 Seconds. Which is still less than the lowest total latency I managed with Adaptive I-Frames being Enabled, and any other Key-Frame Interval. I repeated this test several times, and was able to achieve the same latency over several hours in Firefox, Chrome and Edge. I am a happy Twitch streamer now.

Disclaimer: This discovery is not guaranteed to work everywhere, and may be unique to the Ingest server I used. It may even be different due to changing which GPU is used, which CPU is used, which RAM is used, how your network looks like, etc. It is provided at no guarantee or warranty.