With StreamFX's NVENC you can achieve the quality targets in one of two ways: Constant Quantization Parameters (CQP) or Variable Bitrate with Constant Quality (VBR-CQ). Both have their own advantages and downsides, and it depends entirely on your requirements which one you should use. Modern NVIDIA GPUs (Pascal and newer) also all support 4:4:4 and 4:2:0 encoding at most of the quality levels shown here.
Constant Quantization Parameter (CQP)
Of the two methods available, CQP is by far the faster method, with hardly any noticable impact on other parts of the GPU. It also supports vastly higher framerates compared to the alternative method, thanks to NVIDIA optimizing their presets further. With the "Fastest (P1)" preset it is now possible to record 1920x1080 NV12 at nearly 460 FPS on an RTX 3080 or higher. Unfortunately none of my PCI-E SSDs, not even the PCI-E 4.0 from Samsung, managed to withstand the amount of data requested at once, so the exact upper limit is unknown.
| Option | Lossless | Near Lossless | Indistinguishable | High Quality | Acceptable Quality |
|---|---|---|---|---|---|
| Preset | Fastest (P1) | ||||
| Tune | Lossless | High Quality | |||
| H264 | |||||
| Profile | High 4:4:4 Predictive | High 4:4:4 Predictive (when using I444) High (when using NV12) |
|||
| Level | Automatic | ||||
| H265 | |||||
| Profile | Main | ||||
| Tier | High | ||||
| Level | Automatic | ||||
| Rate Control Options | |||||
| Mode | Constant Quantization Parameter | ||||
| Multi-Pass | Single Pass | ||||
| Look Ahead | 0 frames | ||||
| Adaptive I-Frames | Disabled | ||||
| Adaptive B-Frames | Disabled | ||||
| Quantization Parameters | |||||
| I-Frame QP P-Frame QP B-Frame QP |
0 | 0 to 4 | 5 to 10 | 11 to 16 | 17 to 22 |
| Adaptive Quantization | |||||
| Spatial Adaptive Quantization | Disabled | ||||
| Temporal Adaptive Quantization | Disabled | ||||
| Other Options | |||||
| Maximum B-Frames | Any | ||||
| B-Frame Reference Mode | Any | ||||
| Zero Latency | Default | ||||
| Weighted Prediction | Default | ||||
| Non-reference P-Frames | Default | ||||
| Reference Frames | -1 frames | ||||
| Low Delay Key-Frame Scale | -1 | ||||
| Key Frames | |||||
| Interval Type | Frames | Frames Seconds |
|||
| Interval | 1 frame | 1 frame 0.5 seconds |
|||
| OBS Studio Advanced Video Settings | |||||
| Color Format | I444 | I444 or NV12 | |||
| Color Space | Identical to Input (sRGB on PC, Bt.709/sRGB for Console) | ||||
| Color Range | Identical to Input (Full on PC, Partial on Console) | ||||
Variable Bitrate with Constant Quality (VBR-CQ)
The second method for recording is VBR-CQ, which is similar to x264's CRF but not a perfect match. While x264's CRF will improve with higher presets and lower CRF values, NVENC's VBR-CQ has a flat ceiling that simply can't be bypassed, no matter what settings are applied. Still, it has its uses for Indistinguishable, High quality and Acceptable Quality recordings. Performance wise it is far slower and has much more impact on the machine, so it should not be used for Single-PC setups.
| Option | Indistinguishable | High Quality | Acceptable Quality | ||
|---|---|---|---|---|---|
| Preset | Medium (P4) or slower | ||||
| Tune | High Quality | ||||
| H264 | |||||
| Profile | High 4:4:4 Predictive (when using I444) High (when using NV12) |
||||
| Level | Automatic | ||||
| H265 | |||||
| Profile | Main | ||||
| Tier | High | ||||
| Level | Automatic | ||||
| Rate Control Options | |||||
| Mode | Variable Bitrate | ||||
| Multi-Pass | Two Pass at Quarter Resolution or: Two Pass at Full Resolution |
||||
| Look Ahead | 32 frames | ||||
| Adaptive I-Frames | Enabled | ||||
| Adaptive B-Frames | Enabled | ||||
| Limits | |||||
| Target Quality | 1.00 to 10.99 | 11.00 to 17.99 | 17.00 to 22.99 | ||
| Target Bitrate | 0 kbit/s | ||||
| Maximum Bitrate | 0 kbit/s | ||||
| Buffer Size | 0 kbit | ||||
| Quantization Parameters | |||||
| Minimum QP Maximum QP |
-1 | ||||
| I-Frame QP P-Frame QP B-Frame QP |
-1 | ||||
| Adaptive Quantization | |||||
| Spatial Adaptive Quantization | Enabled | ||||
| Spatial Adaptive Quantization Strength | 1 (Lowest) to 15 (Highest) | ||||
| Temporal Adaptive Quantization | Enabled | ||||
| Other Options | |||||
| Maximum B-Frames | As many as your GPU supports | ||||
| B-Frame Reference Mode | Each if supported Half if supported None in any other case |
||||
| Zero Latency | Default | ||||
| Weighted Prediction | Default | ||||
| Non-reference P-Frames | Enabled | ||||
| Reference Frames | As many as your GPU supports | ||||
| Low Delay Key-Frame Scale | -1 | ||||
| Key Frames | |||||
| Interval Type | Seconds | ||||
| Interval | 1.00 seconds or more | ||||
| OBS Studio Advanced Video Settings | |||||
| Color Format | I444 or NV12 | ||||
| Color Space | Identical to Input (sRGB on PC, Bt.709/sRGB for Console) | ||||
| Color Range | Identical to Input (Full on PC, Partial on Console) | ||||
Additional Information
- Multi-Pass at Full Resolution is a new option, and has measurably increased quality in VBR-CQ. However, it is very expensive and should be the first option changed if performance is a problem.
- Look Ahead in VBR-CQ can be freely adjusted as needed, as long as it is not lower than the number of B-Frames and Reference Frames.
- Adaptive B-Frames is required for VBR-CQ, as B-Frames can both increase and decrease quality if used in the wrong place.
- Stronger Spatial Adaptive Quantization will focus more bitrate towards visually complex areas, but may starve smoother areas of the bitrate it needs.
- The Key Frames Interval can be freely adjusted for anything but Lossless, as I-only encoding is required to guarantee the least data loss.
- The old presets were deprecated by NVIDIA and now map to the new P1-P7 presets, which come with massive performance or quality boosts. High Quality is now Medium (P4), while High Performance is closer to Faster (P2).