Thoughts and frustrations. And then some notes on quality comparisons between different hardware for encoding.
I’ve been messing around a whole lot recently with OBS, for streaming. With a particular interest in “low” bitrate streaming, under 4,000kbps.
My experiences with OBS have been strange and frustrating (Not blaming them for it. Simply noting my experiences). The biggest issue, was something I was only able to quantify after hours of experimentation and vague comparison to articles and videos of streaming knowledge, from others.
I have been using a Radeon HD7870 for the past 7 years. Even though that GPU family is the basis for two of the current consoles and both consoles can stream: something has happened on the PC side. It seems the drivers are antiquated, in certain respects.
In short, on my system, I just could not reproduce the general quality and speed which I’ve been seeing in comparison articles and videos. As well as lurking forum posts and even actual streams. It didn’t matter if I was using the CPU for encoding, Quicksync, or the encoder chip in my videocard. All of them seemed to be under performing. I could not do a 720p 60fps stream/record, in any game, with Quicksync, CPU, or GPU. Unless I used some of the absolute lowest quality settings. Which looked awful. Even downscaling was itself a performance hit, offsetting some of the speed benefit I should have been getting, from encoding a lower resolution.
All of my testing suggested that OBS was having relatively more difficult time interrupting the frames from the videocard, in order to encode them, add overlays, downscale, show the preview window, etc. Simply disabling the preview window, gave me a notable boost in performance. Once upon a time when Dark Souls 1 and 2 were new games on PC, I could used DSFix and GeDoSaTo to downscale, with basically zero performance from the scaling, itself.
It seems that either the way the drivers for these old cards is antiquated, or OBS isn’t coded well for the way they present their frames. or a combo of the two.
I basically confirmed my thoughts, because I just installed an RTX 2060 yesterday and my CPU performance in OBS is WAY better. Like twice as good. Streaming at 720p/60 is pretty much no issue, at realistic quality settings. Even 1080p is now viable. I’m now aligned with the performance suggested by all those videos and articles which I have looked at. And you can run Soul Caliber 6 at full speed on a single core. So, its nothing to do with the games I’m using, over taxing my hardware. Dark Souls 2 runs at 720p/40 on my laptop with 2013 Intel Graphics. So, my desktop barely leaves idle temps, with that game. But I couldn’t do a 60fps stream, until I swapped my videocard for something modern.
Now, about quality. I’m interested in “low” bitrates for streaming. Under 4000kbps. I’ve been doing most of my testing around 3000kbps. These low bitrates really expose the compression capability of each different piece of hardware. In OBS, you can encode your stream on your CPU cores, via Intel Quicksync, via Nvidia GPU, and via AMD GPU. They all look different, have different performance considerations, have different settings for quality, and can vary (sometimes a lot) between different games.
Before anything else, whatever encoder you use, you should be using “high” profile. Unless you are very concerned about your viewers using older smartphones/tablets/TV apps which only support “Main” profile or even only “low” for really old stuff. “High” profile enables some options which help with compression/quality per bit. Which I have found to be noticeable, even over “Main”. Not as large a concern at higher bitrates. But at lower bitrates, it seems to be a thing.
X.264/CPU encoding: In terms of varying between different games, X.264 is the most agnostic/has the least variance. Generally speaking, even using realistic settings for a quad core, its the best compression per bit and the most consistent quality at lower bitrates. However, the image quality tends to be on the soft side. Unless you start stacking some special quality options. But, X.264 tends to show less macroblocking, at any given bitrate. Especially if you use “faster” CPU preset or better. So, what it may lack in sharpness, it balances with a more stable image.
X.264 is super mature and has a ton of tweaks for quality, which can further maximize compression per bit and detail. It also has a bunch of settings which you can use to tune for different games (darkly lit/dark colors Vs. brighter, etc). However, the basic presets and “tune” options are very good and I would recommend just using those unless you have a specific game which you are really trying to cater towards or have a very specific quality issue to overcome. There are also some things you can turn off, to gain a lot of extra performance to keep a stable 60fps on your stream. As someone who has spent hours and hours laboring over this stuff, Even at 6,000kbps, there’s only so much visual quality you can gain. After a point, its more about speed.
In particular, I have found the “SSIM” tune and the “Animation” tunes to be very good for visual quality in games. or having no tune option selected at all. These should be good for most things. However, feel free to ping me in this thread, for suggestions and tweaks.
The potential problems with encoding X.264 on your CPU is that it can be a large load on your CPU. So, trying to play a game and encode at the same time, can be a tough balancing act. If the game is particularly demanding, you may not have enough performance to share between the game and the stream. And as you add options to X.264, it gets slower.
The faster paced a game is, the more difficult it is to compress quickly. More motion to track per frame, demands more hardware performance. And of course the higher compression quality you try to use, the more power you need from the CPU. Which can affect even games which don’t require much from the CPU. This is why many streamers have a separate PC to handle the encoding, or are using CPUs with more than 4 cores. So that they don’t have to compromise on quality options.
And if X.264 has a visual weakness, its with really fast motion or visual affects which affect the entire screen (so, lots of motion). Stuff like full screen ripple effects, underwater distortion, etc, exposes some visual weakness in X.264. It can struggle to keep these visuals looking smooth and defined. Often negatively affecting the clarity of whatever is behind these effects, in the frame.
AMD’s GPU hardware encoders: AMD has a few issues with their hardware encoding. First off, they keep changing their encoder hardware. Sometimes it gets a little better, sometimes it gets worse. Its a strange thing, that. As such, OBS doesn’t cater well to AMD encoders. I think in general, but also because AMD presents a constantly moving target, with their hardware.
I have no idea what the quality is like on many of their cards. On my 7870 at the afore stated lower bitrates, there is a lot of granular macroblocking. However, with really fast motion, it tends to focus more of the bitrate to the center of the screen, where your character is. Therefore, even amongst all of the blocking, there can be a pleasing solidity to your character under fast motion. Also, it seems to do well with those full screen effects such as ripples, which I said X.264/CPU can struggle with. And its good with alpha effects. But its not good at all, for slower paced games. The encoder seems to have a lot of trouble in locking down fine details which should be viable in a slower paced game. There are actually a lot of settings exposed for my 7870, OBS. However, it seems OBS struggles to access its framebuffer to actually do the encoding at acceptable speeds.
AMD just released the new 5700 videocards. They supposedly have a new and improved encoder chip on them. However, preliminary tests have awful visual quality. I suspect this may be due to streaming software not yet taking specific advantage of the new chip. I think they are probably just using the old AMD code paths to access the new chip. It remains to be seen how this will end up. AMD hasn’t been too vocal about it, either. They only briefly mentioned the new encoder chip, at E3, for example. *One of AMD’s VP’s has heard this from youtubers/streamers and put a team on improving the situation.
Intel Quicksync (revised opinion)
Quicksync is on most of Intel CPUs. Its a separate portion of the CPU, from the main CPU cores. Using it does use some resources from your CPU cores (it uses system RAM as VRAM. CPU time is needed to swap that data), but, not as much CPU as using the X.264 CPU encoder. On a quad core, its like half the CPU resources. Quicksync is quite fast. You should be able to do 60fps captures at least at 1080p.
Its visual quality can vary noticeably, from game to game or even different in-game lighting conditions. Due to this, it can be worth testing on the game you are trying to stream. to see if it is a good match. Whereas X.264 is generally pretty consistent across different visuals. Something really nice about Quicksync, is that the overal image quality retains a lot of detail and sharpness. Generally more than NVENC. And you’d have to really tweak X.264 and use a preset better than (Faster) to achieve such sharpness and overall detail. It also compresses colors more accurately than NVENC.
The big problem with Quicksync is that it doesn’t handle really fast motion well. And motion in general, can cause a strange combing effect on certain aspects of the picture. Usually flat areas with less details and more brightness. the sharpness of its image quality also makes blocking more apparent than say, X.264. Quicksync generally needs relatively more bitrate, to handle faster motion without losing details in the motion and to minimize the combing effect. However, once you hit 6,000kbps bitrate, the negatives are minimized pretty well. And the overall sharpness, color, and detail, is really nice. I came away pretty surprised after my tests today. And may find myself using Quicksync more often.
I recommend Quicksync for slower paced games or games where you really want to show off the details. I think even something like Bioshock or Halo, would be slowish enough, that the negatives wouldn’t push you toward a different encoder (although the combing effect can be distracting if your particular game shows it more). It would should be excellent for adventure games, TCG card games, etc. I would recommend trying it for slow paced games, rather than messing with X.264. The detail and sharpness is just so easily attained. You have to work a little harder, with X.264
NVENC encoder via Nvidia Graphics cards.
I just got my first Nvidia card, since like 2007. I’ve only had it for a couple of days. But already, testing the NVENC encoder has been pretty interesting.
First thing to note is that from what I have read and seen, Nvidia cards before the Geforce 10** series (1060, 1070, 1080), have noticeably lower visual quality per bit, for encoding. And Geforce 10** and 20** (1650,1660, 2060, 2070, 2080) have better compression quality and very similar in quality to eachother. Its not a large jump from 10** series to 20** series. The newer 20** series do have a small bump in overall sharpness of details. Otherwise, they are very similar in encoding quality.
Using NVENC has been interesting, because it exhibits a much wider range of behavior, from game to game. Final Fantasy 15 (I used the benchmark demo) seems to be a best case scenario. Where I got as good of results as you could ever expect. The compression quality NVENC does on that game, is remarkable.
However, Obduction, is just ok. Losing to X.264 in compression per bit, there can be some more “muddy” parts of the screen, with distant objects. Quicksync again has more detail and better colors. But the combing still shows up. NVENC’s weakness with colors really showed up with Obduction, as there are many areas in the game with little color variety to distract you from the inaccurate presentation.
Dark Souls 2 faired similar, overall. Motion on your character is a bit better with NVENC. But, lots of blocking in the skies and “flat” less detailed areas of the screen. Easily fixed with X.264 via CPU.
Soul Caliber 6 is interesting as well. NVENC does a really great job of keeping a solid visual feel to the characters and the high amount of screen motion for that game. Better than X.264 CPU, in many respects. Characters appear more “solid” and with more “depth” while in motion. And full screen visual effects and fades fair much better. Overall detail is lower. But the trade for the solidity and clarity in motion, is a preferrable advantage for something like Soul Caliber. And likely for a faster paced FPS, as well. I think it probably has algorithms which detect motion and prioritize bits to those parts of the screen.
Comparing frame by frame during high motion with NVENC: SC6 characters have a lot more frames where the characters appear high quality Vs. Quicksync and even X.264. those two encoders tend to pixelate as soon as motion gets really fast. You have to pump up the options to overcome it, with X.264. Quicksyncs only answer is more bitrate. As soon as you stop moving a lot, Quicksync puts more details into the stages. But for this game, NVENC is the superior choice. Even at 6000kbps, I think NVENC has the edge. Due to the importance of motion for this game.
However, in Soul Caliber 6, the skies and clouds do show a fair amount of blocking with NVENC. And if you look at your character frame by frame, the encoder sometimes culls chunks of your character. Like segments of their arms or weapons can be completely missing in still frames. Which I have not noticed with the other encoders. But you don’t notice when actually playing. Its interesting and says a few things about how to allocate screen details, for actually perceived visual quality.
I tested it on Dirt 3 with a snow course, compared to Quicksync and X.264. Quicksync had the highest general detail. Especially on the track surface and on the cars themselves. But the snow blasting behind the cars had that combing effect and some blockyness. X.264 had a softness to it, but had the least blocking. NVENC actually had more overall detail compared to X.264. Due to its positive abilities with high motion. Trackside details had more detail. Snow blasting behind cards looked natural and clear. Cars looked solid.
OBS and Nvidia seem to be working well together. As they have recently added a new codepath for NVENC and a couple of new quality features. As such, NVENC is very fast and even cranking all of the settings available, I’m not sure you could dip under 60fps stream for 1080p or lower. Also, this partnership should also mean using X.264 on your CPU will be as fast as possible. Because OBS is optimized to interrupt Nvidia framebuffers to encode the frames, ad overlays, etc.
NVENC is interesting competition for CPU encoding. Its too inconsistent from game to game, to say it outright matches X.264 CPU. But in maybe many scenarios, it can be about as good, while demanding much less resources from your system. Trading some visual quality, for a large amount of performance. And in specific scenarios (fast motion), it can actually be favorable in visual quality. I have no idea what sort of emphasis Nvidia has for R&D on GPU encoding. But, there’s so much shading power on these things, it seems like they could take this a lot further, if they really wanted. We’ll see!
Overall, I think OBS could do a lot more to guide users on choosing settings. Some of the options have decent explanations. Some do not. And some options don’t really seem to do much. Tying to find out about these things is a really painful process. I hope they find some time to invest in improving documentation and presentation, within the software itself. For example, Handbrake has a fairly extensive compendium on the options they have available. Their issue is that the language is often very technical. Can be tough to follow, even as far as how to activate the settings.
I miiiiiiight post some comparisons. We’ll see.