Need help with improving stream quality in OBS

bedHedd · August 8, 2018, 5:46am

I want to improve the quality of my stream so that the video is sharper.

Do I need a hardware upgrade or is there something in the software I need to change?

The Resolution I would like to hit is 1920x1080 60fps streaming to Youtube.

output stream example gameplay at 9m9s

when I watch it non full screen it is nice and sharp, but when I view the video in full screen it becomes blurrier

Quality I am looking for

Also I am getting an issue of an error of

 encoding overloaded! consider turning down video settings or using a faster encoding

when I alt tab out of any full screen application

output log

00:33:23.458: CPU Name: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
00:33:23.458: CPU Speed: 3299MHz
00:33:23.458: Physical Cores: 6, Logical Cores: 12
00:33:23.458: Physical Memory: 16307MB Total, 10712MB Free
00:33:23.458: Windows Version: 6.3 Build 9600 (revision: 17415; 64-bit)
00:33:23.458: Running as administrator: false
00:33:23.458: Aero is Enabled (Aero is always on for windows 8 and above)
00:33:23.459: Sec. Software Status:
00:33:23.459: 	Windows Defender: enabled (AV)
00:33:23.460: 	Windows Firewall: enabled (FW)
00:33:23.460: 	Windows Defender: enabled (ASW)
00:33:23.460: Portable mode: false
00:33:23.554: OBS 21.1.0 (64bit, windows)
00:33:23.554: ---------------------------------
00:33:23.554: ---------------------------------
00:33:23.554: audio settings reset:
00:33:23.554: 	samples per sec: 48000
00:33:23.554: 	speakers:        2
00:33:23.556: ---------------------------------
00:33:23.556: Initializing D3D11...
00:33:23.556: Available Video Adapters: 
00:33:23.556: 	Adapter 1: AMD Radeon (TM) R9 Fury Series
00:33:23.556: 	  Dedicated VRAM: 4266143744
00:33:23.556: 	  Shared VRAM:    4026531840
00:33:23.556: 	  output 1: pos={0, 0}, size={1920, 1080}, attached=true
00:33:23.556: 	  output 2: pos={1920, -756}, size={3840, 2160}, attached=true
00:33:23.558: Loading up D3D11 on adapter AMD Radeon (TM) R9 Fury Series (0)
00:33:23.565: D3D11 loaded successfully, feature level used: 45056
00:33:24.255: ---------------------------------
00:33:24.255: video settings reset:
00:33:24.255: 	base resolution:   1920x1080
00:33:24.255: 	output resolution: 1920x1080
00:33:24.255: 	downscale filter:  Lanczos
00:33:24.255: 	fps:               60/1
00:33:24.255: 	format:            NV12
00:33:24.255: 	YUV mode:          601/Partial
00:33:24.256: Audio monitoring device:
00:33:24.256: 	name: Default
00:33:24.256: 	id: default
00:33:24.256: ---------------------------------
00:33:24.257: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/chrome_elf.dll' not found, loading of module failed
00:33:24.258: [CoreAudio encoder]: CoreAudio AAC encoder not installed on the system or couldn't be loaded
00:33:24.459: [AMF] The AMF Runtime is very old and unsupported, consider updating your drivers.
00:33:24.459: [AMF] Version 2.3.3 loaded (Compiled: 1.4.4.0, Runtime: 1.4.2.0, Library: 1;4;2;0;17.10.1731;201704242122;CL#1401971).
00:33:24.499: [AMF] <Id: 2> Unable to create H265/HEVC encoder, error AMF_ENCODER_NOT_PRESENT (code 36)
00:33:24.533: [AMF] <Id: 4> Unable to create H265/HEVC encoder, error AMF_ENCODER_NOT_PRESENT (code 36)
00:33:24.533: [AMF] [H265/HEVC] Not supported by any GPU, disabling...
00:33:24.547: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/libcef.dll' not found, loading of module failed
00:33:24.548: LoadLibrary failed for '../../obs-plugins/64bit/libEGL.dll': The specified procedure could not be found.
00:33:24.548:  (127)
00:33:24.548: Module '../../obs-plugins/64bit/libEGL.dll' not loaded
00:33:24.548: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/libGLESv2.dll' not found, loading of module failed
00:33:24.549: [browser_source: 'Version: 1.31.0']
00:33:24.552: LoadLibrary failed for 'nvEncodeAPI64.dll': The specified module could not be found.
00:33:24.552:  (126)
00:33:24.756: Couldn't find VLC installation, VLC video source disabled
00:33:24.759: No blackmagic support
00:33:24.762: ---------------------------------
00:33:24.762:   Loaded Modules:
00:33:24.762:     win-wasapi.dll
00:33:24.762:     win-mf.dll
00:33:24.762:     win-dshow.dll
00:33:24.762:     win-decklink.dll
00:33:24.762:     win-capture.dll
00:33:24.762:     vlc-video.dll
00:33:24.762:     text-freetype2.dll
00:33:24.762:     rtmp-services.dll
00:33:24.762:     obs-x264.dll
00:33:24.762:     obs-vst.dll
00:33:24.762:     obs-transitions.dll
00:33:24.762:     obs-text.dll
00:33:24.762:     obs-qsv11.dll
00:33:24.762:     obs-outputs.dll
00:33:24.762:     obs-filters.dll
00:33:24.762:     obs-ffmpeg.dll
00:33:24.762:     obs-browser.dll
00:33:24.762:     image-source.dll
00:33:24.762:     frontend-tools.dll
00:33:24.762:     enc-amf.dll
00:33:24.762:     coreaudio-encoder.dll
00:33:24.762: ---------------------------------
00:33:24.762: ==== Startup complete ===============================================
00:33:24.766: All scene data cleared
00:33:24.766: ------------------------------------------------
00:33:24.779: WASAPI: Device 'VoiceMeeter Input (VB-Audio VoiceMeeter VAIO)' initialized
00:33:24.787: WASAPI: Device 'Hi-Fi Cable Input (VB-Audio Hi-Fi Cable)' initialized
00:33:24.799: WASAPI: Device 'VoiceMeeter Aux Output (VB-Audio VoiceMeeter AUX VAIO)' initialized
00:33:24.813: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds
00:33:25.375: Switched to scene 'Thanks for tuning in'
00:33:25.375: ------------------------------------------------
00:33:25.375: Loaded scenes:
00:33:25.375: - scene 'Scene':
00:33:25.375:     - source: 'Quake Champions' (game_capture)
00:33:25.375:     - source: 'Overwatch' (game_capture)
00:33:25.375: - scene 'starting/ending':
00:33:25.375:     - source: 'static' (image_source)
00:33:25.375:     - source: 'Stream Starting Soon' (text_gdiplus)
00:33:25.375: - scene 'be back':
00:33:25.375:     - source: 'BRB' (image_source)
00:33:25.375: - scene 'Thanks for tuning in':
00:33:25.375:     - source: 'thanks' (image_source)
00:33:25.375: ------------------------------------------------
00:33:25.570: adding 21 milliseconds of audio buffering, total audio buffering is now 42 milliseconds
00:35:58.224: Settings changed (outputs)
00:35:58.224: ------------------------------------------------

tkoham · August 8, 2018, 5:53am

iirc the fiji builtin encoder isn’t capable of that bitrate at 1080p60

change the bitrate to something like 3500 or reduce the framerate/reso

if you really really want lossless encoding pick up something like a BMD mini recorder or intensity shuttle and duplicate your display out

bedHedd · August 8, 2018, 6:25am

Can I encode with x264 on my CPU and still hit the framerate/resolution

Do you have any recommendations for PCI cards? Would I just use a display port cable from my graphics card and plug it into the pci recorder card(if I got a pci card)? Would the recorder cause any performance gains as could potentially free up gpu/CPU resources?

Is there any difference between a pci capture card And a mini recorder?

tkoham · August 8, 2018, 6:41am

can’t remember what codecs broadwell supports in hardware but you should have 2 cores or so to play with either way

not many people stream 1080/60 lossless with builtin hardware because it’s generally extremely hard to do.

the intensity pro or pro 4k, or the mini capture or capture 4k.

you’d need hdmi output but I think you can passively adapt that

yeah that’s kinda the point (if your cpu/gpu use hw encoding it doesn’t degrade perf to use them but they can’t do lossless except in software typically)

a few yeah but OBS can use either, the mini recorder IS a pcie card

Honestly though if you have a usb3.1 port free the intensity is a perfectly valid option, no pcie slot needed.

FurryJackman · August 8, 2018, 8:25am

What you could try is NDI from your gaming machine to a secondary machine with a GT 1030 GDDR5 and NVENC. You don’t even need that powerful of a processor in that case. A G4560 would do for NVENC.

mihawk90 · August 8, 2018, 5:27pm

Theoretically yes, but 1080p60 takes a lot of resources encoding JIT. You probably wouldn’t be able to game alongside it. You are already getting an encoding overload. The only thing you can do to reduce the overload is either reduce quality directly, or use a faster encoding setting which generally also reduces quality.

Generally for that kind of resolution it’s recommended to have either more then 4C/8T or use a secondary streaming PC. Either via capture card or an nginx setup.

That doesn’t really work. OBS doesn’t use the hardware encoder of any kind of capture card currently as far as I know (maybe blackmagic cards, idk), because there’s no driver support and/or no implementation for it. To use the hardware encoder you need to use their software. Also the hardware encoder is generally not as good quality wise as x264.

At that point the capture card (in the same PC) just becomes a burden as it also just loads the CPU, and the CPU would need to encode anyway. There is absolutely no benefit to a capture card in the same PC unless you can use the hardware encoder, which - again - you can’t with OBS.

Yes, that would work, either with a capture card and then use NVENC in OBS (could even stream consoles at that point), or with an nginx setup. But again, hardware encoders are generally not as good as software encoders.

For information on what “nginx setup” means:

Also I don’t know what bitrate YouTube supports, but that could also be a limiting factor there.
If you have a set limit on bitrate there’s only 2 variables to play with:

resolution
encoding speed

Since your resolution (and fps) is already targeted you really only have encoding speed left. For the best encoding you would want the slowest encoding speed setting, but that of course takes more resources then the fast settings. And since you’re already getting an encoding overload you would need a faster setting then you already have, reducing quality.

FurryJackman · August 8, 2018, 6:00pm

Price to performance vs X264, and if the bitrate is high enough, NVENC still wins.

I recommend you get a Startech USB3HDCAP if you want to skip the NDI method. It’s a legendary HDMI/DVI/VGA AND Component capture card.

A H370 motherboard, a G4560 and a GT 1030 GDDR5 and the USB3HDCAP is a very cheap but good in terms of NVENC solution.

If you want to use Linux though, I’d recommend the Decklink Mini Recorder 4K and a H370 board. For Windows, the Startech is enough though. And you’ll need Windows for Streamlabs Stream Labels, cause they refuse to support Linux until Streamlabs OBS hits 1.0 release.

tkoham · August 8, 2018, 6:30pm

if you do want to offload the streaming a prebuilt appliance might be the best route, but honestly lossless isn’t that important because twitch transcodes down for anyone without a great internet connection. 3-5 meg is definitely “good enough”

lossless compressed bitrate for 1080p60 is from like 90-150mbit, not something you’re gonna be able to upload anyway.

here are some prebuilts to consider:

mihawk90 · August 8, 2018, 9:26pm

Yes, I don’t dispute that. However there are limits on how high you can go on bitrate on a stream

That is… not entirely true. They do that, yes, but not with every channel. You need a certain amount of regular views or they won’t provide that option.

tkoham · August 8, 2018, 9:36pm

so then streaming lossless makes even less sense. If only people with 10+ down and low latency can watch you then you’re limiting your audience.

SgtAwesomesauce · August 8, 2018, 9:37pm

That’s why most people do 720p/30 or 1080p/24.

mihawk90 · August 8, 2018, 9:40pm

Exactly. To start out at least.

When you finally get the option you can crank it up all the way and don’t care anymore.

tkoham · August 8, 2018, 9:40pm

again though, lossless 1080p60 is like 90-150 mbit

it’s a nontrivial expense to make that happen

most places in the US don’t offer that rate up at home, and like half the places in the US don’t offer it down

SgtAwesomesauce · August 8, 2018, 9:41pm

Part of it is there’s a trade off between CPU power and transcode quality for a given bitrate.

Throw more CPU at it and you’ve got more options.

bedHedd · August 10, 2018, 6:54am

I get it when i alt tab out of a application. Once I am in a game there is no issues what so ever.

So finding a used 5960x or 6900k could potentially solve the issue?

So from what I have read in the responses in this thread there aren’t any available hardware encoders with direct support on OBS.

The general consensus is to build a separate dedicated streaming computer for a lossless stream, which looks like a great idea for if I decide to become a pro-streamer.

YouTube supports streaming 1080p 60fps with a CRF of 23. The video is transcoded, and I am fine with it. As long as the output is better than my current I will take it.

I don’t think I worded my op clearly, but my goal is to figure out what is causing the quality in the example clips.

Is it a issue with my CPU/GPU where I need to upgrade to newer hardware that can support a stream and game play? Most streamers(Quake Champions and Overwatch) run a stream off a single system with a 7700k with a GTX 1080.

Is it a issue of YouTube compressing the video to a lower quality? Videos in the YouTube mobile app are sharper than on desktop (possibly due to pixel density on a phone and using firefox as my YouTube viewer).

Or is it a issue with how Quake Champions is optimized and that results in lower quality? I run the game on low settings and just barely get 100 fps. It’s wierd because other players run it in lower end hardware and get better framerates ranging from 140-180 fps on higher settings.

It can be a combination of all, but what I really want to know is if my current hardware is the limitation.

Is there a good reading or video guides for streaming and hardware selection?

tkoham · August 10, 2018, 7:50am

Like I said, you’ve got your bitrate set too high for the AMD hardware. Fiji’s native recorder just isn’t capable of 1080p60 at that bitrate. It will just underrun to try to keep up with the parameters you’re giving it and output very bad quality footage.

mihawk90 · August 10, 2018, 8:20am

I really don’t know what your definition of “lossless” is. But “lossless” doesn’t mean what you think it means. “lossless” would be uncompressed RGB, which I doubt anyone with a sane mind would be streaming.
I think what you mean is just a high quality stream.
h.264/x264 is by the very definition a lossy codec. The same goes for audio. You’re usually streaming AAC in OBS.

Also yes, especially since you can throw everything at it. You could even stream console games easily (would just need to setup a scene for grabbing a capture card).

Honestly I don’t even see an issue in your example clips. The stream looks perfectly fine, a lot better then most OW streams actually. If you mean the blurriness on some edges, that’s perfectly normal for a livestream.
This isn’t a video encoding where the encoder has all the time in the world to figure out the perfect image, this is a time-critical encoding so the encoder takes some shortcuts.
If you want to get rid of that you need to throw an insane amount of computational power at it.
Also keep in mind that both Twitch and YouTube re-encode your stream if the ingest server isn’t happy with your settings. So check their documentation for good settings.

Looking at both your videos you’re anywhere between 110 and 200 fps…
I mean, you’re streaming so of course you’ll be getting lower FPS then without a stream. And if you’re comparing with people that run a 7700K (probably also OC’d), then a) you’re missing clock speed because 7700K is 4.20 GHz base, and b) you’re missing 2C/4T. So of course you’ll be having lower FPS because the CPU is busy with encoding.

Twitch Docs and OBS forum

FurryJackman · August 10, 2018, 1:03pm

Basically, for good 1080p60 with no pixelation, you need X264 at medium/slow preset at 7.5Mbps on a Threadripper 1950X. That is Robbaz’s setup, a Swedish streamer that does a lot of KSP and My Summer Car. He uses dual system with a Elgato 4K 60 on the encode machine.

tkoham · August 10, 2018, 11:43pm

actually it means any file (compressed or otherwise) that when decoded is the same as the original. You can get lossless prores and dnxHD, that are significantly smaller in size than raw YUV or png sequences.

bedHedd · July 29, 2019, 12:43am

This issue can be solved if you set the process priority to anything above normal

I also upgraded to a 2700X and a RTX 2070, so I am pretty happy with quality and streamerside performance.

I tried using x264 with the 2700X, stream was fine, but in game framerates tanked below what I wanted. The new NVENC is stupidly amazing.

If you are building a new PC and you plan to stream frequently, you should stick with a Turing gpu (either pick up a used rtx 20XX/60/70/80 or buy a new super version) just because the encoder and current setup are better.

I hope AMD can get their encoder together on the 5700 and 5700XT. @eposvox mentioned in his AMF video AMD reaching out to him for improving the encoder. I have my fingers crossed and look forward to his videos covering the improvements.