Neural Net Voice filtering for OBS - Plugin in development & Live demo!

Hi Everyone.

As I have been doing more videos and streaming I have been working on a noise suppression filter for the human voice. Currently I have written this as an LV2 plugin, however, I am in the process of porting it into an OBS audio filter.

This is NOT a simple gate/expander/compressor setup, nor does it identify voice by differentiating the stereo channels. The input is first downmixed to mono before processing.

This plugin only incurs approximately 30ms of latency which is low enough for real-time monitoring without speech jamming.

Right now I am currently live re-streaming a friend (bamfsvamps) via the voice plugin.

~~Raw Unprocessed: Twitch
~~Processed Output: Twitch

Edit: Finished, but you can see the recording here:
The recording has expired on twitch.

Note the filter is extremely effective at removing even the game audio which is mixed into the output.

Edit: Also @bamfsvamps is using a laptop to play and is recording via the laptop mic, so the filter is working very hard to remove her keyboard noise. When using a more conventional setup (cheap headset w/mic) the key nose is entirely suppressed.

Note: This is NOT an open-source project and will be for sale when it is ready, price is TBD but I am aiming to keep it affordable for those with a limited budget and are just streaming for fun.

Edit: It has been released and is now available here: https://hostfission.com/gnifs-voice-filter

10 Likes

-1 for not being open source, but cool stuff!

1 Like

People have to make money somehow. Open source in most cases is not the way however.

2 Likes

You can, and many people do, make money on open source projects.

I agree it’s not as easy, but it’s better in the long run for everyone.

agree with @Randumb, I would say -4 for proprietary though

(can we get some :sadstallman: 's and :angrystallman: 's in the chat?)

However +1 for you thinking about the people, it’ll do well on Twitch

1 Like

Just a heads up, the following is explained on OBS Studio’s GitHub page.

“OBS Studio is an open source program licensed under the GPLv2. That means that if you create a plugin for OBS Studio or a modified version of OBS Studio and distribute it in binary form, you must give users a way to access the source code of your binary. Otherwise, you will be in violation of the license. The easiest way to do this is to have a public Github repository of your fork or plugin, and include a text file in your download for the binary containing a link to the repo. The relevant part of the license is under section 3 of the GPLv2.”

Even if you’re selling your OBS plugin, it has to be open source mate.

3 Likes

Remember open source != free software, you just have to license it right

https://dwheeler.com/essays/floss-license-slide.html

This write up also has most ideas for making money on open source projects

1 Like

Not in all cases. It is only if you need to make changes to OBS itself does the plugin need to be GPL compatible(ie open source). If the plugin works with binary versions of OBS, where you can download the binary, add in this plugin, then run it, the plugin does not need to be open source since you are not making changes to the GPL OBS source.

This is how the proprietary Linux Nvidia drivers work with the GPLv2 kernel, they have a few changes to the kernel itself which are compiled by the user, then they have a binary driver module that is not GPL.

Anyway, this looks really nice gnif, even if I personally do not have a use for it.

1 Like

Thanks guys.

Yeah, I know and I would love to open source it, but also need to be able to feed my family. Looking Glass has shown how generous people are when it comes to donations towards open source projects, however the donations barely cover the cost of time, let alone hardware.

I am sorry but this is simply not true, their interpretation of the GPL license is incorrect. It is completely legal to make a binary blob that OBS loads and uses, if this was not the case companies like NVidia who release a binary driver for the GPL’d Linux kernel would get smashed legally. Being binary compatible with a GPL project doesn’t mean you have to be GPL yourself.

Even if this were somehow true, it would be trivial to write an open source loader that is an interface between OBS and a binary blob that contains the IP.

Correct, but the target market (streamers) for the most part are very unlikely to care about a license if they can get it for “free”. If it was a company then bet your ass they would want to cover themselves and get that license, but most people just wouldn’t care.

2 Likes

A nn? What did you train it on?

Sounds like a cool project, I wish we had the sauce to play with :frowning: I hope you can find some way of monetizing it without making it proprietary in the end

Over 6GB of background noise from hundreds of locations recorded from hundreds of sources, from walking around the street with a mobile phone recording, to sitting in noisy offices.

I do too, I have actually been working on this for over a year now, while looking for a way to do this that would mean making it available to the public but still be able to justify it’s cost of development and support that it will entail. Nothing has come up short of making it cheap for people.

1 Like

Yeah, the only idea that comes to my mind is to sell it directly to Twitch and have them pay you for support rather than for the code itself.

1 Like

I have just released version 1.0.0 for Windows OBS 64-bit. This comes with a 30 day trial, no registration required.

https://hostfission.com/gnifs-voice-filter

Note: I have put considerable effort into reverse engineering and creating my own headers that are binary compatible with OBS to be sure that this plugin does not violate the OBS GPLv2 License. Since this is an audio plugin thankfully there wasn’t too much to re-engineer. As such this plugin is not a derived work and does not have to be open source.

1 Like

What’s the latest with this project? I have an outdoor mic setup for a webcam and I need to remove human voices. Running OBS on ubuntu. thank you

This was discontinued to due to lack of interest and better solutions now being available. I suggest you look at the RnNoise filter as a free alternative, or one of the other commercial offerings available.