Noise coming from heaphones and speakers

im hearing a strange kind of background noise coming from my headphones when my computer is on.
Its some kind of electronic noise. Its not static or white noise im not really sure how to describe it. Doing things like moving the scroll wheel on my mouse can cause more noise to come up but not things like using the keyboard or anything like that.

Whats causing this background noise and is there a way to get rid of it?
I have the headphones plugged into the back of the computer through the motherboard.
It only seems to do it with some headphones. Does this mean that the headset im using is breaking? or is there some kind of issue with the computer and certain kinds of headsets?