I'm thinking of doing a small overclock on my FX-8320(Maybe go for 4GHZ) without changing the voltage. But what are the disadvantages of doing this? The performance is not strictly needed (doing it for a bit of fun tbh, I'm happy with my gaming performance).
I have a CM Hyper 212 Evo cooler and an Asus M5A97 evo R2.0
None really. There is a slight increase in power consumption and heat, but the voltage contributes way more to power and heat compared to clock speed. It should be hardly noticeable.
If you were on a stock cooler and you max stable overclocked without raising the voltage you might lower lifespan a little but with a decent cooler like you have you should be okay. You could have gone a little bigger with the cooler but you should be okay.
With my 8120 I get to 4.2ghz before I have to raise the voltage and stock is like 1.31v for me. And it passes AIDA64s stability test.