Unbinding GPU Driver Causes System To Freeze

When I try to unbind the nvidia driver, while it works, it ends up causing parts of the system to freeze. The console that I enter the command in also never ends up finishing. Does anyone have any idea how to fix this? Do I need to unbind something else beforehand?