Those drivers are loaded as "kernel objects" (external libraries with a .ko extension) and thus not part of the GPLed code. The upshot of that is that you can install and uninstall drivers without having to recompile the kernel and even hot load drivers without having to reboot.
That all said, you can patch non-GPL code into GPL code. You're just not allowed to redistribute the result.
Edited by Plan9 - 12/13/12 at 1:53pm