Every time linux's kernel gets a significant update (which seems to be constantly) most of yoru drivers, particularly video and sound drivers have to be re-compiled.
Only if they are kept outside of the Linux kernel, which today would basically be NVidia's proprietary driver. And even if the Linux kernel is updated every 2 or 3 months you don't have to update to it. In fact, unless you are running a bleeding edge distro like I am (Arch Linux) and instead use Ubuntu, Mint, etc et al then this is no problem.
For me personally, Linux drivers work wonderfully, whenever I boot Linux on my machines it automatically sets correct native resolution, loads drivers for my network cards, wacom bamboo, even my NVidia 9800GT is supported out of the box through the excellent Nouveu open source drivers should I not run the proprietary NVidia drivers. It just works, out of the box.
Back when I was on Windows the first thing I had to do after installing was to manually download and install official NVidia/ATI/Wacom etc drivers for my machines in order to have my hardware fully functioning.
As for Linux not having a stable kernel interface, this is a conscious choice brought on by many reasons, not only does it give the developers much more flexibility but also the simple fact that from a developer standpoint proprietary drivers are simply inconvenient.
Particularly given that Linux aims to support tons of hardware 'out-of-the-box' across a wide range of cpu architectures for which it's simply incredibly naive to rely on the good will of companies to provide binary drivers. Not to mention locating fixing bugs, with open source drivers any developer with the appropriate knowledge can fix it, and they do.
So the Linux developers make it hard work to keep a proprietary driver as you will have to do the work of updating it against the changing kernel interface. However, on the flip side, if you release your driver as open source to be included in the kernel then the kernel devs will take care of the kernel interface maintenance for you, aswell as fix bugs, add features, and if possible tune the performance. And most importantly, your hardware will then run on all Linux supported targets out of the box.
This has led to a vast number of hardware companies biting the bullet and provide open source drivers (or the specs necessary to implement them), or even hire full time programmers to work continously on implementing and perfecting Linux in-tree drivers, like Intel for their Sandy/Ivy bridge architectures. They do this because while Linux certainly is small fry on the desktop (no argument there), it's huge in everything from servers, embedded, mobile, even totally dominating in super computers.
Even Microsoft(!) recently submitted open source driver code for Linux so that it would support their Hyper-V virtualization solution. Obviously they didn't do this out of altruism, but because their Hyper-V customers wanted to run Linux efficiently under virtualization.
No other operating system or kernel supports near as much hardware out of the box as Linux does, and it also supports all this hardware across the largest amount of cpu architectures supported by any system.
No, I'm not saying it's perfect, and from a pure desktop user perspective I can definatley see why people in general prefer Windows or OSX. It also has problematic areas like laptops and hibernation where certain hardware causes problems.
But it's also not bad as many of you guys are trying to make it out to be, and distros like Ubuntu make it alot more accessable for beginners.