ARMv6? I doubt KitKat can work on ARMv6 devices.
Yes, there is no driver support. Check out the IRC log. We use old drivers with hacks to make it work with KitKat. The most problematic is the graphics (latest version is for android 4.2) and OMX driver (latest is for 4.1). Remember the green youtube things, tvcatchup bugs etc...
Now youtube HQ playback is still "blocky". We use the old webkit rendering engine instead of chromium webview. etc.
There is a lot of changes and hacks that make this old hardware work with KitKat.
Dazzozo have done the most of the work.
LG P500 is ARMv6 and CM11 works fine on it.