KVM is full virtualization where the VPS operates as its own virtualised server, independently of the host node. All resources are fully virtualised which allows any Operating System to be run.
OpenVZ is a container (or OS-level) style of virtualisation that relies on the host node's kernel. It therefore can only run a selection of Linux-based Operating Systems.
KVM has no restrictions in terms of functionality, but it has more overhead than OpenVZ. OpenVZ is constrained by the host node kernel, but it has less overhead in the containers themselves. OpenVZ tends to have some compatibility issues with Docker (such as cgroups). KVM is generally more stable than OpenVZ.
It should be noted that KVM is not a dedicated environment, both virtualisation types share the host node's resources among all VPS.
