This is a long standing known issue of Mavericks. Unfortunately it's not clear why some systems are susceptible to the issue and some aren't.
As a workaround you could try to disable some full screen transition effects that are connected to some of these Window Server crashes.
Please see
http://www.displaylink.org/forum/showthread.php?t=63239
Carlo