USB sound choppy, USB bus over-saturated?
 

I'm running quad monitor using 2 HP HSTNN-S01X DisplayLink interfaces on a 2010 21.5" iMac Core i3-550 3.2GHz running 10.7.5.

Monitor 1: Built-in
Monitor 2: 1920x1200 DisplayPort
Monitor 3: 1600x1200 HSTNN-S01X DisplayLink
Monitor 4: 1600x1200 HSTNN-S01X DisplayLink

For the most part, everything works perfectly. However, I have USB sound OR mouse issues if I run both DisplayLink adapters at the same time. No problems whatsoever if I just stick to 3 monitors (1 DL adapter).

I use USB sound on one of the HSTNNs because I like to be lazy and keep headphones plugged into my Mac while the USB sound drives my speakers. If I plug speakers into the Mac and use the headphone port on the speakers, I lose the ability to keep separate volume profiles as the Mac will see them as headphones.

This is my USB arrangement:
http://i2.photobucket.com/albums/y15...psd2ae82f7.png

Looks like the iMac has 2 separate USB busses. I had to put both HSTNNs on the same bus, because if I try putting one on the same bus as the keyboard and mouse, then the mouse will jump around and become unusuable. However, I think having to put them on the same bus might be the cause of my USB sound problem.

Is there any way I can rearrange my USB connections to maximize throughput to the DisplayLink adapters without affecting keyboard and mouse? Or am I stuck having to choose between 3 displays with working USB sound, or 4 displays with unusable USB sound?