I think I figured out the culprit behind my USB woes. After trying numerous different configurations on my Macbook Air, I have discovered that the DisplayLink driver does not get along well with Logitech's Control Center v. 3.9.4 system preferences software.
If I have the Logitech software installed without DisplayLink, everything's fine and the same the other way around. But if they are both installed, even if the USB to HDMI adapter is not plugged in to my computer, the USB ports stop working eventually (and sometimes I get logged out right after the ports fail).
So my solution was to uninstall Logitech's software and so far, I've been running for over a day now without the USB ports failing on me.
Just wanted to note that DisplayLink v. 3.1 does not work with Yosemite at all but 3.0 works fine.
Also the Mac DisplayLink support tool app that is found here -
http://support.displaylink.com/knowl...to-displaylink does not seem to work properly in Yosemite. I ran the app and let it sit for 30 minutes and the app never left the "Collecting system information. This process may take a few minutes to complete" screen.