Would you be able to gather logfiles that will show us a bit more about your current setup?
To do that follow the steps mentioned in the following article:
How to Report Macos Issues to DisplayLink
Also please tell me what is the exact brand and model of the DisplayLink powered device that you have.
Thank you.