I was the one to initiate the connection, and they only disappeared about 10 minutes after we closed the connection.
I tried restarting the video driver with Ctrl+Shift+Windows+B, tried restarting windows explorer from the task manager, but neither option fixed it. It just seemed to go away on its own after a while.
I have two monitors, incase that's relevant.
I have the latest Teamviewer updates, as well as an up to date windows installation.
Thanks in advance.