LAN Only - Very sluggish at times

I can't find a rhyme or reason for this.

Sometimes it works great.  I can see my other PC, and when I attach a monitor, I can go through team viewer and control the actual PC through team viewer without noticing latency.

Othertimes, it's a solid second behind.  I have it set to LAN only, connect directly via IP..  Sound turned off and optimized for speed.

RDP Session works well, and I planne don switching.  But it can't use my digital authentication via RDP for a few sites.

Any suggestions on where to begin troubleshootring?