Can I restrain "point to select"?

Hi guys,
be healthy and happy everybody!
Sorry for an offtopic question, but I hope competent people here could help me.
I use "single-click to open, point to select" setting in File Explorer. It's very handy, but when the file list changes (for example, when entering another folder or pressing Home/End etc.), unwanted select happens, just because some (random) file happened to appear under the mouse pointer.
Is it possible to do that when the mouse didn't move, auto-select with a pointer wouldn't happen?
Or, any guesses how to achieve that?
Thank you in advance. :)