Actually — I just managed to find a useful option. Go to Window - "Capturing and Exclusion list" tab - Disable "Capture windows". Solved my issue, when I open the control panel separately it opens in a new window. Probably breaks some other functionality though.
Update - found an alternative. In that same window, keep "Capture windows" enabled, but at the bottom in "Excluded folder paths" just add the Control Panel. Now it works as normal except the Control Panel will always open in a separate window. Unfortunately this doesn't seem to work when launching subsettings from e.g. search, without manually adding every single option under Control Panel. Would be nice if there were an 'all subfolders' option so it could work like this.