Only enable VPN for certain users (disable remote control)

Hi,

I was wondering if the following is possible:

Is it possible to give some family members access to my shared folders on my server (Windows 10), but I don't want them to remote control the server itself (given that the password to connect to the vpn is the same as the one for remote control).

Can I disable the Remote Control feature for certain users?
Thanks!