If you restrict someone or their activities to one thing, they can only do, have, or deal with that thing. If you restrict them to one place, they cannot go anywhere else.
If you restrict something to a particular group, only that group can do it or have it. If you restrict something to a particular place, it is allowed only in that place.