I'm wondering if the Guardian token effect only happens if you're traveling legally, as it were. Unauthorized border crossings don't get the regulation imposed power-limitation and are then more dangerous.

That's an interesting theory! I could buy that, but if it's something that only gets conditionally imposed and not automatically, I then wonder why in moments of crisis (like...Ye Zun!) they can't to go to whoever controls this and say hey, can we maybe let the Envoy have his full powers for a bit to help with this problem?