In episode 10, the Black-Cloaked envoy says (around 23:00) something about not being able to chase down Dixing people by himself on Haixing - the reasons being a) something about gravity? (I meant to ask for help with that line) and b) being able to release only half his power there.

So even before the whole life exchange thing, his power was limited on Haixing, and that could well be to do with the Guardian 'token'.