Right? He always treats it so carefully and tenderly, and gets angry when anyone tries to touch it. Why would that matter if there's no one inside?

I just had a thought - I wonder if it's that he and Nianzhi had corresponding puppets, and he wasn't allowed to take Nianzhi's with him to Haixing, so now every time he sees his own he's reminded of Nianzhi's?