For heavier puppets, I use a ball-and-socket rig. I have a mounting point on the puppet armature, where I can screw in a short rod with a steel ball on it. The rig has a pair of plates that grip the ball. It is usually attached to a flat wooden base that I can screw or clamp onto the set, out of sight.
For lighter, simpler puppets, I just use a rig made of several strands of 3mm armature wire. Sometimes some 1.5mm wire as well, if the puppet is very small. I sand the ends of 2 wires to a point and stab them into the back the puppet, or the side away from camera. Like this: