It's a soft polyurethane foam, often used in upholstery and cushions. I usually get it from Clark Rubber, but that's in Australia and I don't know if there is anything equivalent in whatever country you live in. You may also find it at large fabric stores, or at specialist upholsterer's suppliers.
Edit: I checked your profile, and I see you are in the US. Here's a place, with 2 locations in California: http://www.foamorder.com/foam/upholstery-foam.html I googled "upholstery cut foam USA" to find it.
I've always ordered my upholstery foam from onlinefabricstore.net, but smaller quantities can be found on Amazon (free shipping) or at Home Depot. This is the particular stuff that I order: https://www.onlinefabricstore.net/1-x-24-x-54-upholstery-foam-.htm
I ordered this from ebay.
I was looking for 1/4 sheets but this is the closest I could find.
In addition, try one or two layers of foam under-wrap (also known as pre-wrap or athletic-wrap?) on the outermost surface can help smooth-out the skin. This can reduce bumps showing through underneath the puppet costume or latex build-up. It's a similar sort of sponge foam to furniture/upholstery foam, but more stretchy because it's so thin. The plain foam type doesn't have a woven surface or contain latex/elastic. It comes on a roll about 50-60mm wide, and is less than 1mm thick. I couldn't find it in specialty sports shops in Australia, so imported some from eBay.
Hope that helps,
i just use crochet yarn.
Your local upholstery repair shop probably has plenty to give away.
I also get tons of that stuff from discarded furniture I find in the streets. They don't always smell great, but reusing non-reciclable materials is a civic duty