As I understand it, the wood that is most neutral with respect to food is beech. My next choice would be a hard maple. I'd be worried that cherry would impart some flavor, along with others such as oak or walnut. And as Al says, stay away from soft woods.
I have a couple of followers made out of laminated bamboo, which seems to be pretty neutral with respect to flavor -- but even then I don't put these right next to the cheese; I use these wood followers only to back up a plastic follower.