Yes definitely Murray's for one. Great cheese available there and it is convenient. I've only been to the Grand Central Station location. You certainly should plan to spend a little time at Grand Central, fabulous bit of architecture and history that it is. I understand Murray's has a Greenwich Village location too. Probably worth stopping at both.
I hope some native NYers chime in to identify some shops off the beaten path for you.
Off subject, but we yarn hop when we're in NYC. If you are a knitter or into fiber arts you should make a point of stopping a Habu Textiles in midtown. Takako, the owner, imports all kinds of wildly exotic fiber products from Japan that you won't see anywhere else. Her store is hard to find, an unmarked location at 135 W 29th St # 804. Oh and School Products is right there at street level. Berta Karapetyan is the proprietress, she's there often. They are the importers of Karabella yarns, but they carry all kinds of other great stuff too.
So much to see in NYC. We'll love to hear about your trip.