I used to get 2 gallons of raw milk a week from a local farmer. I was living alone at the time, and did not make cheese then. My gluttony and love of dairy could not compete with that volume of milk, so a couple of times, I attempted to freeze the excess. On thawing, it was always disappointing -- flavor degradation, watery texture, and general unpleasantness all around.
And this was just for drinking and cooking. For cheese, I imagine the changes would be even more devastating.
I also recall reading in Harold McGee's On Food and Cooking around that time that milk is very durable when heated/boiled, but reacts very poorly to freezing; I believe, as Wayne mentioned, that it comes down to the formation of ice crystals, which destroy or damage the micelles.
There's a reason you can buy boxed milk, condensed milk, evaporated milk, and a million other variations on preserved milk, but not frozen milk. It just doesn't work that well.