If you're buying the milk, it may be the dairy's fault (well, it's pretty unavoidable). When dealing with milk in bulk, it tends to be agitated just enough for bits of butterfat to come out, but it's really only obvious when you make the cheese and it floats on top of the whey.
Pumping the milk into the bulk tank, agitating the milk in the bulk tank, stirring while pasteurizing - all of these steps contribute to a small amount of butterfat leaving the milk. That's why homogenization became standard practice.
If it's from your own cow, sometimes even the force of hand milking (squirting milk into a bucket) will be enough to work butterfat out.
Blame it on the high bf% of Jerseys! (and the huge fat globules)