If you use good clean practices when milking and handling the milk you should be fine .
To lessen the chances of contamination try to use the milk every 2 to 3 days , also the lower the ph the less likely that any listeria will servive.
Listeria facts ...... 5 percent of humans are carriers of listeria , listeria multiples better than any other bacteria at refrigeration temps. Listeria is every where in the environment and we are exposed to it every day , usually only unhealthy or immune compromised people are at any risk.