That would be late blowing then. Late blowing occurs between the 3rd and 6th week. There's a higher concentration of spores in the cheese caused by clostridium in your milk or your surroundings.
In order to prevent this, you need to to check the following:
Improving quality of milk:
*Try to prevent feeding your cows silage.
*improve cow hygiene before milking
Clostridium can be airborne (thanks for the info Sailor Con Queso):
*Besides sterilizing your cheese making equipment before making cheese, make sure the surroundings are also clean
*pets in the vecinity can be an intrusion
*Wear clean clothes that haven't gone outside prior to making cheese.
If you buy your raw milk and have no control over it, then try one of these options:
*add some Holdbak LC when you add the cultures
*add sodium nitrate to the milk prior to adding the culture. sodium nitrate is largely regulated or even banned in certain countries.
Hope this helps.