Hugh, it is the milk handling and sanitation protocol that determines that goaty flavor. Sometimes diet and other factors, like buck proximity, make a difference, but I have found this to rarely make a big difference.
Goat milk has a higher ratio of the "goaty" fatty acids in the fat. these are the intermediate chain fatty acids, the C8, C10, and C12. They are easily broken up and liberated from the glycerol base by two things:
1) The natural lipases in goat milk. These are very active at room temp. Meaning any time you milk, that milk must be chilled down as fast as possible.
2) Agitation. Those fats break apart with agitation, which speeds up the reactions of lipases breaking apart the triglyceride.
In short, chill immediately, strip into a strip cup before milking, practice proper udder sanitation (including post dip), and handle the milk gently.
I've made hard cheese from goat milk that has zero goat taste, but that was usually from milk that was less than 12 hours old.
Also, sometimes a particular line of goats has milk that is not very good, or an animal may not have enough nutrients of microelements.
To answer your question, no, during aging, the goaty flavors actually intensify.
