AJ, I think I'd reset the control a little lower...perhaps for a spread between 48-55F. Is it digital or analog? If digital, can't you select the spread? I have an analog Johnson controller and it is not possible to set the spread, but it still maintains a fairly consistent temperature. My newer Ranco digital controller lets me set the spread down to 1 degree. Set it to 51F and if it climbs to 52F, the controller kicks on.
I've never used something like jugs of water to help maintain a constant temp in my caves, but it sounds like a reasonable idea. My effort was confined to filling the available space with wheels of cheese.
Don't know if that helps.