I'm definitely a fan of buying the ingredients you need directly rather than buying a kit. Specifically, I'm not a huge fan of Mad Millie. The prices are relatively high for what you get. Also their rennet tablets are rated at 60 IMCU which is about enough for 1.5 liters of milk. However, their recipes *really* cut down on the amount of rennet to the point where I think it's potentially going to cause problems. Larger volumes of ingredients are only slightly more expensive, but will give you basically 10x the amount of ingredients. I would also recommend buying single strength liquid rennet since that's what most recipes are oriented towards. It doesn't really make much difference, but it means you can follow the recipes directly instead of doing a bunch of math. For bacterial cultures, I recommend getting 1 "mesophilic culture". Something like flora danica is *very versatile*. This is essentially a buttermilk culture. You can use it for Camembert, Brie, Cheddar, Gouda, etc, etc. Then pick up one "thermophilic culture". I tend to like one with bulgaricus, which makes Bulgarian/Greek style yogurt. This will make a variety of Italian cheeses. If you want to make "swiss" style cheeses, then pick up one with helveticus too. Depending on where you buy the cultures, different brands will be available, but they are pretty much interchangeable. I've been using Bienna brands lately because they are a smaller company and the amount/price is convenient. As you get more experienced, you may want to buy specific things, but this will get you started. For Brie and Camembert styles, you'll need to get some penicillium candidum and geotrichum candidum. For blue cheeses, I actually recommend buying some blue cheese and adding a small amount to your milk because penicillium roqueforti is actually very expensive for some reason (upwards of $30 for an envelope -- granted it's a lifetime supply so if you really like blue cheese, it's convenient).
For pots, keep in mind that cheese making temperatures are very low (except for ricotta). Anything that's food grade is fine. It doesn't even need to be a pot. Many cheeses are made with the milk at blood temperature or below -- so the milk is still slightly cool to the touch! Even when you heat it, it only generally needs to get to 42 C, which is bath water temperature. It's not uncommon to make cheese in any old metal container sitting in a sink with a sous vide heater (and really, this is kind of the Rolls Royce of home cheese making equipment).
The other common setup is a pot within a pot, or a pot over a pot. With a pot within a pot, you have 2 pots. The outer one has a bit of water in it. You put the inner one in so that the water makes a kind of jacket around the inner pot. Then you heat the outer pot. This allows you to maintain the temperature easily. For a pot over a pot, you have a large pot sitting on top of a small pot. You heat the small pot and the steam heats the large pot.
Because I make small amounts of cheese (usually only 500 grams or so -- 4 liters of milk max), I just heat my pot directly on an induction stove set to low. If I stir it, it works find. I did the pot inside the pot for a long time, but honestly it's a bit of a faff and it's not that necessary. Especially in my hot climate -- in Sydney in the summer, too, you probably won't even have to heat the milk. More likely you'll have to have the air conditioning on :-) If it's 33 C in the house there are a lot of cheeses you can make without heating the milk. Incidentally this is how many Brie makers work. Instead of heating the milk in a vat they just heat the room to 32 C and wait for the milk to warm up.
Some people don't like to make acidic things with aluminium pots. I make cheese for a long time in one. There is some misinformation based on old, discredited research that links mental illness with aluminium. It's not considered to be a problem these days. However, some people *can* taste it. I can't, but I've known people who can instantly tell if something was cooked in an aluminium pot. If you are one of those people, or know somebody who is, then you should probably stick to stainless steel.
Other than that, my main piece of advice is: don't spend a lot of money. If you are using IH, then concentrate on a pot that has a very flat bottom and that won't warp with heat. Personally I avoid pots with a sandwiched bottom since they tend to come apart eventually as the different metals expand at different rates when heat is applied. I think a pot should last my entire life and I try to buy them appropriately. Try to get a pot that has a bottom that's not much bigger than your burner -- especially for IH. This is quite important for stainless steel (my least favourite material for cookware), because it doesn't conduct heat very well. So if you are buying a big pot, make it taller rather than wider. Heavier guage steel will obviously cost more money, but it's not actually much of an advantage in a stock pot -- other than on the bottom. I also personally avoid pots that are lined, but ceramic lined steel pots can be good for cheese making. If you are careful not to scratch it, it will last a long time.