Put the potatoes in a medium saucepan and add just enough water to cover them. Add 1 teaspoon salt and bring to a simmer over medium heat. Do not boil. Cook for 12 to 15 minutes or until the potatoes are just tender enough to slide off a paring knife when pierced. Drain well and return to the pan. Set the pan over low heat and cook for 1 to 2 minutes to dry the potatoes out.
Remove from the heat and lightly mash the potatoes with a fork just to break them up. Stir in the milk, mashed garlic and olive oil. Run the potatoes through a food mill or mash them with a potato masher to the desired consistency. Season with salt and pepper, to taste, and serve immediately.
Put the cloves in a dry cast iron skillet and cook over low heat for 20 to 30 minutes, or until softened. Turn frequently so that the cloves soften but do not brown. Slide the cloves from the skillet onto a plate to cool to the touch. Squeeze the softened garlic pulp from the individual cloves.