Place the bread in a large bowl and cover with the milk, pressing the bread down into the liquid so it's fully immersed. Set aside.
Place a saute pan over medium-high heat and add one tablespoon of the oil. When the oil is hot, add the garlic and shallots and season with salt. Cook until they become fragrant, about a minute. Add the almonds to toast, about 2 more minutes. Once the almonds are slightly toasted, remove the pan from the heat and set aside.
Lift the soaked bread out of the milk without ringing it out. Just let the excess milk drain off. Put the bread in a blender, reserving the excess milk to thin out the skordalia later if needed. Add the lemon juice, the remaining 1/4 cup olive oil, the toasted almond mixture and salt to taste. Blend the mixture until it is a smooth paste. Add some of the reserved milk if the consistency is too pasty. Taste and adjust for seasoning. Serve with pita chips or crudite (I love this with roasted beets).