Combine all ingredients (including the bean and its pulp), except peaches, in a large saucepan and place over medium heat. Attach a frying or candy thermometer to inside of pan. (see note below) Stirring occasionally, bring the mixture to 170 degrees F. Remove from heat and strain into a lidded container. Cool mixture, then refrigerate mixture overnight to mellow flavors and texture.
Freeze mixture in ice cream freezer according to unit's instructions. The mixture will not freeze hard in the machine. Meanwhile, chop peaches roughly. Once the volume has increased by 1/2 and reached a soft serve consistency, add the peaches and continue turning to incorporate. Spoon the mixture back into a lidded container and harden in the freezer at least 1 hour before serving.
NOTE: If you do not have a thermometer, bring the mixture just barely to a simmer. As soon as you see a bubble on the surface, remove it from the heat.