In a medium saucepan, combine the rosemary sprigs, sugar, and water, and bring to a simmer. Let cook for about 10 minutes until slightly thickened and well-infused.
Remove from heat and strain. Discard rosemary sprigs. Let cool before using.
In a large pitcher, combine cooled syrup, peach nectar, peach slices, and lemon juice ice cubes. Add enough water (about 1 to 1 1/2 cups) to top off, leaving room for additional ice cubes to serve.
Add more ice right before serving. Serve a fresh rosemary sprig in each glass.