Put the purple and sweet potatoes in a medium saucepan with cold water to cover, salt the water generously and bring to a boil. Reduce the heat and simmer until the potatoes are just fork tender, about 6 minutes; drain and cool. Once cool, dice the potatoes into 1/4-inch cubes.
Heat a skillet over medium-low heat. Add the bacon and cook until crisp. Remove from the skillet and place onto a paper-towel-lined-plate. To the bacon fat, add the diced potatoes, toss to coat and brown on all sides. Add the onions and cook until tender, about 4 minutes. Add the garlic and rosemary and cook until fragrant, about 1 minute. Add the celery and red pepper and cook until tender, 3 to 5 minutes. Season with salt and pepper.
To finish, stir in the chopped parsley and cooked bacon.