Heat the oil in a large skillet over a medium-high heat. Season the chicken on both sides with salt and pepper, add to the skillet and cook until browned, about 2 minutes per side. Meanwhile combine the brown sugar, soy sauce, rice vinegar and orange juice in a small bowl and set aside. When the chicken is browned, transfer to a plate and set aside.
Add the ginger and garlic to the pan and cook, stirring, for 30 seconds. Add the chicken broth, the soy sauce mixture, and the peaches to the pan. Turn the heat up to high and cook, uncovered, for about 6 minutes, stirring occasionally until the sauce is nicely thickened and the peaches soften. Add the chicken back to the pan with the sauce, turn the heat down to moderate-low, cover and cook for about 5 minutes, or until chicken is cooked through.
In the meantime, toast the almonds in a dry skillet over a medium-high heat stirring frequently, until golden brown and fragrant, about 2 minutes.
Serve the chicken topped with the sauce and sprinkled with the toasted almonds.