Preheat the oven to 300 degrees F.
In a covered heavy saucepan, bring the rice, milk, apricots, vanilla scrapings, and lemon zest to a simmer over medium heat. Immediately reduce the heat to as low as possible, cover tightly, and simmer gently, stirring occasionally, until the rice is very tender, 20 to 25 minutes. Turn off the heat.
Stir the sugar into the cooked rice mixture. Stir in the egg yolks. Pour the rice pudding into a greased baking dish and bake for 20 minutes until slightly browned on the top. Remove from the oven and let cool. Chill until ready to serve.