Season both sides of the cheese with salt and pepper. Place the cheese in a shallow dish and cover with brandy. Cover with plastic wrap and refrigerate for 2 hours. Season the flour with salt and pepper. Remove the cheese from the dish, reserving 1/4 cup of the brandy, and dredge in the seasoned flour, coating completely.
In a large saute pan over medium heat, melt the butter. Pan-fry the cheese for 2 minutes on each side. Add the reserved brandy and carefully flame the liquor, shaking the pan back and forth several times. Add the lemon juice, to taste. Remove from the pan and serve with the pita bread and olives.