Add the cayenne, caraway, cumin, paprika and salt. With the motor running, slowly drizzle in 1/4 cup of the oil. Process until the harissa is completely pureed and all the ingredients are thoroughly combined. Makes about 2 1/2 cups of harissa.
Mix 3/4 cup of the harissa with the remaining 1 cup canola oil to make the harissa oil.
For the fried olives: Line up three medium bowls. Add the flour to the first, the egg to the second and the panko to the third.
Working in small batches, with the help of a slotted spoon, toss a handful of the olives in the flour. Lift, shake off excess flour and quickly dip the olives in the egg mixture. Dredge them in the panko, shake off excess and add the olives to the egg mixture a second time. Dredge once again in the panko, shaking the olives around to coat evenly. Shake off excess and transfer to an airtight container until ready to use. (The olives can be breaded up to a day in advance.)
In a deep skillet or medium pot, heat 1 1/2 to 2 inches of oil to 375 degrees F.
Serve the fried olives piping hot with labne drizzled with harissa oil on the side.