Using a box grater, coarsely grate the potatoes into a medium bowl. Add the matzo meal, salt, egg yolk and jalapeno, and mix well. Let cool in the refrigerator for 5 minutes to allow the ingredients to bind.
Remove the bowl from the refrigerator and drain off any excess liquid.
Over medium heat, heat 1/2 cup olive oil in a 10-inch skillet until hot but not smoking. Take 1 heaping tablespoon of the potato mixture at a time and place into the pan, being careful not to crowd the latkes. Cook each latke until golden brown on both sides, about 5 minutes total.
Transfer the latkes to a paper towel-lined plate, and season with additional salt if desired. Add 1/4 to 1/2 cup more olive oil (if needed) to the skillet and repeat the process with the remaining latke mixture.
Serve immediately with Horseradish Crema, or applesauce as traditionally served, if desired.
Mix together the crema, horseradish, vinegar and salt.