Preheat your oven to 200°C/gas mark 6. Add the oil to a large baking tray and place in the oven to heat up.
Lay down a wooden spoon and place a new potato in the bowl end. Cut across at about 3mm intervals and repeat with all the potatoes.
Once the oil is hot, take the tray out of the oven and add the potatoes, carefully tossing in the oil. Arrange them so the potatoes are cut side up and season, sprinkling with the lemon zest and rosemary. Roast for 45-50 minutes until golden and crispy.
While the potatoes cook, combine the mayonnaise with the paprika, cayenne, lemon juice, garlic and some freshly ground black pepper. When ready to serve, pour the potatoes into a serving bowl and serve along with the spicy mayo.