Preheat the oven to 180°C, fan 160°C, gas 4. Line a 20cm loose-bottomed square cake tin with baking parchment. Mix together the flour, baking powder and orange zest in a bowl and set aside.
In a separate large bowl, use a hand-held electric whisk to cream together the butter and sugar until pale and creamy. Gradually add the eggs, mixing until well incorporated. Finally, using a spatula, stir in half the pistachios, the flour mixture and milk. Pour into the cake tin and level the top.
Arrange the apricot halves over the top of the cake (they will sink in slightly), then bake in the middle of the preheated oven for 35-40 minutes until a skewer inserted into the middle comes out clean.
Meanwhile, heat the reserved fruit juice in a small pan with the extra 1 tablespoon caster sugar. Simmer gently over a low heat for 5 minutes until it has reduced to about 3 tablespoons.
When the cake is cooked and still warm, spoon the syrup over the top and let it soak into the warm cake. Scatter over the remaining pistachios and leave to cool in the tin. Remove from the tin, cut into portions and serve with Greek yogurt, if liked.