In a large bowl whisk together the condensed milk, cream, crème fraîche, vanilla and brandy until smooth and creamy.
Pour the mixture into a shallow plastic box lined with clingfilm and freeze until slushy (2-3 hours). Beat with a fork, then stir in the crumbled Christmas cake. Pour into the pudding basin and freeze overnight.
When ready to serve, turn the parfait out onto a serving plate and remove the clingfilm.