Heat the orange and lemon zest, ginger, cinnamon stick, cloves, vanilla vodka, cardamom pods, port, red wine and sugar over a low heat until hot, but not boiling. Strain the mixture through a sieve onto the softened gelatine in a large bowl and stir well.
Spoon the soft fruit into a large glass serving bowl and pour enough of the jelly mixture to fill it a third of the way up. Chill in the fridge overnight, or until they have completely set.
Mix half of the whipped cream with the custard and vanilla in a bowl until well combined and spoon this mixture onto the set jelly.
To serve, spoon the remaining whipped cream onto the trifle and scatter over the flaked almonds.
You can make this trifle in 12 individual serving glasses instead.