Put the cardamom pods and cloves in a pestle and mortar and pound for about a minute, or until the pods are split and the cloves are roughly crushed. Tip into a medium non-stick saucepan and add the nutmeg, ginger, salt and milk. Stir in the sugar, vanilla, leaf tea, orange peel and cinnamon stick. Bring to a gentle simmer and cook over a low heat for five minutes, stirring occasionally. Remove the pan from the heat and leave to stand for 10 minutes.
Butter a 1.75-2 litre/3-3½ pint shallow ovenproof dish – a round one looks good. Spread one side of each slice of brioche with softened butter. Arrange the brioche slices in the dish, butter side up and slightly upright but leaning against each other, scattering the mixed dried fruit between the slices as you go.
Whisk the egg yolks, whole eggs and cream together in a large jug or bowl until smooth. Strain the infused milk through a fine sieve onto the egg mixture and stir well to combine. Discard the spices and peel.
Pour the creamy liquid slowly over the buttered brioche. (The bread will float, so press it down gently with a spatula for a few seconds to help it absorb the custard.) Leave to stand for 15 minutes.
Meanwhile, preheat the oven to 180C/350F/Gas 4.
Place the dish in a roasting tin. Pour boiled water into the roasting tin around the outside of dish to come half way up the sides. Sprinkle the pudding with sugar and bake for 35-40 minutes, or until the custard is just set, but still just a little creamy in the centre, and the bread is golden-brown and crisp on top.
Remove from the oven and carefully lift out of the roasting tin. Serve hot with cream, custard or good vanilla ice cream.