For the orange cake, simmer the two whole oranges in a saucepan of simmering water for 45-60 minutes, or until very soft. Remove the pan from the heat and leave the oranges to cool in the water.
Preheat the oven to 180C/350F/Gas 4. Grease a 23cm/9in loose-based, springform cake tin.
Drain the oranges and roughly chop, discarding the pips.
In a food processor, blend 450g/1lb of chopped orange to a purée. Add the sugar, eggs and ground almonds and blend until smooth.
Spoon the mixture into the cake tin.
Bake in the oven for 25 minutes, then cover loosely with aluminium foil and bake for a further 25-35 minutes, or until cooked through (a skewer inserted in the middle should come out clean). Remove from the oven and set aside to cool.
For the caramel and orange sauce, heat a frying pan until hot, add the butter, orange zest and juice and the caster sugar. Add the orange liqueur and brandy and flambé by slowly tilting the pan towards the flame, or lighting with a match. Let the flames flare up then die down. (CAUTION: Keep your face and hair away from the flames. Do not leave unattended.)
Cook for a few minutes until just thickened and bubbling, then add in the segments and remove the pan from the heat.
To serve, slice a piece of cake and place on a plate and pour the sauce over the cake and around the plate. Serve immediately with a scoop of ice cream.