Preheat the oven to 325F and line the bottom of a deep 9-inch round pan (or springform pan) with a double layer of parchment. Butter the pan, including the parchment on the bottom.
In a large bowl, using the paddle attachment of your stand mixer if you have one, beat the butter, sugars, and the orange and lemon zests for a few minutes, scraping down the sides of the pan, until pale and fluffy. Beat in the eggs one at a time - don't worry if the mixture appears curdled. Slowly beat or mix in the ground almonds and hazelnuts, and then the dried fruits, brandy and the juice of the orange and lemon. In a small bowl, stir together the flour, baking powder and salt; add it to the mix and stir or slowly blend just until combined.
Scrape the batter into the prepared pan and smooth the top. Bake for an hour, then reduce the oven temperature to 300F and bake for another hour or so, until the cake is deep golden and springs back when you touch it. A bamboo skewer inserted in the middle should come back with just a few crumbs stuck to it. Cool it completely. If you want to feed it over the course of a few weeks or a month, Nigel suggests poking it all over the surface with a skewer or knitting needle, and pouring over just enough brandy to moisten without making the cake soggy - 3-4 Tbsp at a time. Do this every few days, keeping the cake wrapped in parchment or waxed paper in a tin in between, for up to a month. Makes 1 good-sized cake.