Preheat the oven to 180C/350F/Gas 4.
Divide the cheese into four piles, and roll each one into a tight ball. Set aside.
Place the lamb mince in a large mixing bowl. Add the onion, anchovies, gherkins and capers and season with salt and freshly ground black pepper. Mix the contents of the bowl together, ideally by hand. Divide the mince mixture into four.
One by one flatten each pile of mince into the palm of your hand, leaving a little well in the centre. Place the cheese ball in the well and mould the mince around the cheese to form a tight, plump burger shape that is almost spherical.
Heat the olive oil in a frying pan over a moderate heat and brown the burgers.
If the panhandle is ovenproof, transfer the pan to the oven to bake. If not, carefully transfer the burgers to an oven tray and bake for 30 minutes.
Slice the buns and when the burgers are cooked, soak the cut side of the bread in the pan juices. Fill the buns and serve.