Owner David Hemsley is considering his future at Leicester Lions after three riders quit the Elite League strugglers this month.
Lewis Bridger followed Bjarne Pedersen and Mikkel Michelsen out of the club, which lies bottom of the table having won just twice in 10 fixtures.
Hemsley told BBC Radio Leicester: "I don't know what we're doing wrong and why we're having issues with riders.
"It does make you question where you're going with it. It's embarrassing."
Bridger, who came out of retirement to sign for the Lions last week, left after one meeting at Swindon.
And 20-year-old Michelsen and two-time World Cup champion Pedersen also left the East Midlands team.
"The fans, the management and the riders have been let down by three or four individuals," Hemsley said.
"Certain people value and have a greater hunger for British speedway than other riders who are not prepared to put themselves about so much for a British club.
"I can't understand what Leicester does differently to every other team in the country."
But, despite the high-profile departures, Hemsley accepts blame for the situation.
He added: "I'm ultimately responsible for the delivery of the speedway at Leicester and, though we deliver excellent entertainment week-in, week-out on the track, we're clearly not in a position at the moment where things are as good as they should be.
"I'm at the top of the club's family tree and the buck stops with me, so I have to take responsibility for it."