Ed Miliband dismissed criticism of his leadership of Labour as he set out his vision of the party's future policies on "responsibility" and "fairness".
He vowed to stop Labour being "the party of those ripping off our society" such as benefit cheats and bankers.
It comes amid reports brother David and his supporters are unhappy with the way Ed is leading the party.
But Mr Miliband dismissed such claims as Westminster "tittle-tattle" that was "irrelevant" to voters.
Speaking to reporters after a speech in London, he said he wanted to focus on "the issues that matter to the future of our country".
On Sunday, following press stories of a feud, David Miliband issued a statement insisting he was "fully behind" his brother.
Asked about speculation surrounding his position, and criticism that he is not making a big enough impact, Ed Miliband said: "People aren't interested in who said what to whom in the Labour Party. People are much more interested in the future of the country.
"I'm going to talk about those issues that matter... and in the end the electorate are the boss.
"I am here because I was elected by my party. The gossip and tittle-tattle of Westminster is actually irrelevant to most people's lives."
Mr Miliband also dismissed suggestions that his brother was undermining his leadership, adding: "I think that's totally untrue. I think that's totally unfair."
In his speech, Mr Miliband launched a twin-pronged attack on boardroom excesses and abuses of the benefit system.
"For too many people at the last election, we were seen as the party that represented these two types of people - those at the top and the bottom who were not showing responsibility and were shirking their duty to each other," he said.
"Labour - a party founded by hard-working people for hard-working people - was seen by some, however unfairly, as the party of those ripping off our society."
He told the BBC's Nick Robinson that Labour had got its approach to the wealthy wrong during Gordon Brown's and Tony Blair's time in power.
"We didn't talk about the responsibilities of bankers, we didn't talk about the responsibilities of the rich, and that is one of the ways the Labour Party is going to change under my leadership."
He refused to speculate on whether that meant the 50p top tax rate would be extended, saying people would have to wait for the party's next manifesto.
But he did float the idea of a workers' representative in the boardroom to have a say on executive pay rates.
He also said a future Labour government would require companies to publish the pay gap between their boardrooms and the average earnings of their workers.
He suggested those who work or volunteer should get priority on council-house waiting lists - and backed coalition moves to test more people on incapacity benefit as fit for work.
But he denied he had declared war on "scroungers" - saying he did not like to use that term and it was only a "small minority" who abused the system.
He insisted he wanted to offer rewards, not punishments, to encourage people to "do the right thing", and contrasted his approach with that of the Conservatives, who he said were "demonising" those on benefits.
Backbench Labour MP Frank Field said Mr Miliband was "beginning to mark out a position for the next election" different from New Labour and the coalition - and he hoped Labour's policy reviews would start to produce results.
"I hope historians will see this as a turning point for him personally, but, more importantly, for the country's future as we begin to shape up an opposition that has a real alternative to the coalition government," he told the BBC News channel.