Brandon Lewis has refused to say when the Tories' European election campaign will launch, saying his priority is not to have to fight them at all.
The UK is due to elect new MEPs on 23 May, after Brexit was delayed amid continuing parliamentary deadlock.
Several parties have launched their campaigns already but Conservative chair Mr Lewis told the BBC his focus was on next week's local elections.
The UK is due to leave the EU on 31 October, or sooner if a deal is agreed.
This means the UK must now hold European Parliament elections on 23 May if it wants to avoid leaving the EU without a deal.
But if agreement can be reached among MPs before 22 May, the UK could cancel its participation in the elections.
The Lib Dems launched their campaign on Friday, with a pledge to do what they can to stop Brexit altogether.
Labour's NEC is due to meet on Tuesday to decide on its manifesto for the campaign.
But asked about Conservative campaign plans on BBC One's Andrew Marr Show, Mr Lewis said: "As a government, our first priority is to not have to fight the EU elections.
"I think we should be doing everything we can to respect that 2016 referendum, if and when we are at the point where we are definitely fighting those EU elections then we will take some decisions about that."
He said there was "still time" for the UK Parliament to approve the withdrawal agreement - which it has so far rejected three times - in time so the UK could leave the EU and would not have to take part in the elections at all.
He rejected reports about donors deserting the Conservatives, saying that 2018 had been a "record peacetime fundraising year".
Some Conservative members reportedly plan to back the Brexit Party at the elections, and Conservative councillors in Derbyshire are refusing to campaign at all in protest at the failure to have left the EU.
Mr Lewis said: "I don't deny the frustration people in our party have over where we are on Brexit. I share that frustration, I want to get this done so we don't fight those European elections."
He added: "I hope that Conservative members, colleagues, volunteers, activists will come to want to not just vote for, but campaign for Conservatives to get elected, because ultimately Conservative representation is better than any other party."
On Sky News, Labour frontbencher Rebecca Long-Bailey said the government was still refusing to move on "any of their red lines" in cross-party talks aimed at reaching an agreement on Brexit.
She said: "Honestly I think the discussions so far have been productive, they've gone into a lot of detail, there seems to be a willingness on both sides to move towards some form of consensus.
"But as yet we haven't seen the government move on any of their red lines, we're having further discussions this week and hopefully we'll see some movement."
At the end of March, Prime Minister Theresa May said the public would find it "unacceptable" to have to elect a new group of 73 MEPs almost three years after they voted to leave the EU.
But earlier this month, she agreed a Brexit delay to 31 October with the EU, with the option of leaving earlier if her withdrawal agreement is approved by Parliament.
What do you want to know about Brexit?
Use this form to ask your question:
If you are reading this page on the BBC News app, you will need to visit the mobile version of the BBC website to submit your question on this topic.