The DUP has failed to back the government on a number of amendments to its Finance Bill.
The party also supported an amendment proposed by the Labour Party.
The debate on the Finance Bill is due to last throughout Monday evening and into Tuesday, with the DUP expected to abstain on more amendments.
The Conservative government agreed a financial package with the DUP in exchange for support on certain issues.
About £430m has come to Northern Ireland's budget under the arrangement so far. The remainder of the £1bn is due to come in the 2019-20 financial year, with the York Street Interchange and rural broadband among the projects earmarked for cash.
In recent weeks the relationship between the two parties has been strained following criticism by the DUP of the prime minister's draft Brexit agreement.
The DUP has previously said it will review its arrangement propping up the Conservatives if the draft Brexit plan passes through Parliament.
To date the DUP's 10 MPs have supported the government on key votes, but it is opposed to the Irish border backstop proposal.
The DUP has previously only abstained on one bill.
Sammy Wilson from the DUP has a message for the government: "You’ve got to keep your side of the bargain, otherwise we don’t feel obliged to keep ours" #newsnight | @nicholaswatt pic.twitter.com/Hz9lB6q5T4— BBC Newsnight (@BBCNewsnight) November 19, 2018
The DUP MP Sammy Wilson, the party's Brexit spokesperson, told BBC's The World Tonight that the government has "seriously broken" its commitment that it made with the DUP that Northern Ireland would not be treated differently from the rest of the UK in the Brexit agreement.
"Since the government has not honoured its side of the bargain we tonight tried to spell out some of the consequences of that," Mr Wilson said.
However the East Antrim MP denied that the confidence and supply agreement pointing out that the Monday night's votes were not intended "to damage the government fiscally".
He said that none of the votes in which the DUP voted with the opposition had financial consequences for the government, but they had political consequences.
"The prime minister has undermined her own authority with her own party and with our party by blatantly breaking promises about what she would deliver in the Brexit deal with the European Union," he said.
Senior DUP sources told BBC News NI the DUP had taken the decision to abstain and then vote with Labour because "we are not a happy bunch".
Another DUP source said it felt the government would respond in a "tetchy manner, but then ho hum because we've a few who are good at that too".
The same source said the party does not believe the move breached the confidence and supply arrangement as the amendments were "cost neutral" - and that is now over to the government to respond.
However, the text of the confidence and supply deal states that the DUP agrees to support the government on "all motions of confidence... the budget, finance bills, money bills".
Commenting on the DUP not voting with the government Labour's Shadow Minister for the Cabinet Office, Jon Trickett MP, said: "We no longer have a functioning government. With Brexit only a few months away something has got to give."
Analysis: Laura Kuenssberg, BBC political editor
Forget for a moment the "will they, won't they" numbers game.
Unless and until the head honcho of the Tories' backbench committee receives 48 letters there won't be a vote of no confidence in the prime minister.
As far as we know tonight the total has not yet been reached.
One of those who has submitted their letter told me in no uncertain terms "where are the others?" - frustrated that many of his colleagues seem to have promised to be part of the action, but reinforcements are yet to arrive.
Read more: The DUP's Brexit message to Theresa May