Boris Johnson has said Stormont's new power-sharing deal strikes a "balance" between supporting veterans and giving victims of the Troubles the chance to seek justice.
It will see plans for a new body to investigate Troubles murders revived.
Some Tory MPs have voiced concerns that this will see cases without foundation being brought against veterans.
But the PM said he would keep his manifesto promise to end "vexatious" prosecutions of former servicemen.
The power-sharing agreement commits the UK government to introducing legislation implementing the 2014 Stormont House agreement within 100 days.
The Stormont House agreement includes the creation of the Historical Investigations Unit (HIU) to take on the criminal justice element of looking into the past.
A separate truth recovery mechanism will offer bereaved relatives the chance to learn more about the circumstances of their loved ones' deaths without the prospect of conviction.
The HIU's predecessor, the Historical Enquiries Team, was much criticised, including by the police watchdog, which concluded it treated cases where the state was involved with "less rigour" than those not involving military personnel.
The Public Prosecution Service in Northern Ireland has said that of the so-called Troubles "legacy" cases it has taken decisions on since 2011, 17 relate to republicans, eight to loyalists, five are connected to the Army and three involve police officers.
'No new evidence'
The prime minister is in Belfast to mark the return of devolution in Northern Ireland after a three-year impasse.
He told a press conference: "I think that the parties here who have revived Stormont have done a very good job of finding a balance between giving people who are in search of the truth the confidence that they need, but also giving people who served our country in the armed services the confidence and certainty that they need."
He added: "Nobody thinks that people should get away with crimes. What we're saying is that people should be protected from unfair vexatious prosecutions when there's no new evidence to be found, and those positions, I think, are wholly compatible."
His spokesman also said the government had "always been clear we would implement the Stormont House agreement in a way that provides certainty for veterans and justice for victims".
The spokesman said the commitment goes "no further than the Queen's Speech" and the Conservative Party manifesto, "other than providing a timeline for the legislation".
Number 10 will work with the Ministry of Defence and other Whitehall departments "to develop Stormont House proposals in the coming weeks", he added.
One minister described the discussions as sensitive, while a government source acknowledged talks between departments would be difficult, but said it was not stepping back from any manifesto commitments.
In July it was announced that a new Office for Veterans' Affairs was being created "to provide life-long support to military personnel".
Johnny Mercer MP - who has been one of the most outspoken critics of the historical investigations process - was appointed minister for defence people and veterans.
At the time the MP for Plymouth - a veteran himself - said the prime minister had "tasked me to end the repeated and vexatious pursuit of veterans" over offences allegedly committed in the line of duty.