MSPs have defeated the Scottish government in a vote calling on ministers to "halt" national assessments for five-year-old pupils.
The assessments were introduced across Scotland last year, but some teachers and parents have criticised them.
Ministers insist they are not high-stakes tests, but help schools assess where pupils need help at an early age.
Opposition parties say they are not in line with play-based early learning and could create league tables of schools.
Members voted by 63 to 61 to pass a Conservative motion calling for a "halt" to the tests, with all of the opposition parties uniting against the SNP.
The largely symbolic motion does not bind the government to any action, but Education Secretary John Swinney said he would "reflect" on the defeat.
Opposition parties called on the government to "respect the will of parliament" and scrap the tests.
Standardised assessments were introduced in 2017 in a bid to gather more data about the stages children have reached in their learning, with literacy and numeracy tests at P1, P4, P7 and S3 level.
However there has been opposition to tests for the youngest pupils, with the EIS teaching union claiming they have left some children distressed and parents group Upstart Scotland saying they are administered "at an age when the overwhelming majority of European children aren't even at school".
Mr Swinney contends that it would be "deeply irresponsible" to scrap the tests, saying that "many of the criticisms from opposition parties have ranged from ill-informed to hypocritical".
During the debate, SNP members repeatedly highlighted the fact that the Scottish Conservative manifesto for the 2016 Holyrood elections backed having tests in P1.
Mr Swinney said he was "appalled" at the "deceit" of the Conservative motion, accusing the biggest opposition party of "playing politics".
Ms Smith said her party had "made a mistake" in backing P1 assessments in 2016, and had flagged up "misgivings" in the years since.
She said: "As a party we continue to have these misgivings. We are listening to what is being said by those who are being asked by deliver the tests, and it is why I am proposing this motion, which asks the government to stop and think.
"Halt the P1 tests so that we can reconsider the facts before us, and the whole approach to evaluating pupil progress in P1."
Opposition MSPs were all critical of the assessments, albeit for different reasons, with the Greens and Lib Dems hitting out at the principle of standardised testing in general.
Labour's Iain Gray hit out at the practical implementation, saying young pupils had been upset "to the point of tears" by "daunting" tests and "questions they find incomprehensible".
He also claimed teachers don't find the data generated useful "in any way", saying: "The government has managed to introduce assessments which feel like high-stakes tests to teachers and pupils, but don't produce statistically valid comparative measurements."
Green MSP Ross Greer said his party was "unequivocally opposed" to standardised assessments, and said Mr Swinney was "developing a reputation for casting aside the views of elected members, as well as those of experts, teachers, parents and pupils".
And Lib Dem education spokesman Tavish Scott said the assessments "skew learning away from play", which he said should be at the heart of the early-years curriculum. He added that he was "at a loss to understand why the government is deaf to the practical observations of teachers and parents".
What happens now? Analysis from BBC Scotland Political Editor Brian Taylor
In a point of order, Labour's Richard Leonard challenged ministers to obey the will of parliament. And Tavish Scott of the Lib Dems found a neat formula when he said that a child could work out that 63 is a bigger number than 61.
But Mr Swinney is not, currently, for budging. He has promised to review the situation and to reflect upon the vote. Meanwhile he urged councils to continue with standardised testing, including at P1 level.
My take? He'll come back with further suggested reforms - including, perhaps, revisions to the questions and methodology in an effort to address, again, concerns that the system is stressful.
But, as things stand tonight, I do not see him abandoning the P1 assessments entirely. Which will generate a further debate about educational provision. And also about Parliamentary democracy.
Mr Swinney argued that the assessments were "absolutely vital" to get information about children's learning "as early as possible", contending that they were "designed to fit compatibly" with a play-based curriculum.
He said the tests had been going on for "years" at 29 out of Scotland's 32 local authorities, and were a "key tool to inform a teacher's professional judgement".
However his amendment reflecting this point was voted down by members by 63 votes to 61.
And MSPs subsequently passed Ms Smith's motion by the same margin.
The motion passed calls on parliament to note "the level of concern that has been raised by teachers and other education professionals regarding the introduction and delivery of new testing arrangements for P1 pupils", and calls on ministers to "halt the tests in P1 and to reconsider the evidence and the whole approach to evaluating the progress of P1 pupils".
Following the vote, Mr Swinney told BBC Scotland that the government "remains absolutely committed to standardised assessment".
He said he would "obviously reflect" on the vote, but advised schools to continue with the current system in the meantime.