The watchdog Qualifications Wales has concerns about the practicalities of Welsh Government proposals for exams in 2021, its chief executive has said.
There were still many questions that needed sorting out, Philip Blaker said.
He added a lot of work was required to ensure "consistency and fairness across schools".
Education Minister Kirsty Williams has given an advisory group six weeks to devise how to deliver the "fine detail" of the plans.
On Tuesday it was announced the end-of-year GCSE, AS and A-level exams had been cancelled.
Instead, assessments will be done under teacher supervision, starting in the second half of the spring term. This will include assessments that are externally set and marked.
The Welsh Government has said the situation for vocational qualifications is "more complex" and will require "extra work".
Speaking to the Senedd's education committee on Thursday, Mr Blaker said there was a lot of "novel thinking" that needed to be done "very quickly".
In particular the regulator has concerns about teacher-led assessments - where the outcome is eventually determined by the pupils' teacher - because it thinks that could lead to further inconsistency.
He said Qualifications Wales analysis had found even with the inflated grades of 2020 there was a huge degree of variability between each school or college.
The watchdog had put forward its own proposals, which included plans for some but fewer A-level exams, which he said were deliverable because they were building on known assessment techniques and within an existing structure of classroom learning.
"Instead if we think about the potential for there being, actually, too much assessment... that may actually mean that there is a greater well-being issue for learners," he said.
In particular he questioned whether the right of appeal under teacher-led outcomes could lead to schools having to revise their decisions on pupils' grades.
He also questioned how pupils educated outside school could be assessed fairly.
Reacting to the comments, the education minister told BBC Radio Wales the new advisory group working out the "fine details" would meet on Friday.
"Philip Blaker is a part of that group, to ensure that the system is fair and robust and has his confidence. There is lots of work to be done," Ms Williams said.
'A tad behind'
Also speaking to the committee was Ian Morgan, the chief executive of the WJEC exam board, who said the advisory group needed to work very quickly, and everyone had to be upfront about what is deliverable.
"We are probably a tad behind already in terms of where we need to be to deliver on some of these things," he said.
He said he wanted to avoid a repeat of the widespread grade revision of the summer, and that devising a fair system "within a short timeframe will require significant effort" from the wider education community.
"What we don't want is any fallout from the decisions that are made now happening in summer 2021.
"At the end of the day, certification rests with the WJEC... and therefore [we have to be] comfortable that what is being put in place is tried, tested and fair and balanced to give a fair outcome for learners."
He said the board was committed to doing "whatever it is we need to do" but added "there are still some key issues that need to be bottomed out in order for us to sign up in a clear and effective way".
Ian Roberts, education spokesman for Welsh Local Government Association, said local authorities have given the plans "a very warm welcome" while not pretending "that the way ahead may not be difficult".
He said that GwE, the consortium of councils that run education in north Wales, was writing a letter of thanks to the minister for her decision saying that "this gives the best opportunity to demonstrate a student's ability regardless of how Covid has affected them during the year".
Kay Martin, the principal of Cardiff and Vale college and chair of the Colegau Cymru Curriculum and Quality Group, told the committee the move was "generally welcomed".
However, she expressed concern about the "challenging situation" for vocational qualifications.
"We work with about 40 awarding organisations that operate right across the United Kingdom… and so we have to wait with each of those awarding bodies… to see what changes they will make but we don't have control of that in Wales."