Nicklas Bendtner fired in the opener from close range before immediately going off with an ankle injury.
And in the 90th minute Theo Walcott - who had earlier hit the post - converted Jack Wilshere's sublime pass.
Cardiff are set to name Solskjaer as successor to Malky Mackay.
And the former Manchester United striker was sitting alongside Bluebirds' owner Vincent Tan in the Emirates Stadium directors' box after holding talks to take over as manager.
Arsenal's year to remember
Arsene Wenger's Arsenal collected a Premier League-high 82 points in 2013, including 25 wins and just six defeats. The Gunners failed to win any of their first three matches of the year.
The Norwegian would have been encouraged with what he saw as the well-organised visitors held Arsenal until the dying minutes when goals from substitute Bendtner and Walcott left Cardiff only two points above the drop zone.
For a long time it looked as though Arsene Wenger's men, who started the day top before briefly being displaced when Manchester City won the early kick-off game, would be left frustrated as they enjoyed plenty of the ball without being able to break Cardiff down.
The hosts were restricted to half chances for the majority of the first half, with Walcott side-footing the best chance over the bar.
A lot of the main striker responsibilities at Arsenal have fallen to Olivier Giroud this season but, with the Frenchman missing with an ankle injury, Lukas Podolski made his first start since August.
Arsenal 2-0 Cardiff: Arsene Wenger rues striker shortage
Arsenal have not been prolific at home so far this season, going into the match with only 16 league goals from nine matches.
And the Gunners' lack of striking options will inevitably lead to more questions being directed at Wenger about potential targets this transfer window, with the first half display doing little to convince they have a suitable replacement for Giroud.
Cardiff, going into the game with only one league win in nine matches, defended deep but looked dangerous on the break as Fraizer Campbell worked tirelessly in attack.
The few Arsenal chances that did arrive mostly fell to Walcott, who shot into the side-netting while stretching from a Wilshere cross early on before just failing to get on the end of a Mikel Arteta through ball.
Wilshere had a penalty appeal turned down late in the half after going down under the challenge of Gary Medel, who looked to have made contact with the Arsenal man and not the ball.
An injury-hit Gunners side, who were also missing Aaron Ramsey, Mesut Ozil, Kieran Gibbs and Tomas Rosicky, continued to struggle to create chances in the opening stages of the second half, although they were enjoying the majority of the possession.
Arsenal 2-0 Cardiff: David Kerslake certain of Bluebirds survival
England midfielder Wilshere was attempting to spark his side into life, but it was Cardiff who had a penalty shout turned down when Craig Noone's cross hit Nacho Monreal in the arm.
With Bendtner on in place of Podolski the Gunners finally upped the tempo and almost took the lead when Wilshere cut inside before striking a shot against the post.
Per Mertesacker then headed wide twice when unmarked before the deadlock was broken as Bendtner was quickest to react when David Marshall saved Bacary Sagna's header, the Dane firing home with just two minutes remaining before immediately going off injured.
The victory was sealed as Walcott calmly chipped over Marshall after latching on to Wilshere's flicked through ball.
"We needed to be patient against a very good Cardiff side. They did work absolutely hard, their new manager was in the stands. They deserve credit and were a bit unfortunate to lose in the end.
"I think we were mature, intelligent and resilient. I felt in the first half it took us a while to get in our stride and our pace was a bit slow. We did all right in the second half but didn't take our chances and they defended very well.
"We know we have to play 10 home games and we want to make this place a fortress and take the points no matter what the others do.
"If we can manage to do that we will be in a very strong position."
Cardiff city caretaker manager David Kerslake:
"The players have given their all and I thought they had done enough to get a draw. We knew we would be good enough to soak the pressure, not a lot of teams can do that."
On Ole Gunnar Solskjaer being in stands:
"I'm not even sure how much the players knew. They are very professional boys and gave it their best.
"If I was a gambling man I would say yes, it will be my last game in charge. I am contracted to the club and until the owners and chairman say so that is the case.
"I have 100% belief this squad are good enough to stay in the Premier League."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.