Adam McGurk's last-minute strike gave Burton a point after Bury had led 2-0.
Anton Forrester put the Shakers ahead, cleverly side-stepping two defenders before curling in from 18 yards.
Debutant William Edjenguele headed home a corner before the break, but Brewers striker Michael Symes nodded McGurk's cross in to set up a thrilling finale.
And McGurk turned from provider to scorer when his shot had just enough power to carry it through the hands of Bury goalkeeper Rob Lainton.
VIEW FROM THE DRESSING ROOM
Burton Albion boss Gary Rowett told BBC Radio Derby:
"We've showed so far this season that whatever happens in the game we are capable of handling it.
"I felt it was one of those games where everyone talks us up and I had a sneaking suspicion we were going to let ourselves down a little bit.
"I have to say I thought Bury were excellent, as good a side as we've played so far. They played with bags of energy, almost did what we tried to do to teams but did it just a little bit better than us.
"I said at half time we've got to show character and we have done."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.