Bury are still searching for their first win of the season following defeat at home by Notts County.
The Shakers were behind after just two minutes when Lee Hughes diverted Alan Judge's free-kick into the net.
Lateef Elford-Alliyu then had a great chance for the hosts but headed straight at keeper Bartosz Białkowski.
And with Bury pressing for an equaliser, a quick Magpies break allowed Judge to finish past Trevor Carson and seal victory.
VIEWS FROM THE DRESSING ROOM
Notts County manager Keith Curle told BBC Radio Nottingham:
"Credit to Bury, they stuck to their game-plan, flooding the midfield, good movement, good rotation in their play.
"Even though we got the early start from a free-kick that went in, they caused us problems.
"There's a unity building and you get it from coming to places like Bury.
"Make no mistake, this is not an easy place to come to."
Bury 01 Carson 02 Picken 03 Skarz 05 Lockwood 17 Hughes 06 Sweeney 07 Worrall (Jones - 77' ) 08 Schumacher 13 Carrington (Marshall - 66' ) 12 John-Lewis 19 Cullen (Elford-Alliyu - 58' ) Substitutes 20 Dibble 16 Sodje 04 Byrne 09 Jones 11 Marshall 18 Harrop 15 Elford-Alliyu Notts County 01 Bialkowski 02 Kelly Booked 03 Sheehan 06 Leacock 08 Liddle 04 Bishop (Mahon - 84' ) 10 Judge 11 J Hughes 13 Zoko (Labadie - 66' ) 28 Boucaud Booked 19 L Hughes (Nangel - 90' ) Substitutes 23 Speiss 27 Regan 14 Mahon 16 Labadie 18 Bencherif 31 Nangel 09 Showunmi