On an awful Don Valley pitch of which a third was ruined folowing Thursday's U2 concert, the Millers started brightly as Paul Warne fired the opening drive which was tipped away for a corner by keeper Kenny Arthur.
And in the opening 15 minutes Tom Pope also headed into Arthur's arms from Harrisons flag kick after a jinking run by Nicky Law was blocked by Michael Flynn.
Minutes later Dale Tonge's super cross from the flank was headed wide by a diving Pope as Rotherham continued to press.
Dale, with one win from their opening three games, started to come into proceedings with Will Buckley and veteran Gary Jones coming into the game in midfield.
On 35 minutes Buckley's ball in was just out of Chris Dagnall's reach as the Millers defence, minus captain Ian Sharps through injury, fell under a period of pressure as the visitors forced their only first half corner.
But just before the break the game sprung into life with two goals in two minutes as firstly the Millers broke the deadlock.
Nathan Stanton fouled Warne on the edge of the area and picked up the first booking of the afternoon.
Le Fondre pounced with a flick header to turn home Nicky Law's free kick to pile misery on his old club.
However joy was shortlived as Dale clawed their way back in the second and final added on minute of the half as Dagnall's shot from close range squirmed under Andy Warrington as the teams headed in level at the break.
Dagnall almost had the best chance of a scrappy second half opening after Chris O'Grady's flick on just evaded him with the ball running through to the greatful Warrington.
The game continued to drift along until the ever involved Warne got on the scoresheet himself just after the hour.
Tom Pope laid the ball off for the veteran to fire home into the corner past Arthur as Rotherham regained the lead.
Dale responded as O'Grady set up the dangerous Dagnall once again who sped into the area only to have his shot parried by Warrington as the game livened up once again, with both sides making a couple of substitutions.
With little to shout about and the game following its stop start pattern a Rundle free kick was cleared after a clash between Tom Kennedy and Jamie Green in the final 10 minutes.
Rundle fired over the bar from 10 yards after O'Grady again set up an opening and a brave save from Warrington from Dagnall after a loose free-kick in stoppage time kept the visitors at bay as the Millers triumphed.