The National League side looked set to reach round three for the first time in their history when Jimmy Ball tapped in just after the hour following early goals from Jamey Osborne and Alex Gudger.
But the Millers, who had previously barely threatened, scored four times in the final 14 minutes to turn the tie on its head.
Freddie Ladapo started the comeback when he side-footed home before Michael Ihiekwe curled in to give the League One side hope.
Michael Smith then twice headed in crosses by fellow-substitute Chiedozie Ogbene to earn Rotherham a dramatic victory and secure a home tie against Championship side Hull City.
The Moors began at a frenetic pace and went in front in the sixth minute through Osborne's half-volley from 18 yards before doubling their lead two minutes later when Gudger side-footed home Lee Vaughan's cross from eight yards as Rotherham appealed in vain for offside.
Jimmy Ball then made it 3-0 from Jake Beesley's low cross to seemingly put the tie out of Paul Warne's side's reach.
Even the hope given to Rotherham by Ladapo pulling one back could have been extinguished had Beesley not fired well off target from 15 yards after the ball broke to him kindly.
Instead, within moments it was 3-2 as Ihiekwe struck following an intense spell of Rotherham pressure.
Matt Crooks then hit the post for Rotherham as they pressurised the Moors backline with wave after wave of attack.
That pressure finally told when Smith headed in Ogbene's cross at the far post to level, before the pair combined once again in stoppage time to end Solihull's hopes of at least earning a replay.