Rotherham were more than a match for their third-tier opponents although clear-cut chances were at a premium for most of the game.
Millers goalkeeper Josh Vickers was eventually called into action to tip away Andre Gray's shot and also reacted to deny Charlie Austin after a corner routine.
But the visitors dominated as the match progressed, with Ogbene's header cleared off the line and Hakeem Odoffin bringing a diving save from Archer - who was playing his first game since August.
Rotherham continued to threaten during the extra half-hour and gained their reward when Harding picked out Ihiekwe, who controlled and sidefooted home.
However, substitute Dykes - who had been foiled by Vickers in regulation time - levelled with five minutes left, meeting Albert Adomah's cross with a header that the keeper could only push onto the bar and across the line.
Andre Dozzell was the only QPR player to miss in the shoot-out, with Vickers' save ensuring sudden death before Archer pushed Ogbene's low penalty around the post to send the Championship side through.