The Millers had the better of the first half exchanges after York's bright start as Stephen Brogan hit a post and Adam Le Fondre missed two decent opportunities.
Danny Parslow missed the hosts best chance as he danced into the area but blazed over Jamie Annerson's crossbar from 10 yards.
But all the goals came after half time, just as Rotherham had their best spell of the night. Le Fondre curled an effort just wide of the upright before Rankine crossed for Chris Smith to head the opener just past the hour.
And it was two shortly afterwards as Luke Ashworth hauled Rankine back in the box and the striker converted from the spot himself.
Annerson then tipped Racchi's free-kick onto the crossbar to prevent a third.
But Rotherham's sorry evening was complete when Rankine cut in onto his right foot and planted a shot past Annerson as the home side deservedly sealed a trip to Darlington in round two.
Att 2,644 (377 Millers)