First-half chances had been at a premium, but Coventry fired a couple of early efforts wide of the post before a Ben Pringle cross-shot at the other end forced a save from Joe Murphy.
The best chance fell to Carl Baker midway through the half when striker Callum Wilson pulled back across goal but the Coventry captain's effort was blocked on the line by Joe Skarz.
Rotherham took the lead in the 58th minute thanks to a precise left-wing delivery from Pringle which Nouha Dicko guided into the corner of the net for his first goal since joining on loan from Wigan with a deft flick.
It got even better for the visitors in the 70th minute when Pringle again crossed, Dicko this time headed back across goal and Alex Revell headed past Murphy from close range.
The result was put beyond doubt three minutes later when Dicko was brought down by Aaron Phillips in the box after some tricky play and Kieran Agard made no mistake from the resulting penalty.