The visitors arrived in South Yorkshire with the best part of a starting XI missing but they managed to claim all the points thanks to Aaron Morley's first-half cracker.
Rotherham threatened early with Matt Crooks heading against the post before Ben Wiles's follow-up was blocked.
The visitors had failed to threaten much before they stung Rotherham with the opener on 42 minutes.
It came from the right flank with 17-year-old Luke Matheson picking out Morley, who fired a curling effort into the top corner from the edge of the box.
Crooks again came close for the Millers at the start of the second half but his effort bounced agonisingly wide.
The home side continued to push for a leveller, with Clark Robertson testing Robert Sanchez from Matt Olosunde's cross.
Rochdale came close to a second, first through Matt Done whose long-range effort whistled just wide and then Callum Camps's free-kick which Daniel Iversen had to push over.