The Millers had won their previous five at home but the Gills fully deserved their third league win of the season.
The visitors took a second-minute lead with Josh Parker bundling in from Billy Bingham's corner.
Rotherham's top scorer Kieffer Moore then glanced a cross from Joe Newell just over the crossbar.
Gillingham started the second half strongly and doubled their lead after 48 minutes when Jake Hessenthaler picked out Tom Eaves to head in from close-range.
Moore struck one back in the 57th minute as he latched onto a long ball from Darren Potter and powered past Tomas Holy.
Rotherham's hopes of gaining a point faded when Mattock was sent off with five minutes to go following two yellow cards in quick succession for a foul and dissent.
Gillingham sealed the victory after 86 minutes with Bingham and Parker combining again from a corner.