Andy Barcham scored the only goal of the game 12 minutes from time to move the Gills up to seventh and leave Rotherham six points behind them with two games remaining.
Rotherham managed just one shot on target in the first half, with Tom Pope's strike lacking power and Alan Julian saving comfortably.
Barcham has two good efforts for the visitors, Jamie Annerson saving well after 38 minutes.
And a minute later Barcham latched on to Ade Akinfenwa's pass and rounded Annerson, but Dale Tonge got back to clear off the line.
And in first half stoppage time Annerson made another two good saves, first to deny Akinfenwa and then turning Cody McDonald's follow up away for a corner.
Pope went close for Rotherham 10 minutes after the break and then Ian Thomas-Moore shot just over the bar while at the other end Annerson again denied the visitors, saving with his legs with McDonald clean through on goal.
But Gillingham finally got the breakthrough after 78 minutes when Barcham shot from just inside the area and a wicked deflection off Ryan Cresswell took the ball up and over the helpless Annerson.
Source: DSG