After losing to bottom club Darlington in midweek, the Millers climbed back into fourth place with games in hand thanks to hot shot Adam Le Fondre who bagged another brace.
Ronnie Moore made three changes to his team including Gavin Gunning's debut at left back. Fellow Blackburn loanee Marcus Marshall started from the bench.
But after a scrappy opening which saw Nathan Jarman fire in an early shot for the Mariners, Le Fondre set the ball rolling with the opener on 9 minutes.
Gary Roberts picked up the ball and drove towards goal and when he looked set to shoot, instead cleverly slid the ball into Le Fondre's path for him to slot the ball past Nick Colgan.
Robert Atkinson then fired over on 15 minutes for Grimsby who had their best 20 minute spell of the game, before Roberts shot flew over the top at the other end.
Then 8 minutes before the break, Le Fondre doubled his tally with the Millers second.
Nicky Law's corner wasn't cleared and the striker pounced to finish off the scraps inside the six yard box to send Rotherham into half time with a two goal cushion.
It was a scrappy affair after the break, with Kevin Ellison going closest from a free kick after a foul on Roberts and Town's Peter Sweeney booked just after the hour for a heavy tackle.
On 74 minutes Tom Pope could have added a third to make the game safe, but his left foot angled shot beat Colgan but not the far post as it flew wide.
As new boy Marshall made a brief cameo appearance, Grimsby bagged a consolation in the final added minute of three as Wes Fletcher finished off a through pass with a tidy low drive past Andy Warrington into the corner, but the Millers held out.