Lewis Grabban was dangerous as the visitors made a bright start, but Vale took charge with Richards, on his return after a three-match ban, netting twice in the space of eight minutes.
Richards' 18th minute opener came from a 20-yard free-kick which visiting goalkeeper Conrad Logan allowed to slip through his hands and into the corner of the net.
But Logan was unguarded when Richards clipped his second from eight yards after ex-Rotherham striker Tom Pope headed on a long clearance from Gareth Owen.
The goals knocked the stuffing out of the visitors until Grabban came close to reducing the deficit when Stuart Tomlinson made a smart save as half-time approached.
Grabban had an effort ruled out for offside just after the break when Rotherham started to put Vale under the cosh with a succession of corners and free-kicks.
But Vale stood strong and were content to protect their lead during an uneventful second-half.
Source: DSG