The points loss has seen them fall out of the play-off places and they now look to have lost any real chance of making up the lost ground.
When City were reduced to ten men with 20 minutes to go after Omar Daley received a straight red card for an off-the-ball incident with Graham Coughlan, Rotherham really turned the screw on the visitors but City's fighting spirit saw them through.
Rotherham dominated the first period and Chris O'Grady had the ball in the net after 33 minutes but his effort was ruled out for offside.
And the home side had a good shout for a penalty turned down when Peter Holmes raced through and appeared to be tripped but the referee waved aside Rotherham's claims.
Jamie Yates should also have done better than shoot into the side-netting after 24 minutes when he had been put clean through by O'Grady.
Bradford's main threat before the break came from set-pieces. Eddie Johnson got in a header from a Paul Heckingbottom free-kick and then Johnson followed up by driving a free-kick high over the bar.
Barry Conlon brought a good save from Andy Warrington in the Rotherham goal just before the break.
And within three minutes of the restart the visitors took the lead. Tom Penford fed Joe Colbeck wide on the right and he cut in to hit a shot across goal from the edge of the penalty area which hit the left-hand upright and went into the back of the net.
But City's defence was caught flat-footed after 65 minutes when Rotherham equalised. Marc Joseph swung in a free-kick from the left and O'Grady surprised everyone stealing in on the far side to fire home.
But hard as Rotherham tried in the last 20 minutes they could not find a way through City's resolute back-line.