For the second home game on the bounce, the promotion-chasing Millers left it until the closing minutes to grab a winner.
Wes Thomas set them on their way but Boro grabbed a late equaliser themselves after 86 minutes when Francois Zoko slammed home from close range following the visitors' seventh corner of the second half.
But Rotherham came back strongly and two minutes later substitute Haris Vuckic won the ball on the edge of the area, held off a challenge and then slid a pass to Lee Frecklington, who curled his shot into the bottom corner.
"It was a tough day at the office," Evans said. "I told them it would be and it was.
"We were probably a notch down here on our performance but the one thing they have got is they will run until they drop and they have done that again. It is something footballers are born with - the character and the desire to win - and it is something we have got in abundance.
"We will always go to the death and it was as good a win as we have had all season. It is a good win for us and Frecklington's goal was a stunning finish at that late stage of the game when the pressure is on.
"With 16 games to go, it's getting to squeaky bum time."
Video replays showed Rotherham's first goal was onside despite what appeared to be claims from Stevenage to the contrary.
However, Boro boss Graham Westley stressed afterwards that his players were not claiming offside but felt there was a foul on goalkeeper Chris Day.
Source PA