Blades boss Nigel Clough made wholesale changes to his starting line-up, resting a number of players ahead of Sunday's match against Hull at Wembley.
In a fast-paced first half, Rotherham had the best effort when James Tavernier ran into the area and fired in a shot which was well saved by George Long.
Tavernier also slipped a good ball through to Tom Hitchcock inside the area, forcing Long to block with his legs.
Rotherham's Alex Revell headed straight at Long and then put a thumping drive wide.
The visitors made a couple of penalty appeals early after the restart, first a handball inside the area from Davies and then when Hitchcock went down under a challenge from Kieron Freeman.
The hosts rarely put Adam Collin under any pressure with the goalkeeper comfortably saving Harry Maguire's header.
However, the winner came when Davies tucked away a spot-kick, which was awarded after Rotherham captain Craig Morgan clashed with Maguire at a corner.
Source PA