Former Millers loanee Tom Eaves scored a last-minute winner to complete a hat-trick and knock 10-man Rotherham United out of the FA Cup in the third round.
It was cruel on the Millers, who battled hard after losing defender Adam Thompson to a first-half red card.
The League One side led 2-1 at the break thanks to goals from Michael Smith and Kyle Vassell, after Eaves had put their Championship opponents ahead.
They fought valiantly, but Eaves struck twice more to send Hull through.
A replay would have been just reward for the endeavour shown by Paul Warne's League One side, who were looking to book a place in the fourth round for the first time since 2002, following Thompson's early sending-off for a lunging challenge on Tigers teenager Keane Lewis-Potter.
It would also have meant Vassell's stunning goal was not in vain.
After Eaves turned home a Dan Batty pull-back from close range, Smith's powerful 12-yard strike into the top corner looked like being the goal of the game.
But his striker partner Vassell had other ideas, allowing a throw-in on the left to bounce over his shoulder before he volleyed past the diving Matt Ingram from the corner of the penalty area.
In the second period, Rotherham looked secure in their lead despite their man disadvantage against a higher-division opponent.
But Hull boss Grant McCann, who made eight changes from the Tigers' previous match, turned the match in his team's favour by sending on top-scorer Jarrod Bowen, Kamil Grosicki and Sean Irvine.
Just a few minutes later Grosicki crossed from the left for Eaves to head in and with time running out, he nodded home again to finally end Rotherham's resistance.