The Millers dominated the first half but eventually went down 3-0, conceding goals at crucial times and failing to convert their own pressure into goals.
Patterson told Radio Sheffield afterwards: "I am obviously disappointed with the result, but first half we should be comfortably three or four in front.
"We absolutely battered them, couldn't convert our chances and get done with a sucker punch. I am disappointed second half we conceded such a sloppy goal early on.
"We've come here to win, so we've had a little gamble, put three centre-forwards up front to have a go and we conceded another soft goal."