A Ryan Allsop own goal after a stunning effort from Tommy Rowe had Rovers ahead but Blackpool equalised inside the next minute through a brilliant Sean Longstaff strike.
John Marquis put Rovers back in front in first half stoppage time but Blackpool levelled thanks to a wonderful goal from Oliver Turton.
It looked like Rovers had tipped the game firmly in their favour when substitute Alfie May slotted home with 15 minutes to go.
But Blackpool hit back straight from the kick-off with Rovers failing to touch the ball in a sweeping team move finished by Callum Cooke.
And it left Ferguson frustrated.
"When you're three times in the lead you would have to say it's two points dropped," he said.
"I thought we controlled large parts of the game so I'm slightly disappointed.
"I think we've got ourselves to blame for not winning the match.
"Sometimes you have to say the opposition did this or that but we've done it to ourselves with just lapses of concentration.
"It's rare that you get two goals of the quality of Blackpool's first two in one game.
"But the third goal is terrible. It's less than a minute after we score and we've not touched the ball.
"I thought with concentration and being careless, we've done that too much for my liking.
"It was a game we should have won but they're all tough at this level and Blackpool have got some good players in their team.
"We've started the season well and we're unbeaten. I'd probably say we deserve two more points that we've got."
May grabbed his third goal in as many games after coming off the bench.
And Ferguson says he has earned himself a start in Tuesday night's Carabao Cup tie with Hull City.
He said: "All I've said to Alfie May was if he keeps doing what he's doing then he'll force my hand.
"He'll certainly start on Tuesday, there's no question about that.
"I thought he was outstanding when he came on.
"Him and Rodney Kongolo gave us real energy when they came on and that's what you hope they do."