Blythe, 20, has impressed at the Swans after spending the last month training with their under-23s squad.
A deal is in place for the Leeds-born centre back to join the Championship club at the end of the season.
Rovers boss Gary McSheffrey has confirmed that Rovers will receive a fee and the agreement includes potential add-ons and a sell-on clause.
Blythe made seven first team appearances for Rovers after making his debut on the final day of last season against Peterborough United.
"Ben’s been in at Swansea for about a month and he’s played in a couple of trial games,” said McSheffrey.
"He’ll be with them for the remainder of the season, training with their under-23s.
"He’s impressed and there’s a deal in place for them to sign him and take him into their under-23 programme.
"I obviously played Ben a few times when I first took over,” he added.
“He’s got some good young centre backs ahead of him in the pecking order here.
"Ultimately a Championship club are going to buy one of our young players so we’ve done something right in developing a player.
"For me he still needs a couple of years of development football to fill out and to be physically ready and robust.
“I think in the games he played for us it’s clear to see that physically he wasn’t ready for League One football.
"Ultimately we can’t give him that time and guarantee of game time to have that period to develop and he’s going into a Championship club and category two programme that’s going to serve him well for a couple of years.
“At the same time the club have got a decent deal for him.”