The Black Cats have struggled to make their expected strong push for promotion this season and are currently on a run of one win in their last 12 matches.
James - who made 34 appearances for Sunderland before joining Rovers in the summer - admits he is surprised by their position but is confident they will be challenging come the end of the season.
"I think I expected them to be higher, like everyone did,” he said.
"But I think it just shows the quality of this league and it takes a lot of people by surprise.
"There's teams up there that people wouldn't have expected but they've shown that consistency and they're maintaining it.
"It just shows the level of the league and how good it actually is.
"I think it'll be surprising where they are but I think they'll turn it around and be up there challenging.
"I think they have to be with the size of the club."
James describes it as a privilege to have played for the former Premier League side, with whom he enjoyed a run to the League One play-offs last season.
And he says he is looking forward to renewing acquaintances with former team mates and staff on Sunday.
"I think it'll be interesting, seeing a lot of familiar faces and I still speak to a lot of them,” the 26-year-old said.
"I've got good friends still up there.
"It's obviously a different coaching staff now.
"It was a really good experience for myself to play for a club that size in the division it was in.
"I never thought it would be possible.
"I don't think people realise how big a club it actually is until you go up there and see it.
"Facilities are unbelievable, the stadium is unbelievable.
"It was a privilege to play for them.“
Having seen first team opportunities limited towards the back end of the season, James felt it was the right time for him to leave the Stadium of Light when offered the chance to join Rovers by then-boss Grant McCann.
And he feels he definitely made the right choice having enjoyed a strong start to his time at the Keepmoat.
“I felt from the back end of the season when I picked up an injury that it was going to be tough,” he said.
"They had three left backs and the other two were top quality players.
"I felt that I needed to go out and play a lot of games this season.
"I was made aware at the time by the manager that it might be tough to play week in, week out so I made the decision.
"I spoke to Grant McCann at the time and made the decision to come and play a lot of games and try to win promotion with Doncaster.
"I've absolutely loved it here.
"I've been trying to do my best but I think there's a lot more to come.
"Hopefully I can prove that and show what I'm capable of.“