James Coppinger also struck to see off the West Country outfit and complete the first trio of league victories since February 2015.
Yeovil pulled a goal back through Ryan Hedges but had Bevis Mugabi sent off for a second yellow card before Rovers grabbed their third.
Rovers could have hardly wished for a better start as they took the lead with just 34 seconds on the clock.
James Coppinger swung in a corner, Andy Butler nodded it down, John Marquis poked the ball across goal and Williams smashed a superb volley into the far corner from a tight angle.
It could have been 2-0 almost immediately when Marquis met Williams' low cross only to side-foot straight at Artur Krysiak.
But Rovers failed to build on their red hot start as they became a little scrappy, allowing Yeovil to force their way into the game.
The physicality of the visitors appeared to knock Rovers off their stride and they struggled to formulate attacks.
They did come the closer to scoring in the first period however.
Butler curled a header just wide from ten yards after meeting a Coppinger free kick into the box.
And Marquis was unlucky not to poke home as he arrived on the end of a Williams cross just before Krysiak, who blocked well.
Coppinger fired a low effort from the edge of the box which the Yeovil keeper held well.
Despite having the better of the ball, it took until just before the break for the visitors to truly test Marko Marosi.
Ryan Hedges cut inside from the right and smashed a shot which Marosi saved well.
And the Rovers keeper had to be alert moments later when Tom Eaves was played into space and fired at the near post, with Marosi pushing the ball wide.
The second half was more of an even affair in the early stages though Rovers went the closest to scoring.
Krysiak produced a stunning one-handed save to deny a Marquis headed from point-blank range.
The game truly kicked into life in a three minute period, opened by Rovers extending their lead on 54 minutes.
Matty Blair chased a lost cause and sent a deep cross to Tommy Rowe who picked out Coppinger 15 yards out for a fine volley into the bottom corner.
But within moments, the advantage was back down to a single goal.
Not before both sides made a substitution with Harry Middleton replacing the injured Mitchell Lund for Rovers while Yeovil introduced new signing Izale McLeod.
Blair carelessly sent a header back towards Marosi but Hedges lurked and intercepted before drilling low into the corner.
Rowe was denied by Krysiak from close range before Rovers' afternoon got a little easier when Yeovil went down to ten men.
Centre half Bevis Mugabi hauled down Williams as he was set to break clear and was shown a second yellow card.
Rovers almost immediately increased their lead when Butler flicked on Coppinger's free kick only to see it deflect wide. And from the resutling corner, Blair headed over the bar.
Yeovil showed threat on the break and McLeod should have done better when he wrestled through a challenge from Joe Wright only to clip the ball wide.
But Rovers made the game safe with eight minutes to go.
Coppinger waltzed through the Yeovil defence and cut a pass back to Williams who slotted home with ease.
Williams put the icing on the cake of his own performance as he completed his hat trick in added time.
A rose unmarked to meet Rowe's cross and head home from close range.
Rovers: Marosi, Lund (Middleton 55), Wright, Butler, Evina, Houghton, Blair, Rowe, Coppinger (Mandeville 85), Marquis, Williams. Subs not used: Etheridge, Pugh, Fielding, Calder, Beestin.
Yeovil: Krysiak, Shephard, Mugabi, Smith, Butcher, Lawless, Hedges (Whitfield 75), Dawson, Dolan, Khan (Campbell 61), Eaves (McLeod 55). Subs not used: Maddison, Kopp, Sowunmi, Lea.
Referee: Michael Salisbury (Lancashire)
Attendance: 4,686 (167)