Clifford had had a heart attack and was taken to hospital last week after collapsing in his cell at Littlehey Prison in Cambridgeshire.
Reports suggest he has now died in hospital.
He was sentenced to eight years in prison in 2014 after being found guilty of eight counts of indecent assault on four girls and women aged between 14 and 19-years-old.