PricewaterhouseCoopers LLP - which trades as PwC - is shutting its base in St Paul's Place and five other regional offices in Plymouth, Liverpool, Norwich, Swansea and Northern Ireland from April 2018.
The firm has 74 staff at the flagship city centre development, who were relocated there in 2014 when bosses were keen to emphasize PwC was the only "Big Four" global accountancy firm to be investing in South Yorkshire.
A spokesperson for the company said there are no planned redundancies "as we want to keep our talented staff."
The firm said the intention was to "consolidate our presence into fewer, larger regional offices" and added the restructure is about "the way in which PwC is expanding its presence outside London, and new ways of working."
The move has reportedly sparked concern among some staff who had relocated to the city and bought homes in the area.
It is understood PwC's Leeds site will become the new base office for Sheffield staff from April next year.
Andy Ward, PwC Sheffield's senior partner, said: “We are committed to supporting our clients in Sheffield, and our redeployed employees will work with other PwC colleagues to provide a broader range of services to help clients in Yorkshire with the issues and opportunities they face.
“Given our employees are equipped with a laptop and smartphone, and often work from client offices and other remote locations, we are consolidating our regional presence into a smaller number of large offices around the UK, such as Leeds.
“We’re bringing together our people and technology in these offices to deliver a wider range of integrated services in a way that best suits what our clients want.
“PwC is expanding its presence outside London to match the changing footprint of our clients and make the most of new opportunities in fast growing regions and the firm has accelerated its investment in the UK regions.
“We have recruited nearly a thousand new staff outside London and are committed to new or additional office space to increase our regional headcount by a further 1, 250 people.”