Police were called when rubble started falling from the six-storey structure into Grosvenor Square.

No-one was injured but police described the building as “quite unstable” and it is being assessed by structural experts.

Eyewitness Sue Harrison, who was working in ground-floor office opposite the building, told the BBC she heard a “huge noise” and saw rubble falling on to cars.

She said: “It just bulged out and all the masonry fell on the cars.

“Then our fire alarms went off and we all came out of our building.”

The work is being carried out by Southampton-based Wessex Demolition & Salvage.

A company statement said: “Wessex Demolition and Salvage Ltd would like to offer its sincere sympathy to all of Southampton’s residents and commuters affected by the problems with the site at Cumberland Place.

“We are still looking into what caused the structural failure and do not want to speculate before we have the conclusions of those investigations. In the meantime, we are working to ensure the site is made safe as soon as possible so the inconvenience caused can be minimized as much as possible.

“This is the first incident of this kind we have had in the eighteen years we have been operating; we have previously worked on a number of structures offering similar, if not larger, challenges without any mishaps.

“Safety is always the top priority for Wessex Demolition, as recognized by our clients. Following this most unfortunate incident all we can do is what we are doing: working as hard as possible to make the site safe and reduce its impact on Southampton and its people.

“Once again, we offer our sincere apologies to everyone affected, and our assurance that we are doing all we can to ensure the impact is as short-lived as possible.

“Finally, we would like to offer our heartfelt thanks for the many, many messages of support we have received this week, from the general public, our clients and fellow contractors.”