One man was killed and three are still missing following the boiler house collapse in February.
Coleman and Company boss Mark Coleman said he was “so disappointed” at “a decision which prevents us recovering our men.”
A company statement said: “We have now reached a stage where we will be handing over the remaining recovery operation to another contractor within the next few weeks.
“This is of course hugely disappointing as we all wanted to recover our friends and colleagues and return them to their families.
“We wish to make it clear that this decision in no way diminishes the excellent work that everyone involved in supporting the recovery has done.
“The praise we have received from RWE, the Police and HSE for how we have conducted the recovery since it began, in these tragic and difficult circumstances, reflects the dedication and commitment we have to each other and the business.
“It has been explained to us that this decision has been made for reasons of safe-guarding employee welfare and the preservation of critical evidence.
‘We understand these reasons and our immediate focus is to ensure that we continue with the recovery operation for as long as we are permitted and at the same time, work collaboratively to facilitate a seamless transition so that the recovery operation going forward proceeds without delay.
“While the sequence of future recovery operations is not our decision, all stakeholders are agreed that the safety of those involved in the process going forward is of paramount importance.
“We will remain on-site as Principal Contractor, for the wider site, and over the coming weeks we will play an active role in supporting the recovery by working collaboratively with a team of industry experts, assisting in any way we can.
“Once the recovery is complete we hope to resume our works to complete the contract.”
Specialist website Demolition News understands that Brown and Mason will be the replacement contractor.