Mace was awarded the contract in December to deliver Phase 1 of the scheme by developers Oxley Developments and Ballymore.
But the Enquirer understands work has fallen behind schedule on the site.
One site source said: “Nothing much has been doing there for a few weeks now.
“The lads have been just hanging around most of the time waiting to hear what is happening.”
Mace is now being replaced on the job by Roundstone Construction Services which is believed to be a subsidiary of Ballymore.
A statement from the developers said: “Oxley Developments and Ballymore have confirmed they are continuing with further phases at Royal Wharf, following the successful commencement of Phase 1 by Mace, which includes the completion of the pre-construction phase and procurement of the primary packages.
“This now provides a stable production plan for the delivery phase of the project, which will enable Roundstone Construction Services to deliver the remainder of Phase 1 and Phase 2 of the project.
“Mace will be undertaking a planned transition over the next few months and Oxley and Ballymore would like to thank Mace for their support to date.”
The developers said Royal Wharf will become the biggest new Docklands neighbourhood since Canary Wharf was built.
It will bring a mixture of over 3,000 apartments and town houses, retail space, cafes, a school, restaurants and a state of the art gymnasium as part of the overall regeneration of East London.