A demolition gang working for contractor J F Hunt had been soft-stripping 91-109 Moorgate when the alarm was raised.
Firefighters were called to put out the fire in the building, which is due to be demolished in several weeks.
The fire damage was limited to a water cooler on the roof, the fire service said.
But the extent of smoke and flames led to a full alert and the closure of the A501 City street and Moorgate tube station, which was later reopened.
A spokesman for Crossrail said the company was investigating the cause of the fire.