Seven workers left their jump-forming shift early on Thursday afternoon as winds picked up but they became trapped when the hoist broke down 107m up from the deck section.
It is understood two men panicked fearing for their lives and climbed out of a hatch on top of the lift cage and clambered down the steel running mast using it as a ladder.
The other five remained in the lift and were rescued two hours later.
An enquiry in now underway to find out why the men chose to use the hoist in high winds rather the internal staircase.
A worker on the bridge who asked to remain anonymous told STV News: “They were absolutely terrified. They thought they were going to be killed.
“They had just started coming down for their dinner and were still nearly at the top of the tower when a wire snapped and a wheel came off the lift. The other wires got twisted.
A Transport Scotland spokesman said: “Our contractor FCBC, is currently leading an investigation into the incident.
“Staff were working in safe conditions inside the jump form at the top of Queensferry Crossing’s centre tower.
“As wind speeds picked up they elected to descend the tower via the Alimak, rather than using the safer option of the internal tower staircase.
“Safety procedures are in place to rescue employees from the Alimak in all conditions, these were immediately instigated.”
He added: “The rescue team successfully carried out the practised rescue drill taking the necessary actions to free the controlled descent mechanism, which allowed those on board to return down the tower safely.”
He added that hoists were fully compliant with UK LOLER regulations and routine checks were carried out daily and weekly.
Harry Frew, Scottish Regional Secretary of the Ucatt union, said: “The workers, they panicked. They thought we are in real danger here, this thing could drop.
“Imagine what was going through their minds. People take actions they shouldn’t do, or actions to save their lives.”
He added: “We intend to meet with management at the crossing as soon as possible to build a clear picture of what has happened and we will resolutely support any of our members that have been affected by this incident.”
Earlier picture of central pylon where hoist broke down 107m above the bridge deck
