An air ambulance was called to the No.1 block on the athletes’ village after the worker fell three metres but he was taken by normal ambulance to hospital.
Bovis Lend Lease is the main contractor on the block but the injured man is believed to have been employed by a subcontractor.
Work has now resumed on the site following the incident.
An ODA spokesperson said: “At approximately 10am this morning a scaffolder working on the Olympic Village site fell a short distance, sustaining a suspected broken arm.
“The on-site medical team attended immediately, quickly followed by the ambulance service who have driven the worker to hospital for further treatment.
“The incident is being thoroughly looked into to ensure the high safety record we have achieved across the site so far is maintained.”