The 44-year-old from north-west London was walking with another man when he fell from the sea wall at Friars Bay, Peacehaven at about 0920 BST.
Coastguards said a group of construction workers went to his aid and one managed to grab hold of him but the force of the waves swept him away.
He was found by a lifeboat crew shortly afterwards and airlifted to hospital where he was pronounced dead.
One of the construction workers was taken to Brighton Hospital suffering from hypothermia and head injuries.
The other walker was also taken to hospital for treatment for hypothermia.
A spokeswoman for the Solent Maritime Rescue Coordination Centre said: “A group of construction workers went to the man’s assistance by throwing life rings into the water and due to a large surge the man lost hold of the life ring.
“One of the construction workers entered the water to recover the man and pull him back to the groyne, the waves then broke again on them and the man was lost from the grasp of the construction worker.”
She added: “The man was found shortly afterwards and taken by helicopter to Eastbourne Hospital.
“Our sympathy goes out to the family and friends at this tragic time.”