Figures obtained by Ucatt show there were a total of 9,219 inspections in 2015/16 compared to 9,656 in 2014/15 – a decrease of 4%.
The number of construction inspections has been falling steadily in recent years.
In 2012/13 the HSE undertook 10,577 inspections the latest figures reveal that in three years the number of inspections has declined by 13%.
The majority of regions in the UK saw a decline in inspections in 2015/16.
The greatest drop was in the South East where inspections fell by 26% which comes on top of a 19.6% reduction in the previous two years.
Other regions to record declines in 2015/16 were: Wales 18%, Eastern 13%, Yorkshire 13%, North East 10%, West Midlands 7% and London 5%.
Brian Rye, Acting General Secretary, of construction union UCATT, said: “These statistics are far from meaningless. They paint a serious picture of how a resource starved HSE is increasingly unable to do its job.
“Inspections save lives and prevent accidents. The reduction in inspections is endangering the lives, health and wellbeing of construction workers.”