The May reading fell to 51.2 from 52 in April.
The index has been reading above the critical 50 mark – showing growth – for three years but May’s figure showed the slowest rise during that time.
The May data also recorded an outright reduction in new order volumes for the first time since April 2013.
The report stated: “Anecdotal evidence pointed to a general lack of client confidence, driven by heightened uncertainty about the economic outlook.
“Moreover, a number of firms noted reluctance among clients to place orders and commence contracts until after the EU referendum.”
A third of buyers said their business had suffered during uncertainty surrounding this month’s EU vote.
Looking ahead, UK construction firms are upbeat overall about the year ahead outlook, with just over half of the survey panel forecasting a rise in output.
David Noble, Group Chief Executive Officer at the Chartered Institute of Procurement & Supply, said: “The sector appeared to have taken residence in a waiting room of non-activity, as continuing poor global economic conditions and uncertainty around the EU referendum impacted growth and new orders.
“The chink of light appeared to be the industry’s optimism for the future and an indication that that this may be a short-term situation as employment continued to rise to a four-month high and job creation has been continuous for three years
“Once again the industry holds its breath to see what thecoming months will bring as housing sector growth in particular remains weak.”