The Treasury set-up the Infrastructure Financing Unit in March 2009 as the credit crunch scuppered PFI schemes.
The Unit gave public backing to private projects which allowed them to kickstart finance.
A report by the National Audit Office found that the extra finance costs to the taxpayer were value for money in terms of stimulating the economy.
But the watchdogs now want to see future schemes reviewed on a project-by-project basis to ensure no more taxpayers’ cash is wasted.
Amyas Morse, head of the National Audit Office, said: “By introducing an Infrastructure Finance Unit, the Treasury helped reactivate the market and prevent the stalling of many government projects.
“During 2009, the cost of finance built into the PFI programmes at that time was value for money, but there is no guarantee that it will remain that way.
“Now that the market is providing finance again, a project by project review should be carried out using stricter criteria, to establish the most appropriate funding methods.”
The Unit helped to finalise a large waste treatment and power generation project.
Subsequently, 35 projects, including the contract to widen and maintain the M25, were signed without any further public lending.
In line with policy on acting to stimulate the economy, the Treasury and other government departments gave priority to closing deals at the prevailing market rates, even if this meant the public sector paying more, and the banks carrying less risk.
Analysis by the NAO suggests that higher financing costs increased the annual charge of PFI projects by six to seven per cent and that between £500 million to £1 billion of higher cost has been built in over 30 years.