Staff from the Human Fertilisation and Embryology Authority found patients at Glasgow Royal Infirmary's assisted conception unit had to wait up to six weeks to see the service's only part-time counsellor.
They also found there was no service when the employee was off sick or on holiday.
The criticism emerged in an inspection report of the centre, which had applied to start treating patients again after it suspended carrying out IVF cycles almost two years ago.
Inspectors described counselling provision as a "major area of non-compliance" with the terms of its licence and said staffing levels needed to be increased by the middle of this month.
The report said: "Concerns were raised regarding delays in providing counselling appointments of up to six weeks.
"The counsellor is also only available for 24.5 hours per week and no second counsellor is formally available to cover holiday and sickness.
"The person responsible should ... review the workload and availability of the counsellor against the proposed activity level and document the review in a workforce-activity assessment, as has been done for clinical, nursing and laboratory staffing. This recommendation should be implemented by 15 August 2014."
Many couples struggling to conceive want the support of a counsellor to help them through the stressful IVF process. Men and women donating sperm and eggs also have counselling.
Juliet Le Page, from Edinburgh, who has two children conceived via IVF and runs the support group Fertility Concerns, said counselling was vital. She said: "Very often couples and individuals have a lot of doubts and worries over whether they should be going for IVF at all. The longer they have to wait to speak over their worries with a qualified counsellor the more the likelihood is that their stress levels will increase. This in turn will have a further detrimental impact on their treatment."
The HFEA inspection was carried out following the completion of a £3.5 million refurbishment to the unit. Inspectors approved the go-ahead for doctors to resume carrying out IVF treatments. They will begin consultations this week and begin fertility treatments from mid-September.
Some 1,500 patients from the GRI were switched to the private Nuffield hospital in November 2012 after rates fell from around 30 per cent per cycle to 15 per cent from September to November 2012. An internal investigation suspected pollution from nearby renovation works may have contaminated the fertilisation process.
A spokeswoman for NHS Greater Glasgow and Clyde said: "We have recently received the inspection report and the recommendations from HFEA. We are reviewing the counselling service to identify what resources are required to enable the service to meet the needs of patients."