What got me most about this trip was the fact of Cambo Estate's very existence.

Situated near Kingsbarns on the well-trodden A917 tourist route between St Andrews and Crail in the East Neuk of Fife, this mansion house and grounds have been in the Erskine family for 300 years, yet they only reveal themselves once you've turned off the road and navigated a long and winding tree-lined drive. When you arrive, as we did, in the pitch dark, its mysteriousness is all the more enigmatic.

We entered the house somewhat nervously to seek the key to our self-catering apartment in the Gardener's Wing, which is situated through a big stone archway along from the clock tower from the outside, and from the inside via a series of corridors and locked doors. Then up a couple of flights of stairs, and we were there.

The two-person Orchard apartment is blissfully warm and private, its big wooden door opening into a lounge with open-plan kitchen; the bedroom, complete with little ensuite shower room, is off that again. Large picture windows in the lounge look onto the front gardens of the estate – though we didn't realise that until the next morning.

The Orchard is just one of a series of holiday apartments, which sleep from two to 10 people, within this fine old house, which was rebuilt in 1879-84 after a fire destroyed the late 18th century original. We hadn't brought any food with us and were starving, and with nothing in the fridge we were glad to have had an advance dinner reservation made for us at the glass-fronted Seafood Restaurant in St Andrews, seven miles away.

Waking in the morning after a good night's sleep in the luxurious beds, we discovered a whole new world. Suddenly we could see the gardens that have helped make this place so special. After a breakfast of porridge, fresh raspberries and yoghurt (me) and the full Scottish (him), served in the magnificent dining room of the house proper, we were keen to see more of this fabulous place. I sneaked a look into the adjoining drawing room and was bowled over by its scale and beauty: huge picture windows, oriental carpets, glass chandeliers, massive period fireplace and furniture, and a very strong sense that this is the much-loved home of a family that enjoys frequent company.

So, walking gear on, we set off to explore the walled kitchen garden – a sprawling 2.5 acre site rated as outstanding by Historic Scotland, and which, uniquely for a Victorian walled garden, has a burn running through it towards the sea.

There are six original cast-iron bridges on the estate, and two of these are in the walled garden. Apart from its maze of box hedges, lilacs and roses, and hugely impressive kitchen garden, there is also a small waterfall overlooked by a stone hump-backed bridge, tea house and weeping willow.

All the ancillary buildings here – the five glasshouses (including a peach and vinery with the original vine still growing), plus the round-headed arches, pediments and dovecot – date back to the late 18th and early 19th centuries, when the 9th Earl extended the original house from 1795. The walled garden is also unusual because it's still being cultivated.

Cambo hosts the annual Snowdrops by Starlight festival every February and an attractive feature of a walk around the wider estate is that although breathtaking in its beauty it's not exactly picture-perfect – much of it is in a state of disrepair. There are plans to restore the disused Victorian/Edwardian glasshouses, coach house and stables. The intention is to use these buildings for the Cambo Institute, which was formed in 1998 to provide learning, training and volunteering opportunities for young people.

In the end, it was difficult to leave this haunting place. So I bought some packets of Cambo snowdrop bulbs so that, when they bloom, they'll remind me to go back.

www.camboestate.com