Researchers tested the memory and blood sugar levels of 141 apparently healthy people with an average age of 63.
None were suffering from diabetes, or experiencing pre-diabetic symptoms.
Participants with lower blood sugar levels were likely to have better scores in memory tests. In one test - recalling a list of 15 words 30 minutes after hearing them - higher blood sugar correlated with poorer memory.
Scans showed the hippocampus brain region, which is important to memory, was smaller in those with high blood sugar.
"These results suggest that even for people within the normal range of blood sugar, lowering their blood sugar levels could be a promising strategy for preventing memory problems and cognitive decline as they age," lead researcher Dr Agnes Floel, from the Charite University Medicine in Berlin, Germany, said.
"Strategies such as lowering calorie intake and increasing physical activity should be tested."
The research appeared in the latest online edition of the journal Neurology.
Dr Clare Walton, from the Alzheimer's Society, said: "The research suggests that regulating blood sugar levels might be a way to improve people's memory, even if they don't have diabetes. However, before people without diabetes consider changing their diets or taking medication, more research is needed to test this theory."