The men found a skull and part of a femur as they worked in the burn on Sunday morning.
Police cordoned off the area, around a mile from Wanlockhead in Dumfries and Galloway, and found another bone.
It is thought they may come from paupers' graves outside the nearby Meadowfoot Cemetery, which dates back to 1751.
Heavy rain in recent weeks has caused erosion and the bones may have been washed into the burn.
The gold panners, from the Hampshire area, reported their discovery to the Wanlockhead Museum of Lead Mining, which called police.
The bones were examined and found to be very old, police said.