Secularists say that parents are often unaware of their rights to withdraw their child from activities such as church services or religious assemblies, and in many cases pupils who do opt out are not given meaningful alternative activities.
They want the Scottish Government to change the law so that religious observance becomes an opt-in choice rather than the current opt-out.
Holyrood's Public Petitions Committee considered a petition on the issue from parent Mark Gordon and Secular Scotland.
Mr Gordon said research suggests that "only 20% of parents say the school informed them of their opt-out rights".
Information on parents' rights is missing from numerous school handbooks while, in some cases, teachers are unaware of the options, he told MSPs.
"The law does mandate that 'in no circumstances should a child be disadvantaged as a result of withdrawing from religious observance' and should be given a 'suitable worthwhile alternative activity'," he said.
"In my experience and that of many other parents, this is most certainly not the case.
"My daughter is made to sit in the school office with paper and pencils to draw with and is looked after by the school secretary since there are usually no teaching staff available.
"It seems to me that a far more sensible and fairer approach could be to turn the situation around so that the default position was opted out."
Caroline Lynch, secretary of Secular Scotland, said: "We are saying there is failure endemic throughout the school system on this. It is happening everywhere, and that is why it needs to be urgently reformed in our opinion."
The Church of Scotland has already responded with a letter to the committee.
It wrote that "religious observance is no longer tied to any one faith community's creed or liturgical calendar. Nor should it be. It is instead focused on the beliefs and values that shape and are shaped by each school community".
The Church wrote: "To argue that it should be opt-in rather than opt-out would be to diminish the educational experience for young people in the same way as to remove personal social development would affect severely the capacity of a school to deliver on the four capacities as its primary goal."
But Ms Lynch said that while religious observance is supposed to be non-denominational and should not contravene anyone's personal faith stance, "the reality on the ground is that this is, in most cases, Christian worship".
The committee agreed to write to the Scottish Government and seek the views of various religious and education organisations.