Just found this article by Richard Glen Boire, director and chief legal counsel for the Center for Cognitive Liberty and Ethics (formerly known as the Alchemind Society):
'The opposition is coming from people who might be called bioconservatives. They believe that drugs should only be used to make sick people normal, and should not be used to improve, enhance or selectively modify the mental functioning of healthy people. Their concern is often rooted in religious beliefs that perceive the nonmedical use of drugs as immoral and unnatural. Attempting to improve human functioning, rather than simply correct malfunctions, they say, is to audaciously take on Godlike powers and endeavor to skirt natural limits that the Creator placed on human beings.
Last fall, the US President's Council on Bioethics, an influential panel of 18 scholars appointed by George W. Bush, released a 300-page report titled Beyond Therapy: Biotechnology and the Pursuit of Happiness. Among other topics, the report addressed memory-management drugs.
While the council carefully avoided explicitly structuring its analysis on conservative religious values, these are deeply infused in its report. The majority of the council members are ardently religious.' (Betterhumans article)