Yep I think it is a combination of SIMPLE plus (to many) "distasteful," and I think probaby the "distaste" outweighs the "simplicity" issue.
In the US at least, distrust and distaste of pleasure was baked in from the 1600s with the Puritans and Pilgrims.