Determinism makes shipwreck of "choice and avoidance", for one thing. And I agree with Cassius on the moral question; there are no "moral people", by any reasonable definition of morality. Actions can be moral or immoral, according to convention; people are just people.
"Then why punish them?"
I don't think we should punish perceived immorality--I think the law should intervene in case of harm, defined broadly enough to mean harm done to persons, property, institutions and governments. Fringe cases to include harm to one's self, harm to reputation, harm where consent is not considered to be possible...etc.
With all of the above to be determined, in Epicurus' view, by human convention.
Is this solution perfect? Naturally not. No thing human ever is!