How come we say "by accident" but also "on purpose"?

Why is one on, and the other by? What difference do the words accident and purpose have to change the conjunction in front of it?