I think the secular world looks at a celibate and thinks “If he pulls this off, we’re doomed”, in the sense that if you can give up that much natural pleasure and happiness and still “live well”, the secular humanist vision of the world is more-or-less trashed.

Thus, for the worldly, there is a certain Schadenfreude, but also a little existential relief, in the stories of fallen celibates. By showing the celibate life to be fraudulent and hollow, it justifies the via media they have chosen for themselves.

Likewise, by principle of opposite effect, it brings Catholics terribly close to the abyss of despair, by chipping away our confidence in the basic notion that living for God alone is a worthwhile life, an ideal incarnated by the celibate.