I've had this happen with two locks. There is no set screw. There are a couple of locksmiths on the forum and maybe they'll be able to explain this better than this fable I have just composed.
The wafer that prevents the lock cylinder from being pulled out of the latch housing seems to be held in position by a spring exerting force on the wafer until it is stopped from being ejected by a shoulder cut into the wafer that bears on a stop within the cylinder. You could look at an attached drawing if I had attached one but this is theoretical only. When the spring loses tension or the shoulder wears off, the wafer gets dislocated and can drop into the cylinder or fall out of the cylinder. That enables the cylinder to be pulled out.
I tried to rehabilitate one once but gave in and bought a six-pack of locks instead, one of which I used for replacement.