I don't give much of a damn if the guys notice, or not.
If you did give a damn, would that be because these "guys" had real power over you... power that you could not reclaim at will?
Is the fact that you do not now give a damn a reflection of any change in the world other than your own value choices?
How have these guys whose opinion you no longer give a damn about been convinced to give up their power over you?