If you can find a better word than charisma, I'm in.

Looking around at a few articles on the web I do see that charisma is often used, even for negative personalities. Here is an interesting one, with a statement on the difference (found in a comment on the article): The Psychology of Cult Leaders
Quote
Thoughts on Cult Leaders. We know genuine charisma has elements of morality in it. Pseudo-charisma is the combination of personality traits that mimic true charisma and therefore can be very dangerous for a leader to possess (PSY, 533). Cult leaders have very typical traits: a grandiose idea of who they are and what they can achieve; demand blind unquestioned obedience; require excessive admiration; have a sense of entitlement; are exploitive of others and put others at risk frequently – financially, physically, and mentally; are arrogant and haughty in behavior and attitude; have exaggerated sense of power that frequently results in breaking laws; they take sexual advantage of cult members; they humiliate and devalue members to deliberately make them feel inferior, incapable and not worthy; they ignore the needs of others, including biological, physical, and emotional; and they treat people as though they are objects to be used, manipulated or exploited for personal gain (Navarro, 2012).
One observation about charm - many of Trump's followers don't see him as an a$$hole, they really do think he is charming.

You never change things by fighting the existing reality.
To change something, build a new model that makes the old model obsolete.
R. Buckminster Fuller