Contrasting Adversarial Perturbations: The Space of Harmless Perturbations