You could try that, certainly. But the best thing to do would just be leave them alone. They usually don't bother people.

...It's all theoretical. Some say the fairies hurt you or give you bad luck.