What should i do if someone is so aggravating that i can't deal with them anymore?

Just tell him off, honestly. If you really don't like him and never want to hang out with him again just do it.