It is simple: respect and diplomacy.
When he calls, you can say: "Hi, how are you?" Let him speak for a few seconds.
Then interrupt him and say: "You know what? Now is not a good time. I have to go to ... or deal with..."
If he insists, repeat: "Sorry now is not a good time..."
And say something like: "I have to go now. Talk to you another time. Take care. Bye"
That's just the general idea.
If you meet him, same story: "Hi!", "Bye, see you later"
Cut any unwanted contact you have with him to 1 minute or less.
That's the best way to not let him in.
You'll be amazed of how good it feels when you apply this simple strategy.
To your power!