Why not? I mean, what is so scary about this idea?
The good news is that the good moments you experienced together will never be totally gone.
You can always recall memories if you want to.
The truth is that the more you dwell in the past, the less you are able to focus on your future.
Create space rather than limiting yourself!
You will usually notice that when you see him as a friend it just makes you want him back.
See the contradiction?
This means that right now, it is easier to live without him.
You are not running away from anything by saying "no" to an active friendship for now.
By focusing on your own life, you are just embracing the possibilities which are ahead of you.
Right now, he is like a weight which limits you emotionally and has the power to stop your from moving forward.
You will notice that once you rebuild your new life fully, seeing him won't challenge you at all.
At that time, you will probably notice that even the desire to spend time with him might be totally gone as well.
To your power!