So except his friendship... if you can.
Better to have loved and lost than not to have loved at all.
As long as you in your mind can except just being distant friends then why do you have to loose him, however, if it’s going to effect you going out and finding someone else then let him go.
Those are the two sides of the scales and only you can answer which you can live with.