a secret affair in where does this lead?

I don’t know how to end things with a friend who has a girlfriend. I don’t know if I feel something towards him or if I’m just attached. I don’t know if we could be friends after either or if we’re even friends at this point. I’m scared of losing the friendship but what if there is no friendship anymore. I’m not sure where he stands with this whole thing but I don’t know how to confront him about it without him getting defensive and hurting my feelings. I don’t want to be over sensitive as well. We were friends for a few years before we got intimate, he’d constantly pursue me and at one point when he was single we did start to see each other but I ended things due to how serious it was getting and he ended up getting with his current gf after. Idk why I allowed myself to get into this situation, but fast forward a year later it took one drunk night that I hardly remember to initiate where I’m at now with him. I take full accountability but I would never tell his partner. I’m ashamed. I’m also just tired of the sneaking around, I don’t know how to allow myself to “move on” or talk to anyone else due to the idea I’m being unfaithful to him. He’s asked me why I haven’t got with anyone else since him and I told him I don’t feel the need to. In all honesty I mean that I just don’t see a point in me sleeping with someone else that isn’t him, like why would I ? I feel like while some part of my life is on pause his continues but I just don’t know how to express this towards him.