How to rebuild a long term relationship after infidelity?
In short, unless you can accept him and what he has done. Unless you can truly trust him and believe he is worthy of that trust to be in a relationship where you can depend on him. Unless you see these qualities, you should not stay in the relationship.
You have to have a foundation to rebuild on before you can talk about rebuilding. That foundation should consist of character traits that you think are worthy of investing in. This man, he was having an affair for the last 3-4 years. If the guy acts the same in an affair as he does being committed to the relationship, then what kind of man is this?
Be honest with yourself when you address the questions I mention here. Be honest in looking at this man and what he offers to your life. Be honest in identifying if he can be committed to you.
If he was having an affair for 3-4 years, there is a lot of details that have to be understood. For one, can he break ties cleanly with this other woman? Who is she to him? Do they work together and know each other in some way outside of intimacy? Does he even want to work on your relationship or save face and stay together where he is safe? It's good to know if people choose us because they want to be with us or if they choose us because the other option wasn't obtainable.
The questions I have are intended to invoke thought for you and to realize the context of his actions. He was not just cheating; he was lying to her, you, the kids, himself, and everyone around to be this other guy. All of that energy that wasn't going toward you or the kids was in service to preserve this selfish desire to have his cake and eat it too. There is no sugarcoating it when it comes to long-term infidelity. The man is weak, so I question his weakness in coming to the surface again.
So, is it worth investing in? You need to identify what you want and need, and choose from a place of what is best for you, not him. You are not responsible for his happiness. he lived independently from you in the most intimate ways. Him losing your trust is a consequence of those actions. He is a man who not only lacks honesty by lying for those many years, but he lacks self-assurance and seeks validation from someone else. These characteristics make it hard to recommend building on that foundation unless what I said before holds up.
If you stay, you are accepting him and his behaviors. You are accepting that he cheated and has that potential. Can you stay together? Sure, but you are the one that will have to do the most work to make it work, mainly because you have every reason to be jealous/upset. If you can find peace with this man and he is willing to do his own work on himself because clearly, he lacks insight into himself or a backbone to establish his boundaries, then you can invest in the relationship. If you cannot forgive him, accept him as he is, and accept that he could cheat again, it is best to let it go. Some wounds are just too deep.