How do I break out of avoidant and destructive habits?

You clearly describe a self sabotaging life which has to be exhausting. 

Over time in this pattern of self sabotaging change in your life, you may have acted and continue to act on negative self beliefs about yourself.  Negative self beliefs can develop from anytime in our life, but the ones we develop in childhood are usually the strongest.  If I grew up thinking that I was not good enough, that I was not worthy, that I would always screw up, or your words "I have a lot of potential that I will never live up to" we think, feel, and behave in that belief consciously or subconsciously.  You can think of emotions, thoughts, behaviors, and beliefs as interrelated.   Every activating event is filtered though what you believe about yourself, what you believe about others, and what you believe about the world.  That filter system interprets the current life sitatuation and produces automatic thoughts.  For example, if I believed I am not good enough and people never give me a fair chance:  I see an ad for a job that I am qualified to do but have never done.  I am interested in that job.  So that is filterered through these negative beliefs.  So... I have the thoughts "I am not going to be able to do that job" or "they would never hire me."  I then have the negative emotion of anxiety and feeling depressed and then the action of not applying.  So then of course I don't get the job and the belief continues to be strengthened.   

Or how about your example.  I am going to the gym but miss a couple of days because I am sick.  Now it is time to restart my routine.  I have the negative belief "I have a lot of potential that I do not live up to" so maybe I have the automatic thoughts "There is no point even going to the gym, I will just quit at some point anyway.  I always quit."  Then I have the negative emotion of anxiety or depression and the behavior is that I do not go to the gym.  Then, not going to the gym reinforces the underlying beleif "I have a lot of potential that I so not live up to."   

Now the same situation with a positive belief system:  Been going to the gym and missed 2 days beacuse I was sick.   I feel better and have the thought "I'm glad I feel better today.  I'm going to get back on track with the gym."  I have a more positive emotion and the behavior of going to the gym which reinforces the positive beleif I have about myself.   

So the question becomes how do we challenge the negative self talk (Thought process) and eventually the underlying unhelpful beliefs to change the way we feel and act.  That process often begins in therapy.  We start to recognize what we think about and what we tell ourselves, not run on autopilot only being led by how we feel at any given moment.   As we identify and learn our thinking processes and often identify cognitive distortions (patterns of thinking that cause more anxiety and depression), we can then take active steps to challenge these ways of thinking and learn how to catch our thoughts, challenge our thoughts, and reframe our thoughts.   As we start to change our thinking, we will notice that the way we feel begins to change as well.  As we think and feel differently, we also choose to enegage in healthier and more productive behaviors.  As we have more success, we chip away at the underlying belief system as our lives are the very evidence that what we believe about ourselves is not true.   Over time, as we have more and more success, we really start to challenge that we will not live up to our potential. 

Each day we operate in these healthier and healthier thinking patterns is a day the underlying belief gets weaker and weaker and eventually goes away.  When we have setbacks, we try not to catastrophize what the setback means and look at other alternatives.  We realize that we cannot feel our way into thinking and behaving differently, but rather we must think and behave our way into feeling different.  I also think it is important to take things in manageable steps.  Make short term goals.  Don't commmit to something forever, just commit for a day or few days.   Break bigger jobs down into manageable steps.  Enjoy your successes and open yourself up to believing the things that you have believed about yourself may not be true. 

I would also encourage you to seek and stay in therapy for some time.  It if often difficult to sustain change on your own.  A therapist can guide you on what I talked to you about in this response and can encourage you.  A therapist also can help you get back on track when you have a misstep.  Therapy could be a great investment in the rest of your life.  I wish you the best on your journey.  Let me know if you want to talk more.