Even if you have a good relationship, caregiving can "fix" that.

If you can't do it, it's not good for anyone involved to try. You will end up watching your life go by without you and perhaps actively hating the person you are caring for.

(Also, there's a middle ground. Hiring a caregiver doesn't mean you're abandoning them...)