Letting someone go is one of the most challenging things any leader faces in their career
No one wants to do it, even when the affected employee did something egregious to warrant their termination. It's still a human being across from you and this will impact their lives.
Help them land on their feet.
Whether it's a termination for cause, a downsizing, restructuring or just the realization that this isn't the fit, allow them the opportunity to bow out gracefully. If they would like the opportunity to resign, versus being terminated, it can help them land the new job.
Don't worry about unemployment payments. Those payments come from insurance premiums your company has already paid. It has very little impact on you. No need to contest their filing. They will need it to pay bills while they look for something new.
Offer to write a positive letter for them speaking to their strengths. Once they get past the initial emotions, they will take you up on the offer.
Provide outplacement resources. It's one of the best investments you can make. It reduces the chances of negative backlash. It reassures the other employees that you treat people with dignity and limits the damages the person might do to your client base.