Damaged leaves - to remove or not?

Hi everyone, I am wondering that when dealing with an unhealthy plan, and leaves are turning brown etc., do you prune those leaves from the plant or leave alone?

My feeling is that these damaged leaves will not recover. Is it taking energy away from the plant healing if they are left on?

Thanks.