Your client is considering late fees to be the same as proratable interest. They are not. The fee is a fee, assessed in full as of the day it is due. By your client’s logic, if you assessed late fees daily and he paid at noon, he would claim he owed only half the day’s fee. When would it stop?
Late fees in business are meant to be severe enough to induce the customer to pay on time. In fact, in some jurisdictions, they are specifically categorized as fees because they exceed legal limits on interest rates. What your client has done is use you as a bank. I strongly suspect the late fee is less than it would have cost him in time and money to apply for a loan at a bank to fund the purchase. But your carrying costs have been real, because you did not recover your investment in goods or services and had your own carrying costs, even before the invoice came due.
If you don’t want to use late fees as implemented in Manager, don’t tick the box. Instead, include a statement on the sales invoice saying that interest will be charged on late payments at X% per day (or partial day) or a fixed amount per day. Then issue a secondary sales invoice for the interest charges.
If you want to refund the client a portion of his late fee, issue a credit note. I understand why that might be an attractive choice when a client is only one day late. Just remember you have already carried his expense at least since the date the sales invoice was issued, and usually longer.
But I urge you strongly: unless you want to be a bank, do not go down the path of daily late fees.