I just realised a Limited user with view only access to the Bank section can actually import bank statement and upload successfully.
I guess this should be a bug. The essence of View only is completely defeated here.
Fixed in the latest version (19.11.9)