Attachments Tab still shows up after deactivating

If you have any data in the tab, it takes precedence over what you have configured under Customize.