and this has been the case for multiple days ?
if so we can force the deletion using adsiedit.msc
open the configuration partition -> services-> exchange -> first administrative group -> find the 2003 server -> find the storage group => find the publc folder store right click delete