The "Delete" button will not only remove the profile folder, but the user's profile location entry under "HKLM\Software\Microsoft\W
indows NT\CurrentVersion\ProfileL
ist".
The button is very probably disabled because there are still handles open to the old profile. Reboot the machine, log on with an administrative account, try again to delete it.
Note that the W2k3 Resource Kit Tools contain the "delprof.exe" utility to delete profiles from the command line, including remote profiles. Use the /P argument to avoid deleting profiles you might still need ...
Windows Server 2003 Resource Kit Tools
http://www.microsoft.com/downloads/details.aspx?familyid=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en