When you install any Office 2003 application the final installation dialog asks if you want to keep the local files, if you have more than 1.5 GB of free space on a local drive and did not use a network installation point.
By allowing local installation files, you won't need the CD to install First Run options or add features later. If you decide to remove it later, the FAQ recommends letting disk cleanup remove it install of deleting it from Windows Explorer.
FAQ: KB 830168
Information for Administrators: http://www.microsoft.com/office/ork/2003/two/ch3/DepC06.htm
Published November 14, 2003. Last reviewed on May 16, 2011.