Here are some links to webserver+PHP software designed to run from a flash memory device.
USBWebserver (last updated in 2013)
Portable Windows Demo (experimental)
Keep in mind that portable apps don't need to be installed on a removable device. You can also run portable software from a local hard drive. Portable software is low-impact nature and can be easy to maintain.