You can use Unetbootin or Ubuntu's USB startup disk creator to create a persistent USB Live USB.
in Unetbootin, just assign "space used to preserve files across reboots" to 1GB or more (depending on the size of the usb drive), this will create a compressed OS image, that saves the changes after each reboot.
in Ubuntu's startup disk creator, just set the reserved extra space to your desired level. This will do the same as unetbootin.
If you have a large enough USB flash drive, like 8GB+, you can do an actual install to it.
All these methods are slow tho, but i'm not sure about USB 3.0 as these ports and devices might be fast enough to run a OS from them. the second method of actually installing to it, will put more wear on the USB drive, making it die faster.