It may be this, but it isn't supposed to work that way...
what I'm thinking is your office installation is automatically saving a back-up copy of your document.
These back-ups are usually stored in the same folder as the original, and typically have a different file extension (ie Word uses .wbk)
Open Word and select from the toolbar:
Tools > Options > Go to the "Save" tab... uncheck "Always Create Backup Copy"
Save a document in word, and see if the duplicate to your desktop goes away..