I have recently launched a survey using Version 2.00+ Build 121204 for the first time, new install not an upgrade. When viewing Responses and Statistics I'm seeing around 75% incomplete responses. When I export the data to SPSS and view the responses I'm see large gaps of response data with only the id, submitdate and startlanguage, but nothing else. I have also had respondents comment that they had filled out the survey and after hitting the submit button were re-directed back to the survey that was cleared of their answers. I've tried with many different broswers and computers to try and replicate the experience but have not as of yet.

Does limesurvey, when a user starts a survey, start tracking this as an incomplete survey until the user submits their answers and what I'm seeing are users really not completing the survey?

Or is limesurvey clearing the users survey responses upon submitting?

Status incomplete is only changed to complete after the endpage is reached and finished via the submit button.

It can be a bad response quote. It can be a technical problem (perhaps not limesurvey but something with the server or connectivity). It can be Limesurvey.
Since you can not reproduce it with the different browsers and computers it might be interesting to see if you can find any pattern in the responses. Perhaps a certain range of time where people exited the survey? Looks like many people stopped at the welcome screen.

Since you didn't received feedback with technical problems on the first pages of the survey it doesn't seem to be a full session directory. On many installations these php sessions directories are small. And LS creates very big session files for every respondent hitting the welcome screen.

You should try to take a look at the webserver access file to see what happend.