Popular content in video sharing websites (e.g., YouTube) is usually replicated via identical copies or near-duplicates. These duplicates are usually studied because they pose a threat to site owners in terms of wasted disk space, or privacy infringements. Furthermore, this content might potentially hinder the users' experience in these websites. The research presented in this article focuses around the central argument that there is no agreement on the technical definition of what these near-duplicates are, and, more importantly, there is no strong evidence that users of video sharing websites would like this content to be removed. Most scholars define near-duplicate video clips (NDVC) by means of non-semantic features (e.g., different image/audio quality), while a few also include semantic features (i.e., different videos of similar content). However, it is unclear what features contribute to the human perception of near-duplicate videos. The findings of four large scale online surveys that were carried out in the context of our research confirm the relevance of both types of features. Some of our findings confirm the adopted definitions of NDVC whereas other findings are surprising: Near-duplicate videos with different image quality, audio quality, or with/without overlays were perceived as NDVC. However, the same could not be verified when videos differed by more than one of these features at the same time. With respect to semantics, it is yet unclear the exact role that it plays in relation to the features that make videos alike. From a user's perspective, participants preferred in most cases to see only one of the NDVC in the search results of a video search query and they were more tolerant to changes in the audio than in the video tracks. Based on all these findings, we propose a new user-centric NDVC definition and present implications for how duplicate content should be dealt with by video sharing Web sites.