Abordagens de particionamento utilizando locality-sensitive hashing aplicada a busca heurística na detecção de plágio externo

Abstract

[EN] Natural Language Plagiarism Detection (NLPD) aims to seek textual evidence of plagiarism in documents, in order to generate a list of candidate documents of being plagiarized, to further be analysed by humans. External plagiarism detection (EPD) is a NLPD task in which a set of documents is available to be queried, seeking for plagiarism. DPE is comprised of a few steps, one of them being the Heuristic Search (HR), which is the EPD stage that retrieves a set of plagiarism candidate documents from a large corpus, reducing the workload of the later stages of the EPD. The HR stage is an Information Retrieval (IR) task, and comprises two subtasks, namely, Indexing and Source Retrieval. In order to speed up the execution of IR subtasks, two partitioning methods were proposed, the permutations and vocabulary partitioning. Both use Locality-Sensitive Hashing (LSH) and are based on the mathematical concept known as partition of a set. Partitioning any set can generate remainders, and to address this issue, the Remainder at End (RaE), Remainder at Cell (RaC) and Distributed at Cell (DaC) treatment strategies were proposed. Both partitioning methods were approximately 101% faster than the IR state of art. Moreover, RaE, RaC and DaC achieved better results in document indexing, query extraction and retrieval time in comparison to a standard LSH baseline, the Minmax.