Paralelismo e distribuição na recuperação heurística do plágio externo com locality sensitive hash

Abstract

The heuristic retrieval on external plagiarism identification task is intended to return a list of the documents most likely to have been plagiarized, based on a similarity metric, reducing the workload of the following and highly costly steps. DUARTE (2017)’s work formalized a sequence of steps for performing heuristic retrieval with Locality Sensitive Hash (LSH) methods and demonstrated that due to their ability to preserve similarity, LSH methods are viable options for heuristic retrieval. This work proposed two strategies based on DUARTE (2017)’s sequence of steps, called document parallelism (PnD) and permutation parallelism (PnP), that were implemented in Apache Spark distributed computing system, to support the task of identifying plagiarism in large document collections. The experiments demonstrated that the PnD and PnP strategies were able to reduce, according computational capacity increases, the time of the activities of representing, searching and retrieving documents; as well as achieving a high level of effectiveness for returning effectively plagiarized documents.