deduplicate(): merging variants of the same entry#

deduplicate() is used to merge multiple variants of the same entry into one, for instance typos. Such cleaning is needed to apply subsequent dataframe operations that need exact correspondences, such as counting elements. It is typically not needed when the entries are fed directly to a machine-learning model, as a dirty-category encoder can suffice.