Tool to deduplicate a list. Doubles or duplicates are a repeated redundant information present twice (or more) in a list, usually useless.
Duplicates in a List - dCode
Tag(s) : Data Processing
dCode is free and its tools are a valuable help in games, maths, geocaching, puzzles and problems to solve every day!
A suggestion ? a feedback ? a bug ? an idea ? Write to dCode!
Duplicates in a List
Duplicates Detector
Answers to Questions (FAQ)
What is a duplicate? (Definition)
A duplicate is a redundant element (double or triple or more) in a set of elements (a list).
How to remove doubles in a list?
Indicate / copy-paste items in the List field. dCode detects if the list is a list of terms (one per line) or a list of words (with separator) and finds redundant items repeated more than once in order to un-duplicate.
Example: A,B,C,D,A,B,C,A,B,A can be deduplicated as A,B,C,D (repeated terms were removed)
Be carefull to take into account all writing variants of a same element: ignore accents and diacritics allow to treat words like item and ìtém as a same word twice. Also, ignoring uppercase and lowercase allow to detect/find item and ITEM as a duplication of the same word and delete them.
How to count doubles in a list?
By listing items, dCode detector counts the number of appearance of each, the list can be sorted to detect the most duplicated ones.
dCode counts duplicates/redundancies, so 2 identical items is counted 1 duplicate. N identical elements is counted N-1.
Why counting doubles?
There are many applications where the presence of duplicates can be a problem:
— detect and delete duplicate files to free up disk space
— deduplicate data from a database, retaining only one occurrence of each unique/distinct record.
— sending mailings by checking if a list of email addresses contains duplicates in order to filter the list and delete identical emails.
Source code
dCode retains ownership of the "Duplicates in a List" source code. Except explicit open source licence (indicated Creative Commons / free), the "Duplicates in a List" algorithm, the applet or snippet (converter, solver, encryption / decryption, encoding / decoding, ciphering / deciphering, breaker, translator), or the "Duplicates in a List" functions (calculate, convert, solve, decrypt / encrypt, decipher / cipher, decode / encode, translate) written in any informatic language (Python, Java, PHP, C#, Javascript, Matlab, etc.) and all data download, script, or API access for "Duplicates in a List" are not public, same for offline use on PC, mobile, tablet, iPhone or Android app!
Reminder : dCode is free to use.
Cite dCode
The copy-paste of the page "Duplicates in a List" or any of its results, is allowed (even for commercial purposes) as long as you credit dCode!
Exporting results as a .csv or .txt file is free by clicking on the export icon
Cite as source (bibliography):
Duplicates in a List on dCode.fr [online website], retrieved on 2024-11-21,
