Latest VirusTotal.com Antivirus Report : https://www.virustotal.com/gui/url/9b8cb83c56ee58f56b94ee25e81a9d7bf8755d74a438c2f4aeb9ed4989a61929/detection/
Do you want to remove duplicate lines from text ? Do you want to dedupe list quickly ? Do you want to remove duplicate words from text ? Do you want to ensure that the contents of your list are unique because they can be used later in scripts, databases e.t.c. ? Do you want to ensure that you do not have added a lines twice by accident ?
Then try Remove Duplicates Lines from 4dots Software which can import text from Text files, Microsoft Word files and RTF files and removes duplicate lines or words from them.
The lines can also be sort in ascending or descending order and the line comparison can be case sensitive or not.
Moreover, you can batch remove duplicate text from many files. You can specify multiple files, folders and their contents, or import lists of file paths to batch remove duplicate lines.
The application can be executed also from the command-line, which is useful for inclusion in scripts e.t.c.
Remove Duplicate Lines uses a quick multi-threaed hash sort algorithm to make the process faster.
It is easy to use and supports drag and drop.
The application is multilingual and translated into more than 39 languages.
To sum up, if you want to remove duplicate lines from text then try this application which is quick, supports batch processing of many files and can also be executed from the command line.