Remove Duplicate Lines — FAQ

It reads your text one line at a time, works out a comparison key for each line, and keeps only the first line for each distinct key. The line you get back is the original text, not the key — so if you turned on Ignore case, the surviving line keeps whatever capitalisation it had in your input.

No, unless you pick A → Z or Z → A. By default the surviving lines stay in the order they first appeared. Removing a duplicate does not shuffle anything else.

No. The result recalculates as you type and whenever you change an option. The buttons are only for copying, downloading, and moving the result back into the input box.

Yes by default. Smith and smith are treated as different lines until you tick Ignore case. The default is deliberate: for case-sensitive data like IDs, tokens and passwords, silently merging them would be wrong.

Nearly always invisible characters. The two usual suspects:

• Trailing spaces. Tick Trim spaces (it is on by default) to ignore leading and trailing whitespace when comparing.
• Non-breaking spaces, which are common in text copied from web pages and PDFs. These look exactly like a normal space but are a different character, and trimming will not catch one sitting in the middle of a line.

When a line repeats, it keeps the later copy rather than the earlier one. The position in the output does not move — only which copy's text is used. This matters when your list is in date order and the most recent entry is the one you want to keep.

Unique lines gives you the cleaned list. Only repeated lines shows just the ones that appeared more than once, which is how you find out what was duplicated. Only lines used once shows lines that appeared exactly once and drops everything that repeated — a different question, useful for finding one-offs in a merged list.

Yes. Lines are split on CRLF, CR or LF, so pastes from Excel, Notepad, SAP exports, Mac and Linux all come through as separate lines. The output uses plain newlines, which all of those read back correctly.

Not directly — it works on whole lines. If your items sit on one line separated by commas, replace the commas with line breaks first, then run it.

Nothing fixed. The work happens in your browser tab, so the ceiling is your device's memory. Hundreds of thousands of lines are normally fine. Very large pastes may make the tab pause for a moment, because the processing runs on the main thread rather than in a background worker.

Comparison is on the exact characters in the line. Two strings that look identical but use different Unicode representations — for example an accented letter stored as a single code point versus a plain letter plus a combining accent — are not treated as duplicates. It is uncommon, but it does turn up in text exported from some systems.

No. Everything runs in your browser and the page has no upload endpoint. That also means nothing is stored, so closing the tab loses whatever was in the boxes. See the privacy policy for how the site behaves as a whole.