Skip to content
Allin
Sign up · Sign in

Remove duplicate lines

Delete repeated lines while keeping the original order of the first occurrences.

Related tools

All Transform & clean tools →
Find duplicate linesShow only the lines that appear more than once in your text.Remove duplicate wordsKeep only the first occurrence of each word in your text.Remove blank linesDelete all empty or whitespace-only lines from your text.Duplicate each lineRepeat every line a chosen number of times.Comma-separated ↔ linesConvert a comma-separated list into lines, or lines back into a comma list.Filter linesKeep or remove the lines that contain a given word or phrase.Add prefix / suffix to linesAdd text at the start and/or end of every line.Sort lines by lengthSort lines from shortest to longest, or the other way round.

Remove duplicate lines is free to use as often as you like, directly from this page. You will find it under Transform & clean, with Find duplicate lines and Remove duplicate words for the neighbouring cases.

How to use it

  1. Open the tool — no signup or install needed.
  2. Enter your input or adjust the available options.
  3. Get your result instantly, then copy or download it.

Frequently asked questions

What is Remove duplicate lines?

Delete repeated lines while keeping the original order of the first occurrences.

When would I actually use this?

Cleaning up a paste before it goes anywhere: stripping duplicates from a list, normalising case, removing accents for a filename, or sorting lines.

What is the most common mistake?

Sorting text as if it were ASCII. Accented letters land after Z in a byte sort, which puts "Zurich" before "Émile" and produces a list no reader will accept.

How is Remove duplicate lines different from Find duplicate lines?

They sit next to each other but answer different questions: Find duplicate lines is the one to open when you need it to show only the lines that appear more than once in your text. Pick whichever matches what you're starting from — both are free.

Is there a tool for the next step?

Remove duplicate words is the closest one after this: Keep only the first occurrence of each word in your text.

What else is worth having open alongside it?

Remove blank lines and Duplicate each line — they come up in the same task often enough to be worth a second tab.

Where do the figures come from?

Transformations use the browser's Unicode-aware string handling, so case changes and sorting respect the locale rather than byte order. Nothing is truncated: what comes out is the whole of what went in.

Further reading

All guides →
ExplainerFinding Duplicates in a List Without a SpreadsheetTwo lines that look identical are often not identical. Case, a trailing space, a no-break space and two different encodings of the same accented letter were each run through the duplicate finder, and it reported no duplicates for three of the four.How-toCleaning Messy Text: The Order of Operations That Actually MattersStrip tags before decoding entities, trim before deduplicating, collapse whitespace last. Three orderings run in Node, a nine-step pipeline in the right sequence, and the invisible characters — U+00A0, U+200B, U+FEFF — that survive every naive cleanup.How-toFiltering Lines by a Pattern Without a Command LineThis is grep for people who do not use grep, with one important difference: the match is a plain substring, so a real regular expression returns an empty box and no error. Every claim here was checked by running the tool.GuideCleaning Up a List Pasted from a Spreadsheet or a PDFA paste carries characters you cannot see: no-break spaces, soft hyphens, zero-width spaces, tabs and CRLF. Four cleanup tools were run against each of them, and they use three different definitions of whitespace.GuideFind and Replace: The Regex Features That Bite, Demonstrated One by OneGreedy against lazy on the same string, the dot that skips newlines, $& and $$ in the replacement, a reused /g regex that silently skips a row, and why /i knows nothing about Turkish i. Every failure run in Node, with a count-then-replace routine that catches them.ExplainerSorting Text Is Not One Operation: Four Orderings That All Call Themselves AlphabeticalCode-unit order, natural order and locale collation run on the same list in Node, with the outputs printed. Why Zebra beats apple, why item10 beats item9, and why ä sits next to a in German but after z in Swedish.