How to remove duplicate rows
Load a CSV, TSV, JSON array, Excel spreadsheet, or pasted table, then group records by the full row or selected key columns. Choose which occurrence stays, review duplicate groups, and export retained rows, removed rows, or a JSON report. The page reads file and pasted content in your browser and does not send it to a server.
How to run a duplicate-row check
- Drop a CSV, TSV, JSON, or spreadsheet file into the upload area, click browse files, or paste table data into Table data.
- Choose Full record (all columns) or Selected key columns in Match by.
- Choose Keep first occurrence, Keep last occurrence, or Manual review per group in When duplicates are found.
- Review each duplicate group, select a row when manual review is active, then choose an export button.
What matching duplicate records changes
Full record (all columns) compares every column. The Selected key columns option compares only checked fields, so rows with different IDs can form a group when the selected email or name values match. Select at least one column for this mode.
These comparison options start checked. Ignore letter case when matching treats Alice and ALICE as equal. Normalize whitespace (trim & collapse spaces) trims values and changes each run of whitespace to one space. Treat blank/empty/null values as equal puts those empty values in one comparison group.
Input
id,name,email
1,Alice,[email protected]
1, Alice ,[email protected]
2,Bob,[email protected]
Retained CSV
id,name,email
1,Alice,[email protected]
2,Bob,[email protected]
Removed CSV
Original row,id,name,email
2,1, Alice ,[email protected]With Full record (all columns) and Keep first occurrence, the first Alice row stays, row 2 is removed, and Bob stays. The preview shows the matching group before export.
Choose Export retained (CSV) for kept rows, Export removed (CSV) for removed rows, or Export full report (JSON) for group details. The removed CSV adds each original row number. The retained CSV keeps the table columns.
How CSV, JSON, and Excel rows are read
CSV and TSV input uses the first row as column headers. For pasted text, the tool counts commas and tabs on the first line, then uses the delimiter with the higher count. A JSON array loads as records. A JSON object loads a non-empty array of objects found in one of its properties. XLSX and XLS files use the first worksheet.
The limit is 50,000 records. The tool groups records by exact normalized values after applying the selected case, whitespace, and blank-value rules. Near-duplicates that still differ after those rules stay in separate groups. Empty pasted input and JSON without a usable record array produce an error.
Examples with email, spreadsheet, and JSON data
A contact export may contain the same email twice with different row IDs. Select the email column, review the group, and keep the record with the details you want to retain.
A spreadsheet assembled from two reports may repeat complete rows. Choose Full record (all columns), keep the first or last occurrence, and export a retained CSV.
A JSON export may need row references. Export the removed CSV to see each original row number, or export the full JSON report for group, kept-row, removed-row, and record details.
When another dataset clean tool fits better
Find this page in Dataset Clean Tools. Choose Data Cleaner when the file needs format-aware cleanup across CSV, TSV, Excel, JSON, JSONL, XML, or YAML. Choose Email List Cleaner for whitespace and casing normalization plus malformed-email review.
Use Value Clusterer & Standardizer when values are similar rather than exact and you want to review proposed canonical values. This tool groups exact matches and leaves field values unchanged.
Frequently asked questions
How do I remove duplicate records by email or ID?
Choose Selected key columns in Match by, then check the email or ID column you want to compare. Select at least one key column before reviewing the groups.
How does deduplicate CSV rows handle spaces and case?
It follows the three comparison checkboxes. The default settings ignore letter case, trim and collapse whitespace, and treat blank, empty, and null values as equal.
Can I review duplicate rows before exporting?
Yes. Choose Manual review per group to select the record kept in each matching group. An export with groups still awaiting review asks for confirmation and uses the current default selection for those groups.
Does removing duplicate rows upload my data?
No. The file and pasted table are read in the browser, and the page code does not send that content to a server. Downloads happen only when you choose an export button.