9 Replies Latest reply on Aug 14, 2019 8:19 AM by Don Wise

    How to remove duplicate records from extracted data

    jesh d

      Hello Everyone ,


      My data set has more than 5 million records .Out of which many are duplicates .I have saved the data in an extract .Since there are so many duplicates rows with just different record ID's . I would like to know if there is a way to remove the duplicates and only use one unique row .How can i remove the duplicates before i jump into worksheet to analyse the data or is there a LOD expression which can do this work ?I would prefer to clean up duplicates before i jump into worksheet to analyse.Let me know otherwise .


      Below is the sample records : Additionally :there are rows which are repeated more than 10 times.I want 9 of them to go and keep only 1 row .Refer the screenshot for more details Thanks in advance .