Question : Finding Duplicate rows in a datatable to mark as duplicate and have to append the duplicate row in new datatable

Am using vb.net windows application
I have imported 50K rows from a CSV file to a datatable. There may be some duplicate rows. I have one column "ISDUP" in the same datatable. i have to update the ISDUP column with True or False based on Duplicate row or not. When i find duplicate row i have to append the duplicate row in another datatable.

Here, the performence is most important thing.

Please provide the solution for this Problem.

Answer : Finding Duplicate rows in a datatable to mark as duplicate and have to append the duplicate row in new datatable

Here it is again with some modifications

                       
         
           
             1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:

           
           
             Sub SetDuplicateFlag(ByRef dt As DataTable, ByVal DupColName As String)
    Dim ssort As String = ""
    Dim col As DataColumn
    Dim scol As DataColumn
    For Each scol In dt.Columns
      If scol.ColumnName = DupColName Then Continue For
      If ssort <> "" Then ssort &= ","
      ssort &= scol.ColumnName
    Next 
    dt.DefaultView.Sort = ssort
    Dim i As Integer, bEquals As Boolean, c As Integer, ccount As Integer
    ccount = dt.Columns.Count
    For i = dt.Rows.Count - 1 To 0 Step -1
      bEquals = True
      For c = 0 To ccount - 1
        If dt.Columns(c).ColumnName = DupColName Then Continue For
        If Not i = 0 Then
          If dt.DefaultView(i)(c).ToString() <> dt.DefaultView(i - 1)(c).ToString() Then
            bEquals = False
            Exit For
          End If
        Else
          bEquals = False
        End If
      Next c 
      If bEquals Then
        dt.DefaultView(i - 1).Item(DupColName) = True
        dt.DefaultView(i).Item(DupColName) = True '(i).Delete()
      Else
        If dt.DefaultView(i).Item(DupColName) <> True Then
          dt.DefaultView(i).Item(DupColName) = False
        End If
        End If
    Next 
  End Sub

           
         

Open in New Window

Select All