Question : DTS vbscript task that check for duplicate records using ADO gets empty recordset during off business hours and runs correctly during business hours.

Vbscript task that is part of DTS package is using ADO to connect to Access database on remote server and runs select statement to check for duplicate records; This select statement consistently return empty dataset when DTS runs off business hours (while there ARE duplicate records) and runs as expected during business hours. Vbscript completes without errors;
Code Snippet: 
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:
75:
76:
77:
78:
79:
80:
81:
82:
83:
84:
85:
86:
87:
88:
89:
90:
91:
92:
93:
94:
95:
96:
97:

Function Main()
      dim conn
      dim rs 
      dim strSQL
      dim strPrefix 
      dim b
      dim rcnt
      On Error Resume Next
 
       ' instantiate the ADO objects
      set conn = CreateObject("ADODB.Connection")
      set rs = CreateObject("ADODB.Recordset")
      DTSPackageLog.WriteStringToLog  "in code"  & DTSGlobalVariables("gvFileName").Value
      conn.Provider="Microsoft.Jet.OLEDB.4.0"
      conn.Open DTSGlobalVariables("gvFileName").Value
      if err.number > 0 or IsNull( conn) then
         DTSPackageLog.WriteStringToLog  "open conn SQL Error: " & err.number
         err.raise     
      end if
      DTSGlobalVariables("gvEmail").Value = ""
 
      'DTSPackageLog.WriteStringToLog  "conn: " &  conn.properties("DataSource")
     
      strSQL = "SELECT count(*) as cnt FROM Export_Master " 
          
      rs.Open strSQL, conn
      
      if err.number > 0 or IsNull(conn) then
         DTSPackageLog.WriteStringToLog  "SQL Error: " & err.number 
         err.raise     
      end if
     If Not rs.BOF And Not rs.EOF Then 
        DTSPackageLog.WriteStringToLog  "Count: " & rs.fields("cnt") 
     end if
     rs.close
     ' 
     strSQL = "SELECT ReferralDate, CloseDate, [Last Name], [First Name], Count(*) AS Dup FROM Export_Master " & _
                     "GROUP BY ReferralDate, CloseDate, [Last Name], [First Name] HAVING Count(*) >1;"
        
      rs.Open strSQL, conn
      DTSPackageLog.WriteStringToLog  "SQL" 
      if err.number > 0 or IsNull( conn) then
         DTSPackageLog.WriteStringToLog  "SQL Error: " & err.number 
         err.raise     
      end if 
      DTSPackageLog.WriteStringToLog  "SQL rs BOF:" & rs.BOF
      DTSPackageLog.WriteStringToLog  "SQL rs EOF:" & rs.EOF 
      If Not rs.BOF And Not rs.EOF Then
         ' there are duplicates - prepeare to produce e-mail
         DTSPackageLog.WriteStringToLog  "Duplicates" 
         do until rs.eof
 
               DTSGlobalVariables("gvEmail").Value = DTSGlobalVariables("gvEmail").Value & DTSGlobalVariables("gvFacility").Value & Chr(9) & rs.fields("ReferralDate") & Chr(9) & _
                                                                              rs.fields("CloseDate") & Chr(9) & rs.fields("Last Name") & Chr(9) & rs.fields("First Name") & " - " & rs.fields("Dup") & " records;" & Chr(13) & Chr(10)
               ' for each duplicate records insert line in the e-mail  
              rs.movenext
          loop
       end if 
       'msgbox DTSGlobalVariables("gvFacility").Value & "gvEmail:" &  DTSGlobalVariables("gvEmail").Value
       rs.close 
       set rs = Nothing
       conn.close
       set conn = Nothing  
     
       Dim pkg
       Dim stpLoad
       Dim stpOutlookEmail
       Dim tskOutlookEmail
 
      
       set pkg = DTSGlobalVariables.Parent
       set stpLoad = pkg.Steps("DTSStep_DTSDataPumpTask_1")
       set stpOutlookEmail = pkg.Steps("DTSStep_DTSSendMailTask_1")
       set tskOutlookEmail = pkg.Tasks("DTSTask_DTSSendMailTask_1")       
 
	
       If DTSGlobalVariables("gvEmail").Value = "" then
          DTSPackageLog.WriteStringToLog  "no email" 
          stpLoad.DisableStep = False
          stpOutlookEmail.DisableStep = True
          stpLoad.ExecutionStatus = DTSStepExecStat_Waiting
          'Msgbox "Load"
       else
           DTSPackageLog.WriteStringToLog  "e-mail" 
           stpLoad.DisableStep = True
           stpOutlookEmail.DisableStep = False
           stpOutlookEmail.ExecutionStatus = DTSStepExecStat_Waiting
           
           tskOutlookEmail.Properties("MessageText").value  = tskOutlookEmail.Properties("MessageText").value & DTSGlobalVariables("gvEmail").Value & Chr(13) & Chr(10) & "Automatic UR Load Agent"
           'msgbox "msgtext:" & tskOutlookEmail.Properties("MessageText").value
           tskOutlookEmail.Properties("ToLine").value = DTSGlobalVariables("gvFacility").Value & "[email protected]"
           'tskOutlookEmail.Properties("ToLine").value = "[email protected]"
           'msgbox tskOutlookEmail.Properties("ToLine").value
       End If
       Main = DTSTaskExecResult_Success
  
End Function
Open in New Window
Select All

Answer : DTS vbscript task that check for duplicate records using ADO gets empty recordset during off business hours and runs correctly during business hours.

In summary, MS Access is a client application and using it as a server application (i.e. pushing records into it over a network) is just unreliable. I can't tell you why the issue is appearing at non busines hours but I do know that MS Access is not a good foundation to build this kind of process on. It is fantastic for client based data entry, and fine as an ETL source, but not good for ETL processing.

Is the SQL Task transfering data from MS Access > MS Access? If so then it is sucking data from MS Access to SQL, then pushing it back into MS Access (2 trips). As MS Access is a file (not server) based database, you just can't be 100% certain about the state of the MDB file.

If you change your process to copy the data into a local table, it will be only one trip for the data, and you can be certain that it will finish before it goes to the next step (because the destination is an actual 'server' based database, not just a file), plus you can replace a load if your ActiveX code.

But still I am not convinced that the table isn't empty anyway. There could be some other failure occuring that is not being reported by MS Access.

Why don't you (for now) add another step in parallel that copies this data into a local table. Then you can see whats in the local SQL table, as compared to whats in the MS Access table when the full process runs.
Random Solutions  
 
programming4us programming4us