Question : How do I speed up this function?

Hi, I wrote this program which works okay on smaller tables, but the amount of time it takes gets exponentially longer the larger the table.

For instance, a table of 150,000 records takes about 6 minutes, a table of 550,000 records takes about 1 hour and a table of 3.5 million records takes over 1 day. Obviously, I am not doing something right here.

This is a table of price information. I am trying to populate the buybreak and sellbreak fields, which calculate the number of days since a previous low or high price has been realized. As an example, a file is attached with the proper results.

NOTE: the file in the ZIP file is a DBF file, even though it has the extension PNG. Foxpro will read it if you simply type in the extension, for example:

use file1.png in 0

Code Snippet:

         
           
             1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:

           
           
             alte tabl file1 add colu buybreak f(16,8)
alte tabl file1 add colu sellbreak f(16,8)
sele contract dist from file1 into curs contracts
scan
m.tempcontract=contract
sele date, contract, abshigh, abslow, recno() as rec from file1 wher contract=m.tempcontract orde by rec desc into curs breakout readwrit
alte tabl breakout add colu buybreak f(6,0)
alte tabl breakout add colu sellbreak f(6,0)
sele breakout
scan
m.temprec=rec
m.temprecno1=recno()
m.abshigh=abshigh
m.abslow=abslow
loca rest for abshigh>m.abshigh
m.temprecno2=recno()-m.temprecno1
sele breakout
repl all buybreak with m.temprecno2 for rec=m.temprec
go m.temprecno1
loca rest for abslow
           
         

Open in New Window

Select All

Answer : How do I speed up this function?

Why do you need the temporary cursor. Create an index on contract +str(recno()) in the original table and run the loop there.

Something like

USE file1
INDEX on contract+STR(RECNO(),10) TAG contractrec
DO WHILE NOT EOF()
      m.tempcontract=contract
      SCAN while m.tempcontract=contract
            m.temprecno1=RECNO()
            m.abshigh=abshigh
            m.abslow=abslow
            SET ORDER TO contractrec desc
            SKIP
            COUNT REST WHILE abshigh<=m.abshigh AND m.tempcontract=contract TO lnSkippedRecs
            GO m.temprecno1
            REPL buybreak WITH m.lnSkippedRecs+1
            SKIP
            COUNT REST WHILE abslow>=m.abslow AND m.tempcontract=contract TO lnSkippedRecs
            GO m.temprecno1
            REPL sellbreak WITH m.lnSkippedRecs+1
            SET ORDER TO contractrec ASCENDING
      ENDSCAN
ENDDO

Above code is totally untested.

If you want to add columns to sql cursor , add them in the sql select statement like

sele date, contract, abshigh, abslow, recno() as rec,cast(0 as f(6,0)) as buybreak from file1 wher contract=m.tempcontract orde by rec desc into curs breakout readwrite

Also your cursor would be faster if you indexed in on REC and ABSHIGH and ABSLOW