Question : Word frequency counter without Full text index

I need to be able to count the frequecy of all words in a given field within the specified table. I server in SQL 2005, but do not have access to full text indexing. This is one-time run, so the efficiency, such as use of cursors is not a factor. There
Here is example:
ROW1: Broken laptop screen
ROW2: Broken keyboard
ROW3: Laptop not booting
> I would like to see results as such:
WORD FREQUECY
------------ ------------
broken 2
laptop 2
screen 1
not 1
booting 1
keyboard 1

Answer : Word frequency counter without Full text index

Assuming that all words are separated by a space, create the parmstolist function below and then run the following code:

create #t table (word varchar(500))

select b.Value, count(*) as Frequency
from yourtable a
cross apply dbo.Parmstolist(a.yourcolumn, ' ') b
group by b.Value

drop table #t

                       
         
           
             1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:

           
           
             CREATE FUNCTION [dbo].[ParmsToList] (@Parameters varchar(8000), @delimiter varchar(10) ) 
returns @result TABLE (Value varchar(100)) 
AS   
begin 
    declare @dx varchar(9) 
    -- declare @loops int 
     --set @loops = 0 
  
     DECLARE @TempList table 
          ( 
          Value varchar(100) 
          ) 
  
     if @delimiter is null  set @delimiter = ' ' 
     if len(@delimiter) < 1 set @delimiter = ' ' 
     set @dx = left(@delimiter, case when @delimiter = ' ' then 1 else len(@delimiter) end -1) 
  
     DECLARE @Value varchar(8000), @Pos int 
  
     SET @Parameters = LTRIM(RTRIM(@Parameters))+ @delimiter 
     SET @Pos = CHARINDEX(@delimiter, @Parameters, 1) 
  
     IF REPLACE(@Parameters, @delimiter, @dx) <> '' 
     BEGIN 
          WHILE @Pos > 0 -- AND @Loops < 100 
          BEGIN 
               --set @loops = @loops + 1 
               SET @Value = LTRIM(RTRIM(LEFT(@Parameters, @Pos - 1))) 
               IF @Value <> '' 
               BEGIN 
                    INSERT INTO @TempList (Value) VALUES (CAST(@Value AS varchar)) --Use Appropriate conversion 
               END 
               SET @Parameters = SUBSTRING(@Parameters, @Pos+ case when @delimiter = ' ' then 1 else len(@delimiter) end, 8000) 
               SET @Pos = CHARINDEX(@delimiter, @Parameters, 1) 
  
          END 
     END     
     INSERT @result 
     SELECT value 
        FROM @TempList 
     RETURN 
END

           
         

Open in New Window

Select All