First I tried walking a list using a recursive function and a match pattern. All I wanted to do was pop the head of the list, print it and process the remainder of the list.

The printfn statements in my example use the %A format specifier which provides pretty printing for any value. Also the ; symbol used to separate multiple expressions in a match rule causes the first expression to be evaluated and its result discarded before the second expression is evaluated and used as the result. This method can be used to execute side-affecting statements such as old fashioned debug printing.

#light;

let rec listWalk list =
  printfn "-----";
  match list with 
  | [] -> printfn "Empty List";
  | h::t -> printfn "Head is %A" h; printfn "Tail is %A" t; listWalk t;;

let s = "A string of words";;
let l = s |> String.split [' '];;

The list of words l is used in all the examples for this post.

> listWalk l;;
-----
Head is "A"
Tail is ["string"; "of"; "words"]
-----
Head is "string"
Tail is ["of"; "words"]
-----
Head is "of"
Tail is ["words"]
-----
Head is "words"
Tail is []
-----
Empty List
val it : unit = ()
>

The first match clause matches on an empty list ([]) and the second matches on a list construction. The head element is bound to the identifier h and the tail to the identifier t. Looking at the output from printfn, h is a value of type string (denoted by the surrounding double quotes) and t is a value of type list of string. Notice that when t is an empty list its string representation is the same as the pattern used to denote an empty list.

Next I wanted to a match rule to find the last element in the list, that is where there is a value for head and the tail is an empty list. So I added a rule that combined the previous two and matched on a list constructed with a head and an empty list.

let rec listWalk list =
 printfn "-----";
 match list with 
 | [] -> printfn "Empty List";
 | h::[] -> printfn "Last element is %A" h; 
 | h::t -> printfn "Head is %A" h; printfn "Tail is %A" t; listWalk t;; 

Using the same list of words as above.

> listWalk l;;
-----
Head is "A"
Tail is ["string"; "of"; "words"]
-----
Head is "string"
Tail is ["of"; "words"]
-----
Head is "of"
Tail is ["words"]
-----
Last element is "words"
val it : unit = ()

This time the rule for an empty list ([]) did not match because a recursive call was not made when a list with a single element was detected.

There is an easy mistake to make when using match patterns like this though. The patterns h::t and h::[] will both match the pattern of a list with one remaining element, as [] just means an empty list. So if the rules are sequenced incorrectly my new rule to detect the last element in the list will not work.

let rec listWalk list =
  printfn "-----";
  match list with 
  | [] -> printfn "Empty List";
  | h::t -> printfn "Head is %A" h; printfn "Tail is %A" t; listWalk t;
  | h::[] -> printfn "Last element is %A" h;;

Results in…

> listWalk l;;
-----
Head is "A"
Tail is ["string"; "of"; "words"]
-----
Head is "string"
Tail is ["of"; "words"]
-----
Head is "of"
Tail is ["words"]
-----
Head is "words"
Tail is []
-----
Empty List
val it : unit = ()
>

In fact the compiler is able to work this out outputs a warning when compiling the function.

warning FS0026: This rule will never be matched.

Of course if all I wanted to do was iterate over a list there are many functions in the List Module that would be a better choice. For example using one of the four iter(ation) functions.

let listWalk list =
  printfn "-----";
  list |> List.iter (fun e -> printfn "Element is %A" e);;

> > listWalk l;;
-----
Element is "A"
Element is "string"
Element is "of"
Element is "words"
val it : unit = ()
>

So with that in mind lets get started in F# interactive mode and the #light syntax.

My first draft was not pretty, but it worked and got my head into the game. The algorithm involves examining the elements of a list one by one and comparing the value to the value of a variable.

#light;

> let sentence = "there is a red dog";;
val sentence : string

> let tokens = sentence |> String.split [' '];;
val tokens : string list

> tokens;;
val it : string list = ["there"; "is"; "a"; "red"; "dog"]

> let find = "red";;
val find : string

> let rec search tokens find= 
 match tokens with
 | [] -> "no match"
 | h::t when h = find -> "match"
 | h::t -> search t find;;

val search : 'a list -> 'a -> string

First I defined and initialized the value sentence with the string I wanted to search, letting the compiler correctly infer the type to be string. The value of sentence was then passed as the last argument to the String.split function using the pipeline operator (|>). The first argument passed was a list containing the delimiters to split the string on, in this case just the space character. The result of calling split is another list containing the words split from the sentence; executing tokens;; in the interactive session returns the value.

The value to search for was stored in the variable ‘find’. I’m trying to avoid calling variables ‘variables’ as by default they are immutable in F# and so cannot be changed. So I tend to think of them more like values with a name, but this seems to make things a little confusing at times.

Then I defined the function search using the rec modifier to instruct the compiler to allow recursive calls. The function takes two parameters, the first is the list to search and the second is the value to search for. The match statement has three patterns, one of which makes a (tail) recursive call to continue processing the list of words. Recursion was probably not the best construct to use but I wanted to see how it works in F#. The three patterns are evaluated in order and work as follows.

1. Match on an empty list, execute the expression to return the string “no match”
2. Match on a list with at least one element, bind the head of the list to h and the remainder of the list to t. If the value of h is equivalent to the value of ‘find’ execute the expression to return the string “match”
3. Match on a list with at least one element, bind the head of the list to h and the remainder of the list to t. Make a recursive call to search and pass the tail (t) of the list.

Some simple tests showed to function to be working as expected.

> search tokens find;;
val it : string = "match"

> let find = "blue";;
val find : string

> search tokens find;;
val it : string = "no match"

Function Values

That worked as a first draft but it was not very nice, so I set about it with my refactoring stick.

The first thing I did was to modify the search function to return a boolean. I did not have to explicitly say it returned a boolean value, just change the string return values to the boolean constants and the compiler worked it out. Otherwise the search function itself was fine.

> let rec search tokens find = 
 match tokens with
 | [] -> false 
 | h::t when h = find -> true
 | h::t -> search t find;;
val search : 'a list -> 'a -> bool

As well as inferring the return type the compiler was also able to automatically generalize the function. There was nothing in the search function that required the value of find or the list of tokens to be a strings. So the compiler created a function that accepts a list of a generic type (”‘a list”), a single value of the same generic type (”‘a”) and returns a boolean value (bool).

Next I used partial function application / currying to create a function that accepts a string sentence to search in and returns an anonymous function. The code in the wordSearch function to split the sentence is only executed once, and only when the anonymous function is executed. At the moment I don’t understand the language enough to explain why, but the key element to achieving this result is the anonymous function requiring a parameter to be passed.

> let wordSearch s = 
 let tokens = String.split [' '] s
 fun (word: string) -> search tokens word;;
val wordSearch : string -> (string -> bool)

> let s = wordSearch "there is a red dog";;
val s : (string -> bool)

> s "red";;
val it : bool = true

> s "blue";;
val it : bool = false

Next I hope to modify the function to search the list of tokens for any element from a list.

In C# 3.0 we have Object Initializers which allow for an object to be instantiated via an explicit constructor call and properties to be set in one statement.

Form f = new Form1()
{
    Text = "Hello"
};
Application.Run(f);

And in F# we have initial property settings where named values are first mapped to constructor parameters and the unused values used to set properties.

> open System.Windows.Forms;;

> let f = new Form(Text = "Hello");;

val f : Form

> f.Show();;
val it : unit = ()

One of the neat tricks is when a function is automatically inferred to be generic through a process known as Generalization. This occurs when a function does not constrain or use a parameter is such a way that the value must be of a specific type.

Lets start with something that is not Generalized. A function that takes a tuple of two values and subtracts 1 from each value.

> let sub (a,b) = (a-1, b-1);;
val sub : int * int -> int * int

> sub (2,3);;
val it : int * int = (1, 2)

While I cannot explain why the sub function is not generalizable in terms of the rules laid out in the language specification of Generalization, it’s pretty clear what’s happened. Subtracting the integer 1 from each of the tuple values requires that the tuple be of type int * int, i.e. two integers. So the function sub takes a tuple of two integers and returns a tuple of two integers.

Now something that can be generalized, returning the first value in a tuple of two values.

> let getFirst (a,b) = a;;
val getFirst : 'a * 'b -> 'a

> getFirst (1,2);;
val it : int = 1

> getFirst ("a", "b");;
val it : string = "a"

Types are denoted by the token, so the tuplea * “b is a tuple of a value of type a and a value of type b. Because of the generalization getFirst can be called with two integers (1,2) or two strings (”a”, “b”). This is just how the built in fst operator works. It takes a tuple of two values and returns the first (there is also a snd to return the second value).

>fst;;
val it : ('a * 'b -> 'a) = <fun:clo@0_6>

> fst (1,2);;
val it : int = 1

> fst ("a", "b");;
val it : string = "a"

The first thing you need is a user defined scalar function to that returns an XML type. As with all persisted computed columns the function must be deterministic and as Bob points out that’s fine because all XQuery functions are deterministic. However the UDF must also be marked with SCHEMABINDING to ensure it’s determinism.

SCHEMABINDING guarantees that the dependancies of the function cannot be changed without first altering the function which has two effects. First the function will continue to execute without an exception (e.g. bad column name), which could be considered a type of determinism. Second column types and definitions cannot change; changes could lead to truncation, overflow or a dependancy on a column changed to a non deterministic computed definition.

So first the function, you’ll need to pass all the column values into the function.

create function dbo.udf_event_data
(
    @start_time as smalldatetime, 
    @end_time as smalldatetime  
)
returns xml 
with schemabinding
as
begin

-- times must be cast to formatted strings for XQuery
declare @start_time_char varchar(128)
set @start_time_char = convert(varchar(128), @start_time, 126) + 'Z'    

declare @end_time_char varchar(128)
set @end_time_char = convert(varchar(128), @end_time, 126) + 'Z'    

declare @x xml
set @x = '';

set @x = 
    @x.query('
        declare namespace xsi="http://www.w3.org/2001/XMLSchema-instance";

        
                    
                {   
                    if ( empty(sql:variable("@start_time_char") cast as xs:dateTime? )) then                    
                        attribute xsi:nil {"true"}  
                    else () 
                }
                {
                    sql:variable("@start_time_char") cast as xs:dateTime?
                }
            
                  
                {   
                    if ( empty(sql:variable("@end_time_char") cast as xs:dateTime? )) then                  
                        attribute xsi:nil {"true"}  
                    else () 
                }
                {
                    sql:variable("@end_time_char") cast as xs:dateTime?
                }
            
        
    ');     
return @x
end     

The next step is to create a computed column and mark the XML column as PERSISTED. All the columns values needed by the function must be passed in the definition which should look something like.

...
[event_data]  AS ([dbo].[udf_event_data]([start_time],[end_time])) PERSISTED,

The UDF above uses the mechanism for handling SQL NULL discussed previously.

<event_data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true" />

The semantic equivalent of SQL NULL in XQuery is an empty sequence. To convert a SQL value to a sequence the XQuery cast operator can be used. This will first atomize the value, then according to the XQuery spec if the result is an empty sequence the cast operation will succeed if the target type has the optional occurrence indicator (’?’ meaning zero or one).

Meaning that the cast must fail and throw an error unless the occurrence indicator is ? present when casting an empty sequence. However the SQL implementation is a little different; from the documentation on SequenceType Expressions…

“In SQL Server 2005, the instance of and cast as XQuery expressions on SequenceTypes are partially supported… In SQL Server, the question mark (?) is required after the AtomicType.”

I imagine SQL added this requirement to their implementation to avoid runtime errors and possibly to continue to the idea that emptyness / NULLness is not a bad thing.

So to detect NULL in a SQL value and optionally create an element with xsi:nil is used the following.

declare @data varchar(128)

declare @x xml
set @x = '';

set @x = 
    @x.query('
        declare namespace xsi="http://www.w3.org/2001/XMLSchema-instance";
        
            
                {   
                    if ( empty(sql:variable("@data") cast as xs:string? )) then                 
                        attribute xsi:nil {"true"}  
                    else () 
                }
                {
                    sql:variable("@data") cast as xs:string?
                }
        
    ');     
select  @x

If the else clause of the if is used to construct an xsi:string node the following error is raised.

XQuery [query()]: Heterogeneous sequences are not allowed: found 'xs:string ?' and
'attribute(xsi{http://www.w3.org/2001/XMLSchema-instance}:nil,xdt:untypedAtomic)'

The Atomic Constructor Functions can also be used to convert values, such as xs:string(@data) however I prefer to use cast when casting variables.

It also has an effect on the locking strategy employed under different isolation levels that can result in dramatically reduced concurrency on a table. Of course in the general case a table with a clustered index is preferred to a heap so this should not be a day to day issue, but its a moderately interesting situation.

To test this I created a DB with two tables, one a heap and one a b-tree, and a view that shows the locks for a connection. I also referred to the Lock Compatibility page on more than one occasion.

create database locks;
go

use locks
go

create table noindex 
(
    col1 int ,
    col2 varchar(2)
)
go

insert into noindex
values
    (1,'a');
insert into noindex
values
    (2,'b');

create table indexed
(
    col1 int primary key,
    col2 varchar(2)
)
go

insert into 
    indexed
select
    *
from
    noindex;
go

create view
    vwLocks
as
    select 
        cast(request_mode as varchar(5)) as request_mode ,
        cast(request_status as varchar(10)) as request_status ,
        cast(resource_type as varchar(15)) as resource_type,
        case 
            when resource_type = 'OBJECT' then
                cast(object_name(resource_associated_entity_id) as varchar(20)) 
            else
                cast(resource_associated_entity_id as varchar(20))
        end as resource_name,
        cast(resource_description as varchar(15)) as resource_description
    from 
        sys.dm_tran_locks
    where
        request_session_id = @@spid
go

Each test used the same basic query below, changing the table (from ‘noindex’ to ‘indexed’) and the isolation level.

set transaction isolation level read committed
begin transaction

select
    *
from
    noindex
where
    col1 = 1

print 'after select'

select  
    *
from
    vwLocks

insert into
    noindex
values
    (3,'c')

print 'after insert'

select  
    *
from
    vwLocks

update 
    noindex
set 
    col2 = 'x' 
where 
    col1 = 2

print 'after update'

select  
    *
from
    vwLocks

rollback

Heap and READ COMMITTED isolation

Under READ COMMITTED isolation no locks were maintained after the select. After the insert however an Exclusive (X) lock is held against the newly inserted row identified by its Row ID (RID) 1:152:2. The first number in the RID is the File number, the next the Page number and the last the “slot” number in the page which can be though of as the row index. There are also Intent the Exclusive (IX) locks that were taken on the Table and then the Page before the row was inserted.

After the update the only additional locking is an Exclusive lock on the updated row, again identified by its RID (1:152:1).

after select

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             

after insert

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
X            GRANT          RID             72057594038321152    1:152:2        
IX           GRANT          PAGE            72057594038321152    1:152          
IS           GRANT          OBJECT          vwLocks                             
IX           GRANT          OBJECT          noindex                             

after update

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
X            GRANT          RID             72057594038321152    1:152:2        
IX           GRANT          PAGE            72057594038321152    1:152          
IS           GRANT          OBJECT          vwLocks                             
IX           GRANT          OBJECT          noindex                             
X            GRANT          RID             72057594038321152    1:152:1        
   

If the previous query was run and the transaction left running (remove the rollback statement) another query could still read from the table under certain circumstances. The query would need to be constructed so that it did not to access the locked rows, however all rows may need to be accessed to check if they satisfy the query. For example the next query will not block behind the open transaction.

select top 1 * from noindex where col1 = 1

The first row in the table has col1 equal to 1, and the top 1 operator tells the query engine it only needs to return one row so there is no need to scan for further rows. The next query will block as it must access all rows to test if col1 equals 1, if the rows were ordered by col 1 as soon as it found the row with col2 = 2 the query could stop (as 2 > 1 so any value >= 2 must also be > 1).

select * from noindex where col1 = 1

The point here is that there is still room for other queries to read from the table. If there were Non Clustered indexes on the table they would identify the rows in the heap using the RID and would not need to scan the heap to find candidate rows. Instead the non clustered index would be scanned and the RID values used to index directly to the row in the page. Yes there would be locking on the NC index that could block the scan, but I’ll look at that another day.

Heap and SERIALIZABLE isolation

Under a serializable transaction a table level Shared (S) lock is held after the select that prevents all updates on the table. Under serializable isolation the results of the select need to be protected from Phantom Reads and Unrepeatable Reads. As there is no index on col1 another record could be inserted or modified to have a value of 1, which would invalidate the result of the select. So the only way to support the isolation level is to prevent all changes on the table by holding the shared lock. Other connections can still read from the table, but they will block when trying to insert or update any row.

The insert statement leaves an Exclusive lock on the RID the new row is added to and the accompanying Intent Exclusive lock taken on the Page the row was added to. The Shared lock left in place by the select is promoted to a Shared Intent Exclusive (SIX) lock, this seems to act like having both a S and an IX lock on the table. Other connections may still take an Intent Shared lock on the table and then read rows that do not have Exclusive locks on them. However other connections may not take an Intent Exclusive lock on the table that would be the first step in locking down to the row (RID) level to insert or update a row. Thats may understanding, it’s covered in the Lock Modes page.

So after the select and the insert other connections may still read from the table, but any form of update is prohibited. This is the same as the read committed transaction, however running the update changes things somewhat.

The update promotes the Intent Exclusive lock on the table up to a full Exclusive lock preventing all access from other connections. Right now I cannot crack the logic of why this would happen. If only the update query is run in the transaction the the Exclusive lock is also taken.

after select

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             
S            GRANT          OBJECT          noindex                             

after insert

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
X            GRANT          RID             72057594038321152    1:152:2        
IX           GRANT          PAGE            72057594038321152    1:152          
IS           GRANT          OBJECT          vwLocks                             
SIX          GRANT          OBJECT          noindex                             

after update

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
X            GRANT          RID             72057594038321152    1:152:2        
IX           GRANT          PAGE            72057594038321152    1:152          
IS           GRANT          OBJECT          vwLocks                             
X            GRANT          OBJECT          noindex                                            

B-Tree and READ COMMITTED isolation

As a contrast below is the locks taken when using the indexed table under read committed isolation.

At all times other connections may read from the indexed table so long as they do not attempt to read data inserted or updated by the transaction. This is because the highest lock taken on the table and page is an Intent Exclusive. Which is compatible with the Intent Shared lock a query would take on the same object before attempting to lock down to the key level.

after select

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             

after insert

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IX           GRANT          PAGE            72057594038386688    1:154          
IS           GRANT          OBJECT          vwLocks                             
X            GRANT          KEY             72057594038386688    (03000d8f0ecc) 
IX           GRANT          OBJECT          indexed                             

after update

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IX           GRANT          PAGE            72057594038386688    1:154          
IS           GRANT          OBJECT          vwLocks                             
X            GRANT          KEY             72057594038386688    (03000d8f0ecc) 
IX           GRANT          OBJECT          indexed                             
X            GRANT          KEY             72057594038386688    (020068e8b274) 

B-Tree and SERIALIZABLE isolation

The locks taken below under a serializable isolation level are slightly different owing to the additional isolation guarantees. However the effect on the ability of another connection to read from the table is the same.

after select

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          PAGE            72057594038386688    1:154          
IS           GRANT          OBJECT          vwLocks                             
IS           GRANT          OBJECT          indexed                             
S            GRANT          KEY             72057594038386688    (010086470766) 

after insert

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IX           GRANT          PAGE            72057594038386688    1:154          
IS           GRANT          OBJECT          vwLocks                             
X            GRANT          KEY             72057594038386688    (03000d8f0ecc) 
IX           GRANT          OBJECT          indexed                             
S            GRANT          KEY             72057594038386688    (010086470766) 

after update

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IX           GRANT          PAGE            72057594038386688    1:154          
IS           GRANT          OBJECT          vwLocks                             
X            GRANT          KEY             72057594038386688    (03000d8f0ecc) 
IX           GRANT          OBJECT          indexed                             
X            GRANT          KEY             72057594038386688    (020068e8b274) 
S            GRANT          KEY             72057594038386688    (010086470766) 

Conclusion

So if there is a Heap table and its modified inside a Serializable transaction you may well end up with the entire table locked for all other connections. Another reason I guess to avoid Heaps.

To look into this further I tried a similar approach used previously, setting up the environment as follows.

create database locks;
go

use locks;
go

create table data
(
    col1 char(1) primary key,
    col2 char(4500) not null
)

insert into
    data
values
    ('a', 'a');
insert into
    data
values
    ('b', 'b');
go

--drop view vwLocks
create view
    vwLocks
as
    select 
        cast(request_mode as varchar(5)) as request_mode ,
        cast(request_status as varchar(10)) as request_status ,
        cast(resource_type as varchar(15)) as resource_type,
        case 
            when resource_type = 'OBJECT' then
                cast(object_name(resource_associated_entity_id) as varchar(20)) 
            else
                cast(resource_associated_entity_id as varchar(20))
        end as resource_name,
        cast(resource_description as varchar(15)) as resource_description
    from 
        sys.dm_tran_locks
    where
        request_session_id = @@spid
go

-- drop and re-establish the connection to close any locks 
-- hanging around from creating the db.

This created a table where only a single row can fit on a data page, so each of the two rows added must be stored on a different data page. To verify the number of data pages in the table I used DBCC IND. It’s undocumented but there is some info out there, especially in Inside Microsoft SQL Server 2005: The Storage Engine.

Pages of type 1 are the in row data pages, in my DB the first data page is 143 and the NextPagePID column shows that the next page is 154. To view the contents of the pages use DBCC PAGE after first turing on the trace flag 3604, I’ve used these undocumented methods previously.

dbcc ind (locks, data, -1)

-- to view the contents of the pages
dbcc traceon(3604)
dbcc page (locks, 1,143,3)

The next step was to read from the data table and observe the locks taken.

set transaction isolation level read committed
begin transaction
select data.col1, l.* from data outer apply vwLocks as l
commit 

/*
col1 request_mode request_status resource_type   resource_name        resource_description
---- ------------ -------------- --------------- -------------------- --------------------
a    S            GRANT          DATABASE        0                                   
a    IS           GRANT          PAGE            72057594038321152    1:143          
a    IS           GRANT          OBJECT          data                                
a    IS           GRANT          OBJECT          vwLocks                             
b    S            GRANT          DATABASE        0                                   
b    IS           GRANT          PAGE            72057594038321152    1:143          
b    IS           GRANT          OBJECT          data                                
b    IS           GRANT          OBJECT          vwLocks                             
*/

When reading both of the records the view showed locks on page 143, this was not what I expected. I thought it would show an Intent Shared lock on page 143 when reading row “a” and an Intent Shared lock on page 154 when reading row “b”.

The query plan was helpful in solving this mystery. The results of the query against vwLocks are cached by a Table Spool operation and used to perform the join for both rows from the data table.

The Remote Scan in the lower right corner is the read against sys.dm_tran_locks.

As seen in the properties of the Remote Scan operation reading from sys.dm_tran_locks occurs once and the results are used once (Actual Rebinds = 0 and Actual Rewinds = 0). However the Table Spool operation is evaluated twice (Actual Rebinds = 1 and Actual Rewinds =1).

From the help page on Logical and Physical Operators…

“If an operator is on the inner side of a loop join, the sum of the number of rebinds and rewinds should equal the number of rows processed on the outer side of the join. A rebind means that one or more of the correlated parameters of the join changed and the inner side must be reevaluated. A rewind means that none of the correlated parameters changed and the prior inner result set may be reused.”

So I needed to remove the caching effect of the Table Spool to see the lock state for each row read from the table. I tried using join predicates and table valued functions that took parameters but they all included spooling and returned the same values. Finally I settled on a c# table valued function that queried vwLocks; the query engine did not dare cache the results from something so wacky as a clr routine.

The c# function queries the view using the context connection, this is the connection the request is running on. It then reads the results into objects so it can dispose of the SqlDataReader and associated objects before returning the IEnumerable collection. The Fill method is then called by SQL to project the data in each object in the collection to the scalar column values of the table.

The full c# code was.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;

public partial class UserDefinedFunctions
{

    private readonly static string _sql = "select * from vwLocks"; 

    //the table definition property has been re-formated to fit on the page
    //it should be on a single line
    
    [Microsoft.SqlServer.Server.SqlFunction(
        DataAccess=DataAccessKind.Read, 
        FillRowMethodName="Fill", 
        IsDeterministic=false, 
        Name="tvf_c_locks", 
        SystemDataAccess=SystemDataAccessKind.Read, 
        TableDefinition="request_mode nvarchar(5), request_status nvarchar(10), 
            resource_type nvarchar(15), resource_name nvarchar(20), resource_description nvarchar(25)")]
    public static IEnumerable tvc_c_locks()
    {
        using (SqlConnection conn = new SqlConnection("context connection = true"))
        {
            conn.Open();
            using (SqlCommand cmd = new SqlCommand(_sql, conn))
            {
                using (SqlDataReader r = cmd.ExecuteReader(CommandBehavior.SingleResult))
                {
                    List result = new List();

                    while (r.Read())
                    {
                        result.Add(new Row(
                            safeString(r, 0), safeString(r, 1),safeString(r, 2),
                            safeString(r, 3),safeString(r, 4)));
                    }
                    return result;
                }
            }
        }
    }

    private static SqlString safeString(SqlDataReader reader, int index)
    {
        if (reader.IsDBNull(index))
        {
            return SqlString.Null;
        }
        return new SqlString(reader.GetString(index));
    }

    public static void Fill(object row, out SqlString request_mode, out SqlString request_status, 
        out SqlString resource_type, out SqlString resource_name, out SqlString resource_description)
    {
        Row r = row as Row;

        request_mode = r._request_mode;
        request_status = r._request_status;
        resource_type = r._resource_type;
        resource_name = r._resource_name;
        resource_description = r._resource_description;
    }

    public class Row
    {
        public readonly SqlString _request_mode;
        public readonly SqlString _request_status;
        public readonly SqlString _resource_type;
        public readonly SqlString _resource_name;
        public readonly SqlString _resource_description;

        public Row(SqlString request_mode, SqlString request_status, 
                    SqlString resource_type, SqlString resource_name, SqlString resource_description)
        {
            _request_mode = request_mode;
            _request_status = request_status;
            _resource_type = resource_type;
            _resource_name = resource_name;
            _resource_description = resource_description;
        }
    }
};

With the udf deployed to SQL Server using Visual Studio I queried it as follows.

set transaction isolation level read committed
begin transaction
select data.col1,l.*  from data outer apply dbo.tvf_c_locks() as l
commit

/*
col1 request_mode request_status resource_type   resource_name        resource_description
---- ------------ -------------- --------------- -------------------- -------------------------
a    S            GRANT          DATABASE        0                                   
a    Sch-S        GRANT          METADATA        0                    assembly_id = 6
a    IS           GRANT          PAGE            72057594038321152    1:143          
a    Sch-S        GRANT          OBJECT          tvf_c_locks                         
a    IS           GRANT          OBJECT          data                                
a    IS           GRANT          OBJECT          vwLocks                             
b    S            GRANT          DATABASE        0                                   
b    Sch-S        GRANT          METADATA        0                    assembly_id = 6
b    IS           GRANT          PAGE            72057594038321152    1:154          
b    Sch-S        GRANT          OBJECT          tvf_c_locks                         
b    IS           GRANT          OBJECT          data                                
b    IS           GRANT          OBJECT          vwLocks                             
*/

A number of new locks taken during the query to maintain the stability of the c# function. But the results show that when reading row “a” a Page level Intent Shared lock was taken on page 143 only; and when reading row “b” an IS lock was taken on page 154 only. This looks like the result I wanted; page level locking taken for the duration of the read from the page and not for the duration of the query.

If the page level locks were maintained I would expect to see the lock on page 143 also present when row “b” was read. Such as under a REPEATABLE READ transaction where the contents of page 143 must be protected from change for the duration of the query / transaction as shown below.

set transaction isolation level repeatable read
begin transaction
select data.col1,l.*  from data outer apply dbo.tvf_c_locks() as l
commit

col1 request_mode request_status resource_type   resource_name        resource_description
---- ------------ -------------- --------------- -------------------- -------------------------
a    S            GRANT          DATABASE        0                                   
a    Sch-S        GRANT          METADATA        0                    assembly_id = 6
a    IS           GRANT          PAGE            72057594038321152    1:143          
a    Sch-S        GRANT          OBJECT          tvf_c_locks                         
a    S            GRANT          KEY             72057594038321152    (6100bc414817) 
a    IS           GRANT          OBJECT          data                                
a    IS           GRANT          OBJECT          vwLocks                             
b    S            GRANT          DATABASE        0                                   
b    Sch-S        GRANT          METADATA        0                    assembly_id = 6
b    IS           GRANT          PAGE            72057594038321152    1:143          
b    IS           GRANT          PAGE            72057594038321152    1:154          
b    Sch-S        GRANT          OBJECT          tvf_c_locks                         
b    S            GRANT          KEY             72057594038321152    (6100bc414817) 
b    IS           GRANT          OBJECT          data                                
b    S            GRANT          KEY             72057594038321152    (62000610418e) 
b    IS           GRANT          OBJECT          vwLocks                             

In these results the lock on page 143 is still present when reading row “b” from page 154. As seen in previous posts these locks will be maintained to the end of the transaction.

DBCC USEROPTIONS

DBCC USEROPTIONS

Set Option                           Value
-------------------------------------------------------------------------------
textsize                             2147483647
language                             us_english
dateformat                           mdy
datefirst                            7
lock_timeout                         -1
quoted_identifier                    SET
arithabort                           SET
ansi_null_dflt_on                    SET
ansi_warnings                        SET
ansi_padding                         SET
ansi_nulls                           SET
concat_null_yields_null              SET
isolation level                      read committed

sys.dm_exec_sessions

select
    session_id, 
    case (transaction_isolation_level)
        when 0 then 'Unspecified'
        when 1 then 'Read Uncommitted'
        when 2 then 'Read Committed'
        when 3 then 'Repeatable'
        when 4 then 'Serializable'
        when 5 then 'Snapshot'
    end as transaction_isolation_level_desc
from
    sys.dm\_exec\_sessions 
where
    session_id = @@spid

/*
session_id transaction_isolation_level_desc
---------- --------------------------------
55         Read Committed
*/

sys.dm_exec_requests

select
    session_id, 
    case (transaction_isolation_level)
        when 0 then 'Unspecified'
        when 1 then 'Read Uncommitted'
        when 2 then 'Read Committed'
        when 3 then 'Repeatable'
        when 4 then 'Serializable'
        when 5 then 'Snapshot'
    end as transaction_isolation_level_desc
from
    sys.dm_exec_requests 
where
    session_id = @@spid
    
/*
session_id transaction_isolation_level_desc
---------- --------------------------------
55         Read Committed
*/

Remember that the isolation level can be modified for query / statement through the use of table hints to be different to the level in place when the transaction started. This is supported by the sys.dm_exec_requests view as it has a many to one relationship with a transaction and represents a substring of the sys.dm_exec_sql_text.

The simple answer is its the same when compared to the locking examples I previously identified when running under different isolation levels. Without an explicit transaction in place the query will run under an Auto Commit Transaction, which sound a lot like what used to be called “system transactions”.

A quick execution of some modified code from the previous post shows the same locks taken under the SERIALIZABLE isolation level when an explicit transaction is not in place.

set transaction isolation level  serializable

select * from vwLocks
select * from data cross apply vwLocks
select * from vwLocks 

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             

col1 request_mode request_status resource_type   resource_name        resource_description
---- ------------ -------------- --------------- -------------------- --------------------
a    S            GRANT          DATABASE        0                                   
a    IS           GRANT          PAGE            72057594038321152    1:143          
a    RangeS-S     GRANT          KEY             72057594038321152    (6100bc414817) 
a    IS           GRANT          OBJECT          data                                
a    IS           GRANT          OBJECT          vwLocks                             


request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             

And the same is seen for the READ COMMITTED isolation level.

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             


col1 request_mode request_status resource_type   resource_name        resource_description
---- ------------ -------------- --------------- -------------------- --------------------
a    S            GRANT          DATABASE        0                                   
a    IS           GRANT          PAGE            72057594038321152    1:143          
a    IS           GRANT          OBJECT          data                                
a    IS           GRANT          OBJECT          vwLocks                             

request_mode request_status resource_type   resource_name        resource_description
------------ -------------- --------------- -------------------- --------------------
S            GRANT          DATABASE        0                                   
IS           GRANT          OBJECT          vwLocks                             

The query must still take locks in order to protect against reading data that is currently being modified. And the absence of a transaction means there is no delineated time span in which future modifications must be prevented. So locks are not held after the query has executed.

That got me thinking; if the guarantees about future changes (Repeatable Reads and Phantom Reads) don’t apply because there is no transaction it seems odd that SERIALIZABLE needs to take stronger locks than READ COMMITTED. After all they are both implementing the same guarantee about current modifications (no Dirty Reads).

But its not just a transaction that has duration, a query can as well. What if the query involved multiple reads from the same table; or spooled from a non clustered index and then performed lookups on a heap or clustered index. In those cases would the guarantees of the isolation level apply for the duration of the query?

I could not find a way to show the locks taken during the various phases of a query as I could with the simple examples in the previous post on locks. But I was able to create a test that demonstrated different isolation levels.

First I created a table of numbers so I could create a long running query. The code below uses the technique from chapter 4 of Inside SQL Server 2005: T-SQL Querying which can be found here. I’ve removed the primary key to ensure the queries take a long time to complete; you should adjust the number of rows (using the @max variable) in the table so there is enough time to run the second query.

IF OBJECT_ID('dbo.Nums') IS NOT NULL
  DROP TABLE dbo.Nums;
GO
CREATE TABLE dbo.Nums(n INT NOT NULL);
DECLARE @max AS INT, @rc AS INT;
SET @max = 10000;
SET @rc = 1;

INSERT INTO Nums VALUES(1);
WHILE @rc * 2 < = @max
BEGIN
  INSERT INTO dbo.Nums SELECT n + @rc FROM dbo.Nums;
  SET @rc = @rc * 2;
END

INSERT INTO dbo.Nums 
  SELECT n + @rc FROM dbo.Nums WHERE n + @rc <= @max;
GO

Next I modified the vwLocks view created in the previous post to show locks from all connections as follows.

ALTER view
    [dbo].[vwLocks]
as
    select 
        cast(request_mode as varchar(10)) as request_mode ,
        cast(request_status as varchar(10)) as request_status ,
        cast(resource_type as varchar(15)) as resource_type,
        case 
            when resource_type = 'OBJECT' then
                cast(object_name(resource_associated_entity_id) as varchar(20)) 
            else
                cast(resource_associated_entity_id as varchar(20))
        end as resource_name,
        cast(resource_description as varchar(15)) as resource_description
    from
        sys.dm_tran_locks

I then created a query that took 7 seconds to complete; it finds numbers in the table that have 0 or 1 numbers higher than them by counting all of the numbers lower than the first.

set transaction isolation level read committed

select 
    parent.n
from 
    Nums as parent
where
    1 >= 
    (
        select 
            count(child.n)
        from
            Nums as child
        where
            child.n > parent.n
    )

The query plan for this is nasty; on my machine it involved scanning the Parent alias and scanning then spooling the Child alias 10001 times. When run by it returns two numbers: 9999 and 10000. The next time I ran it I quickly switched to another connection and inserted row into the table as follows:

insert into Nums
values(9999999)

Every time I did this the Insert completed immediately and the select query returned the new row as well as the previous values.

n
-----------
9999
10000
9999999

(3 row(s) affected) 

If the Select query is run after the new row is inserted only two numbers are returned: 10000 and 9999999. Inserting the row while the Select query is running results in a Phantom Read; the new row was not present when the Parent number 9999 was considered otherwise it would not have been included. It is was present and included in the result set later in the execution of the query.

When the test is run under SERIALIZABLE isolation level the Insert statement blocks until the Select has completed. The select returns the numbers 9999 and 10000 as it should, when run again after the Insert has completed it returns 10000 and 9999999.

While I cannot demonstrate the locks during the query it seems clear that the SERIALIZABLE query took and held locks which prevented Phantom Reads occurring, and so prevented the Insert from completing. And that the READ COMMITTED query either did not take or did not hold its locks long enough to prevent the Phantom Reads as it does not guard against them; and so the Insert was able to add a phantom record that was sucked into the query.