Passing multiple rows of data to stored procedures has always intrigued application developers. Many different methods have been used, more evolving around XML or some form of concatenated list. Those approaches require a method for parsing before the data can be used in normalized table format.

SQL Server 2008 adds another option: table valued parameters. This allows sending a table as parameter to the stored procedure. The current implementation still has shortcomings because it is a read-only parameter, and it cannot be used as output parameter.

Here is one example to demonstrate using table valued parameters to pass a table as stored procedure input.

CREATE TABLE Loans (
 loan_nbr INT PRIMARY KEY,
 loan_date DATETIME,
 loan_amount DECIMAL(15, 2));
 
-- User-defined table type 
CREATE TYPE LoanTable
AS TABLE (
 loan_nbr INT PRIMARY KEY,
 loan_date DATETIME,
 loan_amount DECIMAL(15, 2));
 
GO
 
-- Procedure with table valued parameter
-- Must use the READONLY clause
CREATE PROCEDURE InsertLoans 
 @Loans LoanTable READONLY
AS
 
 INSERT INTO Loans
 SELECT loan_nbr, loan_date, loan_amount
 FROM @Loans;
 
GO
 
-- Declare table variable of the new type
DECLARE @NewLoans LoanTable;
 
-- Initialize the table variable with data
INSERT INTO @NewLoans
VALUES (1, '20080101', 10000.00),
       (2, '20080203', 15000.00),
       (3, '20080315', 25000.00);
 
 
-- Insert new loans using the table variable as parameter
EXEC InsertLoans @Loans = @NewLoans;

The really cool part here is that table valued parameters are fully supported on the client side via ADO.NET (.NET Framework 3.5 or later) . That means we could easily have a data bound control like grid on the client side that can be used to feed a table parameter to update database table via a stored procedure. Here is a look at what that client code will be.

DataTable loans;
 
loans = new DataTable();
loans.Columns.Add("loan_nbr", typeof(int));
loans.Columns.Add("loan_date", typeof(System.DateTime));
loans.Columns.Add("loan_amount", typeof(decimal));
 
using(SqlCommand cmd = new SqlCommand("InsertLoans", conn)
{
  cmd.CommandType = CommandType.StoredProcedure;
  cmd.Parameters.AddWithValue("Loans", loans);
  cmd.ExecuteNonQuery();
}

Resources:

Table-Valued Parameters
http://msdn2.microsoft.com/en-us/library/bb510489(SQL.100).aspx

Another powerful T-SQL enhancement in SQL Server 2008 is the ability to write composable DML. It allows to consume the OUTPUT clause result set and to feed it as source for a query.

Here is one example to demonstrate that. The scenario is to merge data from daily updated branch office table to central office static table. Additional requirement is to store auditing data for any changes in the company name column. In prior versions of SQL Server this would require a couple SQL statements to accomplish. In SQL Server 2008 the new MERGE statement allows to perform all merge actions in a single statement, and then feed via the OUTPUT clause data for the auditing table. That way a single SQL statement completes the task.

-- Create central accounts static table
CREATE TABLE CentralOfficeAccounts (
 account_nbr INT PRIMARY KEY,
 company_name VARCHAR(35),
 primary_contact VARCHAR(35),
 contact_phone VARCHAR(12));
 
-- Daily updated branch data
CREATE TABLE BranchOfficeAccounts (
 account_nbr INT PRIMARY KEY,
 company_name VARCHAR(35),
 primary_contact VARCHAR(35),
 contact_phone VARCHAR(12));
 
-- Insert sample data 
INSERT INTO CentralOfficeAccounts
VALUES (1, 'Bayside Motors', 'Jim Smith', '902-203-1234'),
       (2, 'Dallas Industries', 'Joe Doe', '301-663-9134'),
       (3, 'Sky Computer Systems', 'Jane Brown', '201-943-6053');
 
INSERT INTO BranchOfficeAccounts
VALUES (2, 'Dallas Industries, Inc.', 'Rick Gross', '301-663-9134'),    -- changed
       (3, 'Sky Computer Systems', 'Jane Brown', '201-943-6053'),       -- same
       (4, 'Auto Insurance Co.', 'Chris Jefferson', '313-601-6201');    -- new
 
-- Table for auditing
CREATE TABLE AccountsAudit (
 account_nbr INT,
 change_action NVARCHAR(10),
 change_date DATETIME DEFAULT CURRENT_TIMESTAMP,
 old_company_name VARCHAR(35),
 new_company_name VARCHAR(35),
 PRIMARY KEY(account_nbr, change_action, change_date));
 
-- Single SQL statement performing the merge and auditing actions
INSERT INTO AccountsAudit 
 (account_nbr, change_action, old_company_name, new_company_name)
SELECT account_nbr, merge_action, old_company_name, new_company_name
FROM (MERGE INTO CentralOfficeAccounts AS C    
      USING BranchOfficeAccounts AS B        
         ON C.account_nbr = B.account_nbr
      WHEN MATCHED                        
       AND C.company_name <> B.company_name THEN                        
           UPDATE SET C.company_name = B.company_name
      WHEN NOT MATCHED THEN            
           INSERT (account_nbr, company_name, 
                   primary_contact, contact_phone)
           VALUES (B.account_nbr, B.company_name, 
                   B.primary_contact, B.contact_phone)
     WHEN SOURCE NOT MATCHED THEN        
          DELETE
     OUTPUT $action, 
            COALESCE(inserted.account_nbr, deleted.account_nbr),
            deleted.company_name, 
            inserted.company_name
     ) AS T(merge_action, account_nbr, old_company_name, new_company_name);
 
SELECT account_nbr, change_action, change_date,
       old_company_name, new_company_name
FROM AccountsAudit; 
 
/*
 
-- Results
account_nbr change_action change_date old_company_name   new_company_name
----------- ------------- ----------- ------------------ -----------------------
1           DELETE        2008-04-15  Bayside Motors     NULL
2           UPDATE        2008-04-15  Dallas Industries  Dallas Industries, Inc.
4           INSERT        2008-04-15  NULL               Auto Insurance Co.
 
*/ 

Note:

This code has been tested with SQL Server 2008 CTP 6 (February 2008). As of SQL Server 2008 Release Candidate 0 (June 2008) the clause “WHEN SOURCE NOT MATCHED” has been replaced with “WHEN NOT MATCHED BY SOURCE”. This makes the wording clearer. It is good to note that this clause of MERGE is not standard.

One very common requirement is to merge source data to a target table. For example, merge the daily changes to accounts data from branch office table to central accounts table. On SQL Server version 2005 and prior this was accomplished using separate INSERT and UPDATE statements. This involved checking if key column exists to perform update and insert if not, or attempt an update first and then if not rows were affected perform insert. Not to mention if we have to check if account is missing from the branch office data and needs to be deleted from the central accounts table. That way we end up with multiple (sometimes complex) statements to implement one transaction, accessing both source and target tables multiple times.

SQL Server 2008 offers a lot more elegant way using the MERGE statement (MERGE is supported by ANSI SQL). It makes data merging very simple and elegant, as well as efficient.

Here is the base syntax for MERGE:

[ WITH <common_table_expression> [,...n] ]
MERGE 
[ TOP ( expression ) [ PERCENT ] ] 
[ INTO ] target_table [ [ AS ] table_alias ]
        [ WITH ( <merge_hint> ) ]
USING <table_source>
ON <search_condition>
[ WHEN MATCHED [ AND <search_condition> ]
        THEN <merge_matched> ]
[ WHEN [TARGET] NOT MATCHED [ AND <search_condition> ]
        THEN <merge_not_matched> ]
[ WHEN SOURCE NOT MATCHED [ AND <search_condition> ]
        THEN <merge_matched> ]
<output_clause>
[ OPTION ( <query_hint> [ ,...n ] ) ]    
;

Below is the initial setup for the source and target tables.

-- Static table as target for merging data
CREATE TABLE CentralOfficeAccounts (
 account_nbr INT PRIMARY KEY,
 company_name VARCHAR(35),
 primary_contact VARCHAR(35),
 contact_phone VARCHAR(12));
 
-- Dynamic table with daily updates to be merged
CREATE TABLE BranchOfficeAccounts (
 account_nbr INT PRIMARY KEY,
 company_name VARCHAR(35),
 primary_contact VARCHAR(35),
 contact_phone VARCHAR(12));
 
-- Sample centrall office static data 
INSERT INTO CentralOfficeAccounts
VALUES (1, 'Bayside Motors', 'Jim Smith', '902-203-1234'),
       (2, 'Dallas Industries', 'Joe Doe', '301-663-9134'),
       (3, 'Sky Computer Systems', 'Jane Brown', '201-943-6053');
 
-- Daily updated branch office data
INSERT INTO BranchOfficeAccounts
VALUES (2, 'Dallas Industries, Inc.', 'Rick Gross', '301-663-9134'), -- changed
       (3, 'Sky Computer Systems', 'Jane Brown', '201-943-6053'),    -- same
       (4, 'Auto Insurance Co.', 'Chris Jefferson', '313-601-6201'); -- new

Here is how MERGE can be used to perform an update on existing accounts and insert of new accounts.

-- Update existing and add missing
MERGE INTO CentralOfficeAccounts AS C    -- Target
USING BranchOfficeAccounts AS B          -- Source
   ON C.account_nbr = B.account_nbr
WHEN MATCHED THEN                        -- On match update
     UPDATE SET C.company_name = B.company_name,
                C.primary_contact = B.primary_contact,
                C.contact_phone = B.contact_phone
WHEN NOT MATCHED THEN                    -- Add missing
     INSERT (account_nbr, company_name, primary_contact, contact_phone)
     VALUES (B.account_nbr, B.company_name, B.primary_contact, B.contact_phone);
 
SELECT account_nbr, company_name, primary_contact, contact_phone 
FROM CentralOfficeAccounts; 
 
/*
 
-- Results:
 
account_nbr company_name             primary_contact  contact_phone
----------- ------------------------ ---------------- -------------
1           Bayside Motors           Jim Smith        902-203-1234
2           Dallas Industries, Inc.  Rick Gross       301-663-9134
3           Sky Computer Systems     Jane Brown       201-943-6053
4           Auto Insurance Co.       Chris Jefferson  313-601-6201
 
*/

Next step is to enhance the statement including predicates to check and update only accounts that have changed. The only difference here is the additional conditions in the MATCHED clause, results will be the same as before.

-- Update existing that changed and add missing
-- Use of predicates 
MERGE INTO CentralOfficeAccounts AS C   -- Target
USING BranchOfficeAccounts AS B         -- Source
   ON C.account_nbr = B.account_nbr
WHEN MATCHED                            -- On match update
 AND (C.company_name <> B.company_name  -- Additional search conditions
   OR C.primary_contact <> B.primary_contact
   OR C.contact_phone <> B.contact_phone) THEN                        
     UPDATE SET C.company_name = B.company_name,
                C.primary_contact = B.primary_contact,
                C.contact_phone = B.contact_phone
WHEN NOT MATCHED THEN                   -- Add missing
     INSERT (account_nbr, company_name, primary_contact, contact_phone)
     VALUES (B.account_nbr, B.company_name, B.primary_contact, B.contact_phone);

Going one step further, let’s add the option to delete accounts that do not exist in the source table from the target table.

--Update existing that changed and add missing, delete missing in source
MERGE INTO CentralOfficeAccounts AS C   -- Target
USING BranchOfficeAccounts AS B         -- Source
   ON C.account_nbr = B.account_nbr
WHEN MATCHED                            -- On match update
 AND (C.company_name <> B.company_name  -- Additional search conditions
   OR C.primary_contact <> B.primary_contact
   OR C.contact_phone <> B.contact_phone) THEN                        
     UPDATE SET C.company_name = B.company_name,
                C.primary_contact = B.primary_contact,
                C.contact_phone = B.contact_phone
WHEN NOT MATCHED THEN                   -- Add missing
     INSERT (account_nbr, company_name, primary_contact, contact_phone)
     VALUES (B.account_nbr, B.company_name, B.primary_contact, B.contact_phone)
WHEN SOURCE NOT MATCHED THEN            -- Delete missing from source
     DELETE;
 
SELECT account_nbr, company_name, primary_contact, contact_phone 
FROM CentralOfficeAccounts; 
 
/*
 
-- Results:
 
account_nbr company_name             primary_contact  contact_phone
----------- ------------------------ ---------------- -------------
2           Dallas Industries, Inc.  Rick Gross       301-663-9134
3           Sky Computer Systems     Jane Brown       201-943-6053
4           Auto Insurance Co.       Chris Jefferson  313-601-6201
 
*/

There is a lot more to MERGE, but this just shows the tremendous power it provides for merging data.

Note:

This code has been tested with SQL Server 2008 CTP 6 (February 2008). As of SQL Server 2008 Release Candidate 0 (June 2008) the clause “WHEN SOURCE NOT MATCHED” has been replaced with “WHEN NOT MATCHED BY SOURCE”. This makes the wording clearer. It is good to note that this clause of MERGE is not standard.

References:

MERGE (Transact-SQL)
http://technet.microsoft.com/en-us/library/bb510625(SQL.100).aspx