SQL Server 2012 has two new analytical functions, LEAD() AND LAG(). These functions return data from Next row (LEAD) and Previous row (LAG) of the same dataset without using self-join.
Let’s go with an example;
This is my initial dataset;
Here is the query to create this dataset;
IF OBJECT_ID('tempdb..#SampleDataSet') is not null drop table #SampleDataSet
SELECT *
INTO #SampleDataSet
FROM
(
SELECT 1 LineId, '2017-04-01' StockId, 'Toyota' AS Make, 'Brown' AS Color, 'Sale' AS ActionType, '2017-4-15' ActionDate UNION
SELECT 2 LineId, '2017-04-01' StockId, 'Toyota' AS Make, 'Brown' AS Color, 'Quote' AS ActionType, '2017-4-10' ActionDate UNION
SELECT 3 LineId, '2017-05-01' StockId,'Toyota' AS Make, 'Red' AS Color, 'Sale' AS ActionType, '2017-5-15' ActionDate UNION
SELECT 4 LineId, '2017-05-01' StockId, 'Toyota' AS Make, 'Red' AS Color, NULL AS ActionType, '2017-5-10' ActionDate UNION
SELECT 5 LineId, '2017-06-01' StockId, 'Toyota' AS Make, 'Pink' AS Color, 'Sale' AS ActionType, '2017-6-15' ActionDate UNION
SELECT 6 LineId, '2017-06-01' StockId, 'Toyota' AS Make, 'Pink' AS Color, 'Quote' AS ActionType, '2017-6-10' ActionDate UNION
SELECT 7 LineId, '2017-07-01' StockId, 'Toyota' AS Make, 'Green' AS Color, 'Sale' AS ActionType, '2017-7-15' ActionDate UNION
SELECT 8 LineId, '2017-07-01' StockId, 'Toyota' AS Make, 'Green' AS Color, 'Quote' AS ActionType, '2017-7-10' ActionDate UNION
SELECT 9 LineId, '2017-08-01' StockId, 'Honda' AS Make, 'Silver' AS Color, 'Sale' AS ActionType, '2017-8-15' ActionDate UNION
SELECT 10 LineId, '2017-08-01' StockId, 'Honda' AS Make, 'Silver' AS Color, 'Quote' AS ActionType, '2017-8-10' ActionDate UNION
SELECT 11 LineId, '2017-09-01' StockId, 'Honda' AS Make, 'Brown' AS Color, 'Sale' AS ActionType, '2017-9-15' ActionDate UNION
SELECT 12 LineId, '2017-09-01' StockId, 'Honda' AS Make, 'Brown' AS Color, NULL AS ActionType, '2017-9-10' ActionDate UNION
SELECT 13 LineId, '2017-10-01' StockId, 'Honda' AS Make, 'Red' AS Color, 'Sale' AS ActionType, '2017-10-15' ActionDate UNION
SELECT 14 LineId, '2017-10-01' StockId, 'Honda' AS Make, 'Red' AS Color, 'Quote' AS ActionType, '2017-10-10' ActionDate
) src
WHERE 1=1
/*
This is my initial dataset
*/
SELECT * FROM #SampleDataSet src
I am adding SeqCount and Seq column to identify each group of car (make and color) and each action in the group. LAG and Lead values of LineId column are used to demonstrate function output.
SELECT
src.LineId, src.StockId,
ROW_NUMBER() OVER (PARTITION BY StockId ORDER BY ActionDate DESC) Seq,
COUNT(*) OVER (PARTITION BY StockId) SeqCount,
src.Make, src.Color, src.ActionType, src.ActionDate,
LEAD(src.LineId) OVER (ORDER BY src.LineId) LeadValue,
LAG(src.LineId) OVER (ORDER BY src.LineId) LagValue
FROM #SampleDataSet src
This is the output of above query;
You can see the application of LEAD() and LAG() function in last two columns on the right side of above dataset.
One of the possible application of this function. As a business rule every quote action must be preceded by sale action in a car dealer dataset. If quote action is null in a group, fix it.
SELECT
src.StockId, src.Seq, src.SeqCount, src.Make, src.Color, src.ActionType,
CASE WHEN src.LeadValue = 'Sale' AND src.ActionType IS NULL THEN 'Quote' ELSE src.ActionType END dActionType,
src.ActionDate, src.LeadValue, src.LagValue
FROM
(
--Dataset transformation
SELECT
src.StockId,
ROW_NUMBER() OVER (PARTITION BY StockId ORDER BY ActionDate DESC) Seq,
COUNT(*) OVER (PARTITION BY StockId) SeqCount,
src.Make, src.Color, src.ActionType, src.ActionDate,
LEAD(ActionType) OVER (ORDER BY ActionDate DESC) LeadValue,
LAG(ActionType) OVER (ORDER BY ActionDate DESC) LagValue
FROM #SampleDataSet src
) src
WHERE 1=1
Here is the output;
dActionType is a derived column that has all the correct sequence of actions with the help of Lead() and LAG() function.
This is tricky. Azure SAS SQL Database has master database only. If programmer is taking any advantage of MSDB databases or synonmous, it will not work there directly.
Use Deploy Database to Microsoft Azure SQL Database
The simple method to restore to Azure SQL Database is;
This will create a new database on Azure SQL Server. You can rename it to your actual database.
Migrate to Azure SQL by restoring from Database
This method requires Azure storage account.
Create a blob storage. Upload database backup. From SQL Management Studio, right-click to [YourLOCALDatabase] and open Tasks>Export Data-tier Application. You can export your database to Azure Blob Storage on this wizard. After backup, connect your Azure SQL Server from SQL Management Studio. Go to your [YourREMOTEDatabase], right-click Databases folder from treeview and go to Import Data-tier Application. Choose your backup file from Blob Storage and enjoy!
This will create a new database on Azure SQL Server. You can rename it to your actual database.
Use Azure Data Migration Assistant Tool
This is unlike other methods mentioned above. This is the most efficient tool for migration databases to Azure SQL. This gives you a chance to select individual objects (tables, stored procedures, views, functions etc). Here is the link;
The tool allows us to create azure backup on local/remote. We can restore those backups on local/remote machine. I have a schedule job on my local that run everyday 10:00AM to create full backup in Azure blob container “myContainer”.
This tool also allows you to backup on your local computer.
Other methods that can be used with Azure SQL Database;
Only data is imported, all other objects will be lost
Requires SQL Server Management Studio
Manual procedure
Use if you need to move data from Azure to a specific destination (e.g. your old SQL Server) or in a particular format (e.g. flat file) with SQL Server Management Studio tools
Follow these steps to verify Access Database Engine (ACE Driver) installation;
Double check to make sure you did install the x64 version of Microsoft Access Database Engine here – http://www.microsoft.com/download/en/details.aspx?id=13255. Please note that only one version x64 –OR– x86 can be installed, not both.
Here is a SQL Server x64 server with the Microsoft Access Database Engine x32 installed – see it does not show up.
Here is the same server with the Microsoft Access Database Engine x64 installed – see how it shows up.
I do not know of a query to get a list of the providers – however if you look in the registry at
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL10.MSSQLSERVER\Providers and check for the key Microsoft.ACE.OLEDB.12.0
This will tell you that it is installed. If you have a 32-bit version installed on a 64-bit box you would need to look under the Wow6432Node, that key would be HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Microsoft SQL Server\MSSQL10.MSSQLSERVER\Providers.
You would have to make sure that both SQL Server and the ODBC/OLEDB driver are the same bitwise either both x64 or x32.
As far as launching the package, I think we have three reasonably safe options. The first is xp_cmdshell, which would probably work fine and be pretty simple to implement on a queue (probably the best option in my opinion).
The second would be to create a single job agent job for each allowed broker process and have the queue processor call sp_start_job only if the job is not already running (pretty easy to check).
The third option would be to use a CLR procedure to call a web service and have the web service start the SSIS package. You would have to dive into C# a bit and create your own web service to start an SSIS package, but it is not much code. Doing it this way, you would not have to allow unsafe assemblies because the web service would be out of process.
Keep in mind that starting a job is asynchronous, and would require additional coding in both the job and the application that starts the processes, as well as creating a job for every SSIS package that needs to be run. Using xp_cmdshell is synchronous, and requires that xp_cmdshell access be enabled (requires security configuration). It does require the writing of a command line for each package, but that can also be stored in a table and pulled when needed.
BOL shows code for running SSIS packages in an application. I am hoping, though not encouraged, that it may be possible to use a CLR stored procedure to run a SSIS package inside the database by simply passing the name of the package to run. We won’t need 100’s of jobs, or enable xp_cmdshell to accomplish the task. I am, however, pragmatic and will use which ever method turns out to be the best/easiest/quickest/etc way to meet the goals of the system: to reliably import the data from numerous source systems in the most efficient way possible and to recover from hardware/network failures/outages both planned and unplanned.