Alerts in SQL SERVER 2000

June 27, 2013, 8:03 am

≫ Next: enabling 3rd LAN interfaces breaks SQL

≪ Previous: Possible for using profiler for creating workload for Database Engine Tuning Advisor in case of SSRS reports

Severity alerts from 19-25 and data and log file size alerts are not working on SQL SERVER 2000 SP3 Stnd Edn, any idea to fix this?

↧

enabling 3rd LAN interfaces breaks SQL

March 4, 2014, 11:14 am

≫ Next: developing a Database maintenance plan

≪ Previous: Alerts in SQL SERVER 2000

I’m having a problem on a server. Here’s the situation, I have Windows Server 2008 Standard (pre-R2) server with Hyper-V enabled and SQL Server 2005 Standard installed. SQL hosts a couple of large databases. Hyper-V hosts one virtual server. All of this runs on a Dell R 710 server with 24GB of memory and a single Quad-core CPU (w/ hyper threading) and 4 physical NICs. One NIC is dedicated to the LAN. One is dedicated to Hyper-V (setup as external in Hyper-V manager). And one is (trying) to be dedicated to an iSCSI connection.

When I enable the iSCSI interface, with either IPV6 or IPV4 enabled, SQL will no longer respond to network requests (I tried using named pipes and tcp/ip). I can connect to SQL from the local host (over named pipes and tcp/ip). The SQL ERRORLOG shows SQL listening on tcp:1433 and named pipes. It also shows clients logon requests being granted. However, from client systems the logon timeouts. Using Wireshark on both client and server, I can see the initial logon request, but no response is ever sent by the server. I’ve tried disabling ‘remote connections’ and ‘tcip/ip’ on SQL Configuration after starting the iSCSI interface, then restarting SQL services, and re-enabling ‘remote connections’ and ‘tcp/ip’ in SQL Configuration, and concluding with a SQL restart. That didn’t help at all.

Last Friday I tried cleaning up the list of ‘TCP/IP’ addresses in ‘TCP/IP Properties’ in “SQL Configuration Manager”. I made sure that only the real IP addresses that are on the server are listed, and that the IP address for iSCSI port was disabled. Unfortunately, this didn’t help either. I’m stuck and this is preventing me from going forward with an important project.

↧

developing a Database maintenance plan

March 6, 2014, 11:36 am

≫ Next: MDW UCP Upload to Utility Control Point Job Fails On Step 3 With The Following Error:

≪ Previous: enabling 3rd LAN interfaces breaks SQL

Can you provide me links which goes into detail about developing a Database maintenance plan[i.e. taking backups [either Full or Inc] of DBs and transaction logs, deleting the oldest cycles, etcc]. I am not a DBA but at same time I am trying to learn as much as I can.

↧

MDW UCP Upload to Utility Control Point Job Fails On Step 3 With The Following Error:

October 10, 2013, 10:15 am

≫ Next: Table Partitioning Bug? – SWITCH OUT / MERGE RANGE

≪ Previous: developing a Database maintenance plan

I have setup an MDW UCP with 5 enrolled instances. Three of two instances are working as they should and are reporting to the UCP MDW dashboard. On the other two instances, the job sysutility_mi_collect_and_upload fails on step 3 with the following error:

Job Name  sysutility_mi_collect_and_upload
Step Name  Upload to Utility Control Point
Duration  00:00:30
Sql Severity  16
Sql Message ID  37007

Message
Executed as user: Domain\SQLSvc. Starting collection set. [SQLSTATE 01000] (Message 50000) Job 'collection_set_5_noncached_collect_and_upload' started successfully. [SQLSTATE 01000] (Message 14243) Waiting for the collection set to kick off jobs. [SQLSTATE 01000] (Message 50000) Waiting for collection set to finish its previous run. Total seconds spent waiting : 10 [SQLSTATE 01000] (Message 50000) Waiting for collection set to finish its previous run. Total seconds spent waiting : 20 [SQLSTATE 01000] (Message 50000) An error occurred during upload to the SQL Server utility control point. [SQLSTATE 42000] (Error 37007). The step failed.

Now when I go to look at the Job History for the job collection_set_5_noncached_collect_and_upload, it fails on step 1 with the following error:

Job Name  collection_set_5_noncached_collect_and_upload
Step Name  collection_set_5_noncached_collect_and_upload_collect
Duration  00:00:20
Sql Severity  0
Sql Message ID  0

Message
Executed as user: Domain\SQLSvc. SSIS error. Component name: GenerateTSQLPackageTask, Code: -1073548540, Subcomponent: Generate T-SQL Package Task, Description: An error occurred with the following error message: "Get DataType failed for Column 'physical_server_name'. : Index was outside the bounds of the array.". . The master package exited with error, previous error messages should explain the cause. Process Exit Code 5. The step failed.

Now unlike other posts where users are having connections (not enabling TCP/IP etc) or permissions issues (Service Account does not have permissions to upload data to the UCP MDW), this is different. I saw a KB articlehttp://technet.microsoft.com/en-us/library/ee210592.aspx under the "Failed Data Collection section Microsoft recommends:

"If data collection or data upload fail due to timeout issues, update the function dbo.fn_sysutility_mi_get_collect_script() in the MSDB database. Specifically, in the function "Invoke-BulkCopyCommand()" add line: $bulkCopy.BulkCopyTimeout=180"

You cannot modify system function. Is there anyone that knows what Microsoft is talking about when it comes to updating system functions? Besides this timeout doesn't seem to my problem, my issue lies in the failure of the collection_set_5_noncached_collect_and_upload with the error message shown. Any assistance is truly appreciated.

Thanks,

SQL4Life

↧

Table Partitioning Bug? – SWITCH OUT / MERGE RANGE

March 7, 2014, 3:01 am

≫ Next: Sliding Window Table Partitioning Problems with RANGE RIGHT, SPLIT, MERGE using Multiple File Groups

≪ Previous: MDW UCP Upload to Utility Control Point Job Fails On Step 3 With The Following Error:

There is misleading information in two system views (sys.data_spaces & sys.destination_data_spaces) about the physical location of data after a partitioning MERGE and before an INDEX REBUILD operation on a partitioned table. In SQL Server 2012 SP1 CU6, the script below (SQLCMD mode, set DataDrive & LogDrive variables for the runtime environment) will create a test database with file groups and files to support a partitioned table. The partition function and scheme spread the test data across 4 files groups, an empty partition, file group and file are maintained at the start and end of the range. A problem occurs after the SWITCH and MERGE RANGE operations, the views sys.data_spaces & sys.destination_data_spaces show the logical, not the physical, location of data.

--=================================================================================      
-- PartitionLabSetup_RangeRight.sql
-- 001. Create test database  
-- 002. Add file groups and files  
-- 003. Create partition function and schema  
-- 004. Create and populate a test table  
--=================================================================================      
USE [master]    
GO   
-----------------------------------------------------------------------------------  
-- 001 - Create Test Database   
-----------------------------------------------------------------------------------  
:SETVAR DataDrive "D:\SQL\Data\"    
:SETVAR LogDrive "D:\SQL\Logs\"    
:SETVAR DatabaseName "workspace"    
:SETVAR TableName "TestTable"   
-- Drop if exists and create Database    
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL    
BEGIN    
    ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE    
    DROP DATABASE $(DatabaseName)    
END    
CREATE DATABASE $(DatabaseName)    
ON    
    ( NAME = $(DatabaseName)_data,    
    FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',    
    SIZE = 10,    
    MAXSIZE = 500,    
    FILEGROWTH = 5 )    
    LOG ON    
    ( NAME = $(DatabaseName)_log,    
    FILENAME = N'$(LogDrive)$(DatabaseName).ldf',    
    SIZE = 5MB,    
    MAXSIZE = 5000MB,    
    FILEGROWTH = 5MB ) ;    
GO    
-----------------------------------------------------------------------------------  
-- 002. Add file groups and files   
-----------------------------------------------------------------------------------  
--:SETVAR DatabaseName "workspace"    
--:SETVAR TableName "TestTable"    
--:SETVAR DataDrive "D:\SQL\Data\"    
--:SETVAR LogDrive "D:\SQL\Logs\"    
DECLARE @nSQL NVARCHAR(2000) ;    
DECLARE @x INT = 1;     
WHILE @x <= 6    
BEGIN    
SELECT @nSQL =    
'ALTER DATABASE $(DatabaseName)    
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';    
ALTER DATABASE $(DatabaseName)    
ADD FILE    
(    
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',    
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''    
)    
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'    
EXEC sp_executeSQL @nSQL;    
SET @x = @x + 1;    
END    
-----------------------------------------------------------------------------------  
-- 003. Create partition function and schema  
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"        
--:SETVAR DatabaseName "workspace"    
USE $(DatabaseName);    
CREATE PARTITION FUNCTION $(TableName)_func (int)    
AS RANGE RIGHT FOR VALUES    
(    
    0,    
    15,    
    30,    
    45,    
    60    
);    
CREATE PARTITION SCHEME $(TableName)_scheme    
AS    
PARTITION $(TableName)_func    
TO    
(    
    $(TableName)_fg1,    
    $(TableName)_fg2,    
    $(TableName)_fg3,    
    $(TableName)_fg4,    
    $(TableName)_fg5,    
    $(TableName)_fg6    
);    
-----------------------------------------------------------------------------------  
-- Create TestTable    
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"    
--:SETVAR BackupDrive "D:\SQL\Backups\"    
--:SETVAR DatabaseName "workspace"    
CREATE TABLE [dbo].$(TableName)(    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC    
    ) ON $(TableName)_scheme(Partition_PK)    
) ON $(TableName)_scheme(Partition_PK)    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]    
-----------------------------------------------------------------------------------  
-- 004. Create and populate a test table      
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key    
--:SETVAR TableName "TestTable"    
SET NOCOUNT ON;   
DECLARE @Now DATETIME = GETDATE()    
WHILE @Now > DATEADD(minute,-1,GETDATE())    
BEGIN  
    INSERT INTO [dbo].$(TableName)     
    ([Partition_PK]    
    ,[RandomNbr])    
    VALUES  
    (    
     DATEPART(second,GETDATE())    
    ,ROUND((RAND() * 100),0)    
    )    
END  
-----------------------------------------------------------------------------------  
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber
--=================================================================================      
-- SECTION 2 - SWITCH OUT  
-- 001 - Create TestTableOut   
-- 002 - Switch out partition in range 0-14   
-- 003 - Merge range 0 -29   
-----------------------------------------------------------------------------------  
-- 001. TestTableOut   
:SETVAR TableName "TestTable"  
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL 
DROP TABLE [dbo].[$(TableName)Out]    
CREATE TABLE [dbo].[$(TableName)Out](    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC 
    )    
) ON $(TableName)_fg2;    
GO    
-----------------------------------------------------------------------------------  
-- 002 - Switch out partition in range 0-14   
--:SETVAR TableName "TestTable"   
ALTER TABLE dbo.$(TableName)    
SWITCH PARTITION 2 TO dbo.$(TableName)Out;   
-----------------------------------------------------------------------------------  
-- 003 - Merge range 0 - 29   
--:SETVAR TableName "TestTable"   
ALTER PARTITION FUNCTION $(TableName)_func()    
MERGE RANGE (15);    
-----------------------------------------------------------------------------------  
-- Confirm table partitioning  
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber

The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.

The T-SQL code below illustrates the problem.

-----------------------------------------------------------------------------------  
-- PartitionLab_RangeRight
USE workspace; 
DROP TABLE dbo.TestTableOut; 
USE master; 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f3    ; 
-- ERROR 
--Msg 5042, Level 16, State 1, Line 1 
--The file 'TestTable_f3    ' cannot be removed because it is not empty. 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f2    ; 
-- Works surprisingly!! 
use workspace; 
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2; 
--Msg 622, Level 16, State 3, Line 2
--The filegroup "TestTable_fg2" has no files assigned to it. Tables, indexes, text columns, ntext columns, and image columns cannot be populated on this filegroup until a file is added.
--The statement has been terminated.
-----------------------------------------------------------------------------------  
-- RANGE RIGHT
-- Rerun PartitionLabSetup_RangeRight.sql before the code below
USE workspace; 
DROP TABLE dbo.TestTableOut; 
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2; 
USE master; 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f3; 
-- Works as expected!!File Group 2 appears to contain data but it can be dropped. Although the system views are reporting the data in File Group 2, it still physically resides in File Group 3 and isn’t moved until the index is rebuilt. The RANGE RIGHT function means the left file group (File Group 2) is retained when splitting ranges. RANGE LEFT would have retained the data in File Group 3 where it already resided, no INDEX REBUILD is necessary to effectively complete the MERGE operation

File Group 2 appears to contain data but it can be dropped. Although the system views are reporting the data in File Group 2, it still physically resides in File Group 3 and isn’t moved until the index is rebuilt. The RANGE RIGHT function means the left file group (File Group 2) is retained when splitting ranges. RANGE LEFT would have retained the data in File Group 3 where it already resided, no INDEX REBUILD is necessary to effectively complete the MERGE operation

The script below implements the same partitioning strategy (data distribution between partitions) on the test table but uses different boundary definitions and RANGE LEFT.

--=================================================================================      
-- PartitionLabSetup_RangeLeft.sql
-- 001. Create test database  
-- 002. Add file groups and files  
-- 003. Create partition function and schema  
-- 004. Create and populate a test table  
--=================================================================================      
USE [master]    
GO   
-----------------------------------------------------------------------------------  
-- 001 - Create Test Database   
-----------------------------------------------------------------------------------  
:SETVAR DataDrive "D:\SQL\Data\"    
:SETVAR LogDrive "D:\SQL\Logs\"    
:SETVAR DatabaseName "workspace"    
:SETVAR TableName "TestTable"   
-- Drop if exists and create Database    
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL    
BEGIN    
    ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE    
    DROP DATABASE $(DatabaseName)    
END    
CREATE DATABASE $(DatabaseName)    
ON    
    ( NAME = $(DatabaseName)_data,    
    FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',    
    SIZE = 10,    
    MAXSIZE = 500,    
    FILEGROWTH = 5 )    
    LOG ON    
    ( NAME = $(DatabaseName)_log,    
    FILENAME = N'$(LogDrive)$(DatabaseName).ldf',    
    SIZE = 5MB,    
    MAXSIZE = 5000MB,    
    FILEGROWTH = 5MB ) ;    
GO    
-----------------------------------------------------------------------------------  
-- 002. Add file groups and files   
-----------------------------------------------------------------------------------  
--:SETVAR DatabaseName "workspace"    
--:SETVAR TableName "TestTable"    
--:SETVAR DataDrive "D:\SQL\Data\"    
--:SETVAR LogDrive "D:\SQL\Logs\"    
DECLARE @nSQL NVARCHAR(2000) ;    
DECLARE @x INT = 1;     
WHILE @x <= 6    
BEGIN    
SELECT @nSQL =    
'ALTER DATABASE $(DatabaseName)    
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';    
ALTER DATABASE $(DatabaseName)    
ADD FILE    
(    
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',    
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''    
)    
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'    
EXEC sp_executeSQL @nSQL;    
SET @x = @x + 1;    
END    
-----------------------------------------------------------------------------------  
-- 003. Create partition function and schema  
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"        
--:SETVAR DatabaseName "workspace"    
USE $(DatabaseName);    
CREATE PARTITION FUNCTION $(TableName)_func (int)    
AS RANGE LEFT FOR VALUES    
(    
    -1,    
    14,    
    29,    
    44,    
    59    
);    
CREATE PARTITION SCHEME $(TableName)_scheme    
AS    
PARTITION $(TableName)_func    
TO    
(    
    $(TableName)_fg1,    
    $(TableName)_fg2,    
    $(TableName)_fg3,    
    $(TableName)_fg4,    
    $(TableName)_fg5,    
    $(TableName)_fg6    
);    
-----------------------------------------------------------------------------------  
-- Create TestTable    
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"    
--:SETVAR BackupDrive "D:\SQL\Backups\"    
--:SETVAR DatabaseName "workspace"    
CREATE TABLE [dbo].$(TableName)(    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC    
    ) ON $(TableName)_scheme(Partition_PK)    
) ON $(TableName)_scheme(Partition_PK)    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]    
-----------------------------------------------------------------------------------  
-- 004. Create and populate a test table      
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key    
--:SETVAR TableName "TestTable"    
SET NOCOUNT ON;   
DECLARE @Now DATETIME = GETDATE()    
WHILE @Now > DATEADD(minute,-1,GETDATE())    
BEGIN  
    INSERT INTO [dbo].$(TableName)     
    ([Partition_PK]    
    ,[RandomNbr])    
    VALUES  
    (    
     DATEPART(second,GETDATE())    
    ,ROUND((RAND() * 100),0)    
    )    
END  
-----------------------------------------------------------------------------------  
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber
--=================================================================================      
-- SECTION 2 - SWITCH OUT  
-- 001 - Create TestTableOut   
-- 002 - Switch out partition in range 0-14   
-- 003 - Merge range 0 -29   
-----------------------------------------------------------------------------------  
-- 001. TestTableOut   
:SETVAR TableName "TestTable"  
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL 
DROP TABLE [dbo].[$(TableName)Out]    
CREATE TABLE [dbo].[$(TableName)Out](    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC 
    )    
) ON $(TableName)_fg2;    
GO    
-----------------------------------------------------------------------------------  
-- 002 - Switch out partition in range 0-14   
--:SETVAR TableName "TestTable"   
ALTER TABLE dbo.$(TableName)    
SWITCH PARTITION 2 TO dbo.$(TableName)Out;   
-----------------------------------------------------------------------------------  
-- 003 - Merge range 0 - 29   
:SETVAR TableName "TestTable"   
ALTER PARTITION FUNCTION $(TableName)_func()    
MERGE RANGE (14);    
-----------------------------------------------------------------------------------  
-- Confirm table partitioning  
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber

The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.

File Group 2 is not reporting any data and can be removed, File Group 3 is protected and files cannot be removed, no index rebuild required to complete.

The data in the File and File Group to be dropped (File Group 2) has already been switched out; File Group 3 contains the data so no index rebuild is needed to complete. This would not be a problem if the same file group is used for all partitions but when they are created and dropped dynamically it introduces a dependency on full index rebuilds for a Sliding Window strategy. Larger tables are typically partitioned and a full index rebuild might be an expensive operation. I’m not sure how a RANGE RIGHT partitioning strategy could be implemented, while creating and dropping file groups dynamically, without introducing this dependency on ALTER INDEX REBUILD.

RANGE RIGHT in partition functions for a ‘Sliding Window’ strategy need to use one file group for all partitions within a table. Merging a range to drop historic data does not then involve physically moving data between file groups; no index rebuild is necessary to complete a MERGE and system views accurately reflect the physical location of data.

If a RANGE RIGHT partition function is used, the data is physically in the wrong file group, assuming a typical ascending partitioning key, after the MERGE. There are lots of examples how to implement a sliding window partitioning strategy available on the internet. Many include RANGE RIGHT, most use a single file group for all partitions, the examples that use multiple file groups don’t explain the re-indexing requirement.

Thanks to Manuj and Chris for help investigating this, the conclusion is my own uncorroborated opinion which might be wrong!?.

↧

Sliding Window Table Partitioning Problems with RANGE RIGHT, SPLIT, MERGE using Multiple File Groups

March 7, 2014, 3:01 am

≫ Next: SQL server agent send email

≪ Previous: Table Partitioning Bug? – SWITCH OUT / MERGE RANGE

--=================================================================================      
-- PartitionLabSetup_RangeRight.sql
-- 001. Create test database  
-- 002. Add file groups and files  
-- 003. Create partition function and schema  
-- 004. Create and populate a test table  
--=================================================================================      
USE [master]    
GO   
-----------------------------------------------------------------------------------  
-- 001 - Create Test Database   
-----------------------------------------------------------------------------------  
:SETVAR DataDrive "D:\SQL\Data\"    
:SETVAR LogDrive "D:\SQL\Logs\"    
:SETVAR DatabaseName "workspace"    
:SETVAR TableName "TestTable"   
-- Drop if exists and create Database    
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL    
BEGIN    
    ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE    
    DROP DATABASE $(DatabaseName)    
END    
CREATE DATABASE $(DatabaseName)    
ON    
    ( NAME = $(DatabaseName)_data,    
    FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',    
    SIZE = 10,    
    MAXSIZE = 500,    
    FILEGROWTH = 5 )    
    LOG ON    
    ( NAME = $(DatabaseName)_log,    
    FILENAME = N'$(LogDrive)$(DatabaseName).ldf',    
    SIZE = 5MB,    
    MAXSIZE = 5000MB,    
    FILEGROWTH = 5MB ) ;    
GO    
-----------------------------------------------------------------------------------  
-- 002. Add file groups and files   
-----------------------------------------------------------------------------------  
--:SETVAR DatabaseName "workspace"    
--:SETVAR TableName "TestTable"    
--:SETVAR DataDrive "D:\SQL\Data\"    
--:SETVAR LogDrive "D:\SQL\Logs\"    
DECLARE @nSQL NVARCHAR(2000) ;    
DECLARE @x INT = 1;     
WHILE @x <= 6    
BEGIN    
SELECT @nSQL =    
'ALTER DATABASE $(DatabaseName)    
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';    
ALTER DATABASE $(DatabaseName)    
ADD FILE    
(    
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',    
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''    
)    
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'    
EXEC sp_executeSQL @nSQL;    
SET @x = @x + 1;    
END    
-----------------------------------------------------------------------------------  
-- 003. Create partition function and schema  
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"        
--:SETVAR DatabaseName "workspace"    
USE $(DatabaseName);    
CREATE PARTITION FUNCTION $(TableName)_func (int)    
AS RANGE RIGHT FOR VALUES    
(    
    0,    
    15,    
    30,    
    45,    
    60    
);    
CREATE PARTITION SCHEME $(TableName)_scheme    
AS    
PARTITION $(TableName)_func    
TO    
(    
    $(TableName)_fg1,    
    $(TableName)_fg2,    
    $(TableName)_fg3,    
    $(TableName)_fg4,    
    $(TableName)_fg5,    
    $(TableName)_fg6    
);    
-----------------------------------------------------------------------------------  
-- Create TestTable    
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"    
--:SETVAR BackupDrive "D:\SQL\Backups\"    
--:SETVAR DatabaseName "workspace"    
CREATE TABLE [dbo].$(TableName)(    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC    
    ) ON $(TableName)_scheme(Partition_PK)    
) ON $(TableName)_scheme(Partition_PK)    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]    
-----------------------------------------------------------------------------------  
-- 004. Create and populate a test table      
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key    
--:SETVAR TableName "TestTable"    
SET NOCOUNT ON;   
DECLARE @Now DATETIME = GETDATE()    
WHILE @Now > DATEADD(minute,-1,GETDATE())    
BEGIN  
    INSERT INTO [dbo].$(TableName)     
    ([Partition_PK]    
    ,[RandomNbr])    
    VALUES  
    (    
     DATEPART(second,GETDATE())    
    ,ROUND((RAND() * 100),0)    
    )    
END  
-----------------------------------------------------------------------------------  
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber
--=================================================================================      
-- SECTION 2 - SWITCH OUT  
-- 001 - Create TestTableOut   
-- 002 - Switch out partition in range 0-14   
-- 003 - Merge range 0 -29   
-----------------------------------------------------------------------------------  
-- 001. TestTableOut   
:SETVAR TableName "TestTable"  
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL 
DROP TABLE [dbo].[$(TableName)Out]    
CREATE TABLE [dbo].[$(TableName)Out](    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC 
    )    
) ON $(TableName)_fg2;    
GO    
-----------------------------------------------------------------------------------  
-- 002 - Switch out partition in range 0-14   
--:SETVAR TableName "TestTable"   
ALTER TABLE dbo.$(TableName)    
SWITCH PARTITION 2 TO dbo.$(TableName)Out;   
-----------------------------------------------------------------------------------  
-- 003 - Merge range 0 - 29   
--:SETVAR TableName "TestTable"   
ALTER PARTITION FUNCTION $(TableName)_func()    
MERGE RANGE (15);    
-----------------------------------------------------------------------------------  
-- Confirm table partitioning  
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber

The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.

The T-SQL code below illustrates the problem.

-----------------------------------------------------------------------------------  
-- PartitionLab_RangeRight
USE workspace; 
DROP TABLE dbo.TestTableOut; 
USE master; 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f3    ; 
-- ERROR 
--Msg 5042, Level 16, State 1, Line 1 
--The file 'TestTable_f3    ' cannot be removed because it is not empty. 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f2    ; 
-- Works surprisingly!! 
use workspace; 
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2; 
--Msg 622, Level 16, State 3, Line 2
--The filegroup "TestTable_fg2" has no files assigned to it. Tables, indexes, text columns, ntext columns, and image columns cannot be populated on this filegroup until a file is added.
--The statement has been terminated.

If you run ALTER INDEX REBUILD before trying to remove files from File Group 3, it works. Rerun the database setup script then the code below.

-----------------------------------------------------------------------------------  
-- RANGE RIGHT
-- Rerun PartitionLabSetup_RangeRight.sql before the code below
USE workspace; 
DROP TABLE dbo.TestTableOut; 
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2; 
USE master; 
ALTER DATABASE workspace 
REMOVE FILE TestTable_f3; 
-- Works as expected!!

The file in File Group 2 appears to contain data but it can be dropped. Although the system views are reporting the data in File Group 2, it still physically resides in File Group 3 and isn’t moved until the index is rebuilt. The RANGE RIGHT function means the left file group (File Group 2) is retained when splitting ranges.

RANGE LEFT would have retained the data in File Group 3 where it already resided, no INDEX REBUILD is necessary to effectively complete the MERGE operation. The script below implements the same partitioning strategy (data distribution between partitions) on the test table but uses different boundary definitions and RANGE LEFT.

--=================================================================================      
-- PartitionLabSetup_RangeLeft.sql
-- 001. Create test database  
-- 002. Add file groups and files  
-- 003. Create partition function and schema  
-- 004. Create and populate a test table  
--=================================================================================      
USE [master]    
GO   
-----------------------------------------------------------------------------------  
-- 001 - Create Test Database   
-----------------------------------------------------------------------------------  
:SETVAR DataDrive "D:\SQL\Data\"    
:SETVAR LogDrive "D:\SQL\Logs\"    
:SETVAR DatabaseName "workspace"    
:SETVAR TableName "TestTable"   
-- Drop if exists and create Database    
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL    
BEGIN    
    ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE    
    DROP DATABASE $(DatabaseName)    
END    
CREATE DATABASE $(DatabaseName)    
ON    
    ( NAME = $(DatabaseName)_data,    
    FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',    
    SIZE = 10,    
    MAXSIZE = 500,    
    FILEGROWTH = 5 )    
    LOG ON    
    ( NAME = $(DatabaseName)_log,    
    FILENAME = N'$(LogDrive)$(DatabaseName).ldf',    
    SIZE = 5MB,    
    MAXSIZE = 5000MB,    
    FILEGROWTH = 5MB ) ;    
GO    
-----------------------------------------------------------------------------------  
-- 002. Add file groups and files   
-----------------------------------------------------------------------------------  
--:SETVAR DatabaseName "workspace"    
--:SETVAR TableName "TestTable"    
--:SETVAR DataDrive "D:\SQL\Data\"    
--:SETVAR LogDrive "D:\SQL\Logs\"    
DECLARE @nSQL NVARCHAR(2000) ;    
DECLARE @x INT = 1;     
WHILE @x <= 6    
BEGIN    
SELECT @nSQL =    
'ALTER DATABASE $(DatabaseName)    
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';    
ALTER DATABASE $(DatabaseName)    
ADD FILE    
(    
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',    
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''    
)    
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'    
EXEC sp_executeSQL @nSQL;    
SET @x = @x + 1;    
END    
-----------------------------------------------------------------------------------  
-- 003. Create partition function and schema  
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"        
--:SETVAR DatabaseName "workspace"    
USE $(DatabaseName);    
CREATE PARTITION FUNCTION $(TableName)_func (int)    
AS RANGE LEFT FOR VALUES    
(    
    -1,    
    14,    
    29,    
    44,    
    59    
);    
CREATE PARTITION SCHEME $(TableName)_scheme    
AS    
PARTITION $(TableName)_func    
TO    
(    
    $(TableName)_fg1,    
    $(TableName)_fg2,    
    $(TableName)_fg3,    
    $(TableName)_fg4,    
    $(TableName)_fg5,    
    $(TableName)_fg6    
);    
-----------------------------------------------------------------------------------  
-- Create TestTable    
-----------------------------------------------------------------------------------  
--:SETVAR TableName "TestTable"    
--:SETVAR BackupDrive "D:\SQL\Backups\"    
--:SETVAR DatabaseName "workspace"    
CREATE TABLE [dbo].$(TableName)(    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC    
    ) ON $(TableName)_scheme(Partition_PK)    
) ON $(TableName)_scheme(Partition_PK)    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]    
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]    
-----------------------------------------------------------------------------------  
-- 004. Create and populate a test table      
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key    
--:SETVAR TableName "TestTable"    
SET NOCOUNT ON;   
DECLARE @Now DATETIME = GETDATE()    
WHILE @Now > DATEADD(minute,-1,GETDATE())    
BEGIN  
    INSERT INTO [dbo].$(TableName)     
    ([Partition_PK]    
    ,[RandomNbr])    
    VALUES  
    (    
     DATEPART(second,GETDATE())    
    ,ROUND((RAND() * 100),0)    
    )    
END  
-----------------------------------------------------------------------------------  
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber
--=================================================================================      
-- SECTION 2 - SWITCH OUT  
-- 001 - Create TestTableOut   
-- 002 - Switch out partition in range 0-14   
-- 003 - Merge range 0 -29   
-----------------------------------------------------------------------------------  
-- 001. TestTableOut   
:SETVAR TableName "TestTable"  
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL 
DROP TABLE [dbo].[$(TableName)Out]    
CREATE TABLE [dbo].[$(TableName)Out](    
    [Partition_PK] [int] NOT NULL,    
    [GUID_PK] [uniqueidentifier] NOT NULL,    
    [CreateDate] [datetime] NULL,    
    [CreateServer] [nvarchar](50) NULL,    
    [RandomNbr] [int] NULL,    
    CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED    
    (    
        [Partition_PK] ASC,    
        [GUID_PK] ASC 
    )    
) ON $(TableName)_fg2;    
GO    
-----------------------------------------------------------------------------------  
-- 002 - Switch out partition in range 0-14   
--:SETVAR TableName "TestTable"   
ALTER TABLE dbo.$(TableName)    
SWITCH PARTITION 2 TO dbo.$(TableName)Out;   
-----------------------------------------------------------------------------------  
-- 003 - Merge range 0 - 29   
:SETVAR TableName "TestTable"   
ALTER PARTITION FUNCTION $(TableName)_func()    
MERGE RANGE (14);    
-----------------------------------------------------------------------------------  
-- Confirm table partitioning  
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/  
SELECT
    N'DatabaseName' = DB_NAME()  
    , N'SchemaName' = s.name
    , N'TableName' = o.name
    , N'IndexName' = i.name
    , N'IndexType' = i.type_desc  
    , N'PartitionScheme' = ps.name
    , N'DataSpaceName' = ds.name
    , N'DataSpaceType' = ds.type_desc  
    , N'PartitionFunction' = pf.name
    , N'PartitionNumber' = dds.destination_id  
    , N'BoundaryValue' = prv.value  
    , N'RightBoundary' = pf.boundary_value_on_right  
    , N'PartitionFileGroup' = ds2.name
    , N'RowsOfData' = p.[rows]  
FROM
    sys.objects AS o  
    INNER JOIN sys.schemas AS s  
        ON o.[schema_id] = s.[schema_id]  
    INNER JOIN sys.partitions AS p  
        ON o.[object_id] = p.[object_id]  
    INNER JOIN sys.indexes AS i  
        ON p.[object_id] = i.[object_id]  
        AND p.index_id = i.index_id  
    INNER JOIN sys.data_spaces AS ds  
        ON i.data_space_id = ds.data_space_id  
    INNER JOIN sys.partition_schemes AS ps  
        ON ds.data_space_id = ps.data_space_id  
    INNER JOIN sys.partition_functions AS pf  
        ON ps.function_id = pf.function_id  
    LEFT OUTER JOIN sys.partition_range_values AS prv  
        ON pf.function_id = prv.function_id  
        AND p.partition_number = prv.boundary_id  
    LEFT OUTER JOIN sys.destination_data_spaces AS dds  
        ON ps.data_space_id = dds.partition_scheme_id  
        AND p.partition_number = dds.destination_id  
    LEFT OUTER JOIN sys.data_spaces AS ds2  
        ON dds.data_space_id = ds2.data_space_id  
ORDER BY
     DatabaseName  
    ,SchemaName  
    ,TableName  
    ,IndexName  
    ,PartitionNumber

The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.

The data in the File and File Group to be dropped (File Group 2) has already been switched out; File Group 3 contains the data so no index rebuild is needed to move data and complete the MERGE.

RANGE RIGHT would not be a problem in a ‘Sliding Window’ if the same file group is used for all partitions, when they are created and dropped it introduces a dependency on full index rebuilds. Larger tables are typically partitioned and a full index rebuild might be an expensive operation. I’m not sure how a RANGE RIGHT partitioning strategy could be implemented, with an ascending partitioning key, using multiple file groups without having to move data. Using a single file group (multiple files) for all partitions within a table would avoid physically moving data between file groups; no index rebuild would be necessary to complete a MERGE and system views would accurately reflect the physical location of data.

If a RANGE RIGHT partition function is used, the data is physically in the wrong file group after the MERGE assuming a typical ascending partitioning key, and the 'Data Spaces' system views might be misleading. Thanks to Manuj and Chris for a lot of help investigating this.

NOTE 10/03/2014 - The solution

The solution is so easy it's embarrassing, I was using the wrong boundary points for the MERGE (both RANGE LEFT & RANGE RIGHT) to get rid of historic data.

-- Wrong Boundary Point Range Right
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (15);

-- Wrong Boundary Point Range Left
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (14);

-- Correct Boundary Pounts for MERGE
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (0); -- or -1 for RANGE LEFT

The empty, switched out partition (on File Group 2) is then MERGED with the empty partition maintained at the start of the range and no data movement is necessary. I retract the suggestion that a problem exists with RANGE RIGHT Sliding Windows using multiple file groups and apologize :-)

↧

SQL server agent send email

January 6, 2012, 12:13 pm

≫ Next: Performance related Query in sql server

≪ Previous: Sliding Window Table Partitioning Problems with RANGE RIGHT, SPLIT, MERGE using Multiple File Groups

I set up email notification for a job completes. I configured database mail profile and mail account. This works fine and I can send out test email.

However, the SQL agent couldn't send out email and when I tried to restart the SQL server agent, I got the following errors:

Message
[260] Unable to start mail session (reason: Microsoft.SqlServer.Management.SqlIMail.Server.Common.BaseException: Mail configuration information could not be read from the database. ---> System.InvalidCastException: Unable to cast object of type 'System.DBNull' to type 'System.Byte[]'.
   at Microsoft.SqlServer.Management.SqlIMail.Server.DataAccess.DataAccessAdapter.GetAccount(Int32 accountID)
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.SqlIMail.Server.DataAccess.DataAccessAdapter.GetAccount(Int32 accountID)
)

My SQL server is SQL 2008 with service pack 3.

Anybody has any idea what's the problem is?

Thanks

↧

Performance related Query in sql server

March 13, 2014, 7:42 am

≫ Next: Restore one filegroup from DB to another DB

≪ Previous: SQL server agent send email

Hi,

I am having an Performance issue in my sql server Box.

The problem is , when i capture the details after having perf mon counters checked and it has given me some values, where the values are shown below along with the date

Counter	Mon	Tue	Wed	Thurs	Fri	sat	Sun
Page life expectancy	30.36	22.52	23.28	19.6	16.98	28.05	101.9
User Connections	1269.81	1215.02	1112.39	1022.17	1027.37	509.01	447.64
Batch Requests/sec	840.19	382.64	353.09	317.29	337.63	120.11	140.6

Can someone give me your suggestion , that if the user connections are high along with the batch requests per second as shown about, what are the points I should do, do I need to increase the memory or No of CPU’s ? Anything as per your experience you can suggest me to increase the performance of my server

hemadri

↧

Restore one filegroup from DB to another DB

March 14, 2014, 4:11 am

≫ Next: Alias a SQL server?

≪ Previous: Performance related Query in sql server

Hi,
I have two databases like each other that one is the backup of

another. Each DB have 2 filegroups. I want to replace one

filegroup from one db to another. How do I do this? Or how do I

backup and then restore?

↧

Alias a SQL server?

March 14, 2014, 6:04 am

≫ Next: Performance monitoring Metrics required to idntify meamory , IO issues

≪ Previous: Restore one filegroup from DB to another DB

Hi,

I have a situation I hope the smart people
on here can help me with.

We have a local SQL server that is moving to
the SAAS (online) environment. In this new environment, the owners of this SQL
DB will rename the server, the SQL name and the SQL instance name to their
naming standard.

The problem is we have about 20 custom programs that
point to our existing SQL server (with existing name). Not to mention dozens of
SSIS packages, procs, views, etc. When we move our server to the SAAS
environment they will be renaming the server, instance, etc. so- none of our
programs will connect.

Is there a way we can use alias names for:
1)
The Server (host file? - dns redirect?)
2) The SQL server name
3) The SQL
instance name

Or is there some other way that we don't have to re-write
all of our programs/packages?

qeqw

↧

Performance monitoring Metrics required to idntify meamory , IO issues

March 17, 2014, 4:17 am

≫ Next: Install MS SQL Server Web Edition on a server with MS SQL Server Express

≪ Previous: Alias a SQL server?

Hello All,

Can some one , come up with your ideas to identify the metrics required to capture the meamory or I/O issues in sql server .

i would like to know the exact metrics to identify when my clients says meamory or i/o performance issues

thank you

hemadribabu

hemadri

↧

Install MS SQL Server Web Edition on a server with MS SQL Server Express

March 18, 2014, 1:33 am

≫ Next: Rogue backups occurring

≪ Previous: Performance monitoring Metrics required to idntify meamory , IO issues

Hi,

We have a VPS cloud server on which there is an instance of SQL Server Express.

Since we need more resources, I consulted with the hosting provider and they offered us to use SQL Server Web Edition (SPLA Lisence).

My question is - the hosting company claims that it is not possible to keep the SQL Server Express instance along side with the SQL Server Web Edition, and that we must do ugrade and remove the SQL Server Express instance.

This will cause us a long downtime, since there are multiple DBs on the SQL Server Express that we will have to backup, and then restore them one by one on the Web Edition after it is installed.

Is this correct? Can't we install the SQL Server Web Edition Instance and gradually move the DBs to it?

Thanks!

↧

Rogue backups occurring

March 19, 2014, 9:22 am

≫ Next: Cannot view data from ndf file

≪ Previous: Install MS SQL Server Web Edition on a server with MS SQL Server Express

Hello --

We have a SQL Server 2008R2 database running on a Windows 2012 x86_64 server. The databases being backed up use a Full Recovery model, and are subject of Full, occurring on Friday, Differential, occurring Saturday through Thursday, and Transaction log, hourly, backups. Recently the Differential backups that run on Monday were failing due to the following error:

Cannot
 perform a differential backup for database "<database name>", 
because a current database backup does not exist. Perform a full 
database backup by reissuing BACKUP DATABASE, omitting the WITH 
DIFFERENTIAL option.  BACKUP DATABASE is terminating abnormally.". 
Possible failure reasons: Problems with the query, "ResultSet" property 
not set correctly, parameters not set correctly, or connection not 
established correctly.  End Error  Error: 2014-03-10 22:00:02.38     
Code: 0xC002F210     Source: Back Up Database (Differential) Execute SQL
 Task

A one-time full backup job was subsequently run, and that solved the problem until the following Monday. A check of the ERRORLOG file had entries like the following:

I/O is frozen on database     I/O was resumed on database Backup      Database backed up

Further investigating seem to indicate that NT backups (VSS backups) are running on the server that was backing up all databases. Those backups are being run by NT AUTHORITY\SYSTEM.

However, a check of the Microsoft Software Shadow Copy Provider and Volume Shadow Copy services indicated that both are set to manual, and are not running on the system. Also, the Windows Server Backup feature is not installed on the server.

Where are the backups coming from, and how can they be stopped from running?

↧

Cannot view data from ndf file

March 20, 2014, 3:11 am

≫ Next: SQL 2012 A significant part of sql server process memory has been paged out. This may result in a performance degradation

≪ Previous: Rogue backups occurring

Hi,

We split-ed database from a single MDF file [125GB] into multile data files
Main.mdf
M1.ndf
M2.ndf

After split, database is working fine.

Now, everyday we take full back of database and restore it onto the other system for reporting. Since the split, the restore is not showing the latest data. We did the split on 18th and now we cannot view new data after 18th .

Any idea on what might be causing it ?

Shady

↧

SQL 2012 A significant part of sql server process memory has been paged out. This may result in a performance degradation

March 20, 2014, 5:16 am

≫ Next: Database offline (suspect)

≪ Previous: Cannot view data from ndf file

Thank you taking sometime to help !!

First of I have

Microsoft SQL Server 2012 (SP1) - 11.0.3128.0 (X64)
Dec 28 2012 20:23:12
(64-bit) on Windows NT 6.2 <X64> (Build 9200: ) (Hypervisor)

max server memory (MB) 128 2147483647 6400 6400
min server memory (MB) 0 2147483647 16 0

A significant part of sql server process memory has been paged out. This may result in a performance degradation. Duration: 1837 seconds. Working set (KB): 1062992, committed (KB): 2219688, memory utilization: 47%.

I am trying to understand this issue but not having much luck , this seems to happening at the most oddest of times when the environment is pretty much quiet. Is there something I need to do or set ?

Thanks

Robert

↧

Database offline (suspect)

March 21, 2014, 2:09 pm

≫ Next: big transaction log file

≪ Previous: SQL 2012 A significant part of sql server process memory has been paged out. This may result in a performance degradation

My Scom Operations Manager daabase went offline and shows (suspect)

what do i do to fix it?

↧

big transaction log file

January 30, 2014, 1:52 pm

≫ Next: SQL Server connection problem using Windows Authentication

≪ Previous: Database offline (suspect)

If one transaction log file is very big (full recovery model) because it never runs transaction log backup, now it eats all disk space.

How to fix this problem?

↧

SQL Server connection problem using Windows Authentication

March 27, 2014, 2:08 pm

≫ Next: usage of Temp tables in SSIS 2012

≪ Previous: big transaction log file

Hi Folks,

Problem - Unable to connect to SQL Server 2008 using Windows Authentication from Windows XP Client Systems. The SQL Server is configured in a NEC Cluster. The problem occurs only when I connect to the Virtual cluster Database Name/IP Address. But I am able to connect using hostname(SQL_Primary)/IP Address(192.168.10.101) using windows Authentication.

Observation - The virtual hostname is registered in DNS and it is resolving without any problem. But, when I was trying various options, I figured out by adding the following entry to 'c:\windows\system32\drivers\etc\hosts' file in my Windows XP client, it works fine.

192.168.10.203 "SQL_Virtual" SQL_Virtual.ABC.COM

"SQL_Virtual" is my Virtual Failover Database name and it is not the default database instance name.

Questions:

1. I couldn't figure out why it is working with the above entry in my host file?

2. How to add this in DNS server?

Thanks guys,

Lisa

↧

usage of Temp tables in SSIS 2012

March 28, 2014, 5:34 am

≫ Next: Cannot restore database to new database server

≪ Previous: SQL Server connection problem using Windows Authentication

Hello,

We have many SSIS packages (2008 R2) which imports data to a temp table and process it from there.

We are upgrading to SQL server 2012 and facing the issue with temp table as working table and our ssis packages fail in 2012. While investigating found that SQL Server 2012 deprecates FMTONLY option and instead uses sp_describe_first_result_set , which does not support using of temp tables as import table. SSIS works fine in our workstations but not in the DEV box. With SQL 2012, I can execute from my workstation, which has (11.0.2100.60) where as DEV server has SQL Server version 11.0.3000.0

Also when I ran profile with that of the DEV box, it gives two different statements

from workstation (11.0.2100.60)

 CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )

go
declare @p1 int
set @p1=NULL
declare @p3 int
set @p3=229378
declare @p4 int
set @p4=294916
declare @p5 int
set @p5=NULL
exec sp_cursoropen @p1 output,N'select * from #temp',@p3 output,@p4 output,@p5 output
select @p1, @p3, @p4, @p5
go

it works fine

But with the DEV server (version 11.0.3000.0), it executes the below sql and it fails to get the meta data

 CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )

exec [sys].sp_describe_first_result_set N'select * from [dbo].[#temp]'

On checking the assembly difference between the versions, I could only see Microsoft.SqlServer.ManagedDTS.dll being 11.0.3000.0, which I replace by 11.0.2100.60 version. but still getting the same result.

The other different I found is with ,Net framework libraries.

Could you advise whats the assembly causing this issue between our workstation and DEV server i.e 11.0.2100.60 and 11.0.3000.0

Many thanks

↧

Cannot restore database to new database server

March 29, 2014, 2:17 pm

≫ Next: MSSQLSERVER Alert

≪ Previous: usage of Temp tables in SSIS 2012

i backed up a database in sql 2008 and trying to restore that to a new db server sql 2012 but when i do the restore it says

Msg 5133, Level 16, State 1, Line 1
Directory lookup for the file "D:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\citrix4.mdf" failed with the operating system error 3(The system cannot find the path specified.).
Msg 3156, Level 16, State 3, Line 1
File 'citrix4' cannot be restored to 'D:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\citrix4.mdf'. Use WITH MOVE to identify a valid location for the file.
Msg 5133, Level 16, State 1, Line 1
Directory lookup for the file "D:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\citrix4_log.ldf" failed with the operating system error 3(The system cannot find the path specified.).
Msg 3156, Level 16, State 3, Line 1
File 'citrix4_log' cannot be restored to 'D:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\citrix4_log.ldf'. Use WITH MOVE to identify a valid location for the file.
Msg 3119, Level 16, State 1, Line 1
Problems were identified while planning for the RESTORE statement. Previous messages provide details.
Msg 3013, Level 16, State 1, Line 1
RESTORE DATABASE is terminating abnormally.

I am using this command

RESTORE DATABASE citrix4
FROM DISK = 'd:\citrix4backup.bak'
WITH REPLACE

any idea?

↧