Twitter

Follow SQLQuill on Twitter

Subscribe

MN Pass

TSQL Looping Stats

Every once in a while, you need to loop through a bunch of records in TSQL.  It’s not the fastest way to do things, but sometimes it’s necessary.  I’ve been working on a process that requires this and I needed to know the progress of the loop, so I came up with this model.  The output will give you a bunch of statistics including a predicted finishing time every 1000 rows (note you can adjust this).  This should save me from getting some grey hairs when a looping process is necessary.

Declare @counter int,
		@currentRow int,
		@rowsFound int,
		@start datetime,
		@SecSoFar bigint,
		@PredictedTotalSec int,
		@PercentComplete varchar(10),
		@CompleteTime datetime

Set @rowsFound = 200000 -- or set this equal to the count of a table you are working with.

Set @currentRow = 1
Set @start = GETDATE()

while @currentRow <=  @rowsFound
Begin
		--<<DO YOUR WORK HERE>>--

			Set @currentRow = @currentRow + 1  ---Update the counter....
			If (@currentRow % 1000) = 0  --display every 1000 rows or adjust...
			Begin
				Set @SecSoFar = Datediff(s, @start, Getdate())
					Set @PercentComplete = Cast(ROUND(@currentRow/Cast(@rowsFound as float), 3)*100 as varchar(20)) + '%'
					Set @PredictedTotalSec = (@SecSoFar/Cast(@currentRow as float))*@rowsFound
					Set @CompleteTime = Dateadd(s, @PredictedTotalSec, @start)

				Select
					@currentRow as 'Current_Row',
					@rowsFound as 'Total_Rows',
					@start as 'Start_Time',
					GETDATE() as 'Current_Time',
					@SecSoFar as 'SecondsSoFar',
					@PercentComplete as 'Percent_Complete',
					@PredictedTotalSec as 'Predicted_Total_Seconds',
					@CompleteTime as 'Predicted_Complete_Time'

			End
End

The Official SQLFAQ High Level Overview of Data Migration Testing

I’ve been tasked with the SQL portion of our data migration and conversion testing for our big release we’ve got coming up. What’s that mean?  Well, we’ve got an upgrade coming to the application we support.  We’ve got 15 databases on 7 servers.  All of them have database schema changes coming, data moving around from one table to another, new tables getting created, old ones getting dropped, new columns populated from other databases, etc.  I’ve got to make sure all the data that moves makes it there and is correct.  Here was how I attacked it:

The High Level Overview:

  1. Script production databases from our log shipping servers
  2. Create two copies of each database on a test server(s)
  3. Upgrade one database of each set
  4. Compare schema on Pre and Post upgraded databases
  5. List out all the diffences
  6. Catagorize how each new column or table is populated
  7. Ignore the new tables that are populated on use or populated with defaults – no data migration
  8. Create Checksum and row count scripts for all the migrated data
  9. Run the scripts and record the results on our Prod-Stage environment before the upgrade
  10. Upgrade Prod-Stage databases
  11. Run the scripts and compare the results to the Pre-upgrade.
  12. Address any issues

This approached seemed to work well, so my plan is to write a post on a bunch of these individual steps, so stay tuned.