Luke Hayler - SQL Server developer

Better late than never

I’ve been a little under the weather since getting back from SQLBits V so this post has been days in the making instead of just s a few hours. But here it is. This post focuses on the conference sessions, I am going to follow up with another post on people, conference feedback and things I might do differently next time.

Even the birds aren’t up this early…

Friday started swimmingly. Up at the crack o’ dawn,  taxi to a mainline station (my local didn’t have an early enough train to get me into London), only to find that the train has been delayed by 18 minutes. Pants. There goes all my hard work ‘being early’. The good news is that I made it on to the Newport Train (from Paddington). With just 2 minutes to spare. And I got a seat with a power outlet. Bonus! All my good intentions at getting a head start on an article have gone out the window as I am a little tired, so I try to catch up on some much needed shut eye.

Keynote by Donald Farmer

I arrived at the Celtic Manor, having shared a taxi, just in time for registration and a quick bacon bap before the welcome started. The keynote was given by Donald Farmer (Blog | Twitter). He gave us a great preview of the simply awesome PowerPivot and a few features in SQL Server 2008 R2. See this recap by Brent Ozar.

Below is a recap of the sessions that I found I got the most out of.

Friday Session 1 – New Development Features in SQL Server 2008 [Simon Sabin]

The first session of the day, that I attended, was Simon Sabin’s (Blog | Twitter) overview of the new development features that can be found in SQL Server 2008. Simon favours the demo approach for sessions and so had relatively few slides. Not a bad thing. Among other features available in SQL Server 2008, the following were discussed/demoed:

• Variable declaration and initialisation – this can now be done in the same line (Declare @i Int = 1)
• Named regions - You can create these using BEGIN & END blocks & naming it with a comment. Named regions will enable you to collapse the blocks for readability.
• Intellisense – as has been available to .NET programmers for years. My only query here is, why not use the syntax that can be found in C# when writing LINQ queries [where you specify the FROM TABLENAME line first and then the SELECT statement). As such, you find yourself writing the FROM line first and then going back up to the SELECT statement….
• Highlighted Syntax errors - think MS Word red squiggly lines to highlight that something is the matter.
• Code validation at design time -
• Date & Time datatypes - new Date datatype allows dates before 01/01/1753, and only takes up 4 bytes of storage. great for when you don’t actually need time precision. Time datatype handles precision down to the nano second, for when you do. The Time datatype also only refers to the time of day and not a time interval.
• No longer allowed implicit calculations on dates - you now must use Date functions to perform calculations on Date datatypes (i.e. DateAdd(), Not “MyDatetime + 1”).
• DateTime Offset - Allows you to store the date and time that has time zone awareness (based on the 24-hour clock)
• Debugging - You can now debug your code! breakpoints, watch lists, view locals, etc
• MERGE Statement - enables you to perform updates/deletes/inserts all within one statement.

Whilst this list is not conclusive, it certainly highlighted a number of new features that are now available in SQL Server 2008 and incentivises you to at least find out what else is available that could simplify you SQL development.

This was a great session, and if you find out that Simon is doing a presentation near you (UK User Groups), I suggest going.

Friday Session 2 - SQL Server 2008 R2: What’s new in Reporting Services? [Donald Farmer]

Donald Farmer (Twitter | Blog) blew my mind. Not once, but twice. First, in his keynote, he talked about the awesomeness of PowerPivot. Then in this session, he did it again. I loved this session and quickly realised that there’s going to be quite a learning curve and an altering of mindset of how reporting ‘gets done’ in the future. Why? Because the lines between the business & IT are blurring.

With the new Self Service Analysis and Reporting tools, there’s a new approach to finding out and tracking how the business is performing. So, if you are a Reporting Services Developer or Data/MI(S) Analyst. Things are going to change:

• PowerPivot & Excel 2010 bring a lot more functionality ( & power) to business users. Enabling them to analyse data at length straight from the data source.
• The arrival of report components in report builder, means that reports can be built quicker using existing parts of reports or report items. Think sub-reports, but on a more granular scale. Each report item can be a reusable component, which can be tweaked by the user for use in their own report.This includes the reuse of datasets, as components. These components are centrally managed by IT (or a reporting team).

So what did we learn? Here’s a bulleted summary:

• Business users analyse data through tools like PowerPivot & Excel 2010, digging for answers to business questions. This analysis is an iterative process and gives rise to metrics that can be monitored. Enter reporting services. Reports are built to monitor the ‘answers’ discovered by the business analysts.
• Reports can be created using components. Components are pre-built report objects that reside in a library that anyone can reuse. Think of sub-reports, but in a centrally managed way.
• Created reports that are saved to the Server are saved as a report in its entirety, as well as individual report components. These components are then available to other users to include/tweak in their own reports.
• When Publishing reports, you can choose which components of the reports you want to publish.
• Report Data can be subscribed to via feeds (rss anyone?). This can be used to in PowerPivot to run Excel-based analyses using Pivot tables
• New features of Reporting Services:
  • Spark lines – think inline charts. No x/y axes. Just used to illustrate a trend.
  • Data bars – no more having to use expressions to calculate cell padding and the applying a background colour. Pick your element & configure.
  • KPI’s – like those in SSAS, you can now add a KPI and configure it’s value, goal, status & trend. KPI’s can be ‘componentised’ and reused in other reports.
  • Lookup functions
  • Aggregates of Aggregates
  • New data sources (Project Madison, SQL Azure)
  • Ability to reset page numbers based on groups
  • Expression-based page breaks
  • Expression-based chart headers
  • Naming Excel Worksheet tabs (on render to excel)
  • Bottom-top text rotation
  • Session variables
• R2 to be released in first half of 2010 (CTP is already out).

Saturday Session 2 – Jamie Thomson’s Whistle Stop Tour of SSIS Add-ins

In this session Jamie took us through the fact that there are a number of SSIS custom tasks available on the intertubez. Many of them are free and can be found on CodePlex. There were only one or two slides (Jamie’s contact info), as the whole session was taken up demoing each of the tasks he had chosen.

XMLify (http://xmlify.codeplex.com) – takes columns that you specify (from a query/view) and loads them into an xml block. Jamie demoed this for capturing error outputs.

Kimball SCD (http://kimballscd.codeplex.com) – For slowly changing dimensions. Takes all data in the destination and for each column you can define it’s usage ( type 1 change, type 2 change, business key, surrogate key, etc). Type 1 changes will update the destination data based on the source data. This task is a monster. Loads of options for configurability.

Twitter task (http://ssistwittersuite.codeplex.com/) – I think this is awesome. You can automate sending your tweets. Careful though, password is in full view in design mode. You can also get all replies/mentions and then data mine them using the Term Extraction task. I can see this being used as a substitute for text (SMS) messaging notifications, for whatever you need to be notified about (if you’re always on Twitter).

Term Extraction – A data mining task to pick out nouns, verbs, phrases. Although not an add-in (comes with SSIS) this is still a neat task that I had not used previously. Jamie is using it to sift through the Twitter feed to his @ssisdemo Twitter account. Very cool use to see what people are saying (words & phrases), and how often each occurs. Could be useful, I guess.

Normaliser (http://normaliser.codeplex.com) – Normalises the data from an incoming data stream. Pure. Simple. Awesome. Data must be sorted as a prerequisite. This task only handles one to many relationships.

Rank Transform (http://ranktransform.codeplex.com) – Ranks a dataset by Partition. Enables you to view a dataset ordered and ranked (by specified column), as well as being able to specify a partition for the ranking (i,e, month). The dataset must be sorted before use. Rank & Dense Rank have their differences, specifically where you have two ranked values that are equal (a ranking ‘tie’). So, if you have two values that are equal and ranked ‘2’, then for the column RANK the next value will be ranked as ‘4’ whereas the column DENSE RANK the next value will be ranked as ‘3’. Still confused? Go here.

Zip task (http://cozyroc.com/ssis/zip-task)- this task takes a source file and compresses it into a gzip file (.gz). That’s it. Decompression decompresses the file. This will only work with a single file and not for a folder or multiple files. I think I would prefer to use 7zip and an Execute

FileWatcher (http://www.sqlis.com/post/File-Watcher-Task.aspx) – This task ‘watches’ a directory, when a file is detected it will progress the flow to the next task. Placed in an endless loop this will continually execute, and will pick up files as they arrive. Neat.

Trace File (http://www.sqlis.com/post/Trace-File-Source-Adapter.aspx) – understands a sql trace file and enables you to handle the trace file contents.

Data Flow Plus (http://www.cozyroc.com/ssis/data-flow-task) – Enables you to populate two different tables with the same data flow. Neat.

Trash Destination (http://www.sqlis.com/post/Trash-Destination-Adapter.aspx) – Enables you to send any data flow to a trash destination. No configuration required, so ideal for a number development scenarios.

This session, along with those by Donald Farmer, was the most enjoyable and one that I probably took the most away from. It opened my eyes to the world of custom SSIS tasks and that fact that so many of them are freely available (with source code so that you can tweak at leisure). Jamie definitely seems to be the go-to guy for SSIS related  info. If you haven’t checked out his site yet, I suggest doing so: http://sqlblog.com/blogs/jamie_thomson/default.aspx

Other SSIS Add-in Links:

• http://www.cozyroc.com
• http://ssisctc.codeplex.com/
• http://www.sqlis.com/category/Component-Downloads.aspx

Go to Part II of the SQLBits recap.