It's been a few weeks since my last post and I've been busy with customers and Outlook add-ins. Last week, however, I was at Microsoft's internal conference. It's like a TechEd proving ground where presenters get to try out their talks on an internal audience. There is some amazing stuff coming down the line in Windows Server, Visual Studio and SQL Server. I'm really pleased about the data warehousing enhancements across the SQL Server product line.

Way back in 2004 I went to TechEd and blogged a series of articles about new features that would arrive in SQL Server 2005. I plan to do a similar series for SQL Server 2008 but since a lot of stuff we saw last week is still internal I'll only blog about bits in publicly released CTP's.

Finally, the Popfly mashup site is now allowing invitations so the first five people to leave a comment telling me what you plan to do with your mashup gets an invite. You need to leave me your e-mail address as well.

One of the main talking points at Tech Ed was whether to use remoting or enterprise services to talk between service tiers. For the benefit of the guys at work I thought I'd try and collate some of the information thats available and draw some guidelines. I won't cover ASMX/WSE as I believe there is no argument there - for talking between services use ASMX.

Firstly, for an overview of the options for hosting objects and communicating with them, you can't beat Don Box's bullets:

• Services and Communications
• Services and Processes

Don however doesn't mention remoting as he is really against it. Rock Lhotka provides a view that includes remoting:

• Middle-tier Hosting: Remoting vs. Enterprise Services

These two articles from Richard Turner (a program manager working on "Indigo") give you the inside view.

• On the road to Indigo - Is .NET Remoting Dead?
• On the road to Indigo: Prescriptive Guidance for Today's Technologies

Finally, another good roundup is by Aaron Skonnard reviewing Richard Turner's Tech-Ed session:

• TechEd: SO Prescriptive Guidance (Richard Turner)

I think the general guidance from all these articles is that remoting is OK given the following rules:

1. Only use it between tiers on the same machine (Otherwise use DCOM/ES instead)
2. Don't use any of the custom stuff like formatters and sinks
3. Don't use it at the service boundary (Use ASMX instead)
4. You MUST use remoting to communicate between app domains in the same process
5. You may have to do some rework when Indigo arrives

Good as Tech-Ed was, this is one thing that I was a little disappointed by and thats the "Ask the Experts" stand. At first sight it looks promising - lots of Microsoft staff, mini white-boards and plenty of discussion. Unfortunately the reality isn't quite the same - most of the Microsoft staff are deep in conversation with each other and when you do manage to ask a question the answer is less than satisfactory.

Maybe I'm asking the wrong questions but I get the impression that the people staffing the stand are marketing personnel because their knowledge doesn't seem to run very deep. I was ready for a good conversation about threading models and scalability of Reporting Services but the person I spoke to fell at the first hurdle. He didn't think anyone else at the stand would be able to help either and gave me an e-mail address with a promise to find the answer I was looking for.

The same thing happened in Barcelona last year so my advice to Microsoft for next year is to ship some developers out from Redmond and not use marketing and support staff from the UK.

I have a couple of photos of the keynote, but I'm pretty awful with the camera so have a look at someone who is a lot better.

The Keynote.

What is the sound of 6000 people drumming?

The SmartPhone demo.

Brad and I are just sitting in the airport enjoying a coffee before the flight. It's been a hard week - getting up at 6am to ensure we were at the first session on time, six 75-minute sessions a day and some sort of event in the evening to make us late back to the hotel. Has it been worth it? Of course - the main points to take away are:

Don't use .NET remoting - its slow, won't interoperate and will be eventually deprecated.

Use ASMX and/or Enterprise Services instead - a lot of work has gone towards making them highly performant and scalable.

SQL Server 2005 is comming - its going to both make your life easier and, if you're an ISV, trample over your business model. Time to look for that next pardigm shift.

Microsoft are getting even more into the developer tools arena - enterprise source and project control, integrated testing and code coverage tools, also code checking and verification tools. I spoke to both Perforce and Compuware to try and get their point of view and see how worried they were.

Compuware trotted out the company line that they have always built on top of what Microsoft provides and will continue to do so.

Perforce were not quite so upbeat - their response was more of a "lets wait and see what Microsoft ships". They made the point that they have probably the fastest source control tool on the market and wonder if Microsoft can match it. They also showed me some new visualizations they are developing related to how the code changes over time - colouring lines based on how old they are. It looks a useful method to find out how a file has evolved.

Two words - "Database Mirroring"... Almost instant stand-by with very fast (~3 seconds) automatic or manual fail over. The client libraries also detect and fail over automatically too. To ensure this happens correctly a third server called the "Witness" provides majority logic to ensure that only a single side is up at any one time. I also get the feeling that Microsoft would prefer you to not attempt to use replication as a fail over solution as there are a number of issues to overcome - data loss, client fail over, recovery and the amount of custom code required to make it work properly. One thing you can't do on a mirror is use it. Kind of obvious really, but you can't even connect in a read-only mode.

Help is at and though - another interesting feature that will be available is a "Database Snapshot" which uses copy on write semantics to provide efficient views of a database at a point in time. A snapshot does not need to copy the database as it will reference the original database pages until they are modified. These snapshots can be created on mirrors which provide ideal access for reporting. Those that have all their data stored in a single database may want to think about splitting it into reporting databases and online databases. Basically, stick the information into a reporting database but keep the OLTP and application runtime data in another.

One last honourable mention is for SNAPHOT ISOLATION LEVEL - similar to Oracle's row versioning. But remember that tempdb is used to store the version information whilst a transaction is in progress.

Data Transformation Services been completely re-written for SQL Server 2005. The main difference is the separation of workflow and dataflow. But additional niceties include for/foreach looping, debugging, and OR constraints which mean daft hacks are not required. In fact modifying the package at runtime is impossible.

The new dataflow pipeline deserves a special mention as its no longer one row in one row out. The transform can consume data from multiple sources and send to multiple destinations in a single task - this allows for some interesting merging and splitting of data. Set operations are also supported - sort, aggregate,  split, multicast amongst others.

DTS has grown up and can play with the big boys. If DTS 2000 is not enough for your needs right now then consider waiting for SQL Server 2005 and not trying to implement yourself.

Our free 'gift' was an African drum. To warm everyone up for the key note they had a drum band lead the entire set of delegates in a mass drum session. So 45 rhythmic minutes later with arms about to drop off; the session started.

I plan on seeing the following sessions, but as usual things will probably change.

Tuesday:
12:00-13:15    DEV381    J2EE & .NET Interoperability
14:45-16:00    DAT313    SQL Server 2005: Data Transformation Services
16:30-17:45    ARC402    Data in Services-Oriented Architecture
18:15-19:30    ARC302    Metropolis : Building Applications in the Service-Oriented Enterprise

Wednesday:
08:30-09:45    ARC307    Realizing Services-Oriented Architecture
10:15-11:30    ARC220    New Developments in Microsoft Solutions Framework
12:00-13:15    DEV301    Visual Studio 2005 Team System - Part 1
14:45-16:00    DEV302    Visual Studio 2005 Team System - Part 2
16:30-17:45    DAT328    SQL Server 2005 : Inside Analysis Services
18:15-19:30    DAT323    SQL Server 2005: What's New in Replication

I would love to see this as well, but Analysis Services comes first...
16:30-17:45    DEV303    Visual Studio 2005 Team System - Part 3

Thursday:
08:30-09:45    CTS300    Prescriptive Guidance for Building with Web Services
10:15-11:30    DAT315    Intelligent BI - Integrating Data Mining into OLAP, RS and DTS in SQL 2005
12:00-13:15    DEV307    21 Rules of thumb - how Microsoft develops its software
14:45-16:00    DAT431    High Availability Technologies in SQL Server 2000 and SQL Server 2005
16:30-17:45    CTS304    Availability and Reliability - Failure Is Not An Option
18:15-19:30    DAT325    SQL Server 2005: SQL Service Broker

Friday:
08:30-09:45    DAT437    SQL Server Analysis Services 2000: Solving Business Problems using MDX
10:15-11:30    ARC406    Understanding the Grey area Between Service and OO Design
12:00-13:15    DAT329    SQL Server 2005: Web Services
14:45-16:00    DEV413    ASP.NET: Tips and Tricks for Building Server Controls
16:30-17:45    DAT335    Business Intelligence using SQL Server Analysis Services and Office 2003

I've finally managed to get TechEd sorted and paid for so I'll be in Amsterdam on Monday. I must say a big thanks to Tim Sneath helping to expedite things.