This is the start of a series of posts that will hopefully help you understand how you can utilize the warehouse in Team Foundation Server to generate reports on your test management data.

Out of the box you get a number of useful reports that should go a long way in getting you the information you are looking for.  If you look on the SharePoint site that is created with your new team project, under the Excel Reports/Test Management folder, you will see five work books. Build Quality, Test Team Productivity, Test Team Progress and Testing Gaps.

Each of these work books contains a few work sheets that have either tabular or chart data on them and help answer a specific question.

Build Quality – contains two work sheets that focus on answering questions about testing on specific builds

• Build verification testing
  • this shows the results for all of the “BVT” runs for builds. By default all runs published as part of the team build work flow are “BVT” runs
• Testing activity per build
  • this shows all of the results for all runs on a build.  This will include all “BVT”, automated and manual test runs.

Test Team Productivity – contains four work sheets that focus on answering questions about how productive your test team is about executing tests and finding bugs

• Test Activity
  • Shows all of the manual testing that is taking place over time
• Test Activity by User
  • Shows how many tests a given user has executed
• Bugs Created by User
  • Shows how many bugs a given user has created
• Bug Effectiveness
  • Show the resolution of bugs that were filed by a given user.  Resolution would be something like By Design, No Repro, or Fixed.  If many of the bugs filed by a particular user come back as no repro that could be something worth looking into.

Test Team Progress – contains five work sheets that focus on tracking how the team is progressing towards completion of their planned testing.

• Test Plan Progress
  • Shows the status for the tests in a given test plan or set of test plans.  The status of the test might be Never Run, Passed, Failed or Blocked.  Hopefully over time this will move from all blue (Never Run) to all green (Passed)
• Test Case Authoring Status
  • Shows how I am progressing towards authoring all of my planned test cases.  Hopefully your chart will move from all red (Design) to all green (Ready)
• Test Status by Suite
  • Exactly the same as Test Plan Progress, however, instead of looking at the whole plan it breaks the data down by suite.
• Test Status by Area
  • Again, same data as the Test Plan Progress but broken down by area path
• Failure Analysis
  • Shows the analysis of test failures.  This can be useful in determining if you have a lot of test/environmental failures or if your test failures are actual application problems.

Testing Gaps

• User Story Status
  • Shows a rollup of the testing by user story.  This can be useful if you are trying to make sure you have good coverage of your user stories or if there are any gaps
• User Story Status by Config
  • Shows a rollup of your testing by user story across your different configurations.  This can be useful to help you determine if you have good coverage of your user stories for your various configurations.
• Tests not Executed
  • This simply gives you a list of tests cases that you have yet to run for your test plan.

Aside from the excel work books there are a number of other reports that are built using sql server reporting services.  The unique thing about these reports is they can use the power of the MDX query language to bring data together from different parts of your life cycle.  Probably the most useful of these reports is the stories over view report and the description of the report says it all

and the screen shot is even better

This report gives you a singular view into the status of your users stores as it relates to completion (hours), testing (number of cases), test coverage (pass/fail) and quality (bugs). 

I hope this post has given you a quick insight into some of the resources that are available to use out of the box with TFS.  In the subsequent posts in this series I will dig into more details of the structure of the analysis services cube and what the various measures and dimensions mean.  In addition I will show you how you can build some other commonly requested reports, in fact if you are using test management with VS 2010 and you are looking for a particular report please post in the comments and I will see if I can include that as one of my examples.

The July 2008 Power Tools release for Team Foundation Server contains some really great additions to the product.  One of the more interesting features is the addition of a Team Members node to your team explorer.  This feature allows you to keep track of your team members, what they are doing and their presence information right inside Visual Studio.

However, that presence information comes from either Live Messenger or Office Communicator and in some cases if you aren’t logged into the client you will see an error.  To avoid that error, you can disable presence information in the Personal Settings dialog.

Select the Personal Settings… menu item from the context menu on the Team Members node

From the dialog click the “Change…” button in the collaboration group

And from the Choose a Collaboration Provider dialog you can select <None>

At that point you can OK your way out and you should be good to go.

Overall I can’t be more pleased with the critique as I feel this is one of the better portraits I have taken. I really like the composition, the lighting and the post processing.  As it turns out Scott agreed and really only had a few minor things he would change.

Scott Bourne over has posted his take on my portrait and you can read the full critique here.

It is great to have an impartial professional take a look at your best work and not only tell you it is good but show you how you can make it even better. Hopefully as I continue to improve I will eventually make some money with this hobby

I received the following email from Amazon.com today

We regret to inform you that the North Carolina state legislature (the General Assembly) appears ready to enact an unconstitutional tax collection scheme that would leave Amazon.com little choice but to end its relationships with North Carolina-based Associates. You are receiving this e-mail because our records indicate that you are an Amazon Associate and resident of North Carolina.

Please note that this is not an immediate termination notice and you are still a valued participant in the Associates Program. All referral fees earned on qualified traffic will continue to be paid as planned.

But because the new law is drafted to go into effect once enacted – which could happen in the next two weeks – we will have to terminate the participation of all North Carolina residents in the Amazon Associates program on or before that same day. After the termination day, we will no longer pay any referral fees for customers referred to Amazon.com or Endless.com nor will we accept new applications for the Associates program from North Carolina residents.

The unfortunate consequences of this legislation on North Carolina residents like you were explained in detail to key senators and representatives in Raleigh, including the leadership of the Senate, House, and both chambers’ finance committees. Other states, including Maryland, Minnesota, and Tennessee, considered nearly identical schemes, but rejected these proposals largely because of the adverse impact on their states’ residents.

The North Carolina General Assembly’s website is http://www.ncleg.net/, and additional information may be obtained from the Performance Marketing Alliance at http://www.performancemarketingalliance.com/.

We thank you for being part of the Amazon Associates program, and we will apprise you of the General Assembly’s action on this matter.

Sincerely,

Amazon.com

I am not even sure how to comment on this, other than to say ”Don’t you get enough of my money”. 

It is amazing how the state of North Carolina will give away hundreds of millions of dollars to Dell, Google and Apple to get them to create a few hundred jobs here and in the same breath they will pass laws to collect taxes on the very tiny amount of money I may or may not make from some Amazon.com referrals.

Give me a break.

Project Description

This tool takes a WCF trace file and a WCF client proxy, or a WCF interface contract, and generates a unit test that replays the same sequence of calls found in the trace file. The code generated is easily modifiable so that data variation can be introduced for the purpose of doing performance testing.
The tool generates code for both Visual Studio 2005 and Visual Studio 2008. It also installs a wizard into both editions of Visual Studio for creating the trace and processing it from inside Visual Studio. If both editions are present the tool is installed into both editions. The source code is a Visual Studio 2005 project.

Latest Release June 2009

A new beta was released in June 2009 adding support for ASMX web services. Download it from here.

Features

• Replay of captured scenario in a unit test that can be included in a load test.
• Support for the DataContractSerializer.
• Support for message contracts.
• ASMX support (beta)
• Support for proxies generated using svcutil.
• Support for clients that create proxies at run time from contract interfaces.
• Supports calls to multiple services in a single scenario.
• Supports multiple calls to the same service operation.
• Filtering by SOAP action of which messages in the trace to replay.
• Readable and modifiable code is generated.
• Automatic association of trace message with proxy method (requires all operations to have a unique SOAP action).
• Support for client and server side traces.
• A command line tool for processing traces and generating code.
• Visual Studio 2005/2008 integration (Team Developer, Team Test, Team Suite and for 2008 also Professional)) that can be used instead of the command line tool.

Scott Bourne over at photofocus.com has an excellent interview with Scott Kelby.  Scott is known mostly as a Photoshop guy, however, I think he is an excellent photographer and one of my favorites.  I found the interview very interesting and I thought you might as well.

An Interview with Scott Kelby.

I am very excited that a portrait I recently took of my niece is being critiqued by Scott Bourne in the ScottCritiques group on flickr.

As an avid photographer, it is always good to have someone who is impartial look at your work and offer advice on how to make it better.  For the most part my work is seen by friends and family and the overwhelming response is “Fantastic” or “Those pictures are so good”.  While this may feel good, the reality is most of them don’t know what to look for or they are just being nice.

The portrait is part of a larger set I took of my nice for her 16th birthday.  As it turns out she was a very cooperative subject.  We had a great time roaming around the JC Raulston Arboretum finding interesting places that would make a good background for a portrait.

The image to the right is one of my favorites and was taken in the Japanese Garden.  One of the keys to a great outdoor portrait is a very simple background and good lighting. In this case the background was a beautiful  red Japanese maple, which given its color and busy leaves could have been too detailed.  However, to bring the background down and the subject out I chose a very wide aperture of f2.8.  The wide aperture has the effect of blurring the background in any photograph with a very shallow depth of field.  The means that if your focus is on your subject object that are immediately in front or behind them will be out of focus and this will help draw the viewers eyes to the primary subject of the image.

Secondly, I used an inexpensive reflector to direct some light onto my subject.  In this case my brother was standing to my left, subject right, at about a 45 degree angle.  As you can see in the image there is some sunlight coming in from the top right, providing a little rim light on her shoulder and hair.  Using the reflector we were able to take some of that light and direct it back into the front of our subject to give some dimension to her face and provide fill for the shadows thrown by the trees over head.

So far there have been a couple of posts with some very interesting options about what the photographer might change.  I am looking forward to seeing what Scott says at the end of the week and using any advice I get to further improve my work.

Well my new blog is up and running, complete with a theme I like and some good information.  Lets see if I can actually post something worth while here.