This project is read-only.

Labs Framework Documentation

See Also

Overview

The Labs Framework (a.k.a., "Labs") is a facility for experimenting with managed libraries. There are two primary use cases for Labs:
  1. Unit Experimentation
  2. Hands-On Labs
In its most advanced form, Labs provides the tools that you need to create and organize cross-platform experiments in WPF, Silverlight, Windows Phone and console application projects. Labs will help you during every-day application development to quickly execute tests against your own APIs and others, without impacting your application's real code and without having to spend any time building a user interface or writing code that organizes, discovers and executes labs. Hands-on labs can also be compiled into an executable and deployed to end-users.

In its most basic form, Labs can be thought of as a glorified console application project or WPF project that you can add to a Visual Studio 2010 solution to play around with ideas and to learn about third-party APIs that your application needs, outside of your application's real projects. It also frees you from having to build the console UI or WPF UI yourself.

To get started with Labs, see Release Notes and Getting Started.

A Scientific Approach

Labs facilitate a scientific approach to development.

The scientific method (in general) is as follows:
  1. Observation
  2. Replication
  3. Interpretation
  4. Verification

Understanding the Scientific Method

These steps should look at least vaguely familiar to you if you've ever spent any time debugging an application. Debugging usually begins with some observation, which could either be your own or it could come from an end-user stating something like, "When I click button X the application crashes". Our first goal when debugging is to confirm the observation, our second goal is to determine whether it's a problem that needs to be fixed, our third goal is to fix the problem and our final goal is to confirm that the problem no longer exists.

Confirm the observation

The first step to confirming unexpected behavior is to observe it ourselves, or to observe it again if it was our own observation to begin with. We need to ensure that we can reproduce the behavior to know that it's real and to establish any environmental factors that may be involved. This is actually the replication step of the scientific method.

Determine whether it's a problem

Once we've reproduced the behavior and understand the environmental factors involved, we must consider whether the behavior is actually a bug or by design. In either case, we must consider whether the behavior needs to change. This is the interpretation step of the scientific method.

We may want to confer with a colleague before we make any changes to be sure that our observation and interpretation are actually correct. This is the verification step of the scientific method. (However, most of the time when debugging we forgo verification and assume that our observations and interpretations are correct and unbiased. Alternatively, pair programming techniques can provide continuous verification.)

Fix the problem

By fixing the problem we introduce a change, which is expected to affect future observations.

Confirm that the problem no longer exists

After changing the behavior, we must try to reproduce the original behavior again to verify that it's gone. This is an extension to the verification step of the scientific method. We must extend verification in software development since we introduce changes between observations.

In science, verification usually extends beyond the scientist making the observation to avoid bias and to establish strong confirmation. In software development, this is usually accomplished by unit tests, integration tests, system tests, peer review, quality assurance testing and user acceptance testing. This shows just how important verification is.

Applying Science to Programming

As mentioned in the previous section, it is common to apply the scientific method to debugging, but there are others aspects of programming to which science applies. It is especially useful for learning about new APIs, either third-party or proprietary, even if documentation and unit tests are provided.

The primary reason we like to experiment with new APIs is because we tend to learn quicker and more deeply through experimentation. A secondary reason is because we tend not to believe in the stated behavior of an API until we see it work for ourselves.

We use the scientific method to learn about software as follows:
  1. Assumption: We assume the correct usage of a new API while writing our experiments.
  2. Observation: We run the experiments and observe their behavior.
  3. Replication: We may attempt to reproduce the same behavior to confirm that an experiment is correct, especially if environmental factors are involved, such as concurrency or randomness.
  4. Interpretation: We must consider whether an experiment is completed. When testing a new API, an experiment is completed when we feel that we understand how to properly use the API in our real application. When experimenting with new ideas for our own APIs, an experiment is completed when we feel that the behavior is either acceptable or unusable as is, although sometimes we'll do just enough experimentation to get an idea of how it should work and then quickly move on to the real implementation.
  5. Verification: We may refer to a new API's documentation, forums or experts to confirm that our usage is correct before using it in our real application. When experimenting with new ideas, it can be helpful to show our colleagues or end-users to get their feedback.

Alternatives to Experimentation

The alternatives to experimentation aren't ideal for learning. Unit testing is not a good learning tool in general and there is an underlying mistrust about documentation.

Documentation

Documentation is useful and it can illustrate how to properly consume an API, but it has flaws:
  • Documentation is easily outdated.
  • Documentation can be wrong.
  • Documentation is static text, not an interactive GUI or a code editor like Visual Studio that allows you to experiment with an API.

Unit Tests

Unit tests are useful for ensuring that an API behaves as expected, but they do not illustrate how to properly consume an API:
  • A unit test contains many artifacts, such as mocking and assertion statements, that do not appear in real consumer code.
  • A unit test only focuses on one aspect of an API instead of illustrating a common usage scenario, which may span multiple unit tests.
  • A unit test is static because it tests a specific unit, so you cannot quickly and easily change the input or the behavior of the test to get a feel for an API, without breaking the test.
  • Unit tests are typically organized by logical units, not by common usage scenarios, making them difficult to locate.
  • Unit tests must be executed within a constrained testing environment that is designed for automation, not for interacting with a GUI.
  • Unit tests aren't always correct. They are code, which means they can also have bugs.
See Unit Experimentation for a detailed comparison between unit tests and labs.

Last edited Feb 5, 2012 at 9:00 PM by davedev, version 24

Comments

No comments yet.