Dan on Dev

Update: This post refers to NuGet 1.2. Although the techniques described here are still valid, once NuGet 1.4 comes out, Dan Turner’s fork will no longer be necessary. I will be writing an updated post about this shortly. So please keep the date of this post in mind!

I’ve been experimenting with setting up a build server for our dev team at work. I didn’t run into any challenges until I configured the build agent.

When I initiated a test build for one of our apps, it failed with a bunch of these:

Could not resolve this reference. Could not locate the assembly “<assembly>”. Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.

D’oh! Of course, my build agent had no idea where to find my project’s external references.

This is a common scenario, and there are all kinds of suggested solutions. For referencing your own projects, the Microsoft Patterns & Practices group suggests the following:

…you need to get the source or assemblies from that project into the workspace on your build server…edit your TFSBuild.proj file to add the assembly or the solution reference and override the BeforeGet event to get assemblies or sources from each of the team projects on which you have a dependency.

The above solution, however, would always get the latest version of your referenced projects. What if you wanted a specific version? Microsoft Patterns & Practices suggests branching the external team project into your team project.

Although branching and utilizing source control makes sense, putting third-party, compiled assemblies into source control puts a bad taste in my mouth. Surely there must be some other way? A search on StackOverflow reveals a lot of people asking about how to do this, and a lot of different approaches to solving it.

Enter NuGet

NuGet is a relatively new, open-source package management solution for .NET developers that has been getting a lot of support from Microsoft. (The idea is to make the process of adding external, third-party libraries easier — here’s an example.)

Our team has been slowly adopting the use of NuGet for all of our third party libraries. Now that we have a build server up and running, I was wondering: Can we leverage NuGet to resolve our dependencies for us?

Daniel Auger briefly discusses using NuGet for .NET Dependency Management, but he dismisses it entirely in his blog:

Nuget is not the final answer for teams using TFS
I’ve been following the Nuget dev list closely, and Nuget is considered to be a development time dependency resolver only, not a build time resolver. This means that if you use TFS to track your Nuget package folders, you could still run into dll versioning issues.

I admit I don’t understand Daniel’s point here. NuGet should be able to track versions for us, and the versioning should work just as well on our build server as on our development machines.

Edit: I asked Daniel about this and, after further discussion, he believes NuGet can be used as a build time resolver as well.

To understand how we can use NuGet, let’s look at what happens when we add a package to a typical .NET project:

MySolution\
  MyProject\
    bin\
    obj\
    Properties\
    MyProject.csproj
    Program.cs
  MySolution.sln

Now let’s add Ninject v 2.2.1.0:

MySolution\
  MyProject\
    bin\
    obj\
    Properties\
    MyProject.csproj
    Program.cs
    packages.config
  packages\
    Ninject.2.2.1.0\
      lib\
        (the assemblies go here)
      Ninject.2.2.1.0.nupkg
    repositories.config
  MySolution.sln

• The packages.config file exists once per project, and it lists any NuGet packages that are in use for that project (it also includes the version #):

<packages>
  <package id="Ninject" version="2.2.1.0" />
</packages>

• The repositories.config file just points to each packages.config file for the solution.
• The packages\ folder is where all downloaded assemblies for any projects in the solution are stored.
• The MyProject.csproj file is changed to add the actual assembly to your project:

<Reference Include="Ninject">
    <HintPath>..\packages\Ninject.2.2.1.0\lib\.NetFramework 4.0\Ninject.dll</HintPath>
</Reference>

We can leverage the command line NuGet.exe program to download NuGet packages from the online repository to the local packages\ folder, just by pointing it at our packages.config file! Here’s the plan:

1. Each developer would install NuGet.exe on their system path.
2. We’d download NuGet.exe to the build server and add it to the system PATH.
3. When developing any project that used NuGet references, we’d add

   nuget install “$(ProjectDir)packages.config” -o “$(SolutionDir)packages”

   to the project’s Pre-build event. (This will tell NuGet to install any packages listed in the project’s packages.config, and download their files to the ‘packages’ folder under the solution folder.)

Almost perfect. But what if we needed references that weren’t on NuGet? Perhaps we had to reference some obscure vendor .dll that had no place in a public gallery. To handle this case, we had to create a local NuGet repository and agree to store all internal .dlls there.

But wait! The nuget install command only lets us specify one source for our libraries; meaning we can either specify the public NuGet repository or our local repository, but not both.

Fortunately, in a discussion on CodePlex, Dan Turner decided to be amazing and create a fork of NuGet.exe that allows you to specify multiple sources. This makes our entire Pre-build event:

nuget install “$(ProjectDir)packages.config” -o “$(SolutionDir)packages” 
-s “\\asa01\asa_share\Dan\ASA NuGet Repository”;https://go.microsoft.com/fwlink/?LinkID=206669

This solution, to me, feels perfect. We’re not storing any assemblies in source control, and our packages.config file should handle package versioning for us.

A big thank you to Dan Turner for making those changes to NuGet.exe. (Perhaps it’ll get added to trunk?) Hopefully this article will guide someone else in a similar situation!

It was a beautiful weekend, so of course I was spending a good portion of it indoors. Anyway, I had wanted to experiment with setting up Team Foundation Build on a VM, and I figured I’d do it at my leisure from the comfort of my home.

After connecting via RDP to my office computer, I realized that I had forgotten to turn on Hardware-Assisted Virtualization (HAV) before leaving on Friday. Not a big deal, but annoying to think that my VM could be working 10-30% faster, and that all I’d had to do was change a setting in the BIOS.

I started to wonder if there was some way I could configure my office computer’s BIOS remotely. I looked into a variety of tools, all claiming to give you hundreds of configuration/management options (hardly any of them free, of course). I just wanted a simple tool to do one thing and do it well.

Then I stumbled across the Dell Client Configuration Utility:

Dell® Client Configuration Utility lets you create a stand-alone package that you can manually run on a Dell client computer to configure a BIOS, update a BIOS, or capture BIOS settings inventory data.

Granted, it didn’t let me configure the BIOS remotely, but I could certainly use this tool via RDP.

Some stipulations:

• The website lists compatibility as Vista x32 and x64, Server 2003, XP x32 and x64. My office computer is running Windows 7 Pro x64, and I encountered no issues, but YMMV.
• The DCCU webpage lists the Dell models that are compatible with this tool. My model was not on this list, but I took a risk. Again, YMMV.

After installation, it seems that the utility is basically a local ASP.NET website running on port 1000. The interface is completely web-based, and didn’t work properly for me in Google Chrome, so I switched to Internet Explorer:

The steps to configure your BIOS are rather simple:

1. Click the Create BIOS Inventory Package button.
2. A file named inventory.exe will be generated. Run this file. (You may need to run it as an administrator)
3. A file named TaskResult.xml will be created in the same directory as the inventory.exe. This is your computer’s BIOS configuration, in XML format.
4. Now, back in the web interface, under the BIOS Settings heading, browse for the TaskResult.xml file and then click the import button to import the settings.
   
5. Now the settings should all load in the grid below. Go ahead and scroll or search for the setting you wanted to change (so in my case, Virtualization).
   
6. Make the change(s), and then click Create BIOS Settings Package at the very bottom of the screen.
7. A file named settings.exe will be generated. Run this file. (You may need to run it as an administrator)
8. Restart your computer.
9. Your BIOS settings should now be configured appropriately!

Running the HAV Detection Tool, I was able to confirm the change:

I just thought this was awesome; I was able to reconfigure my BIOS completely through remote desktop! You can even use this tool to configure a bunch of BIOS settings on several client machines — of course, they should probably all be running the same BIOS version, and you’ll have to run the settings.exe file manually on each machine (perhaps through a logon script).

I doubt I’ll ever need to use this tool again, but I think it is handy to know that it exists. I wonder if other computer manufacturers offer similar utilities?

Depending on your environment, getting an app running on a production server can sometimes be a daunting task. Generally, the less you have to configure, the better. So when you need to deploy an application that uses Oracle.DataAccess and the right version of Oracle isn’t installed, good luck!

We shouldn’t blame the server admins. Maybe they are overloaded with tasks and simply don’t have the time, or maybe they’re worried that installing a new version of Oracle will break other functionality. Whatever the case, we have to understand their position.

Anyway, I had a similar situation with a recent .NET project I was working on. Fortunately, after doing a bit of research (and being very stubborn), I found a fantastic approach that requires no installation of Oracle on the production machine whatsoever. You simply copy a few *.dlls into your project, push the build, and watch the application run.

The below steps are using ODAC 11.2 Release 3 (11.2.0.2.1):

1. Install the latest version of ODAC on your development machine.
2. From your oracle client home, grab oci.dll and oraociei11.dll.
3. From the \bin folder inside your oracle client home, grab OraOps11w.dll.
4. Add the above three .dlls to your Visual Studio project.
5. From the properties window, make sure that the Build Action is set to Content and the Copy to Output Directory is set to Copy if newer.
6. Make sure your project is referencing Oracle.DataAccess, which it should be anyway if you’re using ODAC in your project.
7. The Copy Local property for Oracle.DataAccess should be set to True.

That’s it! Push your project, and you should see a total of four Oracle-related .dll files get copied into your output folder. Give this to your server admins and tell them you knocked one item off their todo list ;-)

Note: oraociei11.dll is approximately 120MB, which stinks, but isn’t a bad sacrifice to make.

If your team needs to do this on a large number of projects, you could do something neat like create a local NuGet package. I have a blog post about this coming soon.

Credits:

• What is the minimum client footprint required to connect C# to an Oracle database? – Stack Overflow
• Cached blog post from Chris Hulbert

At my current employer, I get thrown a lot of small “write-it-and-forget-it” apps that often involve querying a bunch of databases and producing some result. Since we’re a fairly large institution, this might involve querying a variety of database servers — not all SQL Server.

For these kinds of projects, I love ADO.NET Typed DataSets. Even though there are many newer data access technologies from Microsoft available now (Entity Framework, for example), typed datasets are still useful when you’re interacting with data from non-MS SQL databases. I can easily throw my database tables onto the designer, let Visual Studio generate my table adapters, and get to coding.

However, I think anyone who’s ever worked with ADO.NET datasets has encountered this horrifying error message:

Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.

Great. Which constraints? Which records? If my in-memory dataset is holding 20,000+ records, am I really going to know where my problem is? Although you can sometimes figure it out if you’re in the middle of debugging, when this error occurs in production, you have no way of stepping into the problem and figuring out what went wrong.

You may not know it, but datatables actually contain a neat method called GetErrors() which returns all rows that encountered an error. In addition, each tablerow contains a method GetColumnsInError() which, well, return the columns that encountered the problem. These two methods can go a long way towards helping you solve your problem.

After dealing with this problem for the N^th^ time and deciding to do something about it, I wrote an extension method that can be used to fill an arbitrary Table and, if any constraints are violated, throws a detailed exception listing the errors:

internal static class DataTableExtensionMethods
{
    /// <summary>
    /// This extension method can be used in place of the standard "GetData"
    /// method that is auto-generated in a strongly-typed data dapter.
    /// It internally calls GetData, but checks for constraint errors and,
    /// if they are found, throws a detailed exception
    /// listing all of the errors instead of the default vague error message used
    /// by ConstraintException.
    /// </summary>
    /// <typeparam name="TableType">The type of the table that is expected
    /// to be returned.</typeparam>
    /// <param name="table">A reference to the table that will hold the data.</param>
    /// <param name="adapter">The adapter that will be used to fill the table.</param>
    /// <returns>The same table as was given, but filled with the appropriate data.</returns>
    public static TableType GetDataSafely<TableType>(this TableType table, Component adapter)
        where TableType : DataTable
    {
        if (table.DataSet == null)
        {
            throw new ArgumentException(@"
The table passed to the GetDataSafely method should not have a null DataSet.
When calling GetDataSafely, be sure to pass in a table from a DataSet and
don't use the table's constructor.", "table");
        }
        table.DataSet.EnforceConstraints = false;
        // Unfortunately, we have to use 'Component' as the type for adapter
        // since there is no better 'base type' that adapter inherits from
        // (if someone knows of something else, please let me know!)
        // So, we can use C# 4's dynamic keyword to help us out here
        // (Obviously, this code will crash if there is no Fill() method on adapter)
        dynamic dynAdapter = adapter;
        dynAdapter.Fill(table);
        try
        {
            table.DataSet.EnforceConstraints = true;
        }
        catch (ConstraintException ex)
        {
            var errorQuery = table.GetErrors()
                .SelectMany(x => x.GetColumnsInError().Select(
                    c => x.GetColumnError(c) + " (Value: " + x[c] + ")")
                );
            throw new DetailedConstraintException(
                String.Join(Environment.NewLine, errorQuery), ex);
        }
        return table;
    }
}
internal class DetailedConstraintException : ConstraintException
{
    public DetailedConstraintException(string constraintErrorMessage,
        ConstraintException constraintException)
        : base(constraintErrorMessage, constraintException)
    { }
}

Here is an example of calling this method:

var someData = new SomeDataSet().SomeTypedTable.GetDataSafely(
    new SomeTypedTableAdapter());

This would be equivalent to the following, standard approach:

var someData = new SomeDataSet().SomeTypedTable;
var adapter = new SomeTypedTableAdapter();
adapter.Fill(someData);

And yes, I’m aware that this approach is even simpler, but it doesn’t work with the above safe method:

var someData = new SomeTypedTableAdapter().GetData();

The reason it doesn’t work is because we need the dataset to be created so that we can disable constraints before filling the datatable by calling:

table.DataSet.EnforceConstraints = false;
dynamic dynAdapter = adapter;
dynAdapter.Fill(table);

Then we enable constraints after the datatable has been filled:

try
{
    table.DataSet.EnforceConstraints = true;
}

As soon as we set EnforceConstraints to true, all constraints are checked against every record in the table. If a ConstraintException is thrown, we can:

1. Check for any error’d rows (table.GetErrors())
2. For each error’d row, get the violated columns (GetColumnsInError())
3. Finally, for each error’d row, get the specific error message per column (GetColumnError(column))

We use LINQ here to make the process of iterating over all of these errors very simple; in the end, we’re left with an enumeration of string error messages. We can then pass these detailed error messages to an Exception and our debugging process will be much easier:

var errorQuery = table.GetErrors()
    .SelectMany(x => x.GetColumnsInError()
        .Select(c => x.GetColumnError(c) + " (Value: " + x[c] + ")"));

Hope this helps someone!