Web developer from Sydney Australia. Currently using asp.net, mvc where possible.

Sunday, October 17, 2010

Create a Build File for a Visual Studio Solution - MsBuild Series

Why create a build file for a Visual Studio 
Solution?
A build file automates the process of building, testing, analyzing, packaging, & deploying your project. Build files can be used to give you a single click solution to perform mundane tasks in a consistent way.
It saves you time by automating all the tedious steps necessary to prepare your project for testing or deploying. It reduces risk because you can confidently repeat the process, there's no change you will forgot to rename a file or change to release mode etc etc.

OK but How?
Creating a build file for the first time can be a little tricky so I have prepared a quick tutorial on creating a simple msbuild file for compiling your solution. Just follow the steps below and let me know if you have any issues.

1) Folder Structure

Once we start running builds of our project we need a place to store the results. We may also need extra tools for our build process and of course we need a place for our new build file.

Below is a the folder structure I will be using for this introduction. It allows us to keep all of the build files / folders out of our source tree.

Project Root:/
/Build – result of the build will be placed in here
/Source – all source code (& libraries) for the project
/Tools – collection of tools used for the build process
/build.bat – simple 'double click to build' ms dos batch file
/build.proj– our build file for msbuild, this is where we define the steps for our build process

2) Creating the Batch File

Create a new blank file called build.bat in the root directory of your project. This will be a simple ms dos batch file to kick off our build script. Simple copy the contents below into the batch file.
REM dont remove this line
"%windir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe" /nologo build.proj  %*
Make sure you leave the first line intact, otherwise you may run into problems because of the encoding used to save the file. If you have issues with encoding then you may need to open the file using a specific encoding (850) see: http://msdn.microsoft.com/en-us/library/dxfdkfke(VS.80).aspx.

The batch file simply calls msbuild.exe and passes in our build.proj file. The %* argument passes any command line arguments supplied to the batch file into the msbuild.exe command. This allows us to specify a target at the command prompt like this: build /t:clean

4) Creating the Build File

Lets start with the simplest sample possible.
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

     <PropertyGroup>
      <BuildOutputDir>build</BuildOutputDir>
     </PropertyGroup>

     <Target Name="Clean">
      <RemoveDir Directories="$(BuildOutputDir)" />
     </Target>
 
     <Target Name="Build" DependsOnTargets="Clean">
      <Message Text="Clean"/>
     </Target>
</Project>
 
The build file contains two targets: Clean & Build. Build is our default target so if you run build.bat without any target specified and the build target will be executed by default.

The default target is specified on the 2nd line with DefaultTargets="Build"

The Build target depends on the Clean target, so before the build target is executed the Clean target is executed. The Clean target simply deletes the output directory, giving us a clean working space to preform the build.

The output directory is defined in the ProperyGroup near the top of the file. This simply assigns the variable: BuildOutputDir to the string “build”, which is the name of our output or build directory.
5) Adding the Compile Target
To actually compile the project we need to add another target, which we are going to call Compile.

The updated proj file now looks like this:
<PropertyGroup>
  <BuildOutputDir>build</BuildOutputDir>
  <SolutionToCompile>Source\Jobping.StickBeak.sln</SolutionToCompile>
 </PropertyGroup>

 <Target Name="Clean">
  <RemoveDir Directories="$(BuildOutputDir)" />
 </Target>

 <Target Name="Compile">
  <MakeDir Directories="$(OutputDirectory)" />
  <MSBuild Projects="$(SolutionToCompile)"
     Properties="Configuration=Automated_Build;" />
 </Target>
 
 <Target Name="Build" DependsOnTargets="Clean;Compile">
  <Message Text="Clean, Compile"/>
 </Target>
 
Things to notice:
  • Extra variable for the solution to compile
  • Extra target called Compile
  • Compile target uses msbuild to compile the solution
  • Solution Configuration is set to Automated_Build
  • The Build target now depends on Compile too.
If you run this you should receive an error stating that the Automated_Build configuration does not exists. We need to create this configuration inside visual studio.

6) Creating the Configuration “Automated_Build” in VS

Open up our visual studio solution, then under the Build menu, select Configuration Manager. Create a new configuration called “Automated_Build” as show below, copy the settings from the release configuration.

Creating the Configuration inside Visual Studio


I am going to unselect SampleWeb because I do not want that project compiled as part of the automated build.

Configuration Setup - Removed Sample WEb

Now we will change the output directory for the “Jobping.StickyBeak” project under the “Automated_Build” configuration. As Show below:


With our project configuration complete we can now run our build.

Build Output

As you can see we now have our freshly compiled binaries in the build directory.

By simply adding more targets to your build file you can automate any tedious step that is needed to build and package your project.

This is just the beginning! next we will look at 
Resources:

Shout it kick it on DotNetKicks.com

Saturday, October 2, 2010

Jobping Url Shortener – Version 0.6

We have just checked in our latest version of the Jobping Url Shortener, version 0.6. This version resolves an important issue raised by @nato24. Nato24 asked “Have you guys implemented a solution for obscenities in your short url encoder?” . Thanks nato24 & Good question!

Our short urls are too short at the moment to create any four letter words(yet!) but it was just a matter of time before all possible 4 letter words were used.

So, in this release we have decided to remove all vowels (‘a’, ‘e’, ‘i’, ‘o’, ‘u’, ‘A’, ‘E’, ‘I’," ‘O’, ‘U’) from the possible letters used in the Url shortener. This means that our urls will grow a little faster but we were willing to sacrifice this for wordless urls.

However, we do get a major benefit also, we can now make our own short urls using words (that contain vowels). So we are free to make up short urls like http://jobp.in/example and know that the shortener will never create a clashing url because it contains a vowel (or 3).

We also added a feature that allows us to offset the urls generated. By adding this offset into the configuration we allowed our url shortener to skip over any duplicates that would have been created if we just removed the vowels from our code.

The new version is on codeplex here: http://jpurlshortener.codeplex.com/

The production site for our shortener is here: http://jobp.in/

Also be sure to visit Jobping, which is now Global and advertises positions based on Microsoft Technologies.

Shout it

StickyBeak Version 0.3 Released

Finally found some time to add a couple of features to StickyBeak.

StickyBeak is a logging utility for asp.net websites which can log every request to your site. It provides similar features as IIS log files but provides additional logging information (which just isn’t possible with IIS logs) and easy viewing of logs via a admin page. You can also use the StickyBeak log file parser in your own code.

StickyBeak records request details such as url, ip address, unique session Id,  datestamp, cookies, querystring, form values, session variables etc. StickyBeak allows you to track the requests that lead up to errors/exceptions on your site. So it provides valuable context for figuring out exactly what caused an error. StickyBeak is complimentary to elmah.

We use a modified version of StickyBeak on Jobping .

Version 0.3 is now on codeplex and includes the following additional features:

1) Logs Asp.Net Session keys and values.
Session keys and values are now recorded alongside the existing request data (cookies, querystring, posted form values, headers etc).

2) Allows StickyBeak to be temporarily turned on or off via the admin page.
StickyBeak can be Enabled via the configuration file but now you can temporary override this setting on the admin page. Therefore you can have StickyBeak turned off in the configuration file (which means no logging is taking place) then temporary turn on the logging via the admin page. If the application is restarted StickyBeak will then revert to the config setting.

Upcoming features for StickyBeak

We would love to hear any suggestions you may have for the next version so please send them through. Currently we are planning to include features such as:

Database integration - which will allow log files to be consolidated from multiple sources into a rational db for analysis.

Viewstate (logging/decoding) – allow Viewstate to be logged and decoded (we are using mvc :)

You can read more about StickyBeak here: http://markkemper1.blogspot.com/2010/06/introduction-to-stickybeak.html

StickyBeak is hosted on Codeplex: http://stickybeak.codeplex.com/

Shout it
kick it on DotNetKicks.com

StickyBeak in action below. New features highlighted with red squares.

Tuesday, June 22, 2010

Introduction to StickyBeak

While working on Jobping we wanted a raw record of each request made to our site so IF something happens to go wrong we would have all the data necessary to recreate the event and/or the data itself. Needless to say that it has proved very useful to investigate what has happened on the site.

However the code used for this logging is embed into the main project and not easily portable, I wanted to make a assembly that captured this functionality so I could easily drop it into the next project. So we created  StickyBeak.

StickyBeak is a logging tool for asp.net websites written in c# and currently requires the NLog logging library to run. StickyBeak’s purpose is to log each request to your web server and also provide a easy interface to view these requests.

Looking at these logged requests is extremely useful when you are trying to find the cause of an exception or  even more useful when you are trying to recover some lost data because of an exception.

StickyBeak works as an HttpModule and logs the raw request data into a log file using Nlog. The information recorded for the requests includes, date, http method, url, User.Identity.Name, IP Address, unique session Id, unique browser Id, header values, querystring values, posted form values and cookie values.

Below is a screenshot of the admin viewing tool, which lets you see the logged activity on your site.


How it works

StickyBeak runs as a HttpModule, each time a request is processed by .net the module creates a new RequestLog object and populates all the data using the current request. The RequestLog object is then passed to the LogRepository which saves the Requestlog object.

Currently there is only one LogRepository, this repository uses NLog. The NLogRepository writes the LogRequest object out to the log files in a custom format. The NLogRepository can also read LogRequest objects for viewing using the admin interface.

Configuration Needed For StickyBeak To Work

  1. You need a reference to the StickyBeak and NLog assemblies contained in the binary zip file distribution on CodePlex
  2. Configure the StickyBeak HttpModule

  3. Configure the handler (to display admin interface). The configuration below also secures the handler you may wish to remove this for testing.


  4. Configure NLog to record the logging information that is record by StickBeak


  5. You can also optionally configure exclusions for StickyBeak. For example you could exclude all requests to a certain URL, exclude a querystring/form/cookie/header value by key etc. See the sample configuration for more details.
You can download the source and binaries from StickyBeak on CodePlex. kick it on DotNetKicks.com Shout it

Tuesday, May 18, 2010

Quickly Trim all model's Properties - Reflection Vrs Fasterflect

I've had some great feedback from Buu Nguyen author of the Fasterflect library. He supplied the feedback below as well as some sample code, which I have incorporated into the code base. 

Buu Nguyen says:
Fasterflect only speeds up invocation operations, not query (aka lookup) operations.  In fact, query operations in Fasterflect are convenient wrapper for .NET reflection, so using them will cause the code to run slower.

Suggested usage: avoid query operations if you don’t really need them;  move the GetProperties out of each iteration because it is the same for all methods and thus makes it hard to see the performance difference when using Fasterflect.

The delegates should be reused instead of being regenerated every time – the latter will make the code run even slower than the normal Fasterflect API.
Suggested usage: use a dictionary to cache the generated 

The main change to the code was to cache the call to type.GetProperties(). So instead of calling the GetProperties() method directly a call is made to a caching component which ensures that the GetProperties() call is only made once per type.

Also, as Buu has suggested, the calls for generating the delegate setters and getters are now cached into a static dictionary as well.


Test Methods (2-4 using cached call to type.GetProperties() ):
    1. Long Hand - No reflection.
    2. Initial code - normal reflection
    3. FasterFlect m1 - Uses the FasterFlect library
    4. FasterFlect m2 – Uses the FasterFlect library’s delegates approach

Results

So here are the results (over 1 million object trims):
Long Hand Trim
00:00:01.4150000
Initial code
00:00:08.8280000
Fasterflect m1
00:00:04.8190000
Fasterflect m2
00:00:02.9920000

Ok, now we are seeing Fasterflect perform nearly twice as fast as reflection and about 3 times as fast when using the delegate method of Fasterflect.

It also appears that the delegate method of Fasterflect is only about twice as slow as the long hand trim.
Download the source code.

Long Hand code





Property Cache Helper


Initial code






FasterFlect Method #1


FasterFlect Method #2



Set Default Outgoing Repository with hg (Mercurial)

Firstly if you are new to Mercurial check out http://hginit.com/ its a brilliant little Mercurial tutorial.

When creating a hg repository locally, at some point later, I often need to set the default outgoing repository to push out my changes.

Creating projects locally using hg (hg init) without cloning from anther location is ideal for playing with new projects locally and having your source code version controlled.

There are so many times where I spend about 10 minutes going down a path then find out that I don't want to continue. With mercurial or git as a local version control system, you can simply revert back your changes to the last commit.

When the time comes to publish your changes to the world, setting up a default outgoing repository in hg saves a lot of typing.

Here is how to do it
  1. Go to the .hg folder in the root of your repository
  2. Create a new file called "hgrc"
  3. Enter your default outgoing repository as show below.

That's it. Now you can just enter "hg outgoing" to see the list of change-sets that need to be pushed to your default repository and "hg push" to actually push out your changes.

Codeplex now has Mercurial as an option for version control.

Here is a copy paste sample for the Jobping Url Shortener project on codeplex
[paths]
default = https://hg01.codeplex.com/jpurlshortener

kick it on DotNetKicks.com

Monday, May 17, 2010

Quickly Trim all your model's string properties – Speed results

Update: Posted another follow up with feedback on the usage of Fasterflect

With my Quickly Trim all your model's string properties post causing quite a stir (2 comments), I have decided to post a follow up and actually test the speed.

For the actual benchmarking I used a slightly modified version of this nice little class http://www.yoda.arachsys.com/csharp/Benchmark.cs, which I found from this page http://www.yoda.arachsys.com/csharp/benchmark.html

I tested 4 different methods that use reflection to trim all the string properties on an object and the 'long hand' method that does not use reflection at all.

Test Methods:

    1. Long Hand - No reflection.
    2. Initial code - with the GetIndexParameters call removed (not needed)
    3. Initial code with linq – Initial code but using linq to select the properties to process
    4. TypeDescriptor– used the TypeDescriptor class to get the property list
    5. FasterFlect m1 - Uses the FasterFlect library
    6. FasterFlect m2 – Uses the FasterFlect library’s delegates approach

Results

So here are the results (over 1 million object trims):
Long Hand Trim 00:00:01.4220000
Initial code 00:00:07.7130000
Initial code with linq 00:00:10.7270000
TypeDescriptor 00:00:14.2340000
FasterFlect m1 00:00:06.1330000
FasterFlect m2 00:00:06.1830000

Surprised? I was. Firstly, the refection code varied from 7 - 14 times slower then the direct code, this wasn't a surprise more of a reminder that you really need to be careful when using reflection in your code base.

I knew my initial code would perform better then trying to filter for properties before doing the work. Filtering the properties first using linq or any other means, basically results in another loop over the properties on the object. So the more properties you have on your object the worst this method will perform.

But I was quite suprised with the TypeDescriptor. I had hopes that this would perform better then my initial code, as according to the documentation this method actually caches the meta data about objects. I am suspicious I haven’t used it to its full potential…

The other surprise was that the delegate method in fasterflect is actually slower then the normal method. Again, someone may well point out how to implement this better. The other note is that fasterflect was only able to achieve an 80% reduction. I was hoping for more.

Anyway interesting stuff, if someone has another fast method to achieve the same results let me know I’ll take it for a test drive.
Download the source code.

Long Hand code


Initial code



Initial code with linq



TypeDescriptor



FasterFlect Method #1



FasterFlect Method #2



kick it on DotNetKicks.com