Lately I have been playing around with the go language. One of the first things I look for in a new language or framework is how tests can be written and whether it is feasible to do test driven development (TDD).

Assume that I am building a math package for go called funkymath (I chose the name to be fairly certain I do not conflict with any existing math packages).

To run tests for a package in go you run a simple command

1

go test funkymath

Not surprisingly I get an error results telling me that it cannot load the funkymath package, which makes sense since I did not create it:

1
2
3

can't load package: package funkymath: cannot find package "funkymath" in any of:
  /usr/local/go/src/pkg/funkymath (from $GOROOT)
  /home/nkn/Dropbox/SourceCode/go/src/funkymath (from $GOPATH)

To create a package you put the code in one of the defined paths for packages, which is also conveniently presented in the error message. It suggests the default location for packages from the go installation and an extra one in my dropbox folder I have configured in .profile.

Thus, I create a folder named funkymath and put an addition.go file in there and run the test command once again.

1
2

can't load package: package funkymath: 
addition.go:1:1: expected 'package', found 'EOF'

Again not suprisingly I get an error, as I did not put anything in the file. In many languages you would throw an exception to indicate that a method was not implemented yet.

As far as I have been able to find it seems to be idiomatic go to return both the result and an error (using the fact that go allows multiple return values) instead (maybe I am mistaken).

Thus, I created the following basic ‘not implemented’ implementation of a Plus function:

1
2
3
4
5
6
7
8
9

package funkymath

import (
    "fmt"
)

func Plus(a int, b int) (int, error) {
  return 0, fmt.Errorf("not implemented yet")
}

And run the test command again, this time with the information that it does not contain any tests.

1

?     funkymath   [no test files]

So the next step is to create a test file. Test files are located inside the package folder and are identified by go using the naming convention of the file name underscore test.

Thus, I create an addition_test.go file next to addition.go and put in the first test:

1
2
3
4
5
6
7
8
9
10
11
12

package funkymath

import "testing"

func TestPlus_TwoNumbers_ShouldNotReturnAnError(t *testing.T) {    
  a := 2
  b := 2

  if result, err := Plus(a, b); err != nil {
      t.Errorf("Plus(%v, %v) produces error %v, expected result %v", a, b, err, 4)
  }
}

Test functions are identified by the naming convention that they start with ‘Test’, and are automatically parsed a reference to the testing object. Will get back to the specifics of the test function a bit later.

1

./addition_test.go:10: result declared and not used

Ofcourse I cannot run this test function as I am not using result for anything. In general I really like this feature of the language, but sometimes it catches me ofguard. Thus, I inform go that I am not really interested in the first return value:

1
2
3
4
5
6
7
8
9
10
11
12

package funkymath

import "testing"

func TestPlus_TwoNumbers_ShouldNotReturnAnError(t *testing.T) {    
  a := 2
  b := 2

  if _, err := Plus(a, b); err != nil {
      t.Errorf("Plus(%v, %v) produces error %v, expected result %v", a, b, err, 4)
  }
}

And finally get the test error I was looking for:

1
2
3
4

--- FAIL: TestPlus_TwoNumbers_ShouldNotReturnAnError (0.00 seconds)
  addition_test.go:11: Plus(2, 2) produces error not implemented yet, expected result 4
FAIL
FAIL  funkymath   0.013s

Next step is then to implement the functionality. It can always be argued whether to use triangulation or obvious implementation, but I choose to just return a constant.

1
2
3
4
5

package funkymath

func Plus(a int, b int) (int, error) {
  return 4, nil
}

And the result of running the tests:

1

ok     funkymath   0.006s

YAY! I managed to complete the RED-GREEN TDD cycle in go.

This is an extremely simple example, but there is still a couple of valid points to make.

The convention for tests in go is to locate test files inside the package folder and name them filename_test.go. Test functions inside those files are named TestSomething.

The language does not have a rich DSL for writing tests like you would find in .NET using NUnit (or others) or in javascript using jasmine. You have to test values using built in language constructs like ‘if’ and even format the error messages yourself.

This is a bit cumbersome, but I guess that it is fine that the default way works like this, as 3rd party packages can built DSL’s on top of this like Asserts for go.

Next step is to play some more with building a webserver in go. If it is not on the web it did not happen.

I know more or less completed moving my blog from blogger to octopress. In general the experience has been quite good and I am impressed how easy it was to get everything going.

Why

I was not entirely happy with blogger, as I did not like the way the editing of the blog worked and the syntax highlighting of the code sections was done with javascript and was rather flaky when the code used characters that would cause problems in the pre tag.

Thus, I was motivated to move away from blogger and I had already started writing posts in an editor, before deciding to move, and then copy pasting them online afterwards. I started out with just plain text with my own homemade notation for things like headers etc. Eventually, I discovered markdown, which I quickly fell completely in love with.

Since, I like writing in markdown and prefer doing the editing offline the first criteria was to find a blog engine that can handle markdown.

The second criteria was that I would like to have it hosted in a very easy way. Thus, I would like the blog to only depend on html, javascript, and css once it was running. That way I could even host it on dropbox or something similar if I wanted to.

Finally, I wanted something that could do more or less all the stuff that a modern blog engine should do. Handle comments, categories, and so forth.

When I also discovered that octopress allowed me to do code syntax highlighting with pure html and css and was dead easy to setup and use, I quickly decided to move.

The process

I am fortunate to have a good colleague named Alexander Beletsky, who was also considering making a similar move, and he was kind enough to forward some of the tricks he had found.

First step was to get octopress working, which was fairly straightforward. Actually, the main challenge in this was that the version of ruby I had installed was not really working with octopress, but I simply solved this by installing ruby again using the guidelines given.

Next step was to find somewhere to host it. I settled on using github pages, since I am a fan of the github functionality and have some experience with it. It also seems like a stable, fast, and free hosting option. Configuring this was simple following the guidelines from github and octopress (basically just create a new repository with a certain name in github and run one rake command in octopress and your done).

Now I actually had a working blog, and wrote my first short octopress blogpost to test it out. I really wanted to move my old posts to the new blog so I exported them from blogger using the build in export functionality, and ran a script which I found here import.rb. This left me with my old posts in html and I quickly included them in octopress.

Unfortunately, due to the syntax highligthing some of the code snippets was in script tags instead of pre tags. This caused the exported html posts to mess up and no code being shown. Basically, I had the option of including the syntax highligthing javascript, modify the conversion script, write some code that could cleanup the posts, or do it by hand. Luckily, I only started blogging a few years ago and have not been as productive as I planned, so I was able to convert the posts to markdown in a couple of hours of boring work.

In the start I had used the built in comments system of blogger, and those comments where included in the text of the posts now. That is ok with me as long as it is not lost. I also spend quite some time working on getting my disqus comments moved, but sofar it does not work completely. It is possible to comment, but the old comments does not appear in the new blog, even though, as far as I can see they have been ported succesfully. So this is still ongoing work.

To get the look and feel you want for octopress you can either create a completely theme or find an existing one and modify it as you like. I chose the whitespace theme and changed it a bit to bring back the sidebar. Will probably modify it some more as I go along, but in general I think Lucas Lew has created a really nice looking and clean theme. You can find several 3rd party themes for octopress here themes.

All in all I am more than happy with octopress and really looking forward to working more with it.

Just testing out the octopress blogging engine. I am considering moving away from blogger, mostly to enable me to write in markdown (I am not a fan of editing in the browser) and to get the control of whats going one.

Sofar it has been easy enough to get working, but I imagine it will require quite a bit of work to set it up completely as my main blogging outlet.

Code generation has over time turned into one of my pet peeves when working in .NET (although most of it will apply to similar platforms like JAVA). Let me be very clear from the beginning of this post. I am advocating that you should use extreme caution before even considering using code generation to solve whatever real or perceived problems you might find it useful for. I have yet to see an implementation that has not caused extra problems. That does not necessarily mean that it cannot be valuable tool, but please take a step back and think before continuing down that path.

This is the longest blog post I have yet written. It contains no code, just ranting, and hopefully some insights.

Possible Motivation

Code generation is typically introduced to:

• remove duplication.
• reduce tedious coding.
• enforce security measures (authentication and authorization).
• enforce consistent handling of resources such as database connections and transactions.
• apply cross cutting concerns such as logging or caching.

And probably a number of other reasons. Basically, anything that seems hard to do in a nice way in the language and framework you are currently working with. Most of these things can be handled by using more light weight measures, using more simple and well known designs based on patterns.

Regarding tedious coding it would be wise to step back and consider why there is so much tedious coding? If the number of lines of code compared to the functionality it provides is so high that you consider using code generation, I would put it to you that your design is not very good. Basically, you are trying to hammer a square peg into a round hole, and instead of seeing the error of that you just get a bigger hammer (called code generation).

How to Determine what to Generate

Usually, you would determine what to generate based on an external resource (like XML files), examining the existing code structure using the reflection API, or annotating the existing code somehow (probably using attributes) and then examining that using the reflection API.

No matter how you choose to do this you are defining rules that are not clear from the context. An XML file is a very loose format and what is generated from that XML file is by no means clear to a developer looking at that XML, and it can also be very hard to determine from the code of the code generator. Similar problems arise when using the existing code structure or attributes, it is simply not very clear what rules are being applied and how the resulting code will look.

Step back and consider the number of characters in those XML files. Now consider the number of characters of the code if you wrote it by hand in the same way the code generator does it. Now consider the number of characters for the code of the generator. Often you will find that you end up with more characters using the XML than just using the code. Then consider that XML files cannot be unit tested, the build process will be more complicated, and you will have to actually write the code generator.

If you determine what to generate based on examining the existing code you will introduce the problem of simple code changes causing an unwanted side effect. I still recall changing the name of a parameter in a function in a model class as it was misspelled. Compilation and tests worked fine as they should. What I did not foresee was that suddenly our public API had a breaking change as it was generated from the core model. Our public SDK had suddenly also introduced this change as it was also generated from the code models. So basically all our integration partners had to change their code, or we could rollback the release. I know this is an extreme scenario and probably not something you would ponder, but there really is no limit for the ingenuity (insanity) of developers when trying to solve problems.

If you end up doing code generation despite all my attempts to get you to move away from it, do work dilligently on creating and maintaining documentation for you base generation format, whether it is XML based, annotations, or the structure of the code.

How to Actually Generate the code

Once the motivation is in place and you have somehow figured out how to determine what the output code should look like, it is time to actually generate some code.

There are a number of different tools for doing this in the .NET stack, but all of them basically falls into to different categories: + generate by code API (such as the CodeDOM) + generate by template (such as T4 templates)

Using code to generate code is a feasible approach, but unfortunately the APIs in .NET for this are extremely cumbersome and verbose leading to a lot of incomprehensible code. For instance the combination of CodeDOM and reflection (for reading base format) leads to some of the least readable code I have ever encountered in .NET. This basically means that it is very hard to envision the resulting code from the generator, which leads to random experimentation followed by examination of the resulting code to verify changes (if possible).

Using a template engine is a better alternative. The templates will most likely be a raw text format with placeholders for the code that has variation. This means it is a lot easier to envision the resulting code. Usually, the placeholders should be fairly few, otherwise, the reusability is very low and the code generation should probably never have been implemented at all.

Use a template based engine and put as much as the generation logic in the template.

There is an alternative where you generate the code based on a 3rd party solution, such as PostSharp. In this scenario it is the 3rd party that defines the ‘how to determine what to generate’ and maintain most of the documentation.

Do not start from scratch. Search the market for generation tools that can do the bulk of the lifting for you.

Build Process

Once the code is generated there is a number of ways it can interact with the rest of the codebase:

1. never generate a file just build it straight into a DLL
2. generate a temp file as part of build and delete it afterwards
3. generate a temp file and try to hide it from the developers (the folder it is in is not included in the project etc.)
4. generate a temp file and use it in build (the build is seperate step from generation).
5. only generate the code on demand and check it in with the rest of the code.

The distinction between these methods can seem very narrow. The important part is whether the generated code is an artifact of the build process and how easy it is to see for the developers.

The first 3 options basically assumes that it is not a good thing if the developer can see and modify the generated code. As if bad stuff would come from that, which it might.

The last 2 assumes that the generated code is not directly an artifact from the build process. Item 4 is not that different from 1-3, but at least it keeps the code generation step as a seperate step from the build process. This could be a pre process step in building with Visual Studio and the actual build just picks up the file from the project.

Item 5 is basically a templating engine. You generate something once and it is perfectly acceptable to modify it afterwards. This is just like when you add a class file in Visual Studio it will have a bit of code prefilled such as namespace, classname, and useless using statements. Similarly, with very simple commands you can generate new controllers and models in RAILS.

Make sure the build process is not made more complicated (and slower) due to the code generation. Ensure that the code generation is an optional step that comes before the build. Consider only generating once and keeping the files in source control if feasible.

Examples

I am just going to run through some of the examples of code generation I have encountered sofar. One is bad, but ultimately not that harmful, two have been extremely costly, and one is maybe ok.

Re-use of Internal Model in Other Services

The first example of a custom code generator I encountered was used to share models between a number of different services. Basically, the team had decided to go for a SOA design, but not really unstood how to implement it. It was a relatively simple business application consisting of a an ASP.NET webforms site and no less than 12 SOAP based web services.

We eventually came to the conclussion that probably 3 services would have been just about right, as the services depended on each other to a large degree. The sheer work of maintaining a model that belonged in service A, but was used in service B, service C, and the website quickly became a pain with 12 services. Thus, a code generator had been build to generate these models, making this an example of bad design driving the creation of the code generator.

Luckily, the simplicity of the generated classes, the ease with which they could be extended, and the simplicity of the code generator itself ensured that it was only slightly cumbersome to work with.

Custom ORM NHibernate Wrapper

I could write a series of blog posts on this example alone, but basically this backend system was designed by someone that considered code generation his speciality and consequently applied it to everything possible.

Combine this with a desire to build a custom ORM and you get one of the most convoluted systems I have seen. Basically, more or less the entire backend system was generated from XML. The idea of the homemade ORM was eventually dropped, but its basic API was then used to wrapped NHibernate. So as a developer you are basically stuck with a homemade XML format for queries and entities, which was the used to generate mapping and queries for NHibernate. Due to the complexity of such a system the queries and the mapping are highly ineffective.

The worst part was how hard it was to actually write any kind off meaningful code for the system. The generated entities and queries (called views) were hidden and impossible to extend, actually, the only intended way to implement business logic was static methods were everything was passed as parameters. Some of the hacks developers had implemented on top of this system to be able to deliver features only made it worse.

To top off the problems with this code generation it was full of bugs and very hard to modify. Part of the code generator was even code generated.

Possibly the worst design I have ever seen. For a relatively simple business domain and equally simple functionality.

Service Layer Method Generator

This is another example of a very complex code generation system. I cannot be completely certain of the motivation for this code generation system, but my guess is that it is yet another example of a bad design that caused a lot of manual work.

The core of the system is a number of entities, build by hand and mapped with NHibernate. The entities have specific queries as static method in an Active Record style. This design in itself is not that bad. One could argue back and forth of the merits of Active Record compared to the Repository pattern, but at least it is a well known and tried pattern.

Based on the methods, properties, and certain custom attributes of these entities a number of things were code generated (which was not documented ofcourse):

• A service layer directly exposing all the public methods of the webservice. In this service layer certain cross cutting concerns was added, such as connection and transaction handling, exception handling, and security checks.
• A number of classes that was build into a .NET based SDK.

The idea behind the .NET based SDK was that it should work in a RPC style, and it contained both proxy objects and data objects directly generated based on the data retrieved from the core entities. In itself exposing internal implementation details in this way is never a good idea. As a direct consequence of this I have introduced a bug by changing the naming of a function parameter, which reflected through the service layer and the SDK. This meant that consumers of the API suddenly experienced a breaking change.

Nevertheless, the main problem stems from the design. The way the proxy object works in the SDK is that it does not contain any data. This means if for instance you access the name property of a customer object you will end up going over the wire to retrieve this data. Obviously, this leads to extremely chatty behaviour by the consumers of the SDK and eventhough it is possible to avoid it is not the perceived default. The main problem though is that 50 or so entities with an average of 10 properties leads to 50 X 10 X 2 (both get and set) = 1000 methods on the service layer to implement the proxy object (could have been done with a PATCH like operation for partially updates, but alas it is not). With the standard methods for entities (create, update, delete) and some queries the API ends up having approximately 1500 public methods.

I have removed the code generation completely and changed every method to be a ‘method as an object’ implementation. Combined with the template pattern this means that there is very little redundant code. Unfortunately, it does not change the fact that we have 1500 public methods, which is a nightmare to maintain for us, and equally bad for our consumers to understand and navigate. Fans of the code generation in question would probably argue that it is only a problem to maintain, because the code generation has been removed, and they would be partially right. But, taking into account the number of bugs in the code generation and the number of well hidden side effects I am very happy we got rid of it.

When a design decision leaves you with 1500 different methods on a public API, you should really consider whether this design is a good idea

AOP Support

This last example is actually one I designed myself. In a relatively simple ASP.NET MVC application we use attributes to implement AOP support using PostSharp. This actually works very well as PostSharp has a fairly straightforward API and there is no side effect by changing the code (all is handled by annotating with attributes).

It is used for a number of things:

• checking function parameters for null
• checking function parameters for certain values
• checking security rights on controller actions

I am not sure I would implement it this way again. The addition of the PostSharp tool does add a level of complexity and the fact that it must be installed on each and every developer machine also makes it a pricy dependency.

Alternatively, each controller action could have been implemented as a ‘method as an object’ and used template pattern to achieve similar results. Or maybe just do it by hand, it is not like the size of the application made it that much work. Basically, it probably is an example of overengineering that I am responsible for.

Challenges

There are a number of general challenges with using code generation:

• It makes the system inherently a lot more complex.
• It reduces readability quite a bit.
• It makes it hard to implement business logic. The required features will be hard to determine up front, and thus the code generator will not correctly accomodate them. This leads to hacks to work around the code generator.
• The APIs in .NET that are often used for this really sucks. If you really want to create code that is very hard to read, just combine CodeDOM code generation with the reflection API.

There are a some challenges that will get harder without code generation. What if you have 1500 command classes that needs to be changed? Depending on the change it could be put in a base class (if one is available), but if this is not possible it will be a harder challenge.

How do you change 1500 classes by hand? One option is hiring a lot of very cheap labour to do it for you, but more constructively it is also feasible to create a script that does the modification and is then thrown away, which was how they were generated in the first place.

I have yet to see a really good example of code generation. Even those provided by visual studio, the RAILS framework, or similar, are at best minor convenience functionality.

In late 2007 Google started an open source initiative to develop a new programming language named Go. The goal of the language was to make it as efficient as statically compiled languages, but as easy to use as dynamic languages.

Furthermore, the language aims to solve the challenges of concurrent programming in an intuitive way. This makes a lot of sense as computers are getting more and more CPUs with more and more cores. A lot of current programming initiatives are actually focused on enabling us to easier write concurrent programs.

Node as an example does have some tools for concurrent programs, but its main focus by using the event loop is actually an approach of strictly single threaded programming, instead focusing on reducing the blocking IO. There has been a lot of discussions going around with people claiming this approach to be either ingenious or insane. My personal experience is that it is possible to write very efficient servers in Node working with the single threaded model.

In contrast to Node, Microsoft released the async and await keywords in C# that makes implementing asynchronous code in C# a lot easier – simply by taking away most of the work of writing such code, but it does not change how the programming model works.

As a language Go is mostly related to the C family of languages in my opinion. Both with regards to features and syntax it more or less feels like a cleaned up version of JAVA or C# with some cruft removed and some parts made explicit that is implicit in those language. Not surprisingly Go seems to go in the opposite direction of Node (like most languages do) and has a concurrent model at the forefront of the language focus. The intriguing thing is that Go manages to this in a fairly elegant way.

What Go promises is speed close to that of C and C++, a slim and elegant language that is as easy to use as Node (it is as easy to get a simple http server running), and powerful features for doing concurrent programming.

Initial thoughts

With my usual curiosity I have followed the development of the language from the sideline for a while, but when I saw a very impressive demonstration of building a chat server in Go at the Oredev conference last year I decided I had to take a closer look and do some development on my own. This is my initial experiments with it and thoughts about the language.

Like I mentioned above the language feels like a slim and more elegant version of the languages in the C family. This meant it was fairly easy for me to get started and following the great online tutorial (http://tour.golang.org/#1) I was quickly able to produce code.

Clean Code

I cannot recall how many hours I have wasted looking at code where half of the lines are pure cruft. Functions that are never called, variables that are never used, and so forth. Therefore, I am really excited about the way the Go compiler handles this.

To illustrate this we can do a very small example. I will not go over the details (read that tutorial!), but the key is that 2 integers ‘a’ and ‘b’ are declared and assigned.

1
2
3
4
5
6
7
8
9
10
11

package main

import (
  "fmt"
)

func main() {
  a := 1
  b := 2
  fmt.Println(a)
}

This example will simply not compile! The integer ‘b’ is never used for anything and thus should be removed. The following will compile:

1
2
3
4
5
6
7
8
9
10

package main

import (
  "fmt"
)

func main() {
  a := 1
  fmt.Println(a)
}

I have seen some people argue that this is actually annoying when you are experimenting with the code as stuff might not get used in the beginning while developing and it then has to be commented out. I believe the pros clearly outweighs the cons:

• The production code is cleaner.
• It seems to me that if you are adding code that is not used yet and compiling you might be in a YAGNI situation? Do not write the code until you need it.
• The workaround is dead simple – just comment it out. Then if it turns out you are not going to need it someone will eventually delete that comment.

Interfaces

Go does not have classes. It has structs and functions can be added to them. As such there is no inheritance, which means as a language Go favors composition over inheritance since the later is not even possible. I really like this design as composition is to be preferred over inheritance in most scenarios in my opinion (99%?) even in a language that support inheritance. This also means that the way interfaces are implemented are different from for instance C#.

Interfaces are implemented implicitly, which in my mind almost feels like duke typing (except checked at compile time). Thus, if an object satisfies an interface (has the required functions) it will compile. This means that interfaces can easily be added for existing code even the main library.

Imagine what I could have done with ASP.Net HttpContext if it worked this way in C# (check my last blog post). Invent a new interface matching the few bits of HttpContext that was being used and simply pass HttpContext along. A lot more elegant that what can be done in C#.

Http Server

To be honest I have not yet worked very much with Go as an Http Server, but since it is one of the things I usually end up using extensively in a language I thought I would include an example anyways.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

package main

import (
  "fmt"
  "net/http"
)

type User struct {
  Name string
    Password string
    FavoriteColor string
}

func (u User) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "%s, %s, %s", u.Name, u.Password, u.FavoriteColor)
}

func main() {
  http.Handle("/user", &User{"MyUserName", "OhSoSecret", "green"})
  http.ListenAndServe("localhost:4000", nil)
}

I do not think that is too bad. Almost simpler than doing the same in Node, but still to early for me to say how easy it will be to implement a realistic server.

Concurrency

One of the most important features of Go is the way you write concurrent code. It consists of 2 main components being goroutines and channels. A goroutine is a lightweight thread and a channel is a way to communicate between threads.

A fairly common scenario will be goroutine(s) doing some calculation (or reading data) and another goroutine that then does some further manipulation like in this example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

package main

import (
  "fmt"
)

func getdata(resCh chan<- int) {
  i := 0
  for {
      resCh <- i
      i++
  }
}

func printer(resCh <-chan int) {
  for {
      res := <- resCh
      fmt.Println(res)
  }
}

func main() {
  resCh := make(chan int, 100)
  go getdata(resCh)
  go printer(resCh)
  var input string
  fmt.Scanln(&input)
}

A dumb example I agree, as there are easier ways of printing sequential integers to the console. The point was to showcase the way functions can be started in goroutines and use channels to communicate.

I find this a quite elegant way of doing concurrent programming, and if you want to see a slightly less contrived example take the tutorial of Go and solve the last exercise (you can find mine at my github account: https://github.com/elgsdyret/go_playground/blob/master/fetcher_2.go).

Wrap Up

With the limited experiments I have done so far I think Go has a lot going for it. Which is also why more and more companies are showing of stuff implemented in Go. For instance Mozilla just released a beta on Heka, a quite interesting project, which is written in Go (http://blog.mozilla.org/services/2013/04/30/introducing-heka/).

Next step is to play around with more areas of Go. How to write unit tests? How hard is it to write more advanced web servers?

Often in an ASP.NET WebForms application you will get up with unmanageable depencencies on the ASP.NET web stack expressed through the HttpContext class.

This often appears in the form of code like this:

1
2
3
4
5

public void DoSomeCoreStuff()
{
  var userId = (string)HttpContext.Current.Session["userId"];
  DoSomeMoreCoreStuff(userId);
}

This code could be located anywhere in you codebase, and I have seen examples of such calls to session being done more or less inline in a string describing an SQL query. It does not necessarily have to be the Session property that is accessed though, the problem is the same if Application or other bits of the current http context is accessed this way.

There is a number of problems with this approach.

• Using a string like this is not strongly typed, which easily leads to errors. For instance the session variables might have had a different casing like “CurrentUserId”.
• A static reference to HttpContext.Current causes a number of problems. Firstly, it will be hard to use the code in question outside the ASP.NET stack, and secondly it will be equally hard to write meaningful tests for the code in questions.

Options

There are a number of ways to solve this problem leading to a much cleaner design enabling testing and reuse. All the options involves pushing the actual call to HttpContext.Current as far up in the call chain as possible, meaning in the code behind of the aspx page.

Simple Types

Given that the currentUserId in our example above is an string (or any other simple type or object that can be easily constructed) why not just use that? It can be injected through the constructor or the method in question.

1
2
3
4

public void DoSomeCoreStuff(string userId)
{
  DoSomeMoreCoreStuff(userId);
}

Domain Specific Wrapper Class

Often it is not as simple as a single entry that you want to retrieve from the session. Maybe there is a group of related entries that you want to use.

1
2
3
4
5
6
7

public void DoSomeCoreStuff()
{
  var userId = (string)HttpContext.Current.Session["currentUserId"];
  var userColor = (string)HttpContext.Current.Session["favoriteColor"];
  var userNickname = (string)HttpContext.Current.Session["nick"];
  DoSomeMoreCoreStuff(userId, userColor, userNickname);
}

In that case an interface named according to the relation (for instance UserContext in our example) and injected can solve this problem. The key is to have an implementation of the interface that actually wraps Session and ensure that the string key for the entry is not copied all over the code. The wrapper implementation is instantiated at the top level of the web stack, most often in the code behind class and elsewhere we code against the interface in a inversion of control style.

1
2
3
4
5
6
7

public void DoSomeCoreStuff(UserContext userContext)
{
  var userId = userContext.Id;
  var userColor = userContext.FavoriteColor;
  var userNickname = userContext.NickName;
  DoSomeMoreCoreStuff(userId, userColor, userNickname);
}

The implementation of the interface looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

public class HttpContextUserContext : UserContext
{
  HttpContext context;

  public HttpContextUserContext(HttpContext context)
  {
      this.context = context;
  }

  public string Id
  {
      get
      {
          return (string)context.Session["currentUserId"];
      }
  }

  public string FavoriteColor
  {
      get
      {
          return (string)context.Session["favoriteColor"];
      }
  }

  public string NickName
  {
      get
      {
          return (string)context.Session["nick"];
      }
  }
}

It might seem like a lot of code, but the flexibility is definitely worth the small overhead of adding this.

HttpContextBase

The main problem with the original implementation of the HttpContext class is that it is not based on an interface nor an abstract base class. This means that reusing code that depends on HttpContext can be hard, and testing equally hard. Microsoft realized this problen when they released the MVC framework and added a class called HttpContextBase. The funny thing is that HttpContext does not inherit from HttpContextBase as that would be a breaking change, but it has the exact same API.

Thus, our problem from above with the static dependency could be solved as the following stepwise refactoring:

• Ensure that HttpContext is injected through the constructor or the method in question and ensure that HttpContext.Current is passed in as that value.
• Replace HttpContext in the constructor or method with HttpContextBase and replace HttpContext.Current with new HttpContextWrapper(HttpContext.Current)

And our example would then look something like this:

1
2
3
4
5

public void DoSomeCoreStuff(HttpContextBase context)
{
  var userId = (string)context.Session["userId"];
  DoSomeMoreCoreStuff(userId);
}

This might seem a bit overkill, but we are now able to use our method with a different implementation of HttpContextBase for reuse and testing purposes.

When doing a different implementation you should know that every base implementation in HttpContextBase just throws a NotImplementedException. Thus, you will need to override that different bits in your test and reuse implementations depending on what you really require. Alternatively, there are a couple of generic open source implementations available if you want to pull in that extra dependency.

In general I would recommend using one of the first approaches, but if you are working with a fairly large system that adheres to the big ball of mud “design” antipattern this might be difficult. Maybe it is just to difficult to discern any meaningful relation between the different data retrieved from HttpContext, and with this approach you can at least start writing some tests to get control back of the code.

As I mentioned earlier this is just one simple example as HttpContext have a lot of data and functionality built in.s

I have not yet had a chance to play around with the dynamic features that was introduced with .NET 4, probably because I have yet to find a good scenario for using it and probably because I am mostly stuck on doing work on the .NET 3.5.

Fortunately, I am currently working on a new REST API build in Nancy and the dynamic features of .NET 4 seems perfect to implement HTTP PATCH.

Our way of thinking is that it is hard to model with traditional classes as not all properties are necessarily included. One solution could be to make them all nullable and use that, but then it would not be possible for us to distinguish between a property simply not in the body of the request and a property the consumer deliberately set to null.

My first test was like this:

1
2
3
4
5

public void DynamicValueNoneSet()
{
  dynamic a = new ExpandoObject();
  string name = a.Name;
}

This showed me that I would get a RunTimeBinderException if a given value was not available (yes I could also have read the documentation, but how much fun is that?).

My next attempt included this little implementation of DynamicObject – to avoid getting the exception when using it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public class NullDynamic : DynamicObject
{
  private readonly IDictionary<string, object> expando;
  
  public NullDynamic()
  {
      expando = new ExpandoObject();
  }

  public override bool TryGetMember(GetMemberBinder binder, out object result)
  {
      expando.TryGetValue(binder.Name, out result);
      return true;
  }

  public override bool TrySetMember(SetMemberBinder binder, object value)
  {
      expando[binder.Name] = value;
      return true;
  }
}

Since the declared goal is to be able to distinguish between a value not being there and deliberately being set to null, this would not work either.

Actually what I found myself thinking was that I missed the “undefined” value from javascript, or some other way to express that something was not there, built into the language.

Trying to avoid going to such drastic measure I came up with this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

public class Dynosaur: DynamicObject
{
  private readonly IDictionary<string, object> expando;

  public Dynosaur()
  {
      expando = new ExpandoObject();
  }

  public override bool TryGetMember(GetMemberBinder binder, out object result)
  {
      return expando.TryGetValue(binder.Name, out result);
  }

  public override bool TrySetMember(SetMemberBinder binder, object value)
  {
      expando[binder.Name] = value;
      return true;
  }

  public bool HasValue(string name)
  {
      return expando.Keys.Contains(name);
  }
}

Maybe not the most elegant solution, but I does allow me to write code like:

1
2
3
4
5
6
7
8

public void PrintName(dynamic a)
{
  if (!a.HasValue("Name"))
  {
      a.Name = "dog";
  }
  Console.WriteLine("the name is: " + a.Name);
}

Given that “a” is actually an instance of my Dynosaur class. I would not say that I am entirely satisfied with this implementation, but I actually think that this is as far as the dynamic features of C# will take me.

Any good ideas out there? I am thinking there must be other people that has played around with this and has come up some crazy dynamic stuff in C#.

The devops “thingy” is only a few years old. I use the word “thingy” (if it can be considered a word) as it is seemingly hard to describe precisely what exactly devops is.

I am new to devops, but went to devops days in London anyways. This is my attempt to clear my head of some of the thoughts, ideas, and impressions that have been rambling around my head for these two days, and probably will for quite some time.

I had a blast of a time in London listening to the different speakers and participating in discussions, but I must admit that I am even more confused now than I was before coming. Nevertheless, I did get some impressions:

1. Getting back to my opening sentence, the notion that has struck me as being one of the most important is that devops might not be meant to be described precisely, but is a continuous movement towards a better way of delivering value for the end customer (ups… I just did my own attempt at a description). I have heard several other suggestions during these days and it seems that each individual has his own definition of devops. I had the pleasure of talking with Patrick Debois over a beer about this and he hinted that this was actually on purpose. Maybe the journey is more important than the goal and a large part of the value comes from defining what devops means to you? What is your definition?
2. You can get some value with tools and processes, but if you do not get the people aboard you will not get to the end goal (which seems to be as hard to define as the definition).
3. Changing culture seems to be most powerful pattern for doing devops and maybe the only really relevant one.
4. There are a zillion of tools out there that I had never heard about and I am going to try out. Seems like everybody is using (have used at some point) logstash, graphite and so forth.
5. The people that do devops are awesome and passionate about it, which makes me want to learn more and participate in more devops days (Copenhagen next year anyone?)

Cheers!

Over time JavaScript has gained a lot of traction as it is the default option for running code in the browser, and it has become a valid server option with the rise of node. This progress has largely been fueled by an arms race between the major browser vendor continuously trying to wield the fastest environment for running JavaScript.

Thus, as JavaScript can be run everywhere and with the notion that this language like any other has its rough spots (partly due to the history of it’s creation, a notion supported by the famous book by Douglas Crockford “JavaScript the Good Parts”) it is no wonder that alternatives that compiles to JavaScript have been popping up for a while.

Microsoft recently released TypeScript to join the efforts of Google with their Dart language to define the future of code running in the browser. Simultaneously, initiatives like CoffeeScript (by J Ashkenas) has gained some traction and the community is working (fighting) to get the next version of JavaScript ready. Another option that I cannot leave out is ClojureScript a version of the brilliant Clojure language that compiles to JavaScript.

The options are abundant each with their own goals. Some like TypeScript and Dart aim to bring some form of static typing to the table (I do not specifically want this in JavaScript, but I still welcome these initiatives), while a language such as CoffeeScript aim to improve the syntax and hide some of the bad parts of JavaScript.

J Ashkenas (of BackBone, Underscore and CoffeeScript fame) maintains a quite comprehensive list (https://github.com/jashkenas/coffee-script/wiki/List-of-languages-that-compile-to-JS), and for a good read on the different aspect of languages compiling to JavaScript consider: http://buildnewgames.com/compiling-to-javascript/

Indeed it is a very good time to be a web (Node) programmer. Some notable JavaScript engineers (Such as N. Zakas) claim that time would be better spent improving JavaScript and teaching it than building alternatives, but I believe that every new language will bring inspiration to developers and to the future of JavaScript.

All of the languages compiling to JavaScript has a shared number of cons:

• Build process: Because they actually compile to JavaScript. Any serious JavaScript project will have a build process anyways, combining files, minifying, obfuscating, possibly running test suites automatically, and thus I do not see a big problem with this.
• Debugging: Since what you are sending to the browser is JavaScript your debugging will also be in JavaScript. This means that unless you can get source map to work http://addyosmani.com/blog/the-breakpoint-episode-3-source-maps-shortcut-secrets-and-jsrun, to use one of these languages you need to be fluent in JavaScript. Definitely, a challenge for some, but for experienced JavaScript developers this might not be a big problem (on a side note I believe that CoffeeScript for instance generates quite nice JavaScript).

CoffeeScript

I have spend some energy on CoffeeScript as an alternative (or in addition to!) to JavaScript on my projects. I believe CoffeeScript brings readability and as a consequence of this also maintainability to a project. I cannot stress enough how important readability is for any piece of code and as such doing CoffeeScript to improve that becomes a very intriguing option.

One of the features about CoffeeScript that i most often hear complaints against is meaningful indentation, and it was also something I personally was not sure I would like. Nevertheless, the more I think about it, I am for using the indentation, as this is the way I consume code more than it is the braces. Why should the compiler use a different mechanism than me for interpreting code? Is it not easier to maintain code where your understand the code the same way the compiler does?

Here is just a few examples of features I believe adds readability to code:

Function syntax:

1

var a = function() { return x*x; };

1

a = -> x*x

This especially becomes valuable if you like me use a lot of underscore.js functionality:

1

_([1, 2, 3]).each(function(num){ alert(num); });

1

_([1, 2, 3]).each((num) -> alert(num));

Default values:

1
2
3
4
5
6

function(a, b) {
  if (!b) {
      b = 'b';
  }
  console.log(a, b);
}

In CoffeeScript you could do:

1

(a, b = 'b') -> console.log(a, b)

Basically, CoffeeScript is full of features that reduces the cluttering of your code removing unnecessary noise and thereby improving readability.

I could whip up a ton more examples, but instead if will just point you to http://coffeescript.org/ where you can read more if your interested. No need for me to copying all the examples :-D

Recently, I converted a JavaScript project to CoffeeScript and I litterally ended up removing one third of all the characters just in the conversion. Afterwards, I spends some time using some of the nice features such as default values for parameters and splats, and consequently was able to reduce the amount of code even more.

It is indeed a good time to be a polyglot JavaScript developer and I cheer every single time a new language can compile to JavaScript – bring it on!

The last couple of days I have been trying to implement a pure javascript client that could connect to our SOAP API.

To do that we need to enable Cross-origin resource sharing (called CORS for short). Basically, as a security measure browsers implement Same origin policy ensuring that only programs loaded from the same domain can query resources on that domain.

Thus, JavaScript loaded from http://www.example.com/myprogram.html could query http://www.example.com/api, but no http://www.test.com/api.

As Rich Internet Applications written as javascript clients is seeing an increase in popularity (and ease of implementing due to lots of nice frameworks such as Backbone), more and more providers of services will want to expose those services directly to the clients.

To enable CORS we need to set a series of headers and respond to the HTTP OPTIONS verb request (used by the browser to figure out if the server actually supports CORS). To be honest I have yet to figure out all the headers and exactly why each one is required, but with some trial and error I have gotten it to work.

Allowing CORS from ASP.NET SOAP WebService

The first challenge for our API was to get our ASP.NET SOAP service to set the correct headers (which I just admitted to not knowing…) for only that service.

It turned out this is actually not that hard and throwing in a HttpModule setting the headers for requests to a specific url (the service) with a lot of random (more or less) headers actually enabled me to invoke the service from javascript.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

public void AddCors(HttpApplication httpApplication)
{
  var sb = new StringBuilder();
  var hdrs = httpApplication.Response.Headers;
  var origin = hdrs.Get("Origin");
  if (string.IsNullOrEmpty(origin))
  {
      origin = "*";
  }
  hdrs.Add("Access-Control-Allow-Origin", origin);
  hdrs.Add("Access-Control-Allow-Credentials", "true");
  sb.Append("Content-Type, Cake");

  if (httpApplication.Request.HttpMethod.ToUpper() != "OPTIONS")
  {
      hdrs.Add("Access-Control-Allow-Headers", sb.ToString());
      return;
  }

  hdrs.Add("Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,OPTIONS");
  sb.Append(",authorization,Content-Type,accept");
  sb.Append(",accept-charset,accept-encoding,accept-language");
  sb.Append(",connection,content-length,content-type");
  sb.Append(",host,origin,referer,user-agent, SOAPAction");
  hdrs.Add("Access-Control-Allow-Headers", sb.ToString());
  hdrs.Add("Access-Control-Max-Age", "3000"); //seconds
  hdrs.Add("content-length", "0");
  httpApplication.Response.StatusCode = 204;
}

Consuming SOAP envelopes from JavaScript

I tried out a number of promising leads for consuming SOAP services. The common ground for all of them was that they used the XmlHttpRequest object internally and tried to generate a proxy based on the wsdl of the service. After a couple of tries I gave up on this approach and went closer to the metal.

Using fiddler and an integration that worked (written as a server) I spied on the SOAP envelopes needed to send the requests, and saved them as templates (in the format for Mustache.js). This way I could easily put in the values needed without having to worry about serializing object and manipulating XML. The response was simply parsed using regex’ and thus I was able to handle the envelopes.

Actually, this way of handling although primitive is not half bad. If the service changes the service contract the template/regex has to be updated, but code generated on top of the wsdl would suffer from the same, and the implementation is extremely simple and basic this way.

The last bit was sending the requests to the server which was done through jQuery and the XmlHttpRequest. This took me some fiddling (not using fiddler) and googling to find the exact parameters and such. Again I used fiddler to find the correct headers for the requests and added them using jQuery.

Using ASP.NET Cookie SessionState from JavaScript

Thus, I was able to log in to our web service and retrieve the key used for subsequent request as an access token (actually the ASP.NET session id).

Unfortunately, the web service uses ASP.NET session to maintain whether you are logged in or not. Thus, a Set-Cookie header is returned from the Connect method and that Cookie must be returned in the header of the following requests to ensure access.

Due to security measures it is not possible to add the Cookie header manually to the XmlHttpRequest object. As the session id is also returned in the SOAP response envelope it is not a problem to retrieve it and I managed it to send it in another header called CAKE (part of the explanation for the many weird headers in the CORS implementation above).

Thus, I was stuck with changing the service implementation (not relying on ASP.NET session or somehow hijacking it using my CAKE header) or forcing the potential consumers of our API to use one of the potential hacks using iframes or JSONP (which I have not tested), but at least the last option defeats the purpose of making it easy to access (one could ago that SOAP does the same, but that is another battle).

What is the point you might ask? I actually managed to implement CORS, but due to the implementation using ASP.NET Session it is completely useless.

Anyone has any hacks worth trying?

Next step…. Something more usable than SOAP……. Not using ASP.NET Session