The terrific Garbage Collector in the .NET runtime is a compacting one. This means it moves memory blocks in use closer to each other during garbage collection, thus increasing the overall size of contiguous free memory and lessening the chance of running out of memory due to heap fragmentation.
This is pretty important for memory intensive applications as it is a nice guarantee that free contiguous memory will always be available whenever a large object needs to be allocated.

Whenever you perform send or receive socket operations in .NET, the buffer that holds the transmitted or received data is pinned by the .NET runtime. Pinning basically means that the region in memory that holds the data is locked down and is not eligible to be moved around by the Garbage Collector. This is necessary so that the Windows Socket API which handles the actual socket operation (and lives outside managed code) can access the buffer memory locations reliably. This pinning takes place regardless of if the operation is synchronous or asynchronous.

If you are building a memory intensive server application that is expected to serve thousands of clients concurrently, then you have a simmering problem waiting to blow up as soon as the server starts serving a non-trivial number of clients.
Thousands of sockets performing receive and/or send operations will cause thousands of memory locations to be pinned. If your server stores a lot of data in memory, you can have a fragmented heap that even though collectively has enough free memory to store a large object, is unable to do so, because there isn’t enough memory in a contiguous free block to store it. This is illustrated below: 

 
The diagram above depicts the common scenario where a large object is about to be allocated in memory. There is insufficient contiguous memory to store the object, which triggers garbage collection. During garbage collection, the memory blocks in use are compacted, freeing enough room to store the large object. 

 
The diagram above depicts the same scenario with some objects pinned. The garbage collector compacts objects that are not pinned which makes some room, but not enough room due to the unmovable pinned objects. The end result is that a nasty Out Of Memory exception is thrown by the runtime because there is no contiguous free memory space large enough for the large object to be allocated.

This problem is discussed at length in this blog post and in this blog post. The solution to the problem, as noted in those posts, is to pre-allocate buffers. This basically means setting up a byte array which your sockets will use to buffer data. If all your socket operations use buffers from the designated byte array, then only memory locations within the region where the byte array is stored will be pinned, giving the GC greater freedom to compact memory.
However, for this to work effectively with multiple asynchronous socket operations, there has to be some kind of buffer manager that will dish out different segments of the array to different operations and ensure that buffer segments allotted for one operation are not used by a different operation until they are marked as no longer in use.

I searched for existing solutions for this problem but I couldn’t find any that was sufficiently adequate, so I wrote one that was robust and dependable. The solution is part of my ServerToolkit project, which is a set of useful libraries that can be used to easily build scalable .NET servers. The solution is called BufferPool and is the first ServerToolkit sub-project.

The idea is that you have a buffer pool that is a collection of memory blocks (byte arrays) known as slabs that you can grab buffers from. You decide the size of the slab and the initial number of slabs to create.
As you request buffers from the pool, it internally allocates segments within the slabs for use by your buffer. If you need more buffers than are available within the pool, new slabs are created and are added to the pool. Conversely, as you dispose your buffers, slabs within the pool will be removed when they no longer contain any allocated segments.  
 

USING BUFFER POOLING IN YOUR APPLICATION

Download the code from http://github.com/tenor/ServerToolkit/tree/master/BufferPool

Compile it to a DLL and reference the DLL from your project
OR
Add the project to your solution and reference the ServerToolkit.BufferManagement project from your main project.

In the code file that performs socket operations, add

using ServerToolkit.BufferManagement;

Create a buffer pool when your application starts with

BufferPool pool = new BufferPool(1 * 1024 * 1024, 1, 1);

This creates a pool that will have 1 MB slabs with one slab created initially. Subsequent slabs, if needed, will be created in increments of one.

It is advisable to create the pool as soon as your server application starts or at least before it begins any memory-intensive operations.

To use the pool in synchronous send and receive socket operations

/** SENDING DATA **/

// const int SEND_BUFFER_SIZE is the desired size of the send buffer in bytes 
// byte[] data contains the data to be sent.

using (var buffer = pool.GetBuffer(SEND_BUFFER_SIZE)) 
{ 
    buffer.CopyFrom(data); 
    socket.Send(buffer.GetSegments()); 
}


/** RECEIVING DATA **/

// const int RECEIVE_BUFFER_SIZE is the desired size of the receive buffer in bytes 
// byte[] data is where the received data will be stored.

using (var buffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE)) 
{ 
    socket.Receive(buffer.GetSegments()); 
    buffer.CopyTo(data); 
}

To use the pool for asynchronous send and receive socket operations

Sending data:

/** SENDING DATA ASYNCHRONOUSLY **/

// const int SEND_BUFFER_SIZE is the desired size of the send buffer in bytes 
// byte[] data contains the data to be sent.

var buffer = pool.GetBuffer(SEND_BUFFER_SIZE); 
buffer.CopyFrom(data); 
socket.BeginSend(buffer.GetSegments(), SocketFlags.None, SendCallback, buffer);

//...


//In the send callback.

private void SendCallback(IAsyncResult ar) 
{ 
    var sendBuffer = (IBuffer)ar.AsyncState; 
    try 
    { 
        socket.EndSend(ar); 
    } 
    catch (Exception ex) 
    { 
        //Handle Exception here 
    } 
    finally 
    { 
        if (sendBuffer != null) 
        { 
            sendBuffer.Dispose(); 
        } 
    } 
}

Receiving data:

/** RECEIVING DATA ASYNCHRONOUSLY **/

// const int RECEIVE_BUFFER_SIZE is the desired size of the receive buffer in bytes. 
// byte[] data is where the received data will be stored.

var buffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE); 
socket.BeginReceive(buffer.GetSegments(), SocketFlags.None, ReadCallback, buffer);

//...


//In the read callback

private void ReadCallback(IAsyncResult ar) 
{ 
    var recvBuffer = (IBuffer)ar.AsyncState; 
    int bytesRead = 0;

    try 
    { 
        bytesRead = socket.EndReceive(ar); 
        byte[] data = new byte[bytesRead > 0 ? bytesRead : 0];

        if (bytesRead > 0) 
        { 
            recvBuffer.CopyTo(data, 0, bytesRead);

            //Do anything else you wish with read data here. 
        } 
        else 
        { 
            return; 
        }

    } 
    catch (Exception ex) 
    { 
        //Handle Exception here 
    } 
    finally 
    { 
        if (recvBuffer != null) 
        { 
            recvBuffer.Dispose(); 
        } 
    }

    //Read/Expect more data                    
    var buffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE); 
    socket.BeginReceive(buffer.GetSegments(), SocketFlags.None, ReadCallback, buffer);

} 

There is a performance penalty involved when using the code above. Each time you perform a socket operation, you create a new buffer, use it and then dispose it. These actions involve multiple lock statements, and can become a bottleneck on the shared pool.

There is an optimization that can help mitigate this issue. The optimization is based on the way socket operations work. You can send data any time you like on a socket, but you typically receive data in a loop. i.e. receive data, signal that you are ready to receive more data and repeat. This means that each socket works with only one receive buffer at a time.
With this in mind, you can assign each socket a receive buffer of its own.  Each socket will use its receive buffer exclusively for all its receive operations, thereby avoiding the need to create and dispose a new buffer for each receive operation.
You have to dispose the receive buffer when you close the socket.

I recommend the pattern below.

/** RECEIVING DATA ASYNCHRONOUSLY (OPTIMIZED) **/

// const int RECEIVE_BUFFER_SIZE is the desired size of the receive buffer in bytes.

// const int BUFFER_SIZE_DISPOSAL_THRESHOLD specifies a size limit, which if exceeded by a buffer
// would cause the buffer to be disposed immediately after a receive operation ends.
// Its purpose is to reduce the number of large buffers lingering in memory.

// IBuffer recvBuffer is the receive buffer associated with this socket.

// byte[] data is where the received data will be stored.

// object stateObject holds any state object that should be passed to the read callback. 
// It is not necessary for this sample code.

if (recvBuffer == null || recvBuffer.IsDisposed)
{
    //Get new receive buffer if it is not available.
    recvBuffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE);
}
else if (recvBuffer.Size < RECEIVE_BUFFER_SIZE)
{
    //If the receive buffer size is smaller than desired buffer size,
    //dispose receive buffer and acquire a new one that is long enough.
    recvBuffer.Dispose();
    recvBuffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE);
}

socket.BeginReceive(recvBuffer.GetSegments(), SocketFlags.None, ReadCallback, stateObject);

//...


//In the read callback

private void ReadCallback(IAsyncResult ar)
{
    int bytesRead = socket.EndReceive(ar);

    byte[] data = new byte[bytesRead > 0 ? bytesRead : 0];

    if (recvBuffer != null && !recvBuffer.IsDisposed)
    {
        if (bytesRead > 0)
        {
            recvBuffer.CopyTo(data, 0, bytesRead);

            //Do anything else you wish with read data here.
        }

        //Dispose buffer if it's larger than a specified threshold
        if (recvBuffer.Size > BUFFER_SIZE_DISPOSAL_THRESHOLD)
        {
            recvBuffer.Dispose();
        }
    }

    if (bytesRead <= 0) return;

    //Read/Expect more data
    if (recvBuffer == null || recvBuffer.IsDisposed)
    {
        //Get new receive buffer if it is not available.
        recvBuffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE);
    }
    else if (recvBuffer.Size < RECEIVE_BUFFER_SIZE)
    {
        //If the receive buffer size is smaller than desired buffer size,
        //dispose receive buffer and acquire a new one that is long enough.
        recvBuffer.Dispose();
        recvBuffer = pool.GetBuffer(RECEIVE_BUFFER_SIZE);
    }

    socket.BeginReceive(recvBuffer.GetSegments(), SocketFlags.None, ReadCallback, stateObject);

}

If using the pattern presented above, it’s important to remember to dispose the buffer when closing the socket.
You must explicitly dispose the buffer. It does not define a finalizer, so once a buffer is created, it stays allocated until explicitly disposed.

CONCLUSION

The BufferPool project is the first sub-project of the ServerToolkit project. It is an efficient buffer manager for .NET socket operations.
It is written in C# 2.0 and targets .NET 2.0 for maximum backward compatibility.

BufferPool is designed with high performance and dependability in mind. Its goal is to abstract away the complexities of buffer pooling in a concurrent environment, so that the developer can focus on other issues and not worry about heap fragmentation caused by socket operations.

In my previous post, I described the role of System.Void in the .NET framework and demonstrated some of the restrictions placed on the type.
In this post, I’ll postulate why those restrictions are in place and I’ll try to bypass them in an attempt to create an instance of the type.

The restrictions on the System.Void type are documented in the Common Language Infrastructure (CLI) specification (ECMA-335).
I find two of these restrictions particularly interesting:

“The type System.Void is never boxable.” (Section 8.2.4)
“No location or value shall have System.Void” (Section 8.7)

Why is instantiation of the System.Void type forbidden? After all, as long as methods are not allowed to declare a return type of System.Void, the purpose of the type as discussed in my previous post remains intact, regardless of whether instances are allowed to exist or not.

So, what’s the harm in permitting instances of this type to exist? I proffer the following explanations.

Firstly, from a philosophical standpoint, it doesn’t make much sense to have the ability to instantiate a type that methods cannot declare as a return type, because that is effectively saying that you can create this object but your methods cannot pass its value (unless in a boxed representation or masqueraded as an inherited type) around.

Secondly, some unexpected issues can arise when calling methods declared in generic types. For example, if developers can instantiate System.Void, they’ll expect to be able to compile and run the following code, but by so doing produce a method that has a System.Void return type.

class Program
{
	static void Main(string[] args)
	{
		var instance = new MyClass<System.Void>().GetInstance();			
	}
	
}

class MyClass<T> where T:new()
{
	public T GetInstance()
	{
		return new T();
	}
}

The compiler/runtime can be updated to prevent such code from compiling or running, however it’s cleaner to have a rule that states that System.Void cannot be instantiated and that the expression “System.Void” is illegal anywhere in C#.

The last explanation I proffer has to do with pointers in C#, and I think it is the most functional explanation.
You can get the System.Type object of a value type pointer with the following expression:

typeof(MyStruct*)

For example:

typeof(int*)

evaluates to a System.Type object called System.Int32*

Similarly, typeof(bool*), typeof(DateTime*), typeof(char*) and typeof(System.Guid*) will evaluate to System.Type objects called System.Boolean*, System.DateTime*, System.Char* and System.Guid* respectively.

Now, there is a special kind of pointer called the void pointer which can point to any pointer type.
The type declaration for a void pointer is void*, and the following expression gets the System.Type object of a void pointer:

typeof(void*)

It evaluates to a System.Type object called System.Void*

You can already see how an ambiguity is introduced if instances of System.Void were allowed to exist.
In that scenario, typeof(System.Void*) would evaluate to a System.Type object also called System.Void*.
It would be impossible (or at least error-prone) to tell if the object is for a pointer to System.Void or for the special void pointer.

Nevertheless, and in spite of my failed attempts to instantiate System.Void in the previous post, it is possible to create an instance of System.Void -- by making it a static field.

// Metadata version: v2.0.50727
.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )
  .ver 2:0:0:0
}
.assembly ConsoleApplication1
{
  .hash algorithm 0x00008004
  .ver 1:0:0:0
}
.module ConsoleApplication1.exe
// MVID: {60F1E74C-3123-4EF0-8269-A0C3B8B85ADF}
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003       // WINDOWS_CUI
.corflags 0x00000001    //  ILONLY
// Image base: 0x00670000


// =============== CLASS MEMBERS DECLARATION ===================

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .field public static valuetype [mscorlib]System.Void o
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       1 (0x1)
    .maxstack  8
    IL_0000:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

The IL code listing above is equivalent to the following C# program (if it were possible to compile)

class Program
{
	public static System.Void o;
	static void Main()
	{
	}
}

If I compile the IL code using ilasm and run it, I don’t observe any errors, unlike in my previous attempts.
What does this mean? Did it create an instance of the type? Is the runtime skipping over that System.Void field creation instruction? Is the program dying silently?

This is a good example of the "If a tree falls in a forest and no one is around to hear it, does it make a sound?" philosophical riddle.

What I need is some observability. I need to be absolutely sure that the type was instantiated (or not).

I’ll just modify the code a little bit to get some output:

class Program
{
	public static System.Void o;
	static void Main()
	{
		Console.WriteLine(o.ToString());

	}
}

which translates to the following IL code:

// Metadata version: v2.0.50727
.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )
  .ver 2:0:0:0
}
.assembly ConsoleApplication1
{
  .hash algorithm 0x00008004
  .ver 1:0:0:0
}
.module ConsoleApplication1.exe
// MVID: {EE60F7FD-BD3D-46C8-B353-A459F5119496}
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003       // WINDOWS_CUI
.corflags 0x00000001    //  ILONLY
// Image base: 0x00260000


// =============== CLASS MEMBERS DECLARATION ===================

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .field public static valuetype [mscorlib]System.Void o
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       22 (0x16)
    .maxstack  8
    IL_0000:  ldsflda    valuetype [mscorlib]System.Void Program::o
    IL_0005:  constrained. [mscorlib]System.Void
    IL_000b:  callvirt   instance string [mscorlib]System.Object::ToString()
    IL_0010:  call       void [mscorlib]System.Console::WriteLine(string)
    IL_0015:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

If I run the program, I get a fatal InvalidProgramException error message. The issue is that ToString() is defined on the base System.Object class. When I call System.Void.ToString(), the runtime boxes the value type to an object before calling the ToString() method.
Remember that rule which states that System.Void cannot be boxed. It is enforced by the runtime. There is a check that the value is not System.Void during boxing, and if the check fails, the program is halted.

On a side note, this is another good reason to avoid boxing whenever possible. There is some overhead, besides copying the value to the boxed representation, due to many type checks.

System.Void does not define any members that we can access. The other members (GetType(), ToString(), GetHashCode(), etc.) are all inherited. Calling the inherited methods will lead to boxing which is prohibited. How can we observe its existence?

There’s one way.

// Metadata version: v2.0.50727
.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )
  .ver 2:0:0:0
}
.assembly MultiVoid
{
  .permissionset reqmin
             = {[mscorlib]System.Security.Permissions.SecurityPermissionAttribute = {property bool 'SkipVerification' = bool(true)}}
  .hash algorithm 0x00008004
  .ver 1:0:0:0
}
.module MultiVoid.exe
// MVID: {AC411515-B789-4594-B52B-1EC2668D16C9}
.custom instance void [mscorlib]System.Security.UnverifiableCodeAttribute::.ctor() = ( 01 00 00 00 ) 
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003       // WINDOWS_CUI
.corflags 0x00000001    //  ILONLY
// Image base: 0x00BC0000


// =============== CLASS MEMBERS DECLARATION ===================

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .field public static valuetype [mscorlib]System.Void o
  .field public static valuetype [mscorlib]System.Void p
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       35 (0x23)
    .maxstack  1
    .locals init (valuetype [mscorlib]System.Void& pinned V_0,
             valuetype [mscorlib]System.Void& pinned V_1)
    IL_0000:  ldsflda    valuetype [mscorlib]System.Void Program::o
    IL_0005:  stloc.0
    IL_0006:  ldloc.0
    IL_0007:  conv.i
    IL_0008:  conv.i4
    IL_0009:  call       void [mscorlib]System.Console::WriteLine(int32)
    IL_000e:  ldc.i4.0
    IL_000f:  conv.u
    IL_0010:  stloc.0
    IL_0011:  ldsflda    valuetype [mscorlib]System.Void Program::p
    IL_0016:  stloc.1
    IL_0017:  ldloc.1
    IL_0018:  conv.i
    IL_0019:  conv.i4
    IL_001a:  call       void [mscorlib]System.Console::WriteLine(int32)
    IL_001f:  ldc.i4.0
    IL_0020:  conv.u
    IL_0021:  stloc.1
    IL_0022:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

The IL code above translates to the following uncompilable C# code:

class Program
{
	public static System.Void o;
	public static System.Void p;
	static void Main()
	{
		unsafe
		{

			fixed (System.Void* i = &o)
			{
				Console.WriteLine((int)i);
			}

			fixed (System.Void* j = &p)
			{
				Console.WriteLine((int)j);
			}

		}

	}
}

It displays the addresses of the System.Void instances in memory: If you run the IL code, you’ll see an output with the memory addresses of the two instances like as shown below:

Congratulations! You just created two instances of the forbidden System.Void type!

That may be as far as you may go. The runtime will detect attempts to place the instance on the evaluation stack, which is necessary to return the instance value from a method.

When calling a method in a dynamically created instance of a type that is discovered at runtime, it's useful to discover the types of the method parameters and the method return type. This information is needed to successfully call the method and retrieve its return value.

It's also useful to discover if the method doesn't have a return type, so that we know not to expect a return value.
One way to find out is to check if the MethodBase.Invoke call returns a null, since a dynamically invoked method that has no return type will return a null. However, this is not the best approach for the following reasons.

Firstly, any method that returns a reference type can return a null, rendering such null checks unreliable.

Secondly, this approach cannot be used to determine the return type before invoking the method.

Fortunately, MethodInfo objects have a ReturnType property that developers can inspect to determine the return type before invoking the method. For instance, if the method returns a string type, MethodInfo.ReturnType will evaluate to typeof(string). It would seem that the natural way to determine if a method does not define a return type would be to check if MethodInfo.ReturnType == typeof(null), however, that won't work because the expression typeof(null) is illegal in C#. Null is not a type.

What's needed is a specially marked type that can be used in the typeof() evaluation to determine that the method does not have a return type. Think about that for a few seconds to get a feel of how counter-intuitive that sounds.
The issue with this approach is that if a method returns an instance of the proposed special type, then the test becomes inconclusive and unreliable. To prevent this from happening, the runtime must bar creation of methods that return this special type.

Enter the System.Void structure. This type is specially designated by .NET for just this purpose. In C#, the void keyword is an alias for the System.Void structure, thus MethodInfo.ReturnType == typeof(void) is a reliable means of discovering if a method has a return type. As expected, no method is allowed to define System.Void as its return type.

This solution looks good and seems like a perfect answer to the "no return type detection" problem. However, there may be a lot more involved. C# and the CLR strangely takes it one step further by disallowing instantiation of the System.Void type by any means. Allow me to demonstrate:

Attempt 1: Try to instantiate System.Void normally

static void Main()
{
    object o = new System.Void();
    //object o = new void() is illegal in C# lingo
}

You'll get a "System.Void cannot be used from C# -- use typeof(void) to get the void type object" (CS0673) error message if you try to compile the code.

Attempt 2: Try to instantiate System.Void dynamically

static void Main(string[] args)
{

    object o = Activator.CreateInstance(typeof(void));
            
}	

It compiles, but when you try to run it, you get a "Cannot dynamically create an instance of System.Void." error message.

Attempt 3: Bypass the C# compiler, Try to create it directly from IL

// Metadata version: v2.0.50727
.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )
  .ver 2:0:0:0
}
.assembly ConsoleApplication1
{

  .hash algorithm 0x00008004
  .ver 1:0:0:0
}
.module ConsoleApplication1.exe
// MVID: {47CAF74F-C24A-400E-A0F9-26EB27500120}
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003       // WINDOWS_CUI
.corflags 0x00000001    //  ILONLY
// Image base: 0x00D10000


// =============== CLASS MEMBERS DECLARATION ===================

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .field public static valuetype [mscorlib]System.Void o
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       12 (0xc)
    .maxstack  8
    IL_0000:  ldsflda    valuetype [mscorlib]System.Void Program::o
    IL_0005:  initobj    [mscorlib]System.Void
    IL_000b:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

The IL code above is equivalent to the following C# code

using System;

class Program
{
	public static System.Void o;
	static void Main()
	{
		o = new System.Void();
	}
}

If you compile the IL code with ilasm, It compiles successfully, however it fails verification when you run Peverify against the compiled assembly. If you try to run the assembly, it runs into a fatal InvalidProgramException exception.
If you replace occurrences of System.Void in the IL code to any other struct in the System namespace (besides the ones that map to simple types -- they have a different syntax in IL), the IL code will compile and run smoothly.

From the examples shown above, we have established that the C# compiler and runtime prevents instantiation of the System.Void type. In my next post, We'll explore the System.Void type some more and try to figure out why the runtime doesn't want it to exist.

Happy new year!!

Introduction

A common question that comes up when setting up a web application to use the state server is “what happens if the state server fails?” and the answer is that the web application fails.
This article proposes a failover solution, such that if one state server goes down, the web application switches to another one. In addition, the solution performs state server load balancing by distributing requests across available state servers.

Overview

The proposed failover system monitors a specified list of state servers to determine which ones are running; the web application can then decide on which one to use. The process of monitoring a state server is expensive, so it is handled by a dedicated external service. This service notifies the web application (and other web applications) when state servers come online or go offline. The process is illustrated below.

How it works

The failover system comprises two parts.

The first part is the monitoring service, which polls the status of a given list of servers by simply connecting and disconnecting to the servers continuously. If there are any changes in the availability of servers, for instance, a server that was previously unavailable becomes available or vice versa; one or more status files are updated to reflect the change. A status file contains information about the state servers in use and their online status. ASP.NET applications can detect changes to these status files and react accordingly.

The monitoring service has a configurable time period, within which a monitored server that comes online must stay online before the service will update the status files with the change in status for that server. This helps reduce connections to servers that are successively coming online and going offline -- the so called flapping server phenomenon. The length of this time period is determined by the ServerWarmUpTime configuration setting.
It's important to note that the monitoring service can detect the availability of other types of servers, not just ASP.NET state servers, and so can be used for other purposes.

The second part consists of configuration settings and code in the web application that expands upon Maarten Balliauw's most excellent series on state server partitioning and load balancing. The web application is configured such that the configuration file is extended to the external status file. Changes made to the status file cause the application configuration settings to be re-read and updated with the new values. Thus, the web application always has the latest server availability information and uses the information to distribute requests to available state servers -- achieving both load balancing and failover support.

For example, if there are five state servers in use and they are all running, the status file will indicate that the five servers are available; the web application then distributes state server requests evenly across the five servers. If two of the state servers were to suddenly go down, the status file will be updated to indicate that only three state servers are available; the web application then redistributes state server requests to only those three. The overall effect is that during a state server downtime, users will be able to continue using the application. Some sessions will be lost, but that is a slight annoyance compared to the entire application going down for a long period of time.

To use this load-balanced, failover-supported setup, the web application needs a few configuration changes and two code files in the App_Code folder; namely ServerListSectionHandler.cs and PartitionResolver.cs. ServerListSectionHandler.cs enables the status file to be read as part of the application configuration. PartitionResolver.cs contains a custom state server partition resolver class that decides which state server to connect to. This class also tries to pin users to particular state servers so that changes in the status file only affects users whose session was stored on the failing server.

Using the code

To set up the server monitoring service

1. Download the source files.

   You can download the source code at http://www.codeproject.com/KB/aspnet/stateserverfailover.aspx

   
   
   
2. Open up the ServerMonitor solution in visual studio.
   
   The solution contains two projects. One runs the service as a console application and the other one runs it as a windows service. The ServerMonitorService project compiles as a windows service and can be installed and uninstalled with the included install_service.bat and uninstall_service.bat files. The ConsoleServerMonitor project runs the service as a console application, which is a lot easier to test and debug. Both projects share the same sources and function identically.
   
   
3. Open up the project's application configuration file.
   
   
4. In the Servers section, specify the state servers you want to use, like as shown below:
   
   

   <Servers>
       <!-- List of servers to poll -->
       <add key="Server1"  value="localhost:42424" />
       <add key="Server2"  value="appserver1:42424" />
       <add key="Server3"  value="72.27.255.9:42424" />    
   </Servers>
    

   
   
5. In the StatusFilePaths section, add the full file pathname of the status file.
   This file should be located in the folder containing your web application or in subfolders.
   
   You can add multiple paths, if you want to notify multiple web applications, as shown below:
   
   

   <StatusFilePaths>
       <!-- List of file paths where status files are saved/updated -->
       <add key="Web1" value="C:\Inetpub\wwwroot\MyWeb1\server_status.config.xml"/>    
       <add key="Web2" value="C:\Inetpub\wwwroot\SuperWeb2\server_status.config.xml"/>        
   </StatusFilePaths>
     

   
   
6. Build the project.
   
   
7. If you built the ServerMonitorService project, navigate to the output folder and run install_service.bat to install the service.
   
   
8. If you built and installed the windows service, you can start Server Monitoring Service in the Services list. If you built the console server, run ConsoleServerMonitor.exe or simply start debugging from Visual Studio.
   
   
9. Note that the status files are created in the specified folders.
   
   

To configure your web application

1. Open your web application in visual studio
   
   
2. Add an App_Code ASP.NET folder to your application, if your application does not have one.
   
   
3. Copy PartitionResolver.cs and ServerListSectionHandler.cs from SampleWeb\App_Code folder to your web application's App_Code folder.
   
   
4. Open the project's web configuration file. (Add a new web configuration file if your application does not have one)
   
   
5. Add a new SessionStateServers section element in the configSections collection as shown below:
   
   

   <configSections>
   	<section name="SessionStateServers" type="ServerListSectionHandler" restartOnExternalChanges="false"/>
   </configSections>
   

   
   
6. Configure the newly added SessionStateServers section to be read from an external file as shown below:

   <SessionStateServers configSource="server_status.config.xml"/>
   
   

   (If the status file has a different filename, specify that instead)
   
   
7. In the system.web element, configure the application to use the custom partition resolver as shown below:
   

   <system.web>    
       <sessionState mode="StateServer" partitionResolverType="PartitionResolver"/>
   ...
   
   

   
   
8. Your web application is now set up to use the state server failover system

Points of Interest

I originally wanted the monitoring service to update a single status file, which multiple web applications could share. That plan didn't work because ASP.NET only works with external configuration files that are located in the application folder or in its subfolders. Because of this restriction, different web applications can not share one external configuration file.

It's not necessary to set the restartOnExternalChanges attribute of the section element in the web.config file to true. Setting this attribute to true causes the web application to restart whenever the external config file is updated, which will cause any data stored in the Application object to be lost.
The web application will still read the latest data in the external config file, if the attribute's value is set to false, without restarting the application.

The name of the root element of the status file is determined by the StatusXMLRootTag setting of the monitoring service's configuration.
The name must match the name of the new section you add to your web application's web.config file. The name must also be specified in the state server partition resolver class (PartitionResolver.cs)

Web developers often refer to session state stores and caches interchangeably, while in actuality they serve different purposes.

A cache serves as a caching layer between a web application and an external data source. Caches exist mainly to lighten the load on the external data source, thus improving the performance of the application.

The purpose of a session state store is to store a user’s workspace. Session state stores enable client activity to be persisted consistently across several HTTP requests.

Applications that utilize a cache write to the external data source, typically a database, and read from the cache. This technique can significantly improve the performance of applications that mostly read from the data source.
Caches are designed to be read speedily, simultaneously by many clients and threads. Some cache implementations can link the cached object to the data source, such that if the data source is updated, the cache is invalidated.

Session state stores are not linked to external data sources by design, although it can be consumed by an application to do so. While it may be necessary to store certain parts of a client’s session state to a database, there is usually no need to store all of the session state to a database. In fact, many database-less web applications rely solely on session state to operate.
Session state implementations are designed such that each client has exclusive access to its session data. Even though caches can be consumed by an application to simulate this, exclusivity is not enforced and there is always a chance that a client will be able to access another client’s session data due to either poor design or security flaws.

A distributed cache spreads out an application’s caching layer across many machines, which allow high-traffic web applications to scale out by adding more machines as demand increases. Performance can also be improved by distributing session state data across many machines; therefore, it is worthwhile to examine the requirements and nuances of cached data and session state before deciding on a distributed solution to apply.

Distributed caches, like local caches, are most effective when used to cache data that changes infrequently. They also support simultaneous fast reads of a cached object by many threads. This is where session state sharply differs from cached data.

Session state has little need for speedy multiple-thread access to a single stored resource because a client can only exclusively read or update its session. In addition, the usage pattern of session state is unpredictable. Some applications update session data very frequently while some do not. Session state storage designers normally safely assume that session state is write-heavy.

Caches are configured by default to use either an optimistic concurrency mechanism or no concurrency control at all, to access cached data. This design is driven by the strong requirement to eliminate blocking by any means possible, and works superbly due to the lower proportion of writes to reads.
Session state stores, on the other hand, utilize a pessimistic concurrency mechanism to access stored data. This works effectively because of the exclusive nature of resource access.

The number of concurrent session state accesses to a stored resource can increase if a user opens up several web browser instances of the same application or if the application makes use of numerous AJAX calls. Notwithstanding multiple instances and AJAX-intensive applications, a user’s session cannot have more than a handful of concurrent access attempts.
A pessimistic concurrency mechanism, as used by session state stores, can gracefully handle a few concurrent accesses on a write-heavy resource, and more importantly, provide consistent data to all operations. Inconsistencies in served data can arise, if a cache with no concurrency control is employed to store write-heavy session state. This problem becomes more apparent if the application is AJAX-intensive.

Critical applications that rely on session state require failover and redundancy support. These features are usually built into commercial session state storage solutions.
Caches have no need for failover or redundancy because caches are simply a caching layer: if the requested data cannot be retrieved from the cache, it can always be fetched from the primary source. Therefore, most distributed cache implementations do not support failover or redundancy; issues solution architects seldom remember when moving session storage to a distributed cache.

The conundrum of where to store session state arises when an application needs to scale to accommodate more users.
While there are a few commercial distributed session state storage solutions, there are no free robust alternatives, and the usual consensus is to store session state in freely available distributed cache solutions, or eliminate session state entirely from the application.

Moreover, even when session state is manageably stored in a distributed cache, most often, the same servers that are caching infrequently changing data are used to store session state. Sharing the cache this way leads to performance degradation.
This occurs because whenever the cache server needs to store a new cached object or remove an expired one, it has to momentarily suspend all read operations internally on all other cached objects until the object is added or removed. The overall outcome is sub-optimal reads for cached infrequently changing data.

Developers and architects should carefully weigh the aforementioned issues before moving locally stored session state to a distributed storage and should, whenever possible, opt for a solution that was specifically built for distributed session state storage.

Not too long ago, I needed to piece out the communication protocol between the ASP.NET state server and the web server, because I was working on a peer to peer version of the ASP.NET state server.

I made some effort to obtain the protocol as described in this post. Along the way, I found the protocol documentation by Microsoft, and was delighted. I could now design my implementation of the state server based on this information.

While going through the documentation, I noticed that the format of some of the messages did not match what I had earlier observed, so I decided to verify the protocol to be extra sure – Boy, was I in for a surprise.

First, there are bold, wrong statements in the documentation:

From section 3.1.5.3: “Because a client sends a lock-cookie value along with the session state data, the state server MUST store the lock-cookie value. Internally, the state server MUST also store the date and time when the state server received the lock-cookie value. This information is necessary if the state server ever has to send response-locked messages, as specified in sections 2.2.5.2 and 2.2.5.4.”

This is a wrong, misleading statement. A LockCookie value is used by a Set Request message to unlock a locked session entry (if it is locked) before storing the new data.
The state server has no use for storing the client LockCookie value, in fact, the state service MUST never store any LockCookie value sent by the client or in any way let the client influence LockCookie values as they are exclusively generated by the server.

From section 3.1.5.5: “A client can acquire an exclusive lock on session state by using either a successful GetExclusive_Request or Set_Request message.”

This is another off the mark statement. A client can only acquire an exclusive lock with the GetExclusive Request.
Even a cursory look at the SessionStateStoreProviderBase class is sufficient to confirm that this statement is wrong.

Then, there is important information that is left unstated:

The document does not mention anywhere that the ActionFlags header actually indicates that the server should only store the presented data if the unique session id does not already exist. If the ActionFlags header value is set to 1, and the server already has the presented session id, the existing session data will not be updated with the new one, however the state server will still reply with an OK response (as if it stored the data). This behavior is not easily noticeable to the casual observer, but is important to implement a 100% compatible state server.

There are other inaccuracies and misleading statements in the documentation that makes it virtually impossible for anyone to develop a state server implementation using the Microsoft documentation.
I had to painstakingly piece out the protocol from scratch. I’ve published the correct version of the protocol in PDF format and HTML format for reference purposes.

What's more, if you take a look at the history of the Microsoft document, you’ll notice that it has been edited more than twenty times over the course of almost three years. You’d think that after twenty edits, it will be somewhat accurate, but after almost three years of editing the documentation for a major ASP.NET server, Microsoft still manages to get it wrong.  It’s safe to say that either Microsoft is doing this intentionally or they do not know how their own technology works.

What’s even more disturbing is that this documentation is for a protocol, not for a piece of software. Do protocols change every other month? Imagine the chaos that would ensue if every developer had to second guess each statement in RFC-2821 when writing an email client and then also make sure that the protocol hasn’t changed every other month.

It seems the only reason Microsoft publishes these specs at all is to pay lip service to the European Union because there is no point publishing specs that are innacurate and can't save developers’ time when implementing a technology.

Introduction

ASP.NET web developers have three built in options to store session state, namely, in-process memory, SQL Server and State Server.

In-process memory offers the fastest performance but is unsuitable for use in web server farms because the session data is stored in the memory of the ASP.NET worker process.

SQL Server is an out of process session state storage option that works with web server farms. It stores session data in a SQL Server database. It is the most reliable option but the least performing one. One major issue with this option is that quite often developers want to cache data retrieved from a database in session state, to reduce database lookups. SQL Server session state defeats this purpose, because there is little performance gain in caching data retrieved from a database, in a database.

State Server is an out of process session state storage option that works with web server farms. It stores session data in memory and delivers better performance than SQL Server. This seems like a good compromise between the in-process option and the SQL server options. It has some drawbacks, however.

Firstly, several web servers typically depend on one state server for session state which introduces a critical single point of failure.

Secondly, in a load balanced environment, the load balancer may redirect a user’s request to a web server that is different from the web server that served the user’s previous request. If the new web server communicates with a different state server, the user’s original session state will not be found and the web application may not work properly.
This problem occurs even in persistence-based (aka sticky) load balancers either erroneously or due to server failure.

Thirdly, an issue that many developers are unaware about is that the web server and state server communicate in plain text. An eavesdropper can easily get hold of session state data traveling on the network. This may not be a threat if all servers are running in an internal network but it is certainly cause for concern when web servers and state servers are spread across the internet.

The peer to peer ASP.NET state server presented in this write-up addresses the aforementioned problems while transparently replacing the Microsoft provided state server.

Overview

The idea behind the peer to peer state server is simple -- let state servers on a network securely communicate and pass session state data amongst each other as needed, as shown below.

This design improves scalability because web servers can share multiple state servers, eliminating a single point of failure. Furthermore, if a load balancer erroneously or intentionally redirects a user to a different web server attached to a separate state server, the user’s session state will be requested from the state server that served the user’s previous request.

Security is also improved as peers can be configured to encrypt session data while sharing session state. Data transfers between the web server and the state server remain unencrypted but eavesdropping attacks can be eliminated by keeping web servers and linked state servers in trusted networks or on the same computer.

The peer to peer state server is fully backward compatible with the Microsoft provided state server and comes with all the benefits mentioned earlier.

Installation

To compile and install the state server:

1. Download the source file.

   You can download the source code at https://github.com/tenor/p2pStateServer




2. Open up the solution in visual studio. (Visual Studio 2008 will open up a Conversion Wizard. Complete the Wizard.)
   
   The state server comes in two flavors. One runs as a console application and the other one runs as a windows service. The StateService project compiles as a windows service and can be installed and uninstalled with the install_service.bat and uninstall_service.bat files. The ConsoleServer project runs the service as a console application, which is a lot easier to test and debug. Both projects share the same sources and function identically.



3. Open up the properties window for the project you want to build.



4. a. If using Visual Studio 2005, add NET20 in the conditional compilation symbols field of the Build tab.
   b. If using Visual Studio 2008, select .NET Framework 3.5 in the Target Framework field of the Application tab.



5. Build the project.



6. If you built the StateService project, navigate to the output folder and run install_service.bat to install the service.



7. If you are already running the Microsoft state service on your machine, stop it.



8. If you built and installed the windows service, you can start Peer to Peer State Service in the Services list. If you built the console server, run ConsoleServer.exe or simply start debugging from Visual Studio.



9. You can now test and run any web applications you have with the running state server.

To add peer servers:

1. Copy the compiled executable file and the application configuration file to another computer on your network.



2. Open up the configuration file and add a new peer in the <Peers> section. For instance, to configure the state server to connect to another state server running on a computer named SV3 with a peer port number of 42425, you would add <add key="MyPeer" value="pc2:42425" /> to the <Peers> section.



3. You can start the state server on the computer and it will link up with the other state server(s) on the network.



4. It’s up to you to set up the network in any topology you like. For example, consider a network of three state servers as shown below, each state server on each machine would have the configuration shown below:

You can run multiple console server peers on the same computer but each console server must have a unique web server port and peer port setting.

How it works

The Microsoft provided state server, works as shown below.

The Peer to Peer State Server works exactly as illustrated above, except when the state server doesn't have the requested session state, in which case it requests the session state from the network before responding, as illustrated below:

If the requested session state is not transferred within a set time period, the state server assumes the session state does not exist on the network and proceeds to process the web server request without the session state. The GetTransferMessage class represents the message that is broadcast on the network when a node is requesting a session. Peers maintain connection between themselves principally to forward this message. Session state transfers occur out-of-band of the peer network.

Implementation Notes

Various programming techniques are used to implement different aspects of the state server. Some of the notable ones are highlighted below.

PLATFORM

The state server is written in C# 2.0 but targets the NET 3.5 framework so as to take advantage of the ReaderWriterLockSlim class. If the NET20 symbol is defined, the server uses the slower ReaderWriterLock class instead and is able to target the .NET 2.0 framework.

You can download the source code at https://github.com/tenor/p2pStateServer

PROTOCOL

In order to create a state server that can transparently replace the state server, I needed to obtain and understand the full specification of the communication protocol between the web server and the Microsoft provided state server. The steps taken to piece out the protocol are documented in reverse chronology at http://ahuwanya.net/blog/category/Peer-to-Peer-Session-State-Service.aspx

MESSAGING

The server is largely message driven. The messaging subsystem is illustrated below:

When the server receives data from a socket, the data is accumulated in an instance of the HTTPPartialData class currently assigned to that socket. The HTTPPartialData instance validates the data, determines if the accumulated data is a complete HTTP message and checks for errors in the accumulated data. If there is a data error (for example, if the data does not conform to HTTP), the entire accumulated data is discarded and the socket is closed. If the data is valid but not yet complete, the sockets waits for more data to arrive.
If the accumulated data is a complete HTTP message, the data is sent to a MessageFactory object. The MessageFactory object inspects the data to determine the appropriate ServiceMessage child class instance to create. The ServiceMessage child class is instantiated and its implementation of the Process method is called to process the message.

CONCURRENCY HANDLING

A pessimistic concurrency mechanism is employed while accessing session state in the session dictionary, which is defined by the SessionDictionary class. A piece of session state can only be read or modified by one thread at a time. A thread declares exclusive access to operate on a piece of session state by setting the IsInUse property to true. This is done by calling the atomic compare and swap CompareExchangeInUse method (a wrapper to the .NET Interlocked.CompareExchange method that operates on the IsInUse property). Setting this property to true lets other threads know that another thread is working with that session state.

If another thread wants to access the same session state and attempts to declare exclusive access, the attempt will fail because another thread already has exclusive access. The thread will keep trying to acquire exclusive access, and will eventually acquire it when the other thread releases access. This works pretty well because most of the time, only one thread needs to access a session state, and also because most operations on a session state take a very short time to complete. The export (transfer) operation which takes a much longer time is handled with a slightly different mechanism and is discussed in the contention management section below.

TIMERS – OR THE LACK OF THEM

The code has a lot of objects that expire or time-out and on which certain actions must take place on expiration – objects like individual session state dictionary entries that expire or asynchronous messages that timeout. Instead of assigning a timer or a wait handle to track these objects – they are stored in instances of a special collection class called the DateSortedDictionary. Objects in this dictionary are sorted in place by their assigned timestamps. Specially designated threads poll these date sorted dictionaries for expired items and perform related actions if an item is expired. This design significantly reduces the number of threads needed to keep track of expiring items.

DIAGNOSTICS

The Diags class is used to keep track of messages, log server activity and detect deadlocks. Methods on the Diags class are conditional and will not compile into release configuration code.
The VERBOSE symbol can be defined to view or log all activity taking place at the server. This is particularly useful with the console server which outputs this information to the console window. If the VERBOSE symbol is not defined, only critical information or unexpected errors are displayed.

Security

The Microsoft provided state server transmits and receives unencrypted data to and from the web server. This was most likely done for performance reasons. To be compatible with the Microsoft provided state server, the peer to peer state server transmits unencrypted data to the web server. However the peer to peer state server can be configured to transmit encrypted data between peers. This effectively thwarts network eavesdropping attacks if web server and associated state servers are installed on the same computer or on a trusted network.

For example, take the Web server – Microsoft State Server configuration shown below.

Two web servers connect across the public internet to access a state server.

Using peer to peer state servers, the network can be secured by having the web servers have their own local state servers that connects securely to the remote state server on their behalf as shown below:

   

The local state servers can be installed on the same machine as the web server for maximum security and minimum latency.

This approach can help secure geographically distributed web and state servers.
Peer state servers also mutually authenticate each other while connecting, to ensure that the other party is an authorized peer.

Network Topologies

Connections between peers form logical networks which can be designed with common network topologies in mind.

Network A shown above is a ring network of peer state servers which are individually connected to web servers whereas Network B is a ring network of computers which have both state server and web server connected and running. Existing isolated Microsoft state server networks can be upgraded to form a larger peer to peer network by replacing the Microsoft state servers with peer to peer state servers and linking them up as shown in Network A. Network B benefits from the security counter measures mentioned earlier and is somewhat more scalable since any node on the network is a web server and a peer state server.

Both networks will still function if one node fails, unlike on a bus network, however as more nodes are added to the network, the longer it takes for a message to traverse the network.

 

Network C is a star network. An advantage of having a star network is that no matter how many new nodes are added to the ring network, it takes only two hops for a message to reach any node on the network.

Network D is a network of three star networks that form a larger star network. This network too will require a lesser number of hops for a message to traverse the network. Both networks suffer from the disadvantage that if the central node fails, the entire network fails.

By connecting the leaf nodes on Network D, Network E, a partial mesh network is formed. Network E is a clever combination of a ring network and a star network. If the central node fails, the network will still function and it also takes a fewer number of hops for a message to traverse the network than on a ring network.

As demonstrated, the topology of the peer to peer state server network is limited only by the imagination of the network designer.

Interesting Scenarios

There are a lot of scenarios that occur in the state server that are handled using traditional peer to peer processes such as the time to live header which is used to prevent messages from circulating perpetually on the network, and message identifiers used by peers to recognize messages that have been seen earlier, however, there are two particular scenarios that occur in this peer network that are not so common.

SHUTDOWN TRANSFERS

To ensure that session data is not lost during a server shutdown, the state server proceeds to transfer all its session state data to connected peers in a round-robin fashion when a server shut down is initiated.

REBROADCASTS

A request for a session on a network can narrowly miss the node holding the session if it is being transferred it as illustrated below.

As shown above, node 1 is seeking session A from the network just about the same time node 4 wants to transfer the session to node 2.

When the message from node 1 reaches node 2, node 2 forwards the message to node 3 because it doesn’t have the session.

When the message reaches node 3, the session transfer between nodes 4 and 2 begins and by the time the message reaches node 4, the transfer is complete and node 4 no longer has the session anymore and forwards the message to node 5.

Thus, the message traverses the network without reaching any node with the sought session, even though the session exists on the network.

The state server addresses this issue by having nodes that recently transferred a session rebroadcast the message as shown below.

Here, node 4 rebroadcasts the message so that it also travels back the way it came and eventually reaches node 2 which has the session.

Rebroadcasted messages are duplicates of the original message except that they have a different Broadcast ID header which peers use to know it’s a different broadcast.

Contention Management

As stated earlier, the state server uses a pessimistic concurrency model when accessing session state entries in the session dictionary. This works well because most requests take a short time to process. However, one particular request can take a much longer time to process, and can lead to resource starvation and performance degradation.

A GetTransferMessage message broadcast is initiated by a peer when it needs to work with a session state it does not have. When the broadcast reaches a peer with the requested session state, the session state is transferred to the requesting peer.

Unlike other operations on a session state, a transfer can take a significant amount of time because the peer has to connect to the other peer, possibly authenticate, and transmit (a potentially large amount of) data. It’s important to note that any request from the web server can kick start a GetTransferMessage broadcast.

During a transfer, the session is marked as “in use” and other requests on that session will have to wait as usual. However, since it takes a much longer time, Threads waiting for a transfer operation to complete consume a lot of system resources. They can also timeout if the transfer takes too long or if the session is repeatedly transferred around the network due to flooded messages. A bad case is illustrated below:

In the diagram above, a user is flooding a web application with requests, which in turn is causing session requests to be transmitted to a state server.

Because all requests originate from one user, all session requests reference the same session id. A load balancer or state partitioner distributes these requests among the three state servers.

It is important to note that even though it is unlikely that a load balancer or state partitioner will distribute requests for a session among different state servers, a user can produce the scenario shown above by simply pressing and holding the browser refresh key on a web application that uses a poorly implemented state partitioner or a malfunctioning load balancer.
Also, an organized group of malicious users (or a botnet) can produce this scenario even on properly functioning state partitioners and load balancers.

Each state server has requests waiting to be processed. If the highly in-demand session is say, on state server 3, requests on that state server will be processed one by one very quickly.

State servers 1 and 2 issue broadcasts requesting a session transfer. The message eventually reaches state server 3 and the request is transferred to say state server 2. Requests on state server 3 that were not processed will wait until the transfer is complete.

After the session transfer to server 2 is complete, requests on server 2 are processed, whereas requests on server 3 issue broadcasts requesting the session.

A broadcast that originated from state server 1 reaches state server 2 and the session is transferred to state server 1. This goes on and on and the servers keep transferring the session amongst themselves while most of the requests wait, because even when a session is transferred, the state server is only able to process a few requests before it is transferred to another state server.

To make matters worse, if a state server receives a GetTransferMessage message after it has recently transferred the session, it rebroadcasts the message (as explained earlier), which leads to even more GetTransferMessage broadcasts on the network, more back-and-forth transfers and prolonged resource starvation.

The transfer process is relatively slow and since all requests have to wait to be processed one at a time by each state server, requests start to time out and the web server starts discarding requests. The state server is unaware that the web server has discarded those requests and still proceeds to process them.

These redundant requests, waiting for their turn, eat up valuable server processor cycles and degrade the quality of service.

If plenty of these requests arrive, they'll quickly use up all processor resources and the server comes to a grinding halt.

While it may be impossible to stop any group of users from flooding the state server with requests, the state server guards against contentious sessions with the following principle: any degradation of service due to a contentious session should mainly affect the user of that session, and achieves this goal with the following mechanisms:

1. When a request is to be processed and the server notices the session is being transferred, the request stops being processed and is queued to be processed when the transfer is over. This prevents the request from eating up processor cycles while waiting, and frees up resources, so that other requests from other users can be processed. If the number of requests on a queue waiting for a session to transfer is too long, then all those messages are discarded because it means the session is contentious and the server shouldn't bother processing them.



2. After the transfer is complete and a queued request is ready to be reprocessed and the server notices that the same session is been transferred again by another request, then the request will be discarded and not be processed, because it means the session is highly contentious.



3. Before a request tries to query the network (by broadcasting) for a session, it checks if it is expecting a reply from a previous query for that session, and if so, the request is queued to a list of requests to be processed when the query is received. This reduces the number of GetTransferMessage messages that will be generated on the network, which in turn reduces unnecessary rebroadcasts and lookups. If the number of requests on a queue waiting for a session to arrive is too long, then all those requests are discarded because it means the session is contentious.



4. Finally, all incoming requests are queued up in their session id-specific queue and the message processor polls the incoming request queues in a round-robin manner and processes them one after the other, as shown below:
   
   
   
   This means that all session requests are treated fairly, no single user can significantly disrupt the rate at which messages originating from other users are processed. Additionally, if the queue for a particular session id is too long, that queue is discarded because it means that session is contentious.

All these techniques employed by the state server can only adversely affect the web application of the offending user.

Conclusion

The peer to peer state server is fully backward compatible with the Microsoft provided state server and can transparently replace it. Peer state servers can transfer sessions to each other, improving the reliability of session state dependent web applications. Peer state servers also act as a security layer that protects session data on the network.

This project started out as a simple idea but quickly evolved into a complex task. Hopefully, this implementation and other ideas presented in this article will be valuable to developers interested in distributed systems.  Due to the level of complexity, there will be bugs and kinks to work out. Contributions and bug reports will be appreciated.

I've completely spec'd out the ASP.NET state service protocol and have published my findings.

You can download it as a PDF document or view it in HTML format.
The specifications will be very useful to anyone wishing to implement the state service.

 

In my previous post, I took a first look at the ASP.NET state service communication protocol. In this post I'll describe the techniques I'm using to piece out and understand the protocol.

I needed to monitor the traffic between the web server and the state service so I installed WinDump, a Windows version of tcpdump.
With WinDump, I could capture the low level tcp traffic between the state service and the web server. The only caveat was that I had to monitor a state service on a different computer because WinDump doesn’t work with the loopback device.

After a while I decided WinDump was too low level for the task at hand. What I really needed was a tcp relay server.

A tcp relay server is a more involved version of an echo server that sits between a server and a client and relays transmitted data to and fro. With a tcp relay server, not only can I capture the transmitted data, I can also modify it.



To have the greatest flexibility, I decided to write one. My relay server is a console application. Here's the code:

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;

namespace RelayServer
{
    //State object
    public class StateObject
    {
        // Client  socket.
        public Socket WorkSocket = null;
        // Size of receive buffer.
        public const int BufferSize = 1024;
        // Receive buffers.
        public byte[] Buffer = new byte[BufferSize];
        // Socket that connects to other party
        public Socket HandoffSocket = null;
    }

    class Program
    {
        static int relayPort = 24242; //Port to listen on for client connection
        static string serviceHost = "localhost"; //State service host
        static int servicePort = 42424; //State service port
        static StateObject pollStateObj = new StateObject(); //State object polled for disconnection

        static void Main(string[] args)
        {
            IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Any, relayPort);
            Socket clientListener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
            clientListener.Bind(localEndPoint);
            clientListener.Listen(100);
            clientListener.BeginAccept(new AsyncCallback(AcceptCallback), clientListener);

            while (!Console.KeyAvailable)
            {
                Thread.Sleep(200); //check every 200 milliseconds

                //is the state object instantiated and connected?
                if (pollStateObj.WorkSocket != null && 
                     (pollStateObj.WorkSocket.Connected || pollStateObj.HandoffSocket.Connected))
                {                    
                    //check if work (client) socket is disconnected
                    try
                    {
                        //Test for disconnection
                        bool isDisconnected = false;
                        lock (pollStateObj.WorkSocket)
                        {
                            isDisconnected = pollStateObj.WorkSocket.Available == 0 && 
                                pollStateObj.WorkSocket.Poll(1, SelectMode.SelectRead);
                        }

                        if (isDisconnected)
                        {
                            Console.ForegroundColor = ConsoleColor.Red;
                            Console.WriteLine("Client disconnected.");
                            DisconnectSockets(pollStateObj);
                            continue;
                        }
                    }
                    catch (ObjectDisposedException ex)
                    {
                        //do nothing
                        continue;
                    }
                    catch (SocketException ex)
                    {
                        //Was Socket remotely disconnected?
                        if (ex.ErrorCode == 10054)
                        {
                            Console.ForegroundColor = ConsoleColor.Red;
                            Console.WriteLine("Client disconnected.");
                        }
                        else if (ex.ErrorCode == 10053)
                        {
                            Console.ForegroundColor = ConsoleColor.Red;
                            Console.WriteLine("Client aborted connection.");
                        }
                        else
                        {
                            Console.ForegroundColor = ConsoleColor.White;
                            Console.WriteLine("\nError: " + ex + "\n");
                        }

                        DisconnectSockets(pollStateObj);
                        continue;
                    }
                    catch (Exception ex)
                    {
                        Console.ForegroundColor = ConsoleColor.White;
                        Console.WriteLine("\nError: " + ex + "\n");
                        continue;
                    }

                    try
                    {
                        //Test for disconnection
                        lock (pollStateObj.HandoffSocket)
                        {
                            pollStateObj.HandoffSocket.Send(new byte[1], 0, SocketFlags.None);
                        }
                    }
                    catch (ObjectDisposedException ex)
                    {
                        //do nothing
                        continue;
                    }
                    catch (SocketException ex)
                    {
                        //Was Socket remotely disconnected?
                        if (ex.ErrorCode == 10054)
                        {
                            Console.ForegroundColor = ConsoleColor.Red;
                            Console.WriteLine("Server disconnected.");
                        }
                        else if (ex.ErrorCode == 10053)
                        {
                            Console.ForegroundColor = ConsoleColor.Red;
                            Console.WriteLine("Server aborted connection.");
                        }
                        else
                        {
                            Console.ForegroundColor = ConsoleColor.White;
                            Console.WriteLine("\nError: " + ex + "\n");
                        }

                        DisconnectSockets(pollStateObj);
                        continue;
                    }
                    catch (Exception ex)
                    {
                        Console.ForegroundColor = ConsoleColor.White;
                        Console.WriteLine("\nError: " + ex + "\n");
                        continue;
                    }
                }
            }
        }

        private static void AcceptCallback(IAsyncResult ar)
        {
            //Accept incoming connection
            Socket listener = (Socket)ar.AsyncState;
            Socket handler = listener.EndAccept(ar);
            
            //Display Connection Status
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("Client is connected.");

            //Accept another incoming connection
            listener.BeginAccept(new AsyncCallback(AcceptCallback), listener);

            // Create the state object.
            StateObject state = new StateObject();
            state.WorkSocket = handler;
            state.HandoffSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

            //Connect to other socket
            state.HandoffSocket.Connect(serviceHost, servicePort);

            //Create another state object to receive information from service
            StateObject recvState = new StateObject();
            recvState.WorkSocket = state.HandoffSocket;
            recvState.HandoffSocket = state.WorkSocket;

            //Set up the Service socket to receive information from service
            state.HandoffSocket.BeginReceive(recvState.Buffer, 0, StateObject.BufferSize, SocketFlags.None,
                new AsyncCallback(ReadCallback), recvState);

            //Set up the client socket to receive information from client
            handler.BeginReceive(state.Buffer, 0, StateObject.BufferSize, SocketFlags.None,
                new AsyncCallback(ReadCallback), state);

            //Reference this State object as the object to poll for disconnections
            pollStateObj = state;
        }

        private static void ReadCallback(IAsyncResult ar)
        {
            // Retrieve the state object and the handler socket
            // from the asynchronous state object.
            StateObject state = (StateObject)ar.AsyncState;

            if (state.WorkSocket.Connected == false)
            {
                return;
            }

            try
            {
                // Read data from the client socket. 
                int bytesRead;
                lock (state.WorkSocket)
                {
                    bytesRead = state.WorkSocket.EndReceive(ar);
                }

                if (bytesRead > 0)
                {
                    //Transform received data
                    byte[] transformedData = TransformData(state.Buffer, bytesRead, state);

                    //Transmit transformed data to other Socket
                    if (transformedData.Length > 0)
                    {
                        lock (state.HandoffSocket)
                        {
                            state.HandoffSocket.BeginSend(transformedData, 0, transformedData.Length, 
                                SocketFlags.None, new AsyncCallback(SendCallback), state);
                        }
                    }
                }

                lock (state.WorkSocket)
                {
                    state.WorkSocket.BeginReceive(state.Buffer, 0, StateObject.BufferSize, 0,
                        new AsyncCallback(ReadCallback), state);
                }
            }
            catch (ObjectDisposedException ex)
            {
                //do nothing
                return;
            }
            catch (SocketException ex)
            {
                if (ex.ErrorCode == 10054 || ex.ErrorCode == 10053)
                {
                    //do nothing
                    return;
                }
                Console.ForegroundColor = ConsoleColor.White;
                Console.WriteLine("\nError: " + ex + "\n");

                return;
            }
            catch (Exception ex)
            {
                Console.ForegroundColor = ConsoleColor.White;
                Console.WriteLine("\nError: " + ex + "\n");
                return;
            }

        }

        private static void SendCallback(IAsyncResult ar)
        {
            // Retrieve the socket from the state object.
            StateObject state = (StateObject)ar.AsyncState;

            if (!state.HandoffSocket.Connected)
            {
                return;
            }

            // Complete send
            try
            {
                lock (state.HandoffSocket)
                {
                    state.HandoffSocket.EndSend(ar);
                }
            }
            catch (Exception ex)
            {
                Console.ForegroundColor = ConsoleColor.White;
                Console.WriteLine("\nError: " + ex + "\n");
                return;
            }
        }

        private static byte[] TransformData(byte[] Data, int Length, StateObject State)
        {
            //Just display Transmitted Data
            if (State.WorkSocket == pollStateObj.WorkSocket)
                Console.ForegroundColor = ConsoleColor.Cyan;
            else
                Console.ForegroundColor = ConsoleColor.Green;
            
            Console.Write(Encoding.UTF8.GetString(Data, 0, Length));

            byte[] output = new byte[Length];
            Array.Copy(Data, output, Length);
            return output;
        }

        private static void DisconnectSockets(StateObject state)
        {
            lock (state.WorkSocket)
            {                                        
                if (state.WorkSocket.Connected)
                {
                    state.WorkSocket.Shutdown(SocketShutdown.Both);
                }
                state.WorkSocket.Close();                    
            }

            lock (state.HandoffSocket)
            {
                if (state.HandoffSocket.Connected)
                {
                    state.HandoffSocket.Shutdown(SocketShutdown.Both);
                }
                state.HandoffSocket.Close();
            }
        }
    }
}

It's important to note that the tcp relay server must also relay connections and disconnections from either end, in addition to transmitted data.

To test the relay server:

1. Create a new ASP.NET web application in Visual Studio.

2. Add the following line in the system.web section of the Web.config file:

<sessionState mode="StateServer" stateConnectionString="tcpip=localhost:24242" cookieless="false" timeout="20"/>
<!-- -->

3. Add the following lines of code to the Page_Load event handler.

 Session.Add("Test2", "Test");
 Session.Abandon();

4. Run the relay server.

5. Start the local ASP.NET State Service (Located at Control Panel -> Administrative Tools -> Services) .

6. Run the ASP.NET Application

You'll see the gem below in the relay server console window.



This tool will be extremely useful as I spec out my implementation of the state service. For instance, if I want to see how the state service reacts if there is no Exclusive header, I'll simply modify the TransformData method to detect and remove the header.

I have also found Microsoft’s specification of the ASP.NET state service protocol. It doesn't look coherent but it will help me make my implementation compatible with their specifications.

I have been thinking about developing an alternative ASP.NET State Service which can transparently replace the Microsoft-provided state service.

To be successful, the alternative state service has to convince the web server that it is the state service. That means I need to know the high-level communication protocol used between the state service and the web server.
   
How does the web server talk to the state service? Do they communicate via .NET remoting calls? Is there a proprietary protocol? Or is the state service really a fancy web service?
 
Let's investigate.

I'll create an echo server that will listen on a port. I can then set the echo server’s address as the state service address for a web application. If the state service protocol runs over TCP, I should be able to see data transmitted by the web application.

Here's the code snippet for a simple windows socket server that echoes received data:

int port = 24242; //as opposed to the default 42424 :)

IPEndPoint endPoint = new IPEndPoint(IPAddress.Any, port);
Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream,ProtocolType.Tcp);
server.Bind(endPoint);
server.Listen(20);
Socket clientHandler = server.Accept();
byte[] transmission = new byte[1024];
clientHandler.Receive(transmission);

//print transmitted data
Console.WriteLine(Encoding.Default.GetString(transmission));
Console.ReadKey();

(To run the code snippet, you'll need to add using statements to reference System.Net and System.Net.Sockets namespaces.)

I then create a test web application and add the following line in the system.web section of the Web.config file:

<sessionState mode="StateServer" stateConnectionString="tcpip=localhost:24242" cookieless="false" timeout="20"/>
<!-- -->

This configures the ASP.NET application to use the state service, except it's actually our echo server.

I also add the following line of code to the web application's Page_Load event handler.

 Session.Add("Test Entry", "Test Data");

Now, I start the echo server (actually a console application) and fire up the web application to see if any transmitted data is captured and displayed in the console window.



Wow. Is that HTTP?
I see a PUT verb followed by some non-standard headers, followed by some binary data.
Well, that certainly makes my task easier considering that if the service was using .NET remoting calls; I'd have needed to delve into the innards of the .NET Remoting API.

I also notice that the transmitted data is not encrypted. I can clearly see "Test Data" in the transmission. Um, Microsoft -- this is not good.

How about the gibberish looking tidbit in the PUT statement? That must be some kind of identifier.

I add the following line in the system.web section of the Web.config file to see if it will cause the data to be encrypted.

<machineKey validationKey='016E9B3DAA748525DCDBAAC999BB390D63E7E1095F56B737887C10291567085B5A3A2142E6C86F06F07558D77260122C1174419212B6A117B6977B285EA8722B' />
<!-- -->

No such luck, however, the encoded data in parenthesis in the PUT verb line changed.
Hmm. Let's try to make some sense out of all these data.

The PUT Verb With Encoded Data:

After URL-decoding the data, I get  /3e50a960(iE+KOE6bwMI7BuHXun98zlcnkb8=)/miztsjiek5gvzu55km3xun55. The backslash is probably a delimiter and the text in parentheses is probably base64-encoded. (the "=" sign gave that away)

After running and observing the web application a few times, I figured the three parts of the PUT verb line are:

Part A: /3e50a960 is constant (I have no clue what it represents. Application ID? Machine ID? Some kind of magic number? I'll investigate)
Part B: (iE+KOE6bwMI7BuHXun98zlcnkb8=) is derived from MachineKey and is used by the state service to differentiate session data from different machines
Part C: /miztsjiek5gvzu55km3xun55 is the session ID

The Headers:

Host: The Hostname of the web application.
Timeout: The number of minutes a session can be idle before it is discarded.
Content-Length: The length of the binary data.
ExtraFlags: No clue. I'll investigate.
LockCookie: Looks eerily familiar. I'll investigate.

Binary Data:

This could be either information about an item to be updated, added or removed from the state service, in which case the web application retrieves and updates items as needed OR it could simply be a serialized list of all items to be stored in the session, in which case the web application reads all items at the beginning of a web request cycle and updates them at the end of the request cycle. To test my assumptions; I'll add a couple more lines of code to the Page_Load event handler.

Session.Add("test entry 2", "some test data");
Session.Add("test entry 3", "even MORE test data");

I then run the web application to see the transmitted data.



I can see the new items I added in the binary data area. I can now conclude that at the start of a web request, the web application retrieves this list, possibly modifies it and sends it back to the service at the end of the request.

Let me pause right here and think about this for a minute.

Is this model efficient? Is it better to read all items in the session at once and update them all at once, or is it better to read and update items as needed?

Well, since the items are stored by session ID and only one session ID is assigned to one user at any point in time. It means the chance that a session item is requested by multiple users/machines is negligible. Therefore there is no need to retrieve and update items as needed because there is no need for concurrency.

Another advantage with this model is that the state service does not need to know about items being stored. It simply stores whatever data is sent to it. This makes development easier.

Also, retrieving and updating items as needed increases the number of connections to the state service. This can greatly affect the scalability of a web application.

The downside to reading and updating all items at once is bandwidth-related: If an application stores a lot of information in the session but uses only a few at a time, the web application will move a lot of unnecessary data to and fro, which may clog the network. This may not be an issue if your web server and state service are physically close or run on the same computer.

A new question comes up; why is the web application NOT querying the state service for the session data when it starts? Does this mean any information already stored by the state service will be overwritten whenever the web application starts?

I ended up with more questions than I began with and so I still need to investigate further but at least I have gleaned some basic facts needed to start working on an alternative state service.

1. The ASP.NET web server and the state service communicate via HTTP.
2. The state service stores the binary data sent by the web server (probably in a large dictionary) using the Session ID + a derivative of the machine key as the key.
3. The state service does not need to itemize the data to be stored because all items are read and updated at once.
4. The transmitted data is not encrypted (at least not by default). The state service is not concerned about this since it simply stores data.