ASP.NET

Why Do We Recycle Our Application Pools?

20 commentsWritten on June 12th, 2010 by
Categories: ASP.NET, Opinions

I’ve never understood why IIS by default configures newly created application pools to recycle every 1740 minutes.  That means that by default, our ASP.NET applications are restarted every 29 hours.  And for what?  Are we, the .NET community, so bad that we really can’t write code that can keep running without problems for more than 29 hours?  Well, considering the overall lack of knowledge regarding memory management in .NET, i actually wouldn’t be surprised if that turned out to be the case for a very large portion of .NET development teams.

And indeed, if you browse the web to find the reason why application pools are configured to recycle automatically periodically, you’ll be hard pressed to find a reasonable answer that doesn’t pertain to memory issues.  It’s like the community in general has pretty much accepted the fact that our web applications (or service layers hosted in IIS) will need to be recycled to avoid memory problems.  To make things worse, i’ve frequently seen people discuss workarounds on how to keep things working properly after an application pool recycle in mailinglists for various projects.  And again i ask myself: why?

I’ve always been of the opinion that if your code requires periodic restarts to keep working correctly, then something is clearly wrong.  There is a bug in your code somewhere and you need to fix that, instead of restarting the process occasionally to make the problem ‘go away’.  Guess what, it’s not going away, and if the load on your application increases, the problem will only pop up more frequently.  So what are you going to do about that? Make the application pool recycle even more frequently?  Put in another server to share the load?  Perhaps we should focus more on fixing the actual code.  You’re leaking resources somewhere, and that is what you need to fix.

In my company, it’s just standard procedure to disable application pool recycling on all application pools.  Our applications simply have to keep working without application pool recycles.  It’s that simple.  If we run into a memory leak that causes problems, we focus on the actual problem and we fix it.  We’ve ran into plenty of memory leaks (though it hasn’t really happened a lot in the past year, guess why?) and we’ve fixed each and every single one of them that we’ve ran into.  Side note: i’m talking about .NET code that is running on the server.  While i’m pretty sure that we do better at memory management in our Silverlight clients than most other Silverlight-using-shops do, you just never know for sure with Silverlight, partly because there are so many memory leaks in Silveright itself.  But for .NET code running on a server, a memory leak is simply inexcusable in my opinion.  I mean, it can happen to the best of us (i’m pretty much responsible for most of ours to be honest), but it’s not about who introduced the memory leak.  It’s all about how you deal with it.  And recycling the application pool periodically is not dealing with it, it’s living in denial.

The only viable reason to recycle application pools that i can think of is in the situation of a shared hosting provider.  If you’re offering shared hosting, it only makes sense to recycle application pools occasionally since it’s quite realistic to assume that most of those application pools will not be in use constantly.  If a pool is recycled, it will not consume any resources until the next request for an application in that pool is received.  In the context of shared hosting, that actually makes sense.  In the context of real applications, that makes little sense unless you’re hosting a lot of them and plenty of them are hardly being used.  Still, is that really a good enough reason to enabling application pool recycling by default?  I obviously don’t think so.

In my opinion, we, as a community, really need to start focusing more on memory management in .NET and on making sure that our applications can keep running without problems.

Reducing ViewState Size

14 commentsWritten on September 1st, 2009 by
Categories: ASP.NET, Performance

I dislike ViewState as much as the next guy, but when you're working with ASP.NET WebForms, you just can't avoid it. In some cases, the size of the ViewState can become so big that it significantly increases the load time of pages due to the extra bandwidth consumption. The correct solution would obviously be to reduce the size of the ViewState in those pages as much as you can, but it's not always feasible to do so. So we wanted a more general 'solution', and i found this post which discusses compressing the ViewState before you send it to the client and decompressing it when the client sends it back. We used pretty much the same approach, but with some differences.

First of all, ViewState is persisted in the resulting HTML page through an IStateFormatter object. We'll provide our own CompressedStateFormatter which implements the IStateFormatter interface, and uses the standard IStateFormatter that ASP.NET uses:

    public class CompressedStateFormatter : IStateFormatter
    {
        private readonly IStateFormatter actualFormatter;
 
        public CompressedStateFormatter(IStateFormatter actualFormatter)
        {
            this.actualFormatter = actualFormatter;
        }
 
        public string Serialize(object state)
        {
            string decompressedState = actualFormatter.Serialize(state);
 
            using (MemoryStream memoryStream = new MemoryStream())
            {
                using (Stream zipStream = new GZipStream(memoryStream, CompressionMode.Compress))
                using (StreamWriter writer = new StreamWriter(zipStream))
                {
                    writer.Write(decompressedState);
                }
 
                return Convert.ToBase64String(memoryStream.ToArray());
            }
        }
 
        public object Deserialize(string serializedState)
        {
            byte[] data = Convert.FromBase64String(serializedState);
 
            using (MemoryStream memoryStream = new MemoryStream(data))
            using (Stream zippedStream = new GZipStream(memoryStream, CompressionMode.Decompress))
            using (StreamReader reader = new StreamReader(zippedStream))
            {
                return actualFormatter.Deserialize(reader.ReadToEnd());
            }
        }
    }

The idea is very simple: when the Serialize method is called, we first call the real formatter's Serialize method, compress its return value and then return the Base64-encoded string of the compressed serialized state. And in the Deserialize method, we do the exact opposite: we first decompress the Base64-encoded string and then we use the real formatter to deserialize the actual ViewState.

In Mamanze's example, he checks to see if the compressed version is actually smaller than the decompressed version and if so, uses the decompressed version instead of the compressed one. And when decompressing he first checks to see if it's a compressed or decompressed version and obviously only decompresses in case of a compressed version. The only page where i found the compressed version of the ViewState to be larger than the decompressed version was in our log in page, so i just got rid of that piece of the code.

Now we still have to plug this into ASP.NET's behavior somehow... first we add a pagestate.browser file to the App_Browsers folder of your web application (if it doesn't exist, just create it) with the following content:

<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter controlType="System.Web.UI.Page" adapterType="Our.Application.CompressedPageStateAdapter" />
    </controlAdapters>
  </browser>
</browsers>

The CompressedPageStateAdapter looks like this:

    public class CompressedPageStateAdapter : PageAdapter
    {
        public override PageStatePersister GetStatePersister()
        {
            return new CompressedHiddenFieldPageStatePersister(Page);
        }
    }

And the CompressedHiddenFieldPageStatePersister class looks like this:

    public class CompressedHiddenFieldPageStatePersister : HiddenFieldPageStatePersister
    {
        public CompressedHiddenFieldPageStatePersister(Page page) : base(page)
        {
            FieldInfo field = typeof(PageStatePersister).GetField("_stateFormatter", BindingFlags.NonPublic | BindingFlags.Instance);
            // retrieving this property instantiates the default IStateFormatter
            var defaultFormatter = base.StateFormatter;
            var formatter = new CompressedStateFormatter(defaultFormatter);
            field.SetValue(this, formatter);
        }
    }

The HiddenFieldPageStatePersister is the class that ASP.NET WebForms will use by default to store your ViewState into a hidden field in the resulting HTML. By default, the HiddenFieldPageStatePersister uses the default IStateFormatter type that ASP.NET uses, which only uses Base64 encoding but no compression. Unfortunately, there is no clean way to instruct ASP.NET to use a different implementation for IStateFormatter, so we need to use a bit of reflection to overwrite the value of HiddenFieldPageStatePersister's _stateFormatter field. Luckily, this also enables us to first get the value of the StateFormatter property so we can pass this reference (which is the 'real' formatter) to our CompressedStateFormatter.

And that is all there is to it... all of your pages will now use this CompressedHiddenFieldPageStatePersister so you get the benefit of ViewState compression in each of your pages. You can also do this selectively if you want, by not using the pagestate.browser file and overriding the PageStatePersister property of your ASPX page:

        private CompressedHiddenFieldPageStatePersister persister;
 
        protected override PageStatePersister PageStatePersister
        {
            get
            {
                if (persister == null)
                {
                    persister = new CompressedHiddenFieldPageStatePersister(this);
                }
 
                return persister;
            }
        }

This way, only the pages that contain this code will use the CompressedHiddenFieldPageStatePersister.

Instead of inheriting from HiddenFieldPageStatePersister, you could also inherit from SessionPageStatePersister. SessionPageStatePersister will store your ViewState in the HttpSessionState, and will only include a little bit of ViewState in your HTML page instead of everything. But you do need to be aware of the fact that using the CompressedStateFormatter when inheriting from SessionPageStatePersister will only result in compressing the little bit of ViewState that is included in the HTML, and not the ViewState that is stored in the HttpSessionState.

In case you're wondering: why should i use this instead of using typical HTTP compression on the IIS level? I believe it has a couple of advantages to HTTP compression. First of all, AFAIK, HTTP compression does not have any benefit on postbacks. And since ViewState is always posted back to the server, this can make a pretty big difference. Also, with this approach, the client will not have to decompress the entire ViewState (which isn't used client-side anyway) and the browser doesn't have to waste time on it in general.

I haven't used this in production yet, but i will very soon... unless someone knows of a good reason why i shouldn't ;)

The Joys Of Debugging ASP.NET Memory Leaks

5 commentsWritten on April 6th, 2009 by
Categories: .NET bugs, ASP.NET, Memory Management

One of my coworkers describes a very interesting memory leak he just fixed. Be sure to read his post to get the entire story.

We basically had a custom UserControl which contained (among other things) a Repeater which would render another custom UserControl. Nothing really weird there, right? The problem was that the custom UserControl that was created by the Repeater explicitly needed to be disposed. No problem, System.Web.UI.Control implements IDisposable so we could just override the Dispose method, perform our cleanup there and then proceed with the base Dispose implementation.

Now, i'm far from an ASP.NET expert but i was under the impression that all Page and UserControl instances would always have their Dispose method called if they were created by other ASP.NET controls (containing pages or controls). After all, what's the point in implementing IDisposable in the base Control class if you're not even going to guarantee proper usage of the pattern, right?

Well, as Kristof describes in his post, there is at least one situation where a UserControl is not disposed of, even though it was created by a Repeater. In our case, the leak was subtle. It took a stress test of 10 hours with a load that would be comparable to real world usage of one month to expose the leak. A lot of people would say "oh, let's just recycle the application pool periodically and everything is A-OK". But the very idea of having to recycle an application pool periodically to keep a system up and running is just a cop out to me. Good code should not require periodic restarts. Period.

I wonder how many ASP.NET WebForms applications might actually have the same (or at least similar) problem(s) like the one Kristof encountered. If everyone keeps recycling their application pools periodically, we unfortunately may never found out.

Abstracting Request State

22 commentsWritten on January 17th, 2009 by
Categories: ASP.NET, WCF

Sometimes it's useful to be able to store an object somewhere so you can easily access it for the duration of the current request, instead of having to pass it around with every method call that you make. That request could be an ASP.NET request, or a request in your WCF service layer. I used to resort to storing these objects in ThreadStatic fields (which is basically a static reference for each thread), thinking that it would be safe because only one thread handles a complete request. Last week i read that some requests can be paused and resumed by another thread. If you're using ThreadStatic fields, this could lead so major issues which would be a royal pain in the ass to debug. In order to prevent this possible problem, i wanted to have a safe way to keep state that should be available for the duration of a single request.

If your code executes in an ASP.NET environment, you can safely use the HttpContext.Current.Items dictionary for this. If your code executes in a WCF environment, you can store these things in the OperationContext. I don't want my code to be tightly coupled to either ASP.NET or WCF, so i wanted some kind of abstraction. This is the approach that i came up with.

First, we have the IRequestState interface:

    public interface IRequestState
    {
        T Get<T>(string key);
        void Store(string key, object something);
    }

This just offers a way to store objects and retrieve them. That's pretty much al we need, right?

Then we have the ASP.NET implementation:

    public class AspNetRequestState : IRequestState
    {
        public T Get<T>(string key)
        {
            return (T)HttpContext.Current.Items[key];
        }
 
        public void Store(string key, object something)
        {
            HttpContext.Current.Items[key] = something;
        }
    }

Very simple stuff... the AspNetRequestState implementation simply uses the HttpContext.Current.Items dictionary underneath to store and retrieve the objects.

For WCF, it is slightly more complicated. Every WCF call is an operation and it has a context as well, which is provided through the OperationContext class. The OperationContext class doesn't have an Items dictionary like HttpContext does, but it does have a way to add extensions to the context. We can use this extensions mechanism to store state which should be kept around for the duration of the current WCF operation. First, we need to define our Extension:

    public class MyExtension : IExtension<OperationContext>
    {
        public MyExtension()
        {
            State = new Dictionary<string, object>();
        }
 
        public IDictionary<string, object> State { get; private set; }
 
        // we don't really need implementations for these methods in this case
        public void Attach(OperationContext owner) { }
        public void Detach(OperationContext owner) { }
    }

The IExtension interface that we must implement defines the Attach and Detach methods but we don't really need them for what we're trying to do. This extension simply initializes a Dictionary instance and exposes it with a public getter. Now we can easily create our WcfRequestState implementation:

    public class WcfRequestState : IRequestState
    {
        private static IDictionary<string, object> State
        {
            get
            {
                var extension = OperationContext.Current.Extensions.Find<StateExtension>();
 
                if (extension == null)
                {
                    extension = new StateExtension();
                    OperationContext.Current.Extensions.Add(extension);
                }
 
                return extension.State;
            }
        }
 
        public T Get<T>(string key)
        {
            if (State.ContainsKey(key))
            {
                return (T)State[key];
            }
 
            return default(T);
        }
 
        public void Store(string key, object something)
        {
            State[key] = something;
        }
    }

Pretty simple as well, and pretty similar to the AspNetRequestState implementation. The AspNetRequestState implementation is able to simply use the HttpContext.Current.Items dictionary, which we can't use here. So when we want to access the 'State' dictionary in this implementation, we look it up in the current OperationContext's Extensions collection. If it's not there yet, we add a new instance of our MyExtension class to the OperationContext's Extensions collection.

Now we can use this wherever we need to store something for the duration of the current request, regardless of whether we're executing in an ASP.NET or WCF context. Just configure your IoC container to create instances of AspNetRequestState whenever an IRequestState instance is needed in your WebApplication, or configure it to return WcfRequestState instances in your WCF service. The code that needs to store some request state will no longer have to resort to using ThreadStatic fields, and it doesn't need to know about it's runtime environment either. It merely needs an instance of IRequestState.

What Would You Choose For New .NET Web Development?

15 commentsWritten on December 29th, 2008 by
Categories: ASP.NET, Software Development

Web development in the .NET world has gotten a lot more interesting in the last couple of years. A few years ago, the only choice we had for a web front-end was ASP.NET WebForms. Nowadays you can add ASP.NET MVC and Silverlight to the mix. Obviously each option has its pro's and con's, so it's not always clear which option should be used when you're starting to build a new Web application.

Let's start with ASP.NET WebForms. This is the option that has been available since the .NET platform was introduced, so there are already a lot of people experienced with it. There is also a ton of information available about the ins and outs of WebForms. And when it comes to commercial tool vendor support, it's clearly miles ahead of the other options since there are lots of commercially supported controls available for it. Unfortunately, WebForms was originally created to offer a similar development model to WinForms developers. The code-behind files and the event-driven way of working is very similar to how you would write code in WinForms applications. Obviously, web applications and windows applications are completely different things, so you can't reasonably expect the same way of working to be suitable for both.

Particularly, the Page and Control lifecycles are rather complex (at least much more complex than i would think they'd need to be) and often cause weird issues on complex pages. The event-driven model of both the Page class and the Control class (and its derivatives) seem to be the major cause of this. I think most WebForms developers have on more than one occasion spent long times debugging weird situations which were ultimately related to certain events triggering unexpected behavior in other controls on the same page. Granted, people with an in-depth knowledge of how these lifecycles really work don't run into this as much, or are capable of avoiding these problems altogether.

You could indeed make the argument that most of the problems that people are experiencing with WebForms are caused by those people's lack of understanding how it really works and how it really should be done. On the other hand, that is a pretty good sign that WebForms development isn't intuitive or clear enough, and that while Microsoft has tried hard to make it easy to use, most people seem to solve their WebForms problems by hacking around their problems, or playing around with settings until it seems to work.

Another major problem with WebForms is that it doesn't easily enable Test Driven Development. True, there are various patterns you can use to make sure you can write testable UI logic (up to a certain point anyway) but all in all, these approaches require more effort than should be necessary and you often end up wrapping a lot of stuff just to be able to test it. WebForms in general was never designed with testability in mind, and you will most definitely be confronted with that if you try to write testable UI code with WebForms.

As you can probably tell, i'm not a huge fan of WebForms. I think it's fine for simple applications, but for anything beyond that i'd like to avoid it as much as possible.

Microsoft's upcoming ASP.NET MVC framework aims to fix much of the issues i've mentioned above. It was designed with testability in mind, and although it's not perfect either (depending on who you ask), it is certainly a huge improvement over WebForms when it comes to writing testable UI code. You also have a lot more options when it comes to having the framework behave the way you'd like it to. Another important benefit is the fact that ASP.NET MVC kinda forces you to structure your code in a much more sane manner. You put as little logic as possible in the views, and you put most of it in the Controllers where it belongs. Obviously, that doesn't mean you should put business logic in the Controllers!

One of the downsides of ASP.NET MVC is that, due to its completely different way of working compared to WebForms, it comes with a much higher learning curve. Experienced WebForms developers might struggle with it at first, and might even be frustrated because most of their hard-earned WebForms experience no longer gives them a benefit. Some will probably enjoy it much more than WebForms, and some will probably dislike it strongly because it's so different. Another downside is that there are far less commercial tool vendors that are offering ASP.NET MVC controls, at least compared to WebForms. Although that might not be that big of an issue, since i suspect that it's easier to develop nice looking and reusable controls yourself when using ASP.NET MVC than it is to do so when using WebForms.

If you're starting out with ASP.NET MVC, i think it's safe to assume that you won't proceed as quickly as you're used to at first. But once you're used to the new way of working, i'm pretty sure that it enables you to rapidly implement new pages and new functionality in a very clean way. For true web applications, i would probably pick ASP.NET MVC everytime, unless you don't have the room to get over the learning curve. Also, when combined with a client-side javascript library like JQuery, this approach seems very compelling. Yes, i know you can use JQuery with WebForms as well, but it just seems to lend itself better to the MVC approach.

And then there's Silverlight. There are already a couple of reasons why people would not want to use Silverlight for web applications. For starters, even though the application is running in a browser, it's not really a true web application is it? Your users can't bookmark pages, using the back button leads to unexpected behavior, there aren't that many options for Search Engine Optimization, etc... However, if your only web-related requirement is that your application needs to run in a browser, without having to worry about any of the typical expected requirements for web applications, then Silverlight is a pretty interesting choice as well. You can very easily create very rich applications, with possibilities that are either impossible, or extremely difficult to do with typical web development platforms.

Our Genesis web front-end is developed in Silverlight. The UI not only looks great, but you can navigate between all of the available data in a manner that is simply much easier to develop than it would be for typical web apps. It's very easy to create that 'wow'-effect with your users in Silverlight. Obviously, that 'wow'-effect isn't the most important part of your application, but it does count for something.

Another interesting aspect of Silverlight development is that you can go back to a more statefull development model compared to the typically stateless nature of web aplications. After all, the cost of that state is no longer something your web server has to bear. It's now the client who has to keep that state around, so you can avoid a few extra roundtrips here and there as well. Then again, you really don't want your Silverlight application using huge amounts of RAM in your user's browser either so you do need to take care not to go overboard with it. The development model is completely different than both WebForms and ASP.NET MVC, but if you already have people with WPF experience the learning curve is probably not that high.

However, testability isn't great (yet) when it comes to Silverlight. It's possible to write testable code, and you can execute tests in the Silverlight runtime, but it doesn't really lend itself to a true TDD approach yet although i hope that will improve in the future.

So there you have it... the three options for .NET web development. I'd stay away from WebForms altogether from now on, and i'd decide between ASP.NET MVC and Silverlight on a case by case basis. What, you weren't hoping for a definitive answer were you? :)

I would be interested in hearing your thoughts about pro's and con's of any of the options... particularly things that i haven't mentioned, or if you just plain disagree with my statements. Which option would you prefer to use, or not to use?

« Older Entries

Meta

Recent Posts

Categories