WCF And Large Amounts Of Data

12 commentsWritten on September 14th, 2008 by
Categories: WCF

If you're using my Request/Response service layer, or any other WCF service that might require sending large amounts of data over the wire, you quickly bump into some limits that WCF enforces by default. Among the billions of configuration options for WCF, there are luckily some options that allow you to easily send large amounts of data from a service to a client.

I typically use the following options:

For my binding configuration i usually set the maxReceivedMessageSize, maxStringContentLength and the maxArrayLength properties to their maximum values:

    <bindings>

      <netTcpBinding>

        <binding name="MyTcpBinding" maxReceivedMessageSize="2147483647" receiveTimeout="00:30" sendTimeout="00:30">

          <readerQuotas maxStringContentLength="8192" maxArrayLength="20971520" />

        </binding>

      </netTcpBinding>

    </bindings>

This example shows these settings for the netTcpBinding... i've also used them with the wsHttpBinding. Not sure how well it works with other bindings though.

I also set the maxItemsInObjectGraph setting of the DataContractSerializer to make sure i don't hit the default limit if i have to send a large object graph over the wire:

    <behaviors>

      <serviceBehaviors>

        <behavior name="MyBehavior">

          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>

          <serviceMetadata />

        </behavior>

      </serviceBehaviors>

    </behaviors>

You have to apply these settings both server and client side to get it working properly and you need to refer to these settings in your service and endpoint settings:

      <service name="Brion.Library.ServerSide.WCF.WcfRequestProcessor" behaviorConfiguration="MyBehavior">

        <host>

          <baseAddresses>

            <add baseAddress="net.tcp://localhost/RequestProcessor"/>

          </baseAddresses>

        </host>

 

        <endpoint contract="Brion.Library.Common.WCF.IWcfRequestProcessor" binding="netTcpBinding"

              bindingConfiguration="MyTcpBinding" />

 

      </service>

    <client>

      <endpoint address="net.tcp://localhost/RequestProcessor" binding="netTcpBinding"

            name="IRequestProcessor" bindingConfiguration="MyTcpBinding" behaviorConfiguration="MyBehavior"

            contract="Brion.Library.Common.WCF.IWcfRequestProcessor" />

    </client>

Now, i don't recommend sending such large amounts of data through WCF services... but in the case of using my Request/Response service layer, the amount of data you're sending over the wire pretty much depends on which kind of requests (and how many of them) you're batching so i think it's better to make sure that at least the configuration allows for it. So obviously, it's best to keep an eye on the size of your messages to make sure you're not doing anything crazy. Being able to send your entire database over the wire doesn't mean it's a good idea to actually do so ;)

« Technical Interviews: What Questions To Ask?Code Health »

This entry was posted on Sunday, September 14th, 2008 at 13:00 and is filed under WCF. You can subscribe to any comments to this entry via RSS 2.0. You can leave a response, or trackback from your own site.

  • Pingback: Dew Drop - September 14, 2008 | Alvin Ashcraft's Morning Dew

  • http://www.ditmer.dk Jon

    Thanks a lot :)
    The info regarding maxItemsInObjectGraph solved a problem that i have spent at lot of time pursuing until now!

  • Hari

    How do I send large amounts of data TO the WCF? I get a Bad request exception when I try to do so.

  • http://davybrion.com Davy Brion

    @Hari

    if you’re hosting through IIS, make sure you enable large uploads in your web.config:
    http://weblogs.asp.net/jgalloway/archive/2008/01/08/large-file-uploads-in-asp-net.aspx

  • http://danny-t.co.uk DannyT

    Sorry to comment on such an old post but it is on top of google for “sending large amounts of data with wcf” unsurprisingly ;)

    Our scenario is that we are using WCF to keep multiple clients in sync with a server-side datastore. Now when a new client is added we need to sync all of the data and from then on just offer incremental updates. This initial sync has led to our finding the above mentioned configuration within WCF (which I believe should be the default with recommendation to reduce – but that’s another discussion). Your last sentance: “Being able to send your entire database over the wire doesn’t mean it’s a good idea to actually do so ;) ” intrigued me to asking what you would suggest instead as this is essentially what we’re doing for new systems?

  • http://davybrion.com Davy Brion

    @DannyT

    well, if you’re doing it to implement some synching-features, then there’s nothing wrong with it

    my comments was targeted to devs who sometimes send a _lot_ more data over the wire than they should for regular service calls ;)

  • http://danny-t.co.uk DannyT

    Cool, thanks for replying always nice to have a bit of reassurance!

    For the benefit of future searchers that land here with a similar scenario, I am also considering getting the server to generate the initial state of the database and then it becomes just a regular file download. This then means the settings in wcf that I’ve just maxed out become more relevant again rather than being set to highest for the sake of one initial transaction.

  • http://simon.brangwin.myopenid.com/ Simon Brangwin

    Hi Davy,

    Thanks for your excellent request/response service layer.

    We are having a problem when using the net.tcp binding and hosting with WAS. How do you normally go about registering the Request and Response assemblies. When hosting in an ASP.NET application using an http binding under IIS, we would normally call AddRequestAndResponseAssembly methods via the Global.asax. Unfortunately this doesn’t get called when running under WAS.

    I’ve attempted to create a custom ServiceHostFactory and subscribe to the Host.Opening event to do the registration. It seems to work and calls my ComponentRegistration.Register() method, however when Agatha tries to deserialize a request from my client, it complains about unknown types. It’s like the ComponentRegistration hasn’t worked.

  • http://simon.brangwin.myopenid.com/ Simon Brangwin

    Davy, my component registration method is as follows. It definitely gets called before anything else, but by the time Agatha tries to deserialize something, it seems to have lost its config. The interesting thing is that it works ok if I set the projects to Start and run it in the VS debbuger.

    public static void Register()
    {
    var agathaConfig = new ServiceLayerConfiguration(typeof(Agatha.Unity.Container));
    agathaConfig.AddRequestHandlerAssembly(Assembly.GetExecutingAssembly());
    agathaConfig.AddRequestAndResponseAssembly(typeof(GetClaimRequest).Assembly);
    agathaConfig.AddRequestAndResponseAssembly(typeof(DischargeCertificateReceivedCommand).Assembly);
    agathaConfig.Initialize();

  • http://simon.brangwin.myopenid.com/ Simon Brangwin

    Hey Davy,
    We’ve figured it out. We were hooking the Opening event on the service host and doing the ComponentRegistration in there. By the time it raises that event though Agatha has already tried to deserialize types.
    What we should be doing is calling the component registration in the CreateServiceHost method itself before even creating the service host! After making this change it works as expected :)

    • http://davybrion.com Davy Brion

      Hi Simon,

      glad to hear your problem is fixed :)

      for future reference: you’re better off asking your questions on the official agatha google groups list: http://groups.google.com/group/agatha-rrsl/

      That way other people might answer when i’m not available, and every answer can be indexed by google or referred to later on ;)

  • Aajay78

    does not make sense and it is not wokring

Meta

Recent Posts

Categories