Monday, January 3, 2011

Streaming files over WCF

With all the REST buzz nowadays, using SOAP/WCF services seems so ‘old school’. But the right tool for the right job, so let’s move on and focus on the problem I want to handle: sending large files over WCF(I blogged about this before with some tips, but people keep asking questions, so I describe the process step by step).

Table Of Contents

  • Service Contract
  • Service Implementation
  • Configuring The Service
  • Hosting The Service in IIS

WCF Service Contract

Let’s start by laying the groundwork for the WCF Service. As stated the service must enable users to upload a file to the web server which hosts it. Thus the resulting service contract only contains one method, named Upload(…).

Open up Visual Studio 2010 and create a new blank solution. Next add a new Empty Web application called Sample.Services. Afterwards go to Add new item, choose  the WCF service template and type FileUploadService.cs as the name. This will automatically generate some files (IFileUploadService.cs, FileUploadService.cs) and update the web.config with some default configuration settings.

Open the IFileUploadService file and replace the generated code with the code in Listing 1:

Listing 1 – Service Contract

   1:  [ServiceContract(Namespace = "http://sample/wcf/services")]
   2:  public interface IFileUploadService
   3:  {
   4:      [OperationContract]
   5:      UploadResponse Upload(UploadRequest uploadRequest);
   6:  }

As you can see in Listing 1 above they mention two other classes, namely:

  1. UploadRequest
  2. UploadResponse

These classes are both decorated with the MessageContract attribute.

Note: WCF requires that the parameter that holds the data to be streamed must be the only parameter in the method.

Note: If your service operation contains a parameter of a type which has been decorated with the MessageContract attribute then all of the parameters and the return type used in this operation must be of a type to which this attribute has been applied.

You cannot mix parameters of a primitive type with message contracts. For instance if you want the Upload(…) method to return a boolean indicating if the upload succeeded or failed then you have to wrap this in another message contract.

The FileInfo class specifies the structure of a SOAP envelope for a particular message.

Listing 2 – FileInfo class

   1:  [MessageContract]
   2:  public class UploadRequest
   3:  {
   4:      [MessageHeader(MustUnderstand = true)]
   5:      public string FileName { get; set; }
   7:      [MessageBodyMember(Order = 1)]
   8:      public Stream Stream { get; set; }
   9:  }

Note: By applying the MessageHeader attribute to the FileName and Length propery you place this information in the header of the SOAP message. When streaming a file the body of the SOAP message must only contain the actual file itself. By applying the MessageBodyMember attribute to the Stream property you place it in the body of the SOAP message.

Listing 3 displays how to setup the return value as a message contract. The response only contains a boolean value to indicate if the upload was successful.

Listing 3 – FileReceivedInfo class

   1:  [MessageContract]
   2:  public class UploadResponse
   3:  {
   4:      [MessageBodyMember(Order = 1)]
   5:      public bool UploadSucceeded { get; set; }
   6:  }

Service Implementation

Now it’s time to provide an actual implementation for the service contract.

Open the FileUploadService.cs file and replace the generated code with the code in Listing 4. The code is pretty straightforward. It reads the incoming stream and saves it to a file using familiar .NET code.

The Upload(…) method’s return type is of the UploadResponse type. If the upload succeeds the UploadSucceeded property is set to true, if it fails this property is set to false.

Listing 4 – Service Implementation

   1:  [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall,
   2:      ConcurrencyMode = ConcurrencyMode.Single)]
   3:  public class FileUploadService : IFileUploadService
   4:  {
   5:      #region IFileUploadService Members
   7:      public UploadResponse Upload(UploadRequest request)
   8:      {
   9:          try
  10:          {
  12:              string uploadDirectory =
  13:                  ConfigurationManager.AppSettings["uploadDirectory"];
  15:              // Try to create the upload directory if it does not yet exist
  16:              if (!Directory.Exists(uploadDirectory))
  17:              {
  18:                  Directory.CreateDirectory(uploadDirectory);
  19:              }
  21:              // Check if a file with the same filename is already 
  22:              // present in the upload directory. If this is the case 
  23:              // then delete this file
  24:              string path = Path.Combine(uploadDirectory, fileInfo.FileName);
  25:              if (File.Exists(path))
  26:              {
  27:                  File.Delete(path);
  28:              }
  30:              // Read the incoming stream and save it to file
  31:              const int bufferSize = 2048;
  32:              byte[] buffer = new byte[bufferSize];
  33:              using (FileStream outputStream = new FileStream(path,
  34:                  FileMode.Create, FileAccess.Write))
  35:              {
  36:                  int bytesRead = request.Stream.Read(buffer, 0, bufferSize);
  37:                  while (bytesRead > 0)
  38:                  {
  39:                      outputStream.Write(buffer, 0, bytesRead);
  40:                      bytesRead = request.Stream.Read(buffer, 0, bufferSize);
  41:                  }
  42:                  outputStream.Close();
  43:              }
  44:              return new UploadResponse
  45:                         {
  46:                             UploadSucceeded = true
  47:                         };
  48:          }
  49:          catch (Exception ex)
  50:          {
  51:              return new UploadResponse
  52:                         {
  53:                             UploadSucceeded = false
  54:                         };
  55:          }
  56:      }
  58:      #endregion
  59:  }

Configuring The Host

Start by opening the web.config file. Next specify the directory in which the service should save the incoming files.

Listing 5 – Upload directory

   1:  <appSettings>
   2:    <add key="uploadDirectory" value="C:\temp\upload" />
   3:  </appSettings>

Now let’s first add a new behavior to our serviceBehaviors node.

Listing 6 – Service Behavior

   1:  <behaviors>
   2:    <serviceBehaviors>
   3:      <behavior name="FileUploadServiceBehavior">
   4:        <serviceMetadata httpGetEnabled="True" httpsGetEnabled="False" />
   5:        <serviceDebug includeExceptionDetailInFaults="False" />
   6:      </behavior>
   7:    </serviceBehaviors>
   8:  </behaviors>

The behavior specifies that the service should not propagate exception details and that it’s metadata should be shared over HTTP.

Next up is the binding for the service. A couple of its properties need to be tweaked to fit the needs of the service, namely:

  1. transferMode: Set it to Streamed to enable streaming.
  2. messageEncoding: The message encoding is set to MTOM encoding which is a mechanism for transmitting binary attachements with SOAP messages.
  3. maxReceivedMessageSize: Set to 64 megabytes to allow large files to be uploaded.
  4. maxBufferSize: Set to 64 kilobytes.
  5. receiveTimeout: Set to 10 minutes. If the file fails to upload within this time frame an exception will be thrown.

Note: Configure these settings according to the needs of your application.

Listing 7 – The Binding

   1:  <bindings>
   2:    <basicHttpBinding>
   3:      <!-- buffer: 64KB; max size: 64MB -->
   4:      <binding name="FileUploadServiceBinding"
   5:               transferMode="Streamed"
   6:               messageEncoding="Mtom"
   7:               maxReceivedMessageSize="67108864" maxBufferSize="65536"
   8:               closeTimeout="00:01:00" openTimeout="00:01:00"
   9:               receiveTimeout="00:10:00" sendTimeout="00:01:00">
  10:        <security mode="None">
  11:          <transport clientCredentialType="None" />
  12:        </security>
  13:      </binding>
  14:    </basicHttpBinding>
  15:  </bindings>

Last but not least is the configuration for the service itself.

Listing 8: FileUploadService configuration

   1:  <services>
   2:    <service behaviorConfiguration="FileUploadServiceBehavior"
   3:      name="Sample.Services.FileUploadService">
   4:      <endpoint address="" binding="basicHttpBinding" contract="Sample.Services.IFileUploadService"
   5:                bindingConfiguration="FileUploadServiceBinding">
   6:      </endpoint>
   7:    </service>
   8:  </services>

Hosting The Service

Now that the service has been setup it’s time to host it. Don’t try to host it in your ASP.NET Development Server(default option), instead choose IIS or the new IIS Express option (if installed). Therefore right click on your web application and choose Properties. Go to the Web tab and change Servers option from Use Visual Studio Development Server to Use Local IIS Web server.

That’s it!


Justin said...

Thanks for the post. I have been wrestling with this for a few days now, and you answered many of my questions. I am having a lot of problems, however, getting my service to be hosted in IIS. I went through all of those steps, but I continuously get errors when trying to add the service reference. Am I doing something wrong there?

Mike Pelton said...

Incredibly helpful and clear. And it works too! Many thanks indeed.

Eyal said...

Just one thing to say: thank you!
The internet should have more articles like this, explaining different parts of Web.config

ziyad said...

I think you have some problem with

listing 4 in line number 24.

string path = Path.Combine(uploadDirectory, fileInfo.FileName);

I guess It should be
string path = Path.Combine(uploadDirectory, request.FileName);

Thank you very much for the nice post

Anonymous said...

FYI, in Figure 2, the class name used in UploadRequest instead of FileInfo as described.

Unknown said...

Thanks for this example. However, I am unable to upload files from a client that uses the service as a Service Reference. Once inside the service, the Stream is empty/corrupt (Stream.Length shows '(request.Stream).Length' threw an exception of type 'System.NotSupportedException' in the debugger).

Any ideas would be helpful.

Unknown said...

Edit: I just read that Length is not supported in this case, however, when I attempt to read from request.Stream, 0 bytes are read (even though on the client, the FileStream that I am sending is a valid file, with 12726 bytes.

rhinoman said...

I concur, image is valid and ends upon the local drive as zero bytes? Going to give the ByteStreamHttpBinding version a crack now.

", when I attempt to read from request.Stream, 0 bytes are read (even though on the client, the FileStream that I am sending is a valid file, with 12726 bytes."

rhinoman said...

Ok Folks, it was a quick fix, a beer to think it over and had it.

Before calling the service, you must set the stream position to zero:

ServiceReference2.FileUploadServiceClient upl = new ServiceReference2.FileUploadServiceClient();
stream.Position = 0;
upl.Upload("kaz0002.jpg", stream);



How to call this service from an ASPX file upload controller

Edwin van Hierden said...

I'm a newby with WCF and MS Visual Studio Express 2012 and found this article very helpfull and clear. But now I would like to generate a WSDL for the client. Is that possible with MS VS Express 2012?

Anonymous said...

Why Microsoft doesn't provide such examples?
They have a very good advertisement but lousy tutorials.
And if they provide information about their products, than those guys are Gurus or what the f....
Thanks very much for this here.

Arthur Ozga said...

Thanks for the post, I just have two issues with trying to impliment the sample:

1) VS 2010 keeps telling me that ConfigurationManager (Listing 4.13) isn't within any of the used namespaces. I tried using
System.Object and System.Configuration, but to no avail. What should I do to get VS 2010 to recognize ConfigurationManager?

2)For the config-related listings (#5-8), it isn't clear to me where to add the text. Should I first delete all the default text? Should i keep the heading? What about the element? Sorry if this sounds like an oversimplistic question--I admittedly have little experience with XML or XAML, whichever one is being used here...

Anonymous said...

I am facinf an iisue of bad request 400 error when file lenght is greater than 64 kb.
any one help me
thanks in Advance

rhinoman said...

Anonymous, please check every setting and make sure there are no errors. I was able to upload/download files which were megabytes in size no problems.

Good Luck

jhonjairoroa87 said...

Hi, I'm trying to upload a file using the attributes in the form:

enctype="multipart/form-data" action="http://localhost/Services/FileUploadService.svc/" method="post"

but i'm gertting the error:

415 Cannot process the message because the content type 'multipart/form-data; boundary=----WebKitFormBoundary0XZn8zAivDWByghj' was not the expected type 'multipart/related; type="application/xop+xml"'.

Anonymous said...

A brilliant example of how it should be done. Many thanks :)

Anonymous said...

Thanks for very nice post. I tried to upload one MB file and getting exception. Can you please give us client side configuration details as well. I am really not sure how to configure that.

Thanks in advance

Anonymous said...

Awesome post, I've been dealing with this issue and this solve it magically. By the way I'm using windows service host, and works great!


Paul Janssen said...

Do you have an example of how the client would invoke the UploadFile method?

rhinoman said...

Paul Janssen - it is a web service so it is called using that technique.


Anonymous said...

Can we transfer large data of 50-100gb using this technique?