Dealing with large data volumes in webservice calls
Once you start sending large volumes of data over a webservice invocation, you often run into performance issue.
There are a range of techniques that can prove useful here:
- Cache some of that information on the client side, instead of sending over everything for each webservice invocation. This is especially useful when some of the information to be sent over is relatively static
- Paginate the information to be sent e.g. instead of sending over entire rows from a database select, send over the first 50, and let the client ask for the next 50 and so on. In this case there is a tradeoff between making multiple remote calls and one single call that returns a large chunk of data.
- Have detail levels. An example of this is the UDDI API, where there a number of 'find' calls for returning things like 'BusinessInfo' list which doesn't have all the data stored about a business- just the more commonly used pieces. Clients that need detailed information about a specific business or set of businesses can 'drill down' for more information. Again, you'd have a tradeoff here between making multiple remote calls and one single call.
- Compress the data before sending it. This is covered in more detail here and here. The issues that this approach throws up relate to this being non-standard, and thus having an ad-hoc mechanism for getting the client and service agree on the compression mechanism.