HPCS2007 poster: Towards an HTTP-based Service Platform with High Scalability

This year, HPCS comes to Saskatoon, and I submitted a poster titled "Towards an HTTP-based Service Platform with High Scalability". The abstract is the following.

HTTP is a message transportation method used by the majority of so-called web services or services on the web, which are implemented by using SOAP or POX/JSON. The application of Ajax has dramatically increased the load on service platforms by long-lived connection and frequent polling. For most multithreading platforms implementing thread-per-request policy, this raises scalability problems for threads since the performance of each thread will be degraded when the total number of threads increases. Service orchestrations can also introduce thread scalability problems when they have long lifetime and involve message exchanges with multiple high-latency partner services.

This poster discusses the issues for an HTTP-based service platform to achieve high scalability. We compare the options for platform implementation of IO, thread pool, asynchrony, and continuation, and propose a design for highly scalable platform for both atomic services and service orchestrations.

Evolution of eBay platform architecture

Thanks to Michael Platt's blog. I downloaded the slides of eBay architecture and just finished reading it. The interesting points to me are as follows.

1. Scalability and maintainability are so important for large-scale distributed systems. The scalability problem is the major reason for the evolution of eBay's platform. And maintainability is the major consideration for the future development.
2. Scaling-out is the proper long-term solution for increasing load.
3. The principle of separation of concerns benefits scalability besides easiness of development and maintenance.
4. Combination of synchronous and asynchronous interactions is a good practice.

Testing, modeling, and simulation of large-scale service-oriented systems

When many enterprises transfer to web services, large-scale intra- and inter-organization service-oriented systems are emerging. Compared to software systems such as stand-alone desktop applications and multi-tier web applications, we almost have no experience about testing large-scale service-oriented systems. A new level of complexity arises from the scale, open, and dynamics of such distributed systems.

When a service-oriented application is deployed, we expect that most aspects of its behavior are by design. However, when these applications are integrated to form large-scale systems, we have no confidence about how the whole system will work based on the knowledge of each application that could be very limited if the integrations are inter-organization. Service-oriented systems are open: A new application following the service contracts can join in and leave the system anytime. The dynamics of these systems are not just reflected on their scales, but also the structures. An approach to elicit knowledge for designing and managing such systems is modeling and simulation.

Frank Cohen writes an excellent article on testing SOA . The purpose of testing is more focused on performance and scalability rather than functionality. Werner Vogels has two posts on scalability and performance that help to understand these terms in the context of distributed systems.

Stateless process manageability

It is a common principle of SOA that services are stateless. The statelessness supports loose coupling, reusability, and composability of services. In the principled Web architecture, stateless communication improves the system's visibility, reliability, and scalability. When the applications of cookies and other approaches in the Web make the communication stateful, the visibility and reliability of Web applications are degraded. Similar to the case of Web architecture, services can be developed to manage state information, but that would detract from some benefits of SOA. However, it does not introduce any conflicts in managing stateful processes and process instances using stateless web services. That directly implies that the process state information is carried by the messages or events going in/out the manageability services, and some applications maintaining the state information are hidden behind the manageability services.



Manageability services communicate with the applications maintaining state information through specific application protocol. On the other side, the communication between manageability consumers and manageability services just need to follow the open service protocol. Databases can store large amounts of state information of processes and process instances during and after their lifetime. Another kind of application, like message topics, maintains a certain group of messages with state information. The management of BPEL processes is essentially monitoring, tracing, and controlling the states through the corresponding manageability service endpoints.

Representing Policies as Aspects

The theories about aspects and aspect-oriented modeling are still developing. An aspect can be described by joint points and corresponding advice in terms of Aspect-Oriented Programming (AOP), which is an approach addresses the code tangling and code scattering issues. A joint point is the place where the concern will crosscut the application. An advice is the action that will be executed at the joint points. Similarly, a joint point of an autonomic policy is the target component or process whose behavior is modified to achieve the goal. The advice is the configuration parameters for managed resources. E.g. for the policy that

for the customers whose orders sum up to more than $3,000 in a day, apply 5% discount for the amount over $3,000.

the joint point is the pricing process of an item ordered by a customer, and the advice is to apply 5% discount to the amount over $3,000 if the sum of the customer's orders in that day exceed $3,000.

We could model policies as aspects. However, there are still many differences between policies and the aspects that we can implement using current AOP languages and tools. Policies are hierarchically organized. The subject of a high-level policy may have no idea about the details how its goal will be realized by low-level policies. The representations of policies may vary for the heterogeneous managed resources. The autonomic managers transform the policies to the versions that their managed resources can understand, and distribute them to the targets. Policy transformation and distribution will occur on each hierarchy of the system. The autonomic managers at higher hierarchies do not need to know all the details about how to deploy the policies. The complexity of these autonomic managers is reduced in this way.

The joint points and advice of a policy are modified in the process of transformation and distribution. E.g. for a policy to increase the service capability for all the web servers, the joint points are all the web servers in the resource pool, and the advice is to increase the service capability. When the policy is distributed to an Apache HTTP server, the joint point turns to the Multi-Processing Module, and the advice will be to increase the value of MaxClients, MinSpareThreads, and MaxSpareThreads with the constraint that MaxClients > ThreadsPerChild * ServerLimit.

The parameters in a policy are initialized in transformation and distribution. Suppose an enterprise has branches in charge of the business of different areas. Now a new policy as follows needs to be executed by all the branches.

For all customers whose orders sum up to more than certain amount in a day, apply certain discount for the amount over the amount.

The policy will be transformed to different versions at branches. For example, the amount and the discount are $3,000 USD and 5% for American customers, and $4,000 CAD and 4% for Canadian customers.

Policies transformation and distribution need a policy specification language. There are a number of running projects related to policy specification languages. XML-based specifications are widely endorsed. However, an industrial standard is in demand.

It will be easy to distribute policies represented in XML among autonomic elements using Web services. The policy will take effect once it is distributed and deployed at load time or runtime. AspectJ is a seamless aspect-oriented extension to the Java programming language. AspectJ defines the syntax for aspects and supplies a weaver to compose the system from classes and aspects. The future AspectJ will support XML representation of aspects and deployment at load time. The policy example in Section 3 may be implemented by the following code snippet in AspectJ.

public aspect TestDiscount {
pointcut checkOut() : call(public double getPrice(..));
after() : discount() {
applyDiscount(AMOUNT, PERCENT);
}
}

The joint point is a call to getPrice(..) method that retrieve the price of an item that the customer orders. The advice is to call applyDiscount(int amount, int percent) to apply the discount specified by certain AMOUNT and PERCENT.

Policy-based Systems Management

Policies are "high-level overall plans embracing the general goals and acceptable procedures" [1]. Policies are common methods of organizations and systems management. Policies reside at the top of the autonomic computing architecture [2]. Autonomic policies specify management requirements and criteria, the mechanism to distribute the policies among autonomic managers, and the configuration parameters for managed resources [3]. Currently, policy-based approaches are widely used in enterprise network management relevant to QoS and security. A team in IBM is developing a policy toolkit (PTK) that comprises modules that parse, edit, validate, transport, transform, and auto-update policies [4]. The policy-based IT management demo in IBM Emerging Technologies Toolkit (ETTK) shows how to manage database is using a policy-based approach in an autonomic computing environment. The policy management for autonomic computing package is released as well.

In a hierarchical autonomic system, the autonomic managers share the goals of high-level policies, while each autonomic manager has a private version of policies according to local environment and task. The same policy will have various representations for diverse autonomic managers and at different hierarchies of the systems. Policies are persistent during short periods. However, policies are also dynamic due to changing business and IT environments during long periods. The dynamic characteristics of policies will be more and more distinguishing with the trend of on-demand business, and request rapid deployment of policies on the fly.

Policies may be recorded in natural languages, specification languages, or general markup languages like XML. In object-oriented era, it is straightforward to represent policies as objects. Policies are represented as objects in some policy tools and policy specification languages like Ponder [5]. Policy objects can have attributes like modality, policy subject, policy target, policy goal, and policy constraints, operations like create, destroy, update, and query [6]. The interactions between policies include policy hierarchizing and policy prioritizing. The policy is a kind of 'cross-cutting' concern, which means a high-level policy will affect the behaviors of several sub-systems; many elements in a system may depend on the same policy to guide their actions; and a low-level policy may play roles in a number of atomic modules comprising an element. The crosscutting of policies results in tangled codes. It is hard to manage the policies that are scattered around modules. We suggest an aspectual view of policies for such issues.

[1] Merriam-Webster Online Dictionary, http://www.webster.com/cgi-bin/dictionary? va=policy visited on Jan. 28, 05.

[2] S.R. White, J.E. Hanson, I. Whalley, D.M. Chess; and J.O. Kephart, "An architectural approach to autonomic computing", Proceedings of International Conference on Autonomic Computing, May 2004, pp. 2-9.

[3] Murch, R., Autonomic Computing, Prentice Hall PTR, 2004.

[4] D.C. Verma and S.B. Calo, "A toolkit for policy enablement in autonomic computing"; Proceedings of International Conference on Autonomic Computing, May 2004, pp. 260-271.

[5] N. Damianou, N. Dulay, E. Lupu, and M. Sloman, "The Ponder Policy Specification Language", Proceedings of the International Workshop on Policies for Distributed Systems and Networks, Lecture Notes In Computer Science; Vol. 1995, Springer-Verlag, London, UK, 2001, pp. 18-38.

[6] J. D. Moffett and M. S. Sloman, "The Representation of Policies as System Objects", Conference proceedings on Organizational computing systems, 1991, pp. 171-184.