Fork me on GitHub

The Job Based Architecture focuses on defining the application as a list of Jobs where each Job type is assigned to a Team to execute. The architecture follows the natural blocks of code in structured programming and enhances this by allowing the various blocks of code (Jobs) to be executed by the most appropriate thread (Team).

Real world design pattern

Looking around an office you can see this design pattern in use. Within an office there are various teams responsible for different jobs - eg sales team for selling, accounts team for invoicing, IT team to support systems. Each team is specifically set up to handle their assigned jobs.

The teams operate efficiently with each other to accomplish the work. An example of this efficiency is interaction via email. Should sales need a customer to be invoiced they can send off an email to accounts requesting this and then get on with the other jobs at hand. At some point later sales will receive email confirmation that the customer was invoiced. The focal point of this interaction is that sales were able to accomplish other jobs while accounts did the invoicing. Sales did not spend their time hitting the refresh on their mail intently waiting for the invoice confirmation email.

Sales hitting refresh waiting for the invoice confirmation email is typically what happens when single servicing thread design patterns such as Thread per Request are utilised. As blocking operations (such as database and web service calls) tie up the thread, the thread sits idle waiting for a response and can not be used for other operations. This requires increasing the overall number of threads to cope with greater loads rather than following the office design pattern and scale only the teams responsible for doing more jobs or bigger jobs.

In essense, having to have a sales person doing invoicing means they are tied up from attending to customers and customers not being attended to are lost sales.

Issues addressed by the Job Based Architecture

Issues addressed by the Job Based Architecture are:

  • Limitations of single servicing thread architecture
  • Inversion of control, not just dependency injection

Limitations of single servicing thread architecture

To illustrate the limitations of the single servicing thread architecture, take for example the below sequential processing steps to service a web request:

    try {

      // Code to parse the request

      // Code to handle request via retrieving content from a database

      // Code to generate response

      // Code to send response

    catch (Exception ex) {
      // Code to handle exception
    }

Following the single service thread architecture (typical of the Thread per Request/Connection design pattern), each step would be executed sequentially by a single thread responsible for servicing the request. This works well and is the basis for most application servers. It however, has the limitation that it ties up a thread to service the request and can become problematic with blocking operations (such as database calls).

To illustrate the blocking operation issue with the single servicing thread architecture, let there for example be a large number of requests coming into the server (larger than the number of threads in the thread pool servicing requests). In this situation, each thread is processing a single request and all remaining requests are queued waiting for a thread to become available. Each queued request must wait until every request in front of it is passed to a thread before it can get serviced by a thread. Should the queue of requests be significantly larger than the number of threads, the request may end up waiting for many requests in front of it to be completely serviced before it has even started to be processed. The issue with the single servicing thread architecture is that it loses its efficiency as the thread spends the majority of its time servicing the request by idly waiting for the database blocking calls to return. In other words, requests are waiting for idle workers to get the job done.

The queuing of requests in single servicing thread architecture becomes even more of a problem when requests are not all alike. Take for example the following processing steps for servicing requests for a website's static home page and static images:

    try {

      // Code to parse the request

      // Code to retrieve page/image from cache

      // Code to send response

    catch (Exception ex) {
      // Code to handle exception
    }

The above code using a cached response will be significantly faster than the database network call response code. In the above situation with a large number of requests coming into the server, these smaller cached requests must also wait for the larger database requests in front of it in the queue to be finished before it is serviced. This becomes a significant problem if the time to process these requests becomes so time consuming that the user is not willing to wait for the home page and goes elsewhere (eg the problem of the sales person who is busy with other work and does not attend to the customer).

The problem of a single servicing thread is not just restricted to web servers. Architectures using queuing, SOA, batching and so forth are all likely to have blocking calls that require increasing overall thread counts to cope with increasing capacity.

Inversion of control, not just dependency injection

Dependency injection relies on a Factory to construct the inital object that requires all dependencies injected into it. Typically this may be taken further by dependency injection solutions to provide frameworks that reduce the need for direct access to the Factories. It however, has the problem that all dependencies must be injected into the object before the object can be used.

To illustrate this take for example the following object (non-important code commented out):

    public class InjectedObject {

        @resource
        private java.sql.Connection connection;

        @resource
        private Cache cache;

        public String getContent(String key) {
            try {
                // Attempt to obtain from cache
                String content = this.cache.getContent(key);
                if (content != null) {
                    return content;  // return content from cache
                }
                // Obtain content from database
                String content = this.getContentFromDatabase(key, this.connection);
                this.cache.storeContent(key, content);
                return content;
            catch (Exception ex) {
                // code to handle exception
            }
        }

        private getContentFromDatabase(String key, java.sql.Connection connection) {
            // code to retrieve database content
        }
    }

Before the above object can be used it must be injected with both a java.sql.Connection and Cache dependency. Should most content be available in the Cache the injection of a java.sql.Connection becomes unnecessary processing, especially if the java.sql.Connection is retrieved from a connection pool causing both blocking to wait for an available java.sql.Connection and contention on the connection pool locks.

It would be better to break the object into two to enable retrieving content from the Cache to not depend on a java.sql.Connection being made available. Doing this would necessitate creating separate objects and subsequently breaking the interface of the client code. Should separate threads be used for cache retrieval and database calls (as described by the issue of single servicing thread architecture) the client becomes involved with the internal behaviour further breaking encapsulation.

Note it is possible to inject a javax.sql.DataSource rather than java.sql.Connection to alleviate this problem, however in doing so:

  • requires additional plumbing code to ensure the java.sql.Connection is closed (and possibly passed around to other functionality involved in the transaction on the java.sql.Connection)
  • still suffers blocking waiting for the java.sql.Connection when one is required

To summarise, just using dependency injection has limitations and the following enhancements are necessary to improve its efficiency:

  • Only the necessary dependencies are injected for each block of code
  • Dependencies are ready to use on injecting

Job Based Architecture

The Job Based Architecture focuses on providing the following:

  • Ability to select the thread responsible for executing each block of code
  • Only the necessary dependencies are injected for each block of code
  • Dependencies are ready to use on injecting

Along with the above there are two additional focuses that have been implied by using blocks of code:

  • Output of a block of code may be input to the next block of code
  • Not all blocks of code sequentially occur one after another. if statements and catch blocks can cause different blocks of code to be executed

The following diagram illustrates each of these focuses providing an overview of the Job Based Architecture:

Job Based Architecture Overview

To better explain the Job Based Architecture, let's go back to the example of obtaining content for a web request in the inversion of control section. Further to this we will use OfficeFloor's ClassWorkSource plug-in that reflectively invokes methods as blocks of code (Jobs).

The example getContent method will be broken down into the following blocks of code:

  1. Obtain content from cache
  2. Obtain content from database
  3. Store database content to cache
  4. Catch block to handle exceptions

The blocks of code will be joined together into the following flow:

Job Based Architecture Example

Obtain content from cache

The following object handles the Job of obtaining content from cache.

The class's methods will be invoked by ClassWorkSource that uses reflection on POJOs to obtain the blocks of code to be executed and is provided with the OfficeCompiler (you may however elect to write your own WorkSource, more detail can be found under OfficeCompiler). The [x] on the left are markers for further explanation below.

        public class CacheFunctions {

[1]         public String getContent(String key, Cache cache, GetContentFlows flows) {
                String content = cache.getContent(key);
                if (content != null) {
[2]                 return content; // content available from cache
                }
[3]             flows.doGetNonCachedContent(key);
                return null;  // ignored as alternate flow will retrieve data
            }

            @FlowInterface
[4]         public static interface GetContentFlows {
                void doGetNonCachedContent(String key);
            }
        }

[1] Is the method signature identifying the necessary inputs to the getContents block of code (Job). It requires a key which is likely the output of the previous block of code, Cache which is dependency injected and GetContentFlows (see [4]). The ClassWorkSource uses reflection to extract this information from the class so that the developer need only write POJOs.

[2] Returns the content as output for the next block of code to use.

[3] The content was not cached so an alternate block of code is invoked to obtain the content. As the class name suggests this object is only responsible for cached content so should not dictate how non-cached content is retrieved. It therefore, passes flow control onto an alternate block of code which becomes responsible for retrieving the content. The return is null as the alternate block of code will handle returning the retrieved content.

[4] This POJO interface informs ClassWorkSource of the alternate flows that may be invoked by getContent. ClassWorkSource will generate a Proxy object for the interface that invokes the flow (provides typed methods wrapping the call to OfficeFrame's method doFlow(int flowIndex, Object parameter)). The annotation @FlowInterface flags for ClassWorkSource to provide the Proxy.

Obtain content from database

The following object handles the Job of obtaining content from the database. This class again uses the ClassWorkSource to enable use of POJOs.

    public class DatabaseFunctions {

        public KeyedData getDatabaseContent(String key, Connection connection) throws SQLException {
            PreparedStatement statement = connection.prepareStatement("SELECT CONTENT FROM CONTENT_TABLE WHERE KEY = ?");
            statement.setString(1, key);
            ResultSet resultSet = statement.executeQuery();
            resultSet.next(); // for simplicity always assume data (may use alternate flow to handle case of no data)
            String content = resultSet.getString("CONTENT");
            statement.close();
            return new KeyedData(key, content);
        }
    }

The KeyedData is an object to allow associating content with its key:

    public class KeyedData {
        public final String key;
        public final String content;
        public KeyedData(String key, String content) {
            this.key = key;
            this.content = content;
        }
    }

The DatabaseFunctions class provides a simple database retrieval block of code to obtain content from table CONTENT_TABLE in the database. The alternate flow (doGetNonCachedContent) triggers this method to be invoked. The key was passed as the argument to the doGetNonCacheContent method, while the Connection is dependency injected.

Note that a proper implementation would include alternate flows to handle missing content in the database. The SQLException however, should be thrown as OfficeFrame allows catching these exceptions and delegate them to another block of code (Job) to handle.

Store database content to cache

Once content is retrieved from the database it should be cached. As caching is not the responsibility of the database code the method will be added to the CacheFunctions class.

    public class CacheFunctions {

        // getContent code ommitted

        public String cacheContent(KeyedData data, Cache cache) {
            cache.storeContent(data.key, data.content);
            return data.content;
        }
    }

cacheContent is invoked with the KeyedData returned from getDatabaseContent, while the Cache is dependency injected. The method stores the content into the Cache and then returns the content so that the next block of code (Job) may be the same as the next block of code (Job) for the getContent method. In other words, this results in the following if statement:

    get content from cache
    if content not in cache
        get from content from database
        store content to cache
    end if
    use content

The resulting useContent method then may have, for example, the following method signature:

    public void useContent(String content, Writer writer) throws IOException

where the useContent will possibly write the content to a Writer that results in sending the content to the HTTP client.

Catch block to handle exceptions

The obtain content from database method throws an SQLException that must be handled. As the handling of an exception (typically know as an Escalation within OfficeFloor) may require a different thread, different dependencies and its own alternate flows, the handling code itself becomes another block of code (a catch block of code).

An example catch block of code for the getDatabaseContent may be as follows:

    public void handleException(SQLException escalation, Writer writer) throws IOException {
        writer.write(escalation.getMessage());
    }

This simple example would write the message of the exception to a Writer which may send this back to the HTTP client. Note that the IOException can also be handled by another catch block, much like adding a nested try block within a catch block.

More complex handing of exceptions (Escalations), could for example:

  • Log failure
  • Provide a more user friendly web page for the error
  • Send alerts
  • Redirect HTTP client to another web server as this web server is currently having connection issues with the database

As the catch block of code is a separate block of code, it provides the following advantages:

  • Decisions about how to appropriately handle exception (Escalations) can be made later with possible input from project leads and the business.
  • Decisions about how to handle exceptions can be changed without code changes.
  • New types of exceptions occuring due to code changes do not need to be handled by calling code (for example if non-cached content was to be retrieved from a Web Service rather than a database the exception thrown could change from java.sql.SQLException to a web.service.WebServiceException). It also encourages developers to use informative method signatures in their code rather than being tempted to use non-checked exceptions.
  • Exception handling code does not clutter the main flows.

Threading (Teams)

As each of the above methods is wrapped as a Job by the ClassWorkSource the Job Based Architecture is free to invoke each method with a different thread (Team). The exact configuration of which thread runs each method can be left until application assembly and deployment time which allows the application to be tuned to the hardware it is running on.

Dependency injection and asynchronous operations

Dependencies (known as ManagedObjects within OfficeFloor) are managed by a Job Based Architecture so that on invoking the Job (block of code) the dependencies are ready for use. Typically database calls are blocking and the Job Based Architecture need only provide a valid java.sql.Connection to the Job. For other operations, such as calling web services with nio, the Job is post-poned from being executed until the web service call returns (or times out) allowing the thread to do other work in the mean time. An example would be:

    public class WebServiceCaller {
    
        public void callWebService(String url, WebServiceCaller caller) {
            caller.invokeAsyncCall(url);
        }
        
        // Job based architecture will hold off calling next method until the WebServiceCaller's asynchronous call has returned
        
        public String getWebServiceResponse(WebServiceCaller caller) {
            return caller.getResponse();
        }
    }

Using a Job Based Architecture

Simplifying the application into blocks of code (Jobs) means that these blocks of code must be connected together to create the execution flows of an application. Manually connecting them together by the developer, either through additional plumbing code or hand written configuration, is error prone and time consuming for the developer. Therefore, even though having a Job Based Architecture improves the efficiency of applications, a Job Based Architecture requires additional tools to aid in the efficiency of rapid application development.

Graphical configuration that connects the blocks of code together into an execution flow is necessary to improve efficiency of development. As the blocks of code provide meta-data about their inputs and outputs, graphical editors are used to connect them together removing the need for developers to write out the configuration by hand. As the execution flows in the application are represented graphically, it is also much easier for developers to review these to gain an understanding of the application. The graphical representation even enables non-technical individuals to review aspects of the application.

OfficeFloor provides both the Job Based Architecture and graphical editors.