An Express middleware to check if a resource exists

After writing a similar code snippets for tens of times, I decided to write a middleware to handle it. The scenario is quite common. You want to check if an resource exists before going to the next step of processing. In my applications,  it is a query in the MongoDB.

/**
 * A middleware to check if id exists in collection
 * app.get('/resources/:id/', exist('id', collection), function(req, res){...}}
 * @param  {String} id            the parameter name of item id in req object
 * @param  {Model} collection     the collection model
 * @return {Function}             the middleware
 */
function exist(id, collection) {
  return function (req, res, next) {
    collection.findById(req.params[id]).exec(function (err, item) {
      if (err) {
        console.error(err);
        return res.send(500, 'Something wrong!');
      }

      if (!item) {
        return res.send(404, 'item ' + req.params[id] + ' not found');
      }
      
      req[req.params[id]] = item;
      next();
    });
  };
}

Resource representation generation on first GET retrieval

Intent

Provide a resource type that has a new resource representation instance generated on the first retrieval. In HTTP, this is often a GET request to a URL that the resource provides.

Also Known As

The closest pattern I can find is the Multiton pattern.

Motivation

I have implement this pattern in two scenarios:

An application provides the remote control of a specific CCD detector. The detector scans a sample and acquires an image on every scan spot. The native image format is not supported by Web browsers. The users can view the scan process on a page. The page will retrieve a new image once the acquisition finishes on a scan spot. The image is then converted to PNG, and sent to the client. The PNG image is saved, and all later requests will be served directly without conversion. The challenge, in this scenario, is that the application should convert the image to PNG only once. That implies that all the requests of that image before the PNG file is available are served together.

In the other scenario, a client retrieves a user's thumbnail photo from an application. The application gets a photo from an Active Directory when the photo is requested from the first time. The application saves the photo and serves it locally thereafter. Similar to the first scenario, the application should retrieve the photo from the AD service only once.

Design

When a resource representation is requested but is not available in the local file system, the request is put into an array storing all the requests for the same representation. Such an array needs to be put into a hash table, and its key is the resource's identifier. When a resource is requested for the first time, the resource is not available in the local file system, and the corresponding key is not in the hash table. Then the key will be created in the hash table, and the first request is pushed into the array. All following requests of the same resource are pushed into the array when the application is generating the resource representation. When the representation is ready, it is saved in the file system, and the key is removed from the hash table. All the requests in the array are served in a batch. The design can be implemented in various programming languages. There is a big difference between the implementation in a non-event-driven programming language like Java and that in an event-driven programming language like node.js.

Challenge 1: synchronization of the hash table

Adding and removing similar resource requests into the hash table have to be synchronized.

Java	node.js
We will have to use a concurrent util class like java.util.concurrent.ConcurrentHashMap(String, List).	A simple object like {"resource-identifier": []} will work.

Challenge 2: asynchronous processing

When the application is generating or retrieving the resource, we want the thread previously allocated to the request to be freed, and the handling of the request continues when the resource is finally available. Before Servlet 3, we will have to use something like Jetty continuation for this in Java. On the contrary, because node.js by nature has only a single thread, the processing is by default asynchronous.

Java	node.js
We will need put a Jetty continuation or a Servlet 3 AsyncContext into the hash map, and write the response from there when the resource is available.	Just put the standard http.ServerResponse instances in the array.

In order to improve performance, we can also add cache control to these generated resources in addition to the copy in the file system.

Make the moment live with livestamp

Livestamp.js is a neat library that update your moment().fromNow() timestamps with js timer. By default, the update interval is one second, which is too short considering that the from now time shows update every minute. If you set the interval by $.livestamp.interval(int), then you will need to call $.livestamp.resume() in order to prevent the delay on a newly created live timestamp. An example is here.

A lightweight JavaScript form data binder

In one of my current projects, I need a JavaScrip module to serialize the user form input in JSON format and then to send it to the server. I also need it to deserialize the data from the server and input into the form in order to allow the user to view or update it. Basically it is a two-way form-JSON binder. AngularJS fits this scenario perfectly, but it is a little heavy from my perspective.

js-binding is the one that I chose. I did a little modification on it in order to avoid serializing blank inputs and deserializing undefined JSON properties. It is the server side that decides if a field should be updated if the request says nothing about it. The modified version is on GitHub.

Using the serialization function, I can also check whether the form was updated before submitting the data to the server.

dygraphs plot x-axis labels align with grids

Recently, I learned to use the dygraphs JS library for 2d chart. Dygraphs is designed to

display dense data sets and enable users to explore and interpret them.

I like it more than Flot, another JS ploting library.

One problem I found was that the x-axis labels do not align with the grids in FireFox, while it is fine in Chrome. The fix is quite simple, just never use align = center in the

where the plot is embedded. A note from the author of dygraphs.

dygraphs are not happy when placed inside a tag. This applies to the CSS text-align property as well. If you want to center a Dygraph, put it inside a table with align = center set.