Friday, November 16, 2012

The performance of a simple PV snapshot application

In order to check the impact of programming style, synchronous and asynchronous, on the performance of a PV snapshot, I set up a simple benchmark and tested it with five different implementations. The benchmark is a snapshot to get the current value of (1) 97 connected PV's, (2) 97 connected and 4 disconnected PV's, and (3) 97 connected and 15 disconnected PV's. Four implementations is programmed with Python and pyepics library. The last one is programmed with node.js and it gets the PV value via system caget command line tool. All the implementations are on github.
The pv and ca implementations are based on the suggestions by pyepics developers. The improvement is huge when all the PV's are connected. For 97 PV's, the time required drops from about 2700 milliseconds to about 200 milliseconds. However, they do not perform good when there are disconnected PV's. In the programs, I set all the connection timeout to be 1 second. The pv version program takes extra 2 seconds for each disconnected PV and the ca version program takes extra 1 second for that. The major part of the pv version is The major part of the ca version is Threading should improve the situation by waiting for the timeout in parallel rather than in sequence. However, the overheads of creating processes and sharing state between processes are also enormous. The major part of the multi-threading pv version is And the corresponding multi-threading ca version is The node.js implementation performs much better than both the pv and ca multi-threading implementations. I think this is because 1) the process in node.js is much lighter than those in python, and 2) there is no shared state between processes in node.js because of the usage of asynchronous callback and closure. On the protocol level, channel access is fully parallel and asynchronous. It seems we should do the same on high-level programming level to leverage that advantage.

Monday, September 17, 2012

Ubuntu 12.04 and nvidia video card

I have updated Ubuntu on Dell Vostro 460 from 10.04 to 12.04. The video driver had no problem when I activated the "version current" driver available in system additional drivers. Later I messed up the system when tried to install a driver from nvidia website. Lightdm failed to load, and I can only log into console mode. It can be fixed by forcing reinstall the nvidia-current package.
sudo apt-get --reinstall install nvidia-current
If this does not work, you might also try
sudo apt-get --reinstall install lightdm
A few days ago, Ubuntu updated its kernel. The lightdm loaded but the configuration was wrong. I realized I had to re-enable the additional drivers. A little surprise was the "version current" stopped working, while the "post-release update" worked. The other thing annoying me was the blue face problem of youtube flash video. The fix was described in this post. The most tricky part was that you can only turn off the hardware acceleration option when the video was in full-screen mode.

Tuesday, August 21, 2012

Two leaky designs related to URL length limit

I discuss two cases of leaky design in this post. This first is Google Calendar, and the second is eBay POST caching.

Google Calendar

If you are using Google Calendar, you might have seen the broken droid with 414 code like this.

The reason for 414 is that the request URI is so long that the server refuses to handle. The maximum length of URL allowed by a server depends on its implementation. No matter it is 255 bytes or 2k characters or even 4k characters, it will reach the limit when URL is used to encoding a complicate request query parameters.

In the case of Google Calendar, when a user clicks the link to add events in GMail. An HTTP GET request is sent to Calender server with the details of the events encoded in the URL. For normal event often represented in iCalendar format, the URL can provide enough room for the data. Some event message might attach other extra text inside, when GMail tries to encode those into the request URI to Calendar, 414 might happen.

A possible solution can be a modified version of GMail that ensure a proper length of the URL with important information encoded inside. However, that is not a right solution. It is because the interface design between GMail and Google Calendar is not RESTful or simply does not follow the HTTP specification. HTTP GET is for retrieving the representation of a resource not for creating a new resource. A side effect of such a design is a duplicate even will be created every time a user clicks the same link and issues the same HTTP GET request.

A straightforward fix is to use POST to create a new event, and the details in the original email can then be easily included in the message body. 414 will never happen. In order to prevent creation of duplicated event, the server might need to verify if it is the same event that already existed on the same time slot.

If, in some cases, the designer really want to create a new resource by a GET request, the request URI should be able to identify the resource to be created. That is, instead return a 404, the server creates a new resource based on the request URI, and then returns the representation of that resource. For a server serves many concurrent clients that might request that URL at the same time, special care needs to be taken to prevent creation of duplication resource on the server and also holds the following requests until the resource representation is available. I developed such a service by using a Servlet Filter to do these tasks with Jetty continuation.

eBay POST caching

In this post, eBay engineer described their careful design of caching POST messages for eBay search. Two major motivation of this design is 1) to leverage the performance and scalability benefits of HTTP caching, and 2) to avoid the length limitation of HTTP request URI. I think the design is leaky because it might introduce extra special header to the message and very naturally not all the intermediaries can understand those headers and help the caching. The forward and reverse proxies in the diagram might only be the customized deployment of eBay.

As Akara Sucharitakul pointed out in his comment, a 303 can be used for the initial POST request,and then caching will work with the following GET. However, this does not reduce the round-trip time. My thought is to generate the MD5 key on the client via JavaScript, and then GET the resource identified by the key. If the user is lucky to be the first to issue such a request, a 404 will be returned, and then the client will do the real POST to create that resource on the server.