What Pixar might have to do with smoother applications?

Hello there,

Long time since we spoke!

First few updates:

• Official (“update button”) Ice Cream Sandwich update in November after Google’s launch in late Oct
• Pixar and smoother animations
• Kernel Update with November built with working camera
• Another Service Center in Germany and Netherlands by December
• Parseltongue for Android?
• Another Phase of Investment Closure

The last one literally took all the time, but was worth it. We are expanding out of India and are setting up a design house in China and, a lab in Taiwan as well soon. Most of the expansion is in for RnD labs, Service centers, content team and in materials and mechanical research. These are long-term goals, but required. Severe bad health (Spondylolysis) and concentrated focus on investment has taken majority of all my time. But now I feel little better and am back on full working mode.

Going back to the other updates, we all know about the impending launch of ICS (IceCream Sandwich) in October. Kernel update is a sort of internal development timeline for pushing better and faster software. New kernels (like .38) supports ARM hibernation and lot more important tools to bring portable devices closer to its desktop counterpart. If you go by the industry signals, OMAP is the preferred platform for next wave of devices. It will also see ICS before Tegra.

Motion blur as seen on first doll in comparison to the one on right. Source: Wikipedia

We have formed a dedicated team to look into the jerks in object (graphics) movements on Android. Obvious standard was to compare the iOS. This problem is also prominent in Game design, and was obvious for the game development team at Notion Ink to also get very involved. We looked at the problem from various angles and are startled at the results. Let me explain.

Suppose we design a UI or game at 60 fps. That should ideally mean around 16.6ms per frame and in total 60 frames per second. Now general mistake we all do is to assume that computer will be able to give you fixed 16.6ms time between every frame and miss out on the part, that possibly the logic of the program need not be computed 60 times a second! I.e. if you program a render at 60fps, it could as well be rendering say 10 consecutive frames at 15, 18, 14, 19, 16, 17, 18, 14, 19 and 18ms timeframe and giving you an average 60fps. The differences are small, but our eyes can’t be fooled and for every violent delta it senses a jitter.

One of a very strong approach in game design is to limit the logic computation to minimum possible to run the game smoothly (so more juice is left for processing graphics) and add what everyone knows as motion blur. More I learnt about motion blur and gaming technologies, more I realized the experience that might have gone into iOS straight from Pixar, their head being same. May be the experience infusion was more because of experience in one field and need on other.

Lets analyze one more angle, the TVs:

Have you ever felt missing movie experience on HD TV specially ones with sizes 50inches and above? You can generalize the experience by saying, “movies on these TVs looked more like TV serials”.

HD TVs render scenes at even 200Hz today. Movies we watch, are generally 24 or 30 fps. You can guess that most of the frames on these TVs are left idle. But that was the case before “true motion TVs” came in. In true motion, TVs fill up these missing frames with interpolated scenes from existing frames. End result is realism which has never seen before specially for movies.

Computers on the other hand, just know what to show at each instant of any frame. While when we capture a movie at a particular FPS, it is better to say “it captures a smaller movies for say 1000/24=41.66ms and superimposes all the image on top of each other and then show a particular frame”. In other words, 41.66ms is the exposure time per frame. The sensor is left open for this amount of time and it captures a moving scene, superimposing objects on top of each other and the end result is captured motion blur. You know how it works when you snap a still image of a moving object.

Coming back to User Interface, say we design a simple animation of 60fps. We have been trained to think that 60 fps is a good frame rate. Let’s define this simple animation as an image moving from left to right on Adam (Eden 1 panel movements for example). We have 1024 pixels to cover in one second and the frame rate chosen is 60. 1024/60 is 16.67 pixels. That is a lot of pixels. If i just show you a line moving across Adam from left to right 1024 pixels, but jumping 16pixels every frame, your eyes will tell you “hey, thats jittery!”.

These results are not our inventions as game designers are very well aware of this. We are not the only one who faced this issue, as jerkeyness is all over Android OS. But the experience in game design and animation can definitely be used to improve the graphic performance on Android. Results clearly speak out that it doesn’t matter if you design on OpenGL or on pure java canvas, smoothness is not dependent on a platform, but on the logic. One very important result we can derive is that every sequential animation (pop up, screen movement from left to right, translations, transitions, etc) can have very different frame rates which makes them look smoother. Smoothness has a lot of components like motion blur, area covered while in motion (or pixels moved per second), logic frame rate and yes even color (our eyes are more sensitive to yellow-green and any jerkeyness in it can be immediately captured, unfortunately you saw more of this color on older androids).

Our research is still not finished, and there are many other beautiful areas of research where we want to spend time to understand these issues from all possible angles.

For reasons which will soon be obvious, our love for a parsentongue, python has grown exponentially. This is one platform we sub-consciously wanted and where we invested as well. Nearly everyone at NI is working on this as their main programming language. If anyone of you there understand what a scrpiting language can do on a platform like Android, you must be as excited as we are!

With Warm Regards

Rohan Shravan