29/03 How fast?

One of the aspects of video that doesn't get talked about much is frame rate. So I will now proceed to talk about it - at length. If you work in the TV or film industry you may be aware of the delights of 3:2 pulldown, PAL vs NTSC incompatibilities and the delightful judder much sought-after by TV and video directors wanting "the film look". If you don't, you may be aware that film is shot at 24 frames per second and old-fashioned TV at 50 fields per second (in Europe), but that's probably it.

If you don't understand any of the aforementioned, let me just explain that "frame rate" is the term used to describe the rate at which the individual pictures that make up a video sequence are shown to the viewer. Pick a frame rate that's too slow, and you can't display motion accurately - things move jerkily around the screen, or blur unrecognisably, or appear as double images - the faster the motion, the bigger the problem.

One of the things we're looking into is what the frame rate for a hypothetical successor to HDTV should be. In order to understand why the figures in use today were chosen, I've been reading lots of documents from the early days of television and film - both from our library and from the web. For you, the lucky reader, I will summarise them here.

James Card wrote an article entitled "Silent Film Speed" in the October 1955 issue of "Image" that is available online here. In it, he gives the projection speeds for about thirty silent films made between 1916 and 1928. There is considerable variation: between eleven and fourteen minutes for a 1000-foot reel (400-foot for 16mm stock). This is due in part to the use of hand-cranked cameras, but also due to advice given to the projectionist to vary the speed in different sections of the film:

'Silent films were usually released with musical cue sheets supplied in many cases by the producing company itself. As early as 1916, Triangle published special instructions to the projectionist. Here are some samples: "The best effects in The Captive God will be had by timing the film to run from 13 to 13 1/2 minutes to the reel. The two big battle scenes... should be speeded up considerably. Following the sub-title 'The Alarm,' shoot it through fast."' - James Card

At 16 frames to the foot (for 35mm), that implies an estimated average frame rate of between 19 and 24 fps for the films of the period quoted.

In his "Projection Department" column in "The Moving Picture World" dated 02/12/1911, F R Richardson advises:

'Speed is of very very great importance and a comprehension of this fact is absolutely necessary to do really fine projection. The operator "renders" a film, if he is a real operator, exactly as does the musician render a piece of music, in that, within limits, the action of the scene being portrayed depends entirely on his judgment.'

and

'Watch the scene closely and by variation of speed bring out everything there is in it. No set rule applies. Only the application of brains to the matter of speed can properly render a film. I have often changed speed half a dozen times on one film of 1000 feet.'

With the development of sound-on-film processes in the 1920s by inventors such as Lee de Forest and Theodore Case, film speeds and hence frame rates standardised at the now ubiquitous 24 fps - an uneasy compromise between cost and audio quality. The subsequent exact adherence to this figure may be due to the limitations imposed by the more complex audio-capable projectors, but I suspect that desires to de-skill the role of the projectionist and mechanise (and electrify) the cranking of the camera may also have been issues. Provision of a uniform viewing experience - quality control - may also have been a priority. To avoid visible flicker (probably the most annoying of all the temporal aliasing artefacts), a double or treble-bladed shutter was used to display each image two or three times in quick succession.

The 30-line television system developed by Baird (and the BBC) in the late 1920s and early 1930s ran at 12.5fps. It may be worth noting that the system used an optomechanical scanning system whereby the "scene" was illuminated by a scanning beam of light from the camera, however. More versatile cameras were adopted relatively quickly, but not before Baird had experimented with an infra-red version that was less irritating to performers.

The Marconi-EMI television system (now known as "405-line", the old monochrome standard that the BBC continued broadcasting until as recently as 1986) was adopted by the BBC in 1937 after broadcast trials against an improved 240-line (progressive-scan!) Baird system. These systems were described contemporaneously as "high-definition television"... This and all subsequent TV systems have used a frame rate that is the same as the mains frequency (50Hz in Europe). The reasons given for this in the BBC's "Technical Manual, M-EMI System of Television, London Television Station", published in April 1939 are that it is to avoid "beating" against the 100Hz brightness fluctuation in AC-driven studio lights and the 50Hz fluctuation induced by poor ripple-suppression in the HT generation circuitry of early CRT televisions. In addition, a 50Hz refresh rate is about the minimum necessary to avoid visible flicker in the displayed image; a fact well-established by cinema at that time. At least on small screens of limited brightness - TV manufacturers in the 1980s started experimenting with 100Hz refresh rates (120Hz in 60Hz-mains countries) to reduce flicker on large, bright CRT televisions. (At the expense of accurate motion portrayal, in many cases, although Philips claim to have cracked that problem in the previously linked article.)

29/03 3D Television in World War II

I've just been to the rather useful library that we have on-site here at Kingswood Warren and obtained an article from "Wireless World". The cover of the issue in question ("Vol XLVIII No. 2") features a photo of "Women in Wireless" - three girls in their twenties, wearing headphones, operating a Morse key, etc. All of them in uniform. The date of the issue is February 1942, and there are references to the war on every page. Adverts contain little poems mocking Hitler. The subject of the article I wanted, though? "Stereoscopic Colour Television".

Yes, that indefatigable inventor John Logie Baird, after his company was liquidated at the outset of war and the television transmitters were turned off to prevent the Luftwaffe using them as navigation beacons, went back to his lab and decided to improve television for a post-war world. By 1944, when Lord Hankey was chairing a committee to look into that very subject, Baird was ready to propose not only colour, but stereoscopic 3D, too. The committee took a slightly more cautious view, recommending merely that "vigorous research work" into an improved television system be carried out, with colour and stereoscopy mere possible avenues of investigation. Of course, it wasn't until 1967 that the UK got colour television, and 3D TV remains only a novelty. Unfortunately Baird's proposals were entirely unsuitable for mass-market adoption: his approaches either relied on red/blue glasses (which don't lead to a pleasant viewing experience, and preclude colour imagery) or required the viewer to sit in a very specific location (which doesn't really work in the living room).

It's interesting to note that one of the other recommendations the Hankey committee made led to an increase in the BBC's investment in scientific and engineering research and the establishment of Research Department at Kingswood Warren. So perhaps I'll be the person developing Baird's proposals for 3D television into something practical, sixty years later...

19/03 3D?

While looking for a decent "3D" monitor, I happened across an interesting piece in Gizmodo from last September reviewing the iZ3D "3D gaming monitor". Interesting for two reasons - firstly, a bit of further research suggests that the third big Hollywood push for 3D cinema is starting to turn into a bandwagon, and secondly because the author of the piece can't see the "3D" effect that the monitor is supposed to provide. The iZ3D display uses the 3D visualisation technique known as stereoscopy, in which a 3D effect is produced by sending different images to each eye. Kaufman asserts that around 8% of people are incapable of perceiving "3D" in a stereoscopic image, due to lazy eyes and other binocular vision dysfunctions. Interestingly, this figure is similar to the fraction of the population who are "colour blind", with impaired colour perception of one kind or another. I reckon that the Gizmodo author probably falls into that former 8%.

To me, this calls for a broader look at what we think "3D" is. Obviously that 8% manages to navigate through life without bumping into things all the time, or the streets would be full of slapstick and the eyesight component of driving tests would be somewhat more rigourous. Indeed, people who have lost their sight in one eye entirely can drive perfectly safely and legally in the UK and many other countries. Those people can see 3D things just fine. And frankly, so can anyone who watches camera-originated (as opposed to hand- or computer-animated) films and television on old-fashioned "2D" screens. Most of the time the inability of the viewer to use their binocular depth perception doesn't hinder their appreciation of the programmes. (Having said that, I find watching many sports to be quite frustrating on television - when a ball spends a long time in the air I can never quite work out where it's going to land.)

The general question is, how do we perceive depth? And how can we improve television to give a better impression of depth? I'll answer those questions (at least partially, because they're quite complicated) in a future post.

13/03 Beyond HD

One of the things that I haven't mentioned yet here (or anywhere else) is that I work for the BBC's science and technology R&D department, BBC Research. Up until recently I didn't have a reason to, because the project I was working on was unlikely to be of interest to a wider audience. Recently I moved to the "Beyond HD" project though, which I think is tremendously exciting: a once-per-generation opportunity to take a step back and ask "OK, what's next?"

The rationale behind the project is simple: technology-wise, HD is a done deal. OK, so there are still plenty of questions to be answered about how people actually watch HD media - the extent to which HDTV will be available on terrestrial television is currently a very political matter, for example, and the HD-DVD vs Blu-ray war is still raging. But the fundamental parameters of the technology itself, the resolutions, frame rates, scanning formats, colourimetry etc, have been thoroughly standardised.

In ten or twenty years time though, I think that HD will be starting to show its age. People will start to get dissatisfied with it. (Some people are yet to be satisfied by "HD" televisions today, but that's another story...) Technology will have moved on, and the quality of the experience people have when they "watch television" could be much higher in consequence. How the quality should be improved is the main thing we're looking at. Hollywood is undergoing something of a renaissance in "3D" filmmaking at the moment - if that takes off in a big way, perhaps people will want to be able to watch 3D television in their homes. If TV screens continue to grow, perhaps the resolution of TV will need to increase. If you increase the resolution, you need to increase the frame rate too if you want to stop moving objects from blurring or juddering. Is wide-screen TV the right width? Would it be better if it had the same aspect ratio as most films do? And what about haptics, smellovision, and a huge range of other wacky, leftfield possibilities?

Obviously some things are going to be higher priorities for research than others, but the great (and hardest) thing about starting a blue-sky research project is that you have to consider all the possibilities, at least briefly. It's going to be very interesting, and I hope to find the time to write about it here.

(As a sidenote, I expect that the concept of "watching TV" will be a very diffuse one in a decade or so: people will want "content" to be available to them in whatever format suits them at the time they want to enjoy it. We're starting to see that now, with audio and video podcasts, mobile TV, Internet video-on-demand services, the Slingbox, and so forth. But lots of other people, including some of my colleagues at the BBC, are working on that. I'm assuming that in a decade or so people will still want to crash on the sofa and veg out in front of a screen, or get their friends or family together and watch something fun/informative/important. The question which we're trying to answer is, how will advances in technology let us make that experience better?)