NEC provides 3.1'' 3D TFT LCD Module for Mobile3DTV terminal prototype

NEC LCD Technologies announced that it will provide samples of its 3.1-inch-diagonal 3D polysilicon thin-film-transistor (TFT) liquid crystal display (LCD) module for the MOBILE3DTV terminal prototype. The MOBILE3DTV prototype will be on public display at the Mobile World Congress 2009 (MWC2009), the world’s largest mobile technologies-related event, in Barcelona February 16– 19.

MOBILE3DTV is one of the projects funded by the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007–2013) in the context of its Information & Communication Technologies (ICT) Cooperation theme. The three-year project began in January 2008 and aims to demonstrate the feasibility of mobile 3D TV broadcasting.

The MOBILE3DTV terminal prototype is being developed by a project consortium partner, MM Solutions AD, and shall be displayed at the Texas Instruments booth (Hall 8, Booth 8A 84), which provides the prototype’s CPU.

A sample of the 3.1-inch diagonal LCD module with wide quarter video graphics array (WQVGA) resolution and excellent 3D picture image for bare-eyes viewing will also be displayed in cooperation with NEC Electronics in the NEC booth (Hall 8, Booth 8A 125).

The MOBILE3DTV terminal prototype is being featured at MWC2009 to demonstrate real-time decoding and displaying of 3D video content. The 3D LCD module has been adopted because of its wide viewing areas and high-performance display which features high-resolution 3D images based on NEC LCD Technologies’ unique horizontal double-density pixel (HDDP) system.

Looking forward, NEC LCD Technologies will continue to promote research and development in pursuit of practical uses and mass production of 3D LCDs.
Read more!

www.mobile3dtv.eu updated

The website www.mobile3dtv.eu has been updated recently. On the website you can find interesting news about the project research, conference papers, technical reports, and software about mobile 3D television and video.
Read more!

Standards related with coding of Mobile 3DTV content

Standardization of digital audio and video is investigated by the Moving Picture Experts Group (MPEG), a working group of ISO/IES and the corresponding standards are issued with ISO/IES designations [1].

MPEG-C, Part 3
The purpose of ISO/IEC 23002-3 Auxiliary Video Data Representations (MPEG-C part 3) is to support all those applications where additional data needs to be efficiently attached to the individual pixels of a regular video. In ISO/IEC 23002-3 it is described how this can be achieved in a generic way by making use of existing (and even future) video codecs available within MPEG. ISO/IEC 23002-3 consists of an array of N-bit values which are associated with the individual pixels of a regular video stream. These data can be compressed like conventional luminance signals using already existing (and even future) MPEG video codecs. The format allows for optional subsampling of the auxiliary data in both the spatial and temporal domain. This can be beneficial depending on the particular application and its requirements and allowing for very low bitrates for the auxiliary data.

The specification is very flexible in the sense that it defines a new 8-bit code word aux_video_type that specifies the type of the associated data, e.g., currently a value of 0x10 signals a depth map, a value of 0x11 signals a parallax map. New values for additional data representations can be easily added to fulfill future demands. The specification is directly applicable to 3D video as it allows specifying such video in the format of single view + associated depth, where the single channel video is augmented by the per-pixel depth attached as auxiliary data. As such, it is susceptible to efficient compression. Rendering of virtual view (at least one in case of stereo) is required at the receiver side. The specification has been standardized since 2007 [2], [3], [4].

MVC
3D video (3DV) and free viewpoint video (FVV) are new types of visual media that expand the user’s experience beyond what is offered by 2D video. 3DV offers a 3D depth impression of the observed scenery, while FVV allows for an interactive selection of viewpoint and direction within a certain operating range. A common element of 3DV and FVV systems is the use of multiple views of the same scene that are transmitted to the user. Multiview Video Coding (MVC, ISO/IEC 14496-10:2008 Amendment 1) is an extension of the Advanced Video Coding (AVC) standard that provides efficient coding of such multiview video. The overall structure of MVC defines the following interfaces: The encoder receives N temporally synchronized video streams and generates one bitstream. The decoder receives the bitstream, decodes and outputs the N video signals. The video representation format is based on N views. For the case of stereo-video, that is two separate views coded together. A promising extension is to study view subsampling, i.e. one full resolution view + one subsampled view. The idea behind this approach is that the human visual system is capable to retrieve the stereo with the quality of the better channel. MVC is standard since 2008 (version 1) [5], [6].

3D Video Coding
3D Video Coding (3DVC) is a standard that targets serving a variety of 3D displays. Such displays here in focus present N views (e.g. N = 9) simultaneously to the user, so-called multi-vied displays. For efficiency reasons only a lower number K of views (K = 1,..,3) shall be transmitted. For those K views additional depth data shall be provided. At the receiver side the N views to be displayed are generated from the K transmitted views with depth by depth image based rendering (DIBR). This application scenario imposes specific constraints such as narrow angle acquisition (<>K out of N views, augmented with K depth sequences. This representation related to stereo-video generalizes the possibilities of MPEG-C, Part 3 and MVC, i.e. the two separate views can be coded together or can be reduced to single view + depth with the second view to be synthesized at the receiver. 3DVC is an ongoing MPEG activity, and a standard is expected in 2011 [7], [8], [9].

References:

1. Moving Pictures Experts Group, http://www.chiariglione.org/mpeg/
   
2. ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC FDIS 23002-3 Representation of Auxiliary Video and Supplemental Information”, Doc. N8768, Marrakech, Morocco, January 2007.
   
3. ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC 13818-1:2003/FDAM2 Carriage of Auxiliary Data”, Doc. N8799, Marrakech, Morocco, January 2007.
   
4. ITU T and ISO/IEC JTC 1, “Advanced video coding for generic audiovisual services”, ITU-T Rec. H.264 and ISO/IEC 14496-10 AVC, 2003, most recent Version: 2005.
   
5. ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC 14496-10:200X/FDAM 1 Multiview Video Coding”, Doc. N9978, Hannover, Germany, July 2008.
   
6. ISO/IEC JTC1/SC29/WG11, “Joint Multiview Video Model (JMVM) 8”, Doc. N9762, Archamps, France, May 2008.
   
7. ISO/IEC JTC1/SC29/WG11 “Overview of 3D Video Coding”, Doc. N9784, Archamps, France, May 2008.
   
8. ISO/IEC JTC1/SC29/WG11 “Description of Exploration Experiments in 3D Video Coding”, Doc. N9783, Archamps, France, May 2008.
   
9. ISO/IEC JTC1/SC29/WG11 “Applications and Requirements of FTV”, Doc. N9466, Shenzhen, China, Oct. 2007.

Read more!

Interesting talk: "Our cell phones, ourselves"

What does your cell phone mean for you? Here is an interesting talk of Jan Chipchase who works as researcher at Nokia about the way we interact with our mobile phones.

See Jan Chipchase on TED.

Enjoy this interesting talk about Jan's research and his results and learn about you and your mobile phone. And feel free to leave a comment...
Read more!

Technical Report: Adaptation and optimization of coding algorithms for mobile 3DTV

Adaptation and optimization of coding algorithms for mobile 3DTV
Philipp Merkle, Heribert Brust, Kristina Dix, Yongzhe Wang, Aljoscha Smolic, Technical report, November 2008

We achieved up to 35% bitrate savings from inter-view prediction compared to stereo simulcast. Inter-view prediction whether as H.264 SEI or MVC does not add substantial complexity. A representation as video plus depth is an interesting alternative for 3D video. It allows to adjust the stereo rendering at the decoder and to optimally adapt the 3D impression for any given display. However, this extended functionality comes at the cost of an increased complexity. MPEG-C Part 3 is suitable for encoding of video plus depth data. Good depth quality is essential for good overall quality.

The publication is available on www.mobile3dtv.eu.
Read more!

Conference Paper: Mobile 3DTV System: Quality and User Perspective

Mobile 3DTV System: Quality and User Perspective
A. Gotchev, S. Jumisko-Pyykkö, A. Boev, D. Strohmeier, "Mobile 3DTV System: Quality and User Perspective", European Workshop of Media Delivery (EUMOB’08), Oulu, Finland, July 2008.

While mobile 3DTV system components such as stereo video compression techniques, transmission channels and auto-stereoscopic displays are available and with good level of maturity, their joint work crucially depend on the quality and user acceptance. We address these two key factors by rigorously investigating all possible artifacts appearing at all processing stages and influencing different layers of the human 3D vision, and by applying a novel user-centered design to the critical components of the system.

The publication is available on www.mobile3dtv.eu.
Read more!

Conference Paper: Designing for User Experience: What to Expect from Mobile 3D TV and Video?

Designing for User Experience: What to Expect from Mobile 3D TV and Video?
S. Jumisko-Pyykkö, M. Weitzel, D. Strohmeier: "Designing for User Experience: What to Expect from Mobile 3D TV and Video?", uxTV Con 2008, Silicon Valley, USA, October 2008.

A long path has been taken in the side of technology to develop 3D video for consumer products, but user studies to underline the actual users’ needs and expectations are disregarded. The object of this study is to examine users’ needs, expectations and requirements for mobile 3D television and video. We conducted three user studies applying a methodological triangulation of the extensive survey, focus groups and probe studies to gather the requirements. The results are presented in the form of the requirement guideline which highlights the aspects of users, desired system and service including the interesting contents and usage contexts. Both academia and industry can benefit from these requirements when designing the further studies and development work in the user experience of 3D television and video.

The publication is available on www.mobile3dtv.eu.
Read more!

Meet the users: Sanna Virtanen (28)

We met Sanna for the first time in a cozy cafe in the center of Tampere. She was watching "Finland's Next Supermodel" on her mobile 3DTV device. Of course, we were courious to hear about her experiences with mobile 3D television.

Sanna is 28 years old and she has worked as Managing Consultant at WeCan OY – a medium sized software company – since two years. Sanna has a boyfriend who lives in Madrid, Spain.

Sanna is a highflyer in her job. She loves the job, although it is very exhausting and she’s always busy at work. But in the same manner in which she loves her job, she lives her life after work. She is a lifestyle woman, always up-to-date and trendy. She bought her mobile 3D phone 5 months ago. Her last phone contract was ending and as she had the possibility to choose a new phone, she decided for mobile 3D. The advertisement and the latest reports in the lifestyle magazines convinced her that this phone fulfills all her demands on functionality and quality.

“I enjoy sitting in the cafe and watch the latest lifestyle magazines or all these cute shows on TV like this ‘Finland’s next supermodel’. 3D makes everything so realistic and I enjoy diving into the content so that I can forget the day for a short time.” Having a cup of Iced Mocha she chilled on the sofa and watched the series when we met her for the first time. She passed some time after work in the cafe before her Pilates workout started.

"When I come home later, I will call Raúl. He is my boyfriend since 4 month. I hate the long-distance relationship, but luckily we now found 3D videophoning", Sanna tells us. Secretly, the 3D videophony was the final reason to decide for mobile3DTV. “You know, it is just this intimacy that I experience when Rául appears in front of me three-dimensionally. For a short time I even forget the annoying distance between us”, Sanna says with a smile on her face.

Do you know Sanna? Until now, Sanna is just a persona, a user archetype. She represents one of the user groups of mobile 3DTV according to our studies into user requirements for the new system [1][2]. You may ask how other users will look like? Read on, we will present them later.

References:

1. Jumisko-Pyykkö, S., Weitzel, M., Strohmeier, D. "Designing for User Experience: What to Expect from Mobile 3D TV and Video? " . Proceedings of the First International Conference on Designing Interactive User Experiences for TV and Video. October 22 - 24, 2008, Silicon Valley, California, USA.
2. Strohmeier, D., Weitzel, M., Jumisko-Pyykkö, S. Use scenarios: mobile 3D television and video. Proceedings of SPIE Electronic Imaging 2009, San Jose, CA, USA (2009)
   

Read more!

Current user research in the context of Mobile3DTV

Related user studies in the field of 3DTV and especially in the field of mobile TV, one can realize a big change in the field of user-centered design. Current research has introduced user-centered approaches to study user’s needs and expectations about novel systems and services. This allows better understanding of the impact of critical system components. The cyclic character of user-centered design is standardized in ISO 13407 [1].

Subjective quality has usually been studied quantitatively based on requirements of the International Telecommunication Union (ITU) [2] offering guidelines to study quality perception studies in controlled, laboratory environments. In the context of mobile television, these studies, which base on quantitative measurements of ITU recommendations, have focused on evaluating the impact of coding errors, low bitrates, and impact of changes in image size or framerates as well as bitrate share between audio and visual modalities (e.g. [3], [4], [5]). Further, there are studies into transmission errors over DVB-H channel and their impact on the perceived quality [6], but still these studies are rare.

With respect to stereo-coding, there are studies on stereo-coding errors in images and videos [7]. However, user studies in the field of 3DTV have focused more on the impact of different factors on presence, the users feeling of being there. In stereo-video content presentation, the most prominent additional features that influence the perceived quality are depth information, perceived sharpness, or naturalness (i.e. the most truthful or realistic reproduction of content) [8][9]. All these factors contribute to the concept of presence, the involvement of the user in the content; the feeling of being there [10]. Nevertheless, some work has reported simulator sickness symptoms caused by the use of stereo-video presentation. This particular impact of stereo vision adversely affects enjoyment and factors creating simulator sickness are not yet known in depth [11].

In contrast to the recommendations of the ITU, current studies in the field of mobile TV include user characteristics, usage contexts and the user’s goal of viewing. Currently there are available results of field trials or studies conducted in several countries [12][13] as well as results of prospective focus groups and online surveys [14][15]. In contrast, user studies into the field of 3DTV are still rare and Freeman’s focus group [16] is the only study which examines user requirements of 3DTV more broadly.

From the methodological point of view, there is a lot of work in progress to close the shortcomings of ITU recommendations. Researchers have introduces contextual quality evaluation procedure that is conducted in parallel to the controlled laboratory assessments [17]. Additionally, evaluation of the experienced quality of critical system components targets the future acceptability of the quality of components which were developed in isolation from the end-product [18]. A third step to understand experienced quality is to understand the interpretation of constructed quality and experienced quality factors [19].

Finally, there is one available study into the user experience of mobile 3D television. Following a methodological triangulation of online survey, focus groups and probe study to elicit user requirements, Jumisko-Pyykkö et al. [20] present a broad view on user experience of mobile TV. In all the three studies, they targeted the user experience factors of user, system and services, and contexts. Concluding their study, they present first guidelines to design user experience of mobile 3D television.

References:

1. ISO 13407. 1999. Human-centered design processes for interactive systems. International Standard, the International Organization for Standardization.
2. ITU-R BT.500-11, Methodology for the Subjective Assessment of the Quality of Television Pictures, Recommendation ITU-R BT.500-11, ITU Telecom. Standardization Sector of ITU, 2002.
3. H. Knoche, J. D. McCarthy, and M. A. Sasse, “Can small be beautiful?: assessing image resolution requirements for mobile tv,” in MULTIMEDIA ‟05: Proceedings of the 13th annual ACM international conference on Multimedia. New York, NY, USA: ACM Press, 2005, pp. 829–838.
   
4. S. Winkler, S. C. Faller, “Maximizing audiovisual quality at low bitrates,” Workshop on Video Processing and Quality Metrics for Consumer Electronics. Scottsdale, United States of America: January 2005.
   
5. S. Jumisko-Pyykkö, “I would like to see the subtitles and the face or at least hear the voice”: Effects of Picture ratio and Audio-video Bitrate Ratio on Perception of Quality in Mobile Television. To appear in Personalized and Mobile Digital TV Applications in Springer Multimedia Tools and Applications Series, 2007
6. S. Jumisko-Pyykkö, Vadakital, V., Liinasuo, M. , Hannuksela M. M., Acceptance of Audiovisual Quality in Erroneous Television Sequences over a DVB-H Channel. Proceedings of Second International Workshop in Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, USA, January 2006.
7. L. B. Stelmach, Tam, W. J. (1998). Stereoscopic image coding: effect of disparate image-quality in left- and right-eye views. Signal Processing: Image Communications, 14:111–117.
8. H. de Ridder, (1996). Naturalness and image quality: Saturation and lightness variations in color images of natural scenes. Journal of Imaging Science and Technology, 40:487–493.
9. A. Berthold (1997). The influence of blur on the perceived quality and sensation of depth of 2D and stereo images. Technical report, ATR Human Information Processing Research Laboratories.
   
10. W. IJsselsteijn, de Ridder, H., Freeman, J., and Avons, S. (2000a). Presence: Concept, determinants and measurement. Proceedings of the SPIE, 3959:520–529.
11. Häkkinen, J., Liinasuo, M., Takatalo, J., and Nyman, G. 2006. Visual comfort with mobile stereoscopic gaming. Proceedings of SPIE. Vol. 6055
12. Chipchase, J., Yanqing, C., Jung, Y., and Design, N. 2006. Personal Television: A Qualitative Study of Mobile TV Users. Lecture Notes in Computer Science. Vol. 4471, pp.195-204
13. Södergård C. (ed.). 2003. Mobile television – technology and user experiences, Report on the Mobile –TV Project. Espoo: VTT Publications 506.
14. Knoche, H. O., McCarthy, J. D. 2004. Mobile Users Needs and Expectations of Future Multimedia Services. In Proceedings of the WWRF12. Nov. 2004, Toronto, Canada
    
15. Carlsson, C., Walden, P. 2007. Mobile TV - To Live or Die by Content, Proc 40th HICSS 51b.
16. Freeman, J., Avons, S. E. 2000. Focus Group Exploration of Presence through Advanced Broadcast Services. In Proc. SPIE 3959 , 3959-3976.
17. Jumisko-Pyykkö, S., Hannuksela, M. M. Does Context Matter in Quality Evaluation of Mobile Television? To appear in the Proceedings of 10th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2008)
18. Jumisko-Pyykkö, S. Kumar Malamal Vadakital, V., Hannuksela, M.M. Acceptance Threshold: Bidimensional Research Method for User-Oriented Quality Evaluation Studies. To appear in International Journal of Digital Multimedia Broadcasting, 2008.
19. Strohmeier, D. Wahrnehmungsuntersuchung von 2D vs. 3D Displays in A/V-Applikationen mittels einer kombinierten Analysemethodik, Diploma Thesis, Ilmenau University of Technology, Germany, 2007
20. Jumisko-Pyykkö, S., Weitzel, M., Strohmeier, D. "Designing for User Experience: What to Expect from Mobile 3D TV and Video? " . Proceedings of the First International Conference on Designing Interactive User Experiences for TV and Video. October 22 - 24, 2008, Silicon Valley, California, USA.
    

Read more!

Mobile3dtv artefact rendering framework is available for download

We have developed a framework for simulation of mobile 3DTV artifacts. It is able to introduce an arbitrary combination of stereoscopic artifacts to a given video stream, with controlled amount of impairment for each artifact. 

It is organized as a collection of Matlab functions, each one responsible for introducing a specific artifact. The configuration is stored in a text file, which describes the input and output video streams, the set of artifacts to be introduced, and the parameters for each artifact. One configuration file can specify a set of artifact parameters, which to be applied over several input video files in “batch mode”.

The framework operates on stereo-video streams (where left and right channel are provided as separate video files) or dense depth video streams (where video and depth channel are provided as two separate video files). Video is decoded into a set of frames, each frame is processed and the result is encoded in a video stream again.

The current version (1.3a) is available for download on our website, in the software download section

Read more!