The video applications on mobile communication devices have usually been designed for content creation, access,
and playback. For instance, many recent mobile devices replicate the functionalities of portable video cameras
and video recorders, and digital TV receivers. These are all demanding uses, but nothing new from the consumer
point of view. However, many of the current devices have two cameras built in, one for capturing high resolution
images, and the other for lower, typically VGA (640x480 pixels) resolution video telephony. We employ video to
enable new applications and describe four actual solutions implemented on mobile communication devices. The
first one is a real-time motion based user interface that can be used for browsing large images or documents such
as maps on small screens. The motion information is extracted from the image sequence captured by the camera.
The second solution is a real-time panorama builder, while the third one assembles document panoramas, both
from individual video frames. The fourth solution is a real-time face and eye detector. It provides another type
of foundation for motion based user interfaces as knowledge of presence and motion of a human faces in the view
of the camera can be a powerful application enabler.
KEYWORDS: Video, Energy efficiency, Operating systems, Multimedia, Chemical elements, Video acceleration, Clocks, Computer simulations, Data processing, Mobile devices
This paper presents a comparison of two systems that can simultaneously decode multiple videos on a simple
CPU and dedicated function-level hardware accelerators. The first system is implemented in a traditional way,
such that the decoder instances access the accelerators concurrently without external coordination. The second
system implementation coordinates the tasks' accelerator accesses by scheduling. The solutions are compared
by execution cycles, energy consumption and cache hit ratios. In the traditional solution each decoder task
continuously requests access to the needed hardware accelerators. However, since the other tasks are competing
on the same resources, the tasks must often yield and wait for their turn, which reduces the energy-effciency.
The scheduling-based approach assumes that the accelerator latencies are deterministic and assigns time slots for
accelerator accesses required by each task. The accelerator access schedule is re-designed for each macroblock at
run-time, thus avoiding the over-allocation of resources and improving energy-effciency. Deterministic accelerator
latencies ensue that the CPU is not interrupted when an accelerator finishes. The contribution of this study is
the comparison of the accelerator timing solution against the traditional approach.
Image stitching is used to combine several images into one wide-angled mosaic image. Traditionally mosaic
images have been constructed from a few separate photographs, but nowadays that video recording has become
commonplace even on mobile phones, it is possible to consider also video sequences as a source for mosaic images.
However, most stitching methods require vast amounts of computational resources that make them unusable on
mobile devices.
We present a novel panorama stitching method that is designed to create high-quality image mosaics from
both video clips and separate images even on low-resource devices. The software is able to create both 360
degree panoramas and perspective-corrected mosaics. Features of the software include among others: detection
of moving objects, inter-frame color balancing and rotation correction. The application selects only the frames
of highest quality for the final mosaic image. Low-quality frames are dropped on the fly while recording the
frames for the mosaic.
The complete software is implemented on Matlab, but also a mobile phone version exists. We present a
complete solution from frame acquisition to panorama output with different resource profiles that suit various
platforms.
KEYWORDS: Video, Energy efficiency, Video acceleration, Mobile devices, Profiling, Multimedia, Multiplexing, Control systems, Operating systems, Computing systems
The multimedia capabilities of emerging high-end battery powered mobile devices rely on monolithic hardware
accelerators with long latencies to minimize interrupt and software overheads. When compared to pure software
implementations, monolithic hardware accelerator solutions need an order of magnitude less power. However, they are
rather inflexible and difficult to modify to provide support for multiple coding standards. A more flexible alternative is
to employ finer grained short latency accelerators that implement the individual coding functions. Unfortunately, with
this approach the software overheads can become very high, if interrupts are used for synchronizing the software and
hardware. Preferably, the cost of hardware accelerator interfacing should be at the same level with software functions. In
this paper we study the benefits attainable from such an approach. As a case study we restructure a MPEG-4 video
decoder in a manner that enables the simultaneous decoding of multiple bit streams using short latency hardware
accelerators. The approach takes multiple video bit streams as input and produces a multiplexed stream that is used to
control the hardware accelerators without interrupts. The decoding processes of each stream can be considered as
threads that share the same hardware resources. Software simulations predict that the energy efficiency of the approach
would be significantly better than for a pure software implementation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.