Topic 4: Multimedia – Computer Notes Form 4

Welcome to our website scholars.co.tz. In this article, are you looking for Topic 4: Multimedia – Computer Notes Form 4

MULTMEDIA (GRAPHICS, ANIMATIONS, SOUNDS AND VIDEOS)

Multimedia is the combination of sound, graphics,  animation, and video.

In the world of computers, multimedia is a subset of hypermedia, which combines the elements of multimedia with hypertext so as to link the information.

Overview

In this topic, we discuss issues concerned with using multi-media components. At the end of this section, you will be able to

• Discuss formats for graphics, sound and video and select the appropriate format for a specific purpose
• Compare formats for graphics, sound and video
• Have an appreciation for other technology tools for the display of information

Multimedia is the combination of sound, graphics, animation, and video.

In the world of computers, multimedia is a subset of hypermedia, which combines the elements of multimedia with hypertext so as to link the information.

1. Graphics

There are many formats for storing pictures in digital format. Images associated with personal computer systems are stored in pixel format. A pixel is an individual dot or element with associated intensity and color information. Pixels are combined to form images. The size of each pixel is a function of the quality of the monitor. For most monitors, the pixel size is about 0.28mm.

Color Depth

Some storage formats for pictures involve compression, a technique that reduces the amount of storage space required for the picture, often at the expense of quality. Other formats alter the color depth of the image, as less color will require less storage space. Color depth refers to the number of unique colors used by the image. Using more colors in a picture will require more storage space (bytes). For instance, a single bit can be either on or off.

This would correspond to two colors, black and white. Using 8 bits for each pixel would result in a color depth of 256 colors. The more colors you have, the more realistic and natural the image looks. The following table shows the connection between color depth and the number of bits representing that color depth.

You can observe a direct relationship between the quality of the image, color depth, and image size in bytes. The larger the image size in bytes, the longer it takes to download.

Image size is always expressed as x and y (width followed by height). The above image has a dimension size of 150 by 100. The larger the image size in pixels, the larger the file size is. For example, increasing the same file to double the pixel size (300 by 200) increases the file size of the image to 41,026 bytes.

From this we can draw some basic principles.

• work with images in large sizes and increased color depths
• when producing images for the web, reduce both the color depth and image size

Image-size

How big should images be on a web page is a good question. A rough guide is to look at the print medium (such as books and newspapers). Seldom is the image size so big that it takes up all the space on the page. We would argue that the web is an entirely different medium, and some of the rules applicable to the print medium are not applicable here.

However, one must keep in mind that some users may have limited bandwidth and others may be using monochrome monitors or systems that only display 256 colors. Other users may only have a screen size of 800 by 600 pixels. Designing for the web often means designing for the lowest common denominator. We would make the following suggestions.

• color depth of 256 colors
• image size on average no more than 320 by 240 pixels
• file size to be 15KB or less (the more images on a single page the more closely this needs to be monitored)

Graphic Formats

There are a number of graphic formats available for the storage of images. The following table lists some of the more common graphic formats currently available.

Other image formats are available. It is worth saving an image in both JPEG and GIF format, then selecting the image based on your criteria (either the best quality or the smallest file size). JPEG offers varying levels of compression.

The more compression applied, the smaller the file size, and the lower the image quality. This is illustrated in the images below.

Altering the image contrast and brightness
There are times that images should be manipulated for better results. Some images are better displayed as grayscale (shades of gray with no color). Examples of these are related to the medical profession, such as x-rays. Often, detail can be hidden in an image, and adjustment of the image characteristics will reveal this hidden information. Consider the following image on the left. It was taken with a digital camera. The image on the right has been enhanced by image processing software (Microsoft Image Composer), and shows more detail than the original image.

Special Effects
In general, most image editing software provide special effects that can be applied to an image. A web designer should consider these effects carefully.

What is the desired message that the author wishes to convey to the receiver by using such effects? Every element, time and bandwidth is precious. Effects should be used sparingly so that the receiver does not become desensitized to their use. The following images illustrate some of the effects possible with Microsoft Image Composer.

 Summary

Images are an important part of any  multi-media project. Images are manipulated in color depth, size and other aspects before they are saved in a format suitable for inclusion on web pages. Consideration should be given to image size and file size when producing images for the web. The two most popular image formats for the web are JPEG and GIF.

A good designer will manipulate the image to extract the best parts of the image in order to maximize its impact on the receiver. This can involve altering the image size, adding special effects, cropping the image (selecting only a part of the image) or combining the image with other image elements.

2.  Animations

Animations are moving graphic images (normally without sound) that consist of a series of images played one after the other. These images are stored in a single file. The current format of these animations (for the web) is animated GIF. The following is an example of an animated GIF.

This animation comprises a series of 10 frames, each slightly different from each other. When played as a sequence, they simulate movement. The ten frames that make up this animation are shown below.

Each image is displayed for a specified time interval, and then replaced by the next image. The persistence of the images on the screen and perceived by the human eye give the impression of smooth movement. This is identical to the way that cinema movies are viewed.

Images must be the same size and should use the same color depth. There are a large number of sites on the Internet dedicated to animated GIF files. In addition, many CD-ROM can be purchased from computer stores that are full of animated GIF’s for use on web pages.

Construction of animated GIF images

In order to construct animated GIF images, special tools are required. Two such tools are Microsoft GIF animator and “GIF construction set for Windows XP. These tools allow you to

• import a sequence of GIF files
• specify the duration in time that each image will be displayed
• specify the total time duration of the image or the number of repetitions
• rearrange the order of the images
• apply transparency rules (specify how the background will appear)
• specify the output image size
• save the image to an animated GIF file

Other Issues

Animated GIF’s can be large in size. The more animations that are included on a web page, the longer the page will take to download. The larger the animation size, the larger the file size will be. It is important when creating animated GIF’s to keep the file size at a minimum.

Too many animated GIF’s on a web page can be distracting. In addition, the major use of animated GIF’s appears to be primarily a method of trying to improve the look of the page. Flashy graphics may not add to the recall or effectiveness of the message. However, a well thought out  animation will aid recall and be very effective at communicating the idea or concept to the viewer.

3. Sounds

This section focuses on the technology of sound. An important aspect of multi-media applications, sound is very effective for creating mood and dramatization. In addition, certain types of music, namely from the Baroque period, have been shown to improve mental concentration and aid later recall.

We use sound every day in communication. Our early ancestors used sound to convey campfire stories, and aided by gestures and other dramatic effects, kept generations spellbound. The power of sound, used properly, should not be underestimated.

Types of sound files
This is a brief discussion of the two main sound formats for personal computers.

MIDI

The Musical Instrument Digital Interface was developed during the 1980’s as a mechanism of controlling electronic musical instruments such as keyboard synthesizers. MIDI files support a wide range of synthetic instruments, such as drum, clarinet, violin, piano and guitar. Musical sequences can both be recorded as the instrument is played, or composed on the PC and then downloaded to the instrument for later playing.

To play MIDI on your computer, you must have a MIDI interface. Most modern sound cards support MIDI. MIDI files consist of information that relates to the instrument to be played, the note, and its duration. Files in MIDI format are significantly smaller in size than other formats such as WAV or AU.

WAVE:-

Waveform Audio is the most popular format available today for the personal computer. Waveform files consist of actual samples of the sound file. This tends to more accurately reflect the original sound, but file sizes are a lot larger. The more samples taken of the original sound, the better the quality is. Sound files consist of a header and a number of samples. The header defines the format of each sample and any compression technique that was applied to each sample. When the sound file is created, the sound is sampled at a regular rate and each sample converted into a digital number that represents the sound at the instant of sampling.

Sample Rates

Sound is digitized (converted to digital format) and stored as a sound file. Sample rate refers to the number of times a second that the original sound was sampled. Higher sampling rates mean better quality of sound. However, as the sample rate increases, the file size increases also. You must make a decision to sometimes trade quality at the expense of file size. The following table lists three common sound qualities and their sample rates.

Again, we are reminded of the fact that bandwidth is still scarce for most users of the Internet. This means designing for the lowest common denominator in mind. The highest percentages of users on the Internet are those with PC’s, so this means that WAV format will be more common.

In addition, due to bandwidth restrictions, you should ensure that sound files are kept short and are small in size. For instance, a telephone quality sound file using ten seconds of speech is 110KB. This will take a minimum of 30.55 seconds to download at 28.8Kbps.

8 bit and 16bit

This refers to the size of each sample. It is thus obvious that a 16-bit sample is twice as large as an 8-bit sample. The advantage of using 16-bit sampling is that a wider range of values is possible, so the resultant sound appears to have better tonal characteristics (it sounds more realistic).

Again, sound files sampled at 16-bit will be twice the size in bytes as the same sound sampled at 8-bit. You will need to be able to justify the larger sample size, and must remember the negative effect this will have on download times.

Stereo and mono

We have two ears that we use to receive sound. The airwaves striking our eardrums are converted into electrical impulses and interpreted by our brain. Similarly, most  computers use two speakers to generate sound, similar to the sound speakers of home sound systems. These are called left and right channels. They are designed to simulate the way we hear, by providing distance separation of sound.

Stereo uses two channels called left and right. The original sound is sampled as two separate channels and stored as two lots of information, one for the left channel, and one for the right channel. This means every sample contains both channels.

Mono refers to using a single sound sample. If you play this back on a stereo system, the single sound sample is sent to both channels. It is not possible to create the appearance of distance separation using mono. However, mono files are smaller in size (half the file size) compared to an equivalent stereo file.

Stereo files are twice the size in bytes as a mono file due to each sample containing left and right channel information. They have the advantage of appearing to give distance separation and are useful to create special effects. This must be traded against the available bandwidth and how long the file takes to download. Mono files are the most common format. The following table shows the comparative file sizes (approximate) for one second of recorded speech.

Compression(Codecs)

A codec is a “compression decompression” device. It is often implemented in software, though many new accelerated cards have codecs implemented in hardware for faster coding and decoding of signals.

A codec is used to alter the signal in order to reduce the bandwidth requirements (i.e., the size of the file is compressed making it smaller and thus quicker to transfer or download). The codec converts the file format to a format suitable for playing on the destination hardware.

The following is a brief discussion on two popular speech codecs

ADPCM (Adaptive Differential Pulse Code Modulation)

This codec reduces the file size by over 50%. However, any compression causes loss of sound quality. There is no generally accepted standard for ADPCM, with codecs generally provided by manufacturers of sound cards that are installed using a software install program.

PCM (Pulse Code Modulation)

Another common codec supported by all Windows operating systems.

Streaming Audio

The major problem associated with the previous format of waveform audio is the file size. When you use a waveform audio file on a web page, the browser must download the entire file before it can be played.

For files of reasonable size, this leads to long delays for the receiver. Streaming audio addresses the issue of this delay by allowing the file to be played whilst it is still being downloaded. Special players are required for viewing and playing streaming audio clips.

Streaming audio is primarily associated with Real Networks. “Real Player” supports the playing of streaming audio over the Internet. The advantages of using streaming audio are

• the use of a special codec that reduces the bandwidth requirements for audio down to about 5Kbps
• instant playing of the audio clip
• suits limited bandwidth connections (i.e., a dial-up user with a 28.8Kbps modem)

When a user selects a streaming audio clip, the streaming audio player contacts the server and establishes the bandwidth and delays between the user  computer and the host server. A few seconds of the clip is then downloaded into temporary buffer storage before the clip begins to play.

As the clip begins to play, more content is downloaded and placed into temporary storage. If there is a congestion delay in the Internet connection, the clip continues to play using the contents of the buffer storage.

Hopefully, before the buffer storage is exhausted, the congestion delay will cease and thus the buffer storage will fill up again. In this manner, the streaming player tries to achieve smooth playback of the clip. Without using storage, gaps or breaks in the audio would be evident when a congestion delay occurred.

A special content provider program is necessary for producing streaming audio clips. A limited free version can be downloaded from the Real Networks web site. In addition, to provide streaming audio clips from a server requires the use of a streaming server.

Recording and editing of sound files

To record WAV sound files you need a microphone (or sound source such a tape recorder), sound card, and sound recorder program.

Topic 4: Multimedia – Computer Notes Form 4

Other considerations

As always, bandwidth is a premium for the Internet. This means that you must tradeoff quality and size for quick download speed and fast display. New technology like streaming audio can deliver more content utilizing the same or less bandwidth.

However, streaming audio content is not currently supported (as at Dec 1998) within a web browser. This means using streaming audio poses the problem of requiring the receiver to have the streaming player installed on their  computer. For this reason alone, you should ensure that an Internet link to where the player can be downloaded from is available on the web page.

In addition, using a separate player rather than the browser (which supports WAV format) means greater memory requirements. This is because the player needs to be started and loaded into memory, in addition to the web browser.

This also raises a contextual issue, as the user is now interfacing with two packages, the browser and the player. This can lead to confusion and reduce the effectiveness of the presentation or learning experience.

If producing for a CD-ROM, many of the issues related to sound file sizes and sampling rates do not apply to the same extent. A CD-ROM can provide information at a much faster rate than a dial-up modem. Typically, a 2X CD-ROM can transfer data at 300KB per second. This means higher quality sound can be used due to the larger available bandwidth.

 4.  Videos

Video Recording is the process of recording still or moving images electronically, rather than photochemical as in photographic film.

The delivery of video on web pages has tilled recently been fraught with difficulty. Lack of standards and support a few years ago were major issues. The reality of the Internet and lack of available bandwidth means that delivery of video content is severely restricted to simple slow moving content (such as people speaking face to face or teachers delivering lectures) and in very small frame windows (of 320×240 or less).

The frame rate (the number of images displayed per second) has also been a problem, with a lot of real-time video (such as net meeting or video conferencing software) achieving rates of 10 frames per second (fps) or less. Around 15fps is the minimum frame rate required for smooth movement. Television frame rates are 30fps for NTSC (American standard) and 25fps for PAL (Europe, NZ standard).

The more frames per second the more realistic movement appears. However, this requires greater bandwidth. With video we are stretching the limits considerably. The features of video are

• content keeps changing so we need a reasonable frame rate else the motion appears jerky
• the window size needs to be a reasonable size to ensure enough detail is perceived
• sound is added for maximum effect

Every increase in a video parameter means you require more bandwidth. For instance, more frames per second require more bandwidth. Larger window sizes require more bandwidth. A  video with audio needs more bandwidth than a video that has no audio.

Just what are the bandwidth requirements of a video image? Well, that depends on the size of image, the color depth, the frame rate and a number of other factors. Consider a 15fps video using 256 color (8-bit color depth) of 320×240 in size, lasting 10 seconds in duration. The bandwidth per second required for this is

15 * ((320 * 240) * 8) = 9.216Mbps

This does not even include audio yet. As you can see, this video cannot be played over the Internet using a dial-up 28.8Kbps connection. In fact, it would not even play on some personal computers. What is needed is a mechanism of reducing the information in the video to a much smaller size (i.e., reduce the bandwidth required).

This is achieved by eliminating redundant information from the video, and compressing what’s left. This is called compression. The device (either in software or hardware) that does compression of the video and decompresses it ready for playing is called a codec.

Frame Sizes and Formats

There is a number of common video formats for personal  computers. The three most common formats available today are

• MOV (Developed and supported by Apple Inc.)
• AVI (Developed and supported by Microsoft.)
• MPEG

Video has standardized on a 4:3 ratio. This ratio is an accepted standard. Typical video sizes are

• 320 x 240
• 160 x 120

When producing video for multi-media productions such as CD-ROM and the Internet, you should adhere to common standards. This means using commonly accepted frame sizes, frame rates, color depths, compression codecs and video formats. Using a non-standard format may cause problems in playback on the client computer, such as the inability to play the video.

Codec

Codec is the hardware that can convert audio or video signals between analog and digital forms.

Codec compresses and decompresses the video stream. When a codec is applied, it looks at the original video stream and changes it, saving the result to a file. When the file is played back, the codec interprets the changes and attempts to reconstruct the original video stream.

A lossless compression scheme preserves the original data, so that the resultant video after decompression looks the same as that before it was compressed. Most lossless compression schemes use run length encoding (RLE).

This works well with computer generated images, but is not very effective when applied to digitized  video. The reason for this is because RLE works by discarding continuous regions that have the same colors. Computer generated images tend to have more solid colors than digitized video, so RLE is more effective for these.

A lossy compression scheme removes data from the video stream that the viewer is unlikely to notice. The amount of information that is lost depends on how much compression is applied.

A spatial compression scheme compresses the information in every frame. This can result in blurring, blockiness (small blocks of the same color) and streaking (lines of the same color).

A temporal compression scheme compresses the data by comparing frames over time. Frame differencing is a type of temporal compression that stores data for only the frames that contain changes. This can result in blackness.

The following is a summary of the more common video codecs supported by personal computers (as at Dec 1998).

Key frames

A key frame contains the entire image information and is used as a reference for subsequent frames in a sequence. In the illustration below, key frames are repeated every seven frames. Intermediate frames that occur between key frames only contain the changes between that frame and the key frame. This method reduces the file size of the video.

The intermediate frames are called delta frames.

 AVI (Audio Video Interleave)

AVI is a format Microsoft introduced with the Windows 3.1 operating system. It intermixes both audio and video content within the file. The video and audio streams in the file are stored sequentially, without time stamps. A header in the file indicates the rate at which the video and audio streams are to be played. The audio stream is stored as a WAV file.

MPEG (Moving Picture Experts Group)

The MPEG-1 standard was adopted in 1991. The goal of MPEG-1 was to develop a way of compressing a  video signal and then play it back using a CD-ROM or telephone line. A digital bit rate of less than 1.5 Mbps was considered necessary. The standard specifies 352×240 pixels at 30 frames per second. The quality is acceptable for the home consumer market.

The MPEG-2 standard was designed to deliver better quality images and larger video size, suitable for broadcast quality. MPEG-2 specifies a data rate of 6.0Mbps.

The MPEG-4 standard is currently under development (as at Dec 98).

• Compression of both audio and video
• encodes changes in image motion that occurs from frame to frame
• audio is split into 32 sub-bands
• video compression levels of 50:1 to 200:1
• audio compression levels of 5:1 to 10:1
• MPEG-1 data rate up to 1.5Mbps
• MPEG-2 data rate up to 10Mbps

Streaming Video

Streaming video is very similar to streaming audio. We are faced with the same problem of file size and the delay involved in downloading the entire video file before it can be played on the client  computer. Streaming video addresses the issue of this delay by allowing the file to be played whilst it is still being downloaded. A special player is required for viewing and playing streaming video.

Streaming video is primarily associated with Real Networks. “Real Player” and “Real Player G2” support the playing of streaming video (and audio) over the Internet. The advantages of using streaming video are

• the use of a special codec that reduces the bandwidth requirements for video
• instant playing of the video clip
• suits limited bandwidth connections (i.e., a dial-up user with a 28.8Kbps modem)

When a user selects a streaming video clip, the streaming video player contacts the server and establishes the bandwidth and delays between the user computer and the host server. A few seconds of the clip is then downloaded into temporary buffer storage before the clip begins to play. As the clip begins to play, more content is downloaded and placed into temporary storage. If there is a congestion delay in the Internet connection, the clip continues to play using the contents of the buffer storage.

Hopefully, before the buffer storage is exhausted, the congestion delay will cease and thus the buffer storage will fill up again. In this manner, the streaming player tries to achieve smooth playback of the clip. Without using storage, gaps or breaks in the video would be evident when a congestion delay occurred.

A special content provider program is necessary for producing streaming  video clips. A limited free version can be downloaded from the Real Networks web site. In addition, to provide streaming video clips from a server requires the use of a streaming server.

Video Production tools

The following is a very brief description of three common video production tools for the creation and conversion of video content suitable for CD-ROM or Internet production.

Adobe Premiere

This video editing and production software is excellent for designing and producing video content.

 XING MPEG Encoder

This package converts an AVI file to MPEG format.

Real Producer G2

This package converts an existing AVI file to streaming video format.

Other Considerations

The following is a video file (© J. Winchester) of just over 4 seconds duration. It comprises a 320×240 animation. The common settings for each AVI file has been set to 15fps, 80% compression level with key frames every 15 frames. The data rate has been limited to 100KB/s. The audio has been set to 11KHz 8-bit mono.

View each of the  video files as shown above. Compare them for quality of image and smoothness of motion. One observation you will find is that for streaming video, smoothest motion is best for content that is moving, but the image quality is lower. This illustrates that tradeoffs are made in compression between motion and quality. Compare also the data rate of 28.8K versus 56K. Is there a discernable difference between the two clips?

The type of options and codec you choose directly relate to the content of the video you are generating.