Signal processing plays a part in nearly all modern media — every time we listen to recorded music or view a photograph or a video, it is virtually certain that signal processing has played a part in its creation, editing or storage. It is not even a slight exaggeration to say that the present system of media content creation and delivery — all modern visual and sound media — would not be possible without the methods described in this article.

This article provides an overview of this technology, discusses the mathematical techniques, then provides source-code examples for the key methods with examples of their use.

The basic idea of signal processing is that a complex, time-domain waveform like this —

Graph 1

— can be fully and completely recreated using a frequency-domain representation like this:

Graph 2

It's difficult to overemphasize the significance of this transformation. Graph 1 above represents an old-style AM ("Amplitude-Modulated") radio signal, with a "carrier wave" (the center point) and some information in sidebands (the two adjacent points). In the early days of electronic technology, if you wanted to create a radio signal, you simply generated the time-varying waveform in Graph 1. If you wanted to preserve a hour's worth of the signal, you were obliged to record the waveform in Graph 1 for an hour.

By learning modern signal processing techniques, we discover that the waveform in Graph 1 is actually fully represented by the three points in Graph 2. If we want to create Graph 1 for any period of time, we only need the three data points in Graph 2. The original waveform has been reduced to a handful of numbers.

Modern signal processing consists of performing a Fourier Transform to create the most efficient representation for a complex signal, then saving only that representation. When it is time to recreate the original signal, we apply an inverse Fourier Transform that converts the frequency-domain information (Graph 2 above) into the original time-varying waveform (Graph 1).

Here is another use for signal processing. Let's say some aliens are trying to send us the secret to eternal happiness, but because they live many light-years away, their signal is unreadably weak, buried in noise like this:

Graph 3

This looks like an impossible task — even though the aliens might tell us something interesting, because of the noise we don't even know they're trying to reach us. But we can use signal processing to convert the original time-domain signal into a frequency-domain representation —

Graph 4

— and the alien signal pops right out of the noise. By the way, this is exactly how SETI (the Search for Extraterrestrial Intelligence) searches for alien signals, and it is why they need lots of computer power to accomplish their mission.

A number of natural phenomena can be modeled and predicted using signal processing methods, and one of these is ocean tides. Tides follow a complex repetitive pattern with an interplay of daily, monthly and yearly cycles, which makes the signal processing methods described in this article an ideal data reduction technique.

First, tidal observations for a particular location are collected and placed in a data set that correlates tidal heights with time. Then a Fourier Transform is applied to create a compact representation of the original data. Just as in the earlier examples, the compact representation contains within it the essence of the original wave pattern, but in a much smaller size, and with the important property that the original waves can be modeled for any time in the past or future. Finally, an inverse Fourier Transform is applied to the compact form to provide a tidal prediction for a particular time and place.

Here is an example prediction:

Tidal height (colors) versus date and time (horizontal and vertical axes). Source: JTides

In this example, three axes of information are provided — date (vertical axis), time (horizontal axis) and tidal height (colors). This diagram was created using a program (JTides) that relies completely on the Fourier signal processing methods described here. Because of the efficiency inherent in Fourier processing, a relatively small program can be relied on to provide worldwide tidal forecasts for a period of decades.

Le's say we want to save bandwidth in a communication channel by transmitting the absolute minimum of information required to reproduce a television picture (which is just a complex radio signal as in Graph 1 above). In the early days of television, one often saw a TV test pattern like this:

Old-style TV test pattern

In the original TV system, to transmit this image across the airwaves, it was necessary to send the exact same information 30 times per second. In modern times, because of signal processing and other improvements, we are only obliged to transmit a group of mathematical values representing the original image, then we only need to transmit new values if the image changes. This is why the old analog TV system is giving way to digital — the analog system was incredibly inefficient.

The above plots were created using software provided with this article — by downloading and using the software, readers can create similar results of their own. The software is fully described in this article, is open-source and is released under the GPL.

After creating FFTExplorer some years ago, I have received a number of requests from educators to unbundle the code and make the Fast Fourier Transform (FFT) routines more accessible. This article answers that request.

This multi-page article explains the highlights of signal processing technology, discusses the technical underpinnings, and provides source code and programs the reader may use to experiment with signal processing. To navigate, use the arrows or the drop-down lists at the top and bottom of each page.