Skip to main content
Loading Explore guides…
Origins of Signal Processing · Acedra
Home
Explore
Engineering
Origins of Signal Processing
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
127 segments
Copy
Hide video
0:00
Origins of signal processing. Before the digital age reduced the world to a stream of binary digits, the processing of information was a physical visceral endeavor. A signal was not an abstract mathematical sequence stored in a silicon memory cell. It was a tangible force of nature. It was the rise and fall of an electric current surging through a copper cable on the floor of the Atlantic Ocean. It was the mechanical rotation of a brass gear in a differential analyzer. It was
0:34
the compression of air waves striking a concrete parabolic mirror on the coast of Kent. The history of signal processing is at its fundamental level the history of humanity's struggle to impose order on the chaos of the physical world. It is the story of how scientists and engineers learned to manipulate the fundamental properties of matter and energy, mass, momentum, capacitance, inductance, and resistance to extract meaning from noise. This review explores the rich and
1:08
complex origins of signal processing, tracing its evolution from the abstract mathematical theories of the enlightenment to the electromechanical zenith of the Second World War. It is a narrative that spans centuries and disciplines, weaving together the insights of reclusive mathematicians, the ambitions of industrial monopolists, and the desperate ingenuity of nations at war. In this pre-digital era, the algorithm was a physical machine and the computation was a continuous flow
1:41
of voltage or torque. To understand modern digital signal processing, one must first understand these analog roots. For the fundamental theorems that govern today's wireless networks and artificial intelligence models were forged not in code but in the fires of the analog age. Mathematical roots antiquity to the 19th century. The intellectual scaffolding of signal processing was erected long before the technology existed to implement it. The human desire to quantify and model the
2:16
natural world led ancient mathematicians to develop methods of approximation that would eventually become the basis of sampling and quantization. As early as antiquity, the method of exhaustion used by Archimedes to calculate the area of a circle treated the continuous curve as a polygon with an infinite number of discrete sides. This conceptual leap viewing the continuous as a limit of the discrete is the philosophical ancestor of the sampling theorem. However, the
2:49
true mathematical language of signal processing began to take shape in the 17th and 18th centuries with the development of calculus and the study of planetary motion. Astronomers and surveyors were the first to grapple with the problem of noise. No two observations of a stars position or planetary orbit were ever identical due to atmospheric distortion and instrument error. The challenge was to extract the true signal from the noisy measurements. In the early 19th century,
3:23
Carl Friedrich Gaus, the prince of mathematicians, formalized the treatment of error. While working on the orbital prediction of the dwarf planet series, Gaus developed the method of least squares, he posited that errors in measurement were not random chaos, but followed a specific probabilistic distribution, the bell curve or Gaussian distribution. By minimizing the sum of the squares of the differences between observed values and a theoretical model, Gaus showed
3:55
that one could mathematically estimate the most probable true value. This was a watershed moment. Gaus essentially defined the concept of estimation theory. He treated the observation as a composite entity. The signal, the truth, plus the noise, the error. The goal of mathematics was to filter out the noise to reveal the signal. This probabilistic approach would lie dormant for a century until it
4:24
was resurrected by Norbert Weiner to solve the problem of anti-aircraft fire control. But the seed was planted. Information is not absolute. It is statistical. While Gaus dealt with error, Jean Baptiste Joseph Forier dealt with complexity. In 1822, Forier published his magnum opus theory analytic de la shaler, the analytical theory of heat. His goal was to model the diffusion of heat through solid bodies, a physical process governed by partial differential equations. In the
5:01
course of this work, Forier made a startling claim that would become the cornerstone of all signal processing. Any continuous periodic function, no matter how complex or irregular, could be represented as the sum of simple sinosoidal waves. At the time, this idea was controversial. Prominent mathematicians like Lrangee were skeptical that a signal with sharp corners like a square wave could be constructed from smooth sine waves. Borier insisted that with
5:32
an infinite series of harmonics, the approximation would converge. He was right. The Forier transform provided the bridge between the time domain, how a signal changes moment by moment, and the frequency domain, how much energy the signal contains at each frequency. Forier's insight allowed engineers to deconstruct reality. A complex sound like a chord played on a piano or the vowel sound of a human voice was not a monolithic entity. It was a recipe of frequencies.
6:06
By adjusting the proportions of these ingredients, boosting the bass, cutting the treble, one could alter the signal's character. This was the birth of spectral analysis and filtering. Concepts that would remain abstract until electricity provided a medium fluid enough to manipulate. While Forier focused on periodic steadystate signals, the real world is full of transients, sudden shocks, decaying pulses, and starting currents. Pierre Simone Lelass, building on the work of Oiler,
6:39
developed a transform that extended Forier's analysis to these transient phenomena. The lelass transform mapped time domain functions into a complex frequency domain, the s plane, allowing for algebraic solutions of differential equations. For the future electrical engineer, the lelass transform was the key to stability. It provided a way to analyze how a system, whether a suspension bridge or a telephone amplifier, would react to a sudden disturbance, a step function.
7:12
It could predict whether the system would settle down smoothly or oscillate wildly out of control. This mathematical tool would become essential in the design of feedback control systems and analog filters serving as the rigorous proof behind the intuition of the circuit designer. Telegraphy and the tyranny of the medium. The transition from mathematical theory to engineering reality began
7:38
in earnest with the electric telegraph. In the mid 19th century, the telegraph was the internet of its day. A Victorian internet that shrank the world from weeks to minutes. However, as telegraph lines grew longer, extending from city to city and eventually under the ocean, engineers encountered a phenomenon that defied the simple DC circuit theories of the time, signal distortion. On short landlines, a dot was a dot and a dash was a dash. The current turned on and off almost
8:13
instantly. But on long submarine cables, the signal behaved differently. The sharp click sent from the transmitter arrived at the receiver as a long, slurred, lethargic rise in current. The dots and dashes blurred together into an unintelligible smear. This was the retardation of the signal, a problem that threatened to bankrupt the investors of the first transatlantic cable projects. The man
8:40
who diagnosed this malady was William Thompson, later Lord Kelvin. In the 1850s, Thompson analyzed the submarine cable not as a simple wire but as a massive capacitor. The copper conductor was the inner plate. The seawater was the outer plate and the gutaperta insulation was the dialectric. When a voltage was applied, the current could not simply flow to the other end. It first had to charge this immense distributed capacitance. Thompson applied Furier's heat diffusion equations
9:14
to the flow of electricity. He derived the law of squares, which stated that the time required for a signal to reach a detectable voltage at the far end was proportional to the square of the cable's length. A 2,000mi cable was not twice as slow as a,000m cable. It was four times as slow. This theoretical barrier suggested that trans oceanianic communication might be impossibly slow. The controversy that followed the Great Eastern saga was a battle between two philosophies
9:48
of signal processing. On one side was Wildman White House, the project's chief electrician, a proponent of brute force. White House believed that if the signal was weak and distorted, the solution was to hit it harder. He constructed massive induction coils to blast the 1858 cable with thousands of volts. He treated the cable as a passive pipe that was clogged, attempting to force the fluid through by pressure. Conversely, Thompson advocated for signal conditioning and
10:21
sensitivity. He understood that the information was still present in the smeared current. It just needed a detector capable of resolving it. Thompson invented the mirror galvanometer. a masterpiece of analog instrumentation. A tiny mirror was suspended by a single silk fiber inside a coil of wire. Even the faintest trickle of current from the Atlantic cable would cause the mirror to twist slightly. A beam of light reflected off the mirror onto a scale magnified
10:52
this movement, allowing the operator to read the derivative of the signal, the initial rise of the current, rather than waiting for the full charge. The tragedy of the 1858 cable, its failure after only a few weeks of operation, was caused by White House's high voltage pulses, which compromised the insulation. But the brief success of the cable, which carried Queen Victoria's 98word message to
11:19
President Buchanan, a transmission that took 16 hours, vindicated Thompson's theory. When the permanent cables were laid in 1866, they were operated according to Thompson's principles of low voltage and sensitive detection, establishing the first victory of signal theory over power engineering. While Thompson mastered the capacitive nature of the cable, he treated the signal essentially as a diffusion phenomenon like heat. It was the reclusive and eccentric genius
11:51
Oliver Heavyside who realized that electricity on a wire was a wave governed by the laws of electromagnetism. Living in isolation and often at odds with the scientific establishment, Heavyside reformulated Maxwell's equations into the vector calculus form used today. In 1876, he derived the telegraphers's equations, which took into account not just resistance and capacitance, but also inductance and leakage. Heavyside discovered a counterintuitive truth. Inductance, usually seen
12:27
as a source of resistance to changing current, was actually the cure for distortion. Heavyside proved that the distortion of a signal, the way different frequencies traveled at different speeds, dispersion, and attenuated at different rates, was caused by the imbalance between the magnetic and electric fields around the wire. He formulated the heavyside condition. If the ratio of inductance to
12:54
resistance equaled the ratio of capacitance to leakage, the line would be distortionless. On such a line, all frequencies would travel at the same speed and attenuate equally, preserving the shape of the voice or telegraph signal perfectly. Heavyside solution was to load the line by adding inductance. This could be done by wrapping the wire in iron tape or by inserting coils at regular intervals. Heavyside, a man of pure science, published his findings, but did not
13:27
patent a practical device. Decades later, at the turn of the century, the implementation of this idea became a fiercely contested commercial prize. Michael Poopin, a Serbian American physicist at Colombia University, patented a precise method of inserting these loading coils at calculated intervals along a telephone line in 1899. Poopin's analysis provided the engineering specifications that Heavyside's general theory lacked. The impact was immediate and lucrative. AT&T,
14:03
desperate to extend the range of its telephone network without resorting to prohibitively expensive thick copper wires, purchased Poopin's patent. The sums involved were staggering for the time. Poopin received approximately $455,000, equivalent to over $11 million today. The loading coil saved AT&T an estimated $100 million in the first quarter of the 20th century. This episode highlighted a recurring theme in the history of signal processing. The lag between mathematical
14:39
insight and engineering application. Heavyside's abstract vector calculus laid the groundwork, but it took Poopin's practical realization and AT&T's capital to turn the distortionless line into a reality that allowed human voices to travel between cities with clarity. Era of amplification and telefan. Passive components, resistors, capacitors, and inductors could shape a signal, filter it, and delay it, but they could not add energy to it. The passive era of signal processing
15:13
reached its limit with the physical resistance of the wire. No matter how perfectly a line was loaded, the signal eventually faded into the thermal noise. The quest for a telephone repeater, a device that could rejuvenate a dying signal, led to the invention of the vacuum tube. In 1906, lead to Forest added a third element, a zigzag wire grid to the existing Fleming valve diode, creating the AON triode. This device was the first electronic amplifier by applying
15:49
a weak fluctuating voltage to the grid. Deforest could control a powerful stream of electrons flowing from the heated filament to the plate. It was a valve for electricity, allowing a small signal to modulate a large power source. The audon was initially a mysterious and unreliable device. Deforest believed incorrectly that the presence of gas ions was essential to its operation. It was the researchers at Bell Labs and General Electric, including Irving Langmir, who
16:22
refined the device into the high vacuum triode, a stable and predictable amplifier. By 1915, AT&T had installed these vacuum tube repeaters along a line stretching from New York to San Francisco, inaugurating Transcontinental Telephone Service. For the first time, signal processing was active. The signal could be refreshed indefinitely, decoupling communication distance from signal energy. This opened the floodgates for complex processing chains. Signals could be amplified,
16:57
inverted, mixed, and filtered without the catastrophic loss of power that plagued passive networks. With the ability to amplify signals came the desire to economize the medium. Copper wires were expensive to string across a continent. Telephone engineers realized that a single pair of wires had a bandwidth far exceeding what was needed for a single human voice which requires only about 300 to 3400 hertz. This realization birthed frequency division
17:29
multipplexing FDM. By using the vacuum tube as an oscillator in mixer, engineers could shift the frequency of a voice signal. One conversation could be carried at its bassband frequencies. A second conversation could be modulated onto a carrier wave at 12 kHz, a third at 16 kHz, and so on. These stacked signals traveled together down the wire and were separated at the receiving end. The success of FDM depended entirely on the quality of the separation. This required filters
18:05
of unprecedented precision. The crude resonant circuits of the radio pioneers were insufficient. They allowed cross talk between adjacent channels. In 1915, George Campbell of Bell Labs and Carl Willie Wagner in Germany independently developed the theory of the wave filter using Heavyside transmission line theory. They designed latter networks of inductors and capacitors that could mathematically slice the frequency spectrum. They defined the concepts of pass band,
18:38
the frequencies to let through, and stop band, the frequencies to block. The design of these analog filters became a high art. Engineers used image parameter theory to cascade sections of filters, creating circuits with steep rolloffs that acted like brick walls in the frequency domain. These passive LC filters were the silent sentinels of the telecommunications network, sorting the cacophony of millions of voices into orderly lanes of traffic. The mathematical sophistication
19:12
required to design these networks involving complex impedance matching and hyperbolic functions represented the pinnacle of precomputer circuit theory. World War era detection, radar, and sonar. The outbreak of the Second World War transformed signal processing from a commercial utility into a technology of existential survival. The need to detect enemy aircraft and submarines in the fog of war drove the development of entirely new theories of detection and estimation.
19:49
Before the dominance of electromagnetic radar, nations experimented with acoustic detection. The most ambitious of these projects was the British Network of Sound Mirrors or listening ears. Built along the coasts of England, most notably at Denge in Kent. These were massive concrete structures, some standing 20 or 30 ft tall, shaped into precise parabolic or spherical sections. They functioned as acoustic telescopes. The hard concrete surface reflected incoming sound waves
20:27
from enemy aircraft engines, focusing them to a focal point where a microphone or a human listener was stationed. The physics of the sound mirror was sound, pun intended, but its utility was doomed by the medium. Sound travels at approximately 340 m/s. In the era of slow biplanes, this was sufficient. But as aircraft speeds increased in the 1930s, approaching 300 mph, the latency of sound became fatal. By the time the sound of a bomber reached the mirror, and the operator
21:06
calculated a bearing, the aircraft was miles away from its apparent position. The sound mirrors were abandoned, standing today as silent concrete monuments to the limitations of analog processing speeds. A similar battle was being fought beneath the waves. The threat of the Ubot in WWI led to the development of Azdic, anti-submarine detection investigation committee, later known as sonar. The breakthrough in sonar was the transducer. In 1917, Paul Ljivan, a French physicist,
21:46
utilized the po electric effect discovered by the Cury brothers. He constructed a sandwich of quartz crystals between steel plates. When an electric pulse was applied, the quartz expanded, generating a high frequency ultrasonic ping. When the echo returned, it compressed the quartz, generating a voltage. Sonar signal processing was a struggle against the complex acoustics of the ocean. The
22:15
ocean is not a homogeneous medium. Layers of water at different temperatures. Thermoclines refract sound waves creating shadow zones where submarines can hide. Early sonar operators had to be skilled signal processors, listening to the Doppler shift of the echo to determine if a submarine was approaching or retreating, and mentally filtering out the background noise of shrimp, whales, and waves. The limitations of acoustic detection led to the adoption of radio waves,
22:50
which travel at the speed of light. This was the birth of radar, radio detection and ranging. Radar is in essence an active signal processing system. A transmitter sends out a known signal, a pulse, and a receiver looks for a reflection, an echo. The core challenge was the signal to noise ratio, SNR. The echo from a distant aircraft is infiniteesimally weak, often buried in the thermal
23:22
noise generated by the movement of electrons in the receivers's own vacuum tubes. The early radar systems like the British chain home used flood light style transmission and relied on operators to stare at an oscilloscope and a scope picking out blips in the grass of noise. But as the war progressed, the need for automated and optimized detection became paramount. In 1943, Dwight North, a researcher at RCA Laboratories, wrote a classified report that revolutionized radar
23:58
receiver design. He asked a fundamental question. What is the optimal filter to maximize the SNR of a received pulse in the presence of white noise? North proved mathematically that the best possible filter was one whose impulse response was the time reversed and complex conjugate of the transmitted pulse. This became known as the matched filter or north filter. The concept was profound. It meant that the receiver shouldn't just look for energy. It should look for the specific
24:31
shape or pattern of the transmitted signal. A matched filter acts as a template matcher. It slides the memory of the transmitted pulse over the incoming noisy signal. When the two align, the output spikes to a maximum. In the analog domain, implementing a perfect mathematical correlation was difficult. However, North showed that for a simple rectangular radar pulse, the matched filter could be approximated by a standard intermediate frequency bandp pass filter with a
25:03
specific bandwidth. This theoretical guidance allowed engineers to tune their radar sets for maximum range, effectively pulling targets out of the noise floor using math. Another plague of radar was clutter, strong reflections from stationary objects like mountains, buildings, or the sea surface. These static echoes often swamped the smaller echoes of moving aircraft. To solve this, engineers developed moving target indication or MTI. MTI exploits the
25:36
Doppler effect. A moving target shifts the phase of the radio wave slightly from pulse to pulse. A stationary target returns the exact same phase every time. To detect this change, the radar needed memory. It needed to store the echo from pulse one and compare it to the echo from pulse 2. If the echo was identical, stationary, it was subtracted out. If it changed, moving, it was displayed. In the 1940s, there was no digital RAM to store the radar signal.
26:09
The solution was the mercury delay line. This device consisted of a tube filled with liquid mercury with po electric transducers at both ends. The electrical radar signal was converted into a sound wave, a mechanical vibration that traveled through the mercury. The length of the tube was precisely calibrated so that the travel time of the sound exactly matched the interval between radar pulses, typically a few milliseconds. The delayed pulse emerging from the mercury
26:40
was then subtracted from the new incoming live pulse in a vacuum tube differential amplifier. The mountains canceled out. The planes remained. This delay line cancellor was a triumph of hybrid signal processing using the slow speed of sound in liquid metal to create a time buffer for the speed of light. Era of prediction and cybernetics. Signal processing in WW2 was not just about detection. It was about prediction and control. The problem of anti-aircraft
27:15
fire control, hitting a fastmoving plane with a slowmoving shell, required solving differential equations in real time. The Bell Labs M9 gun director, deployed in 1943, was the pinnacle of electronic analog computing. Unlike previous mechanical directors that used cams and gears, the M9 used electricity to model mathematics, the M9 represented physical quantities, range, azimuth, elevation as continuous voltages. It used op amps, high gain vacuum tube DC amplifiers to
27:57
perform mathematical operations. To integrate a function, the M9 charged a capacitor through a resistor. To differentiate, find velocity. It measured the current across a capacitor. The M9 closed the loop between the radar sensor and the gun actuator. It took the radar data, calculated the velocity and acceleration of the target, computed the ballistic trajectory of the shell, and moved the gun to the predicted intercept point. The system was so effective that during
28:32
the defense of London against the V1 flying bombs, which flew straight predictable paths, M9 directed batteries shot down over 90% of the targets in the final weeks of the campaign. While the M9 handled the ballistics, the theoretical problem of prediction was being tackled by Norbert Weiner at MIT. Weer was asked to improve the tracking of aircraft that were not flying straight lines, but taking evasive action. Weer realized that the pilot's motion was a stochcastic or random
29:08
process. It wasn't completely random. The pilot couldn't turn the plane instantly, but it wasn't fully deterministic either. It was a statistical time series. In 1942, Weiner produced a classified report titled the extrapolation, interpolation, and smoothing of stationary time series. Due to its yellow cover and the impenetrable difficulty of its mathematics, it was nicknamed the yellow peril. In this document, Weiner formulated the Weiner filter. Unlike the north filter,
29:44
which optimized for a known pulse shape, the Weiner filter optimized for the statistical properties of the signal and the noise. Weer framed the problem as minimizing the mean square error between the predicted position and the actual position. Weer's work was a paradigm shift. He moved signal processing from the deterministic world of circuit design to the probabilistic world of information theory. He treated the signal as a carrier of information and noise as a corruptor
30:18
of that information. His realization that the mathematics of control systems machines and the nervous systems biology were effectively the same led him to coin the term cybernetics. The yellow peril laid the foundation for modern estimation theory influencing everything from communications engineering to the common filters used in space navigation. Analog computing the mechanical brains
30:48
before the transistor computer often referred to a machine that modeled physics using mechanical analogies. These machines were the heavy lifters of signal processing before the digital dawn. One of the earliest and most beautiful analog computers was Lord Kelvin's tide predicting machine developed in the 1870s. Tides are complex waveforms created by the gravitational pole of the moon and sun composed of many overlapping periodic cycles or harmonics. Kelvin's machine was a
31:24
mechanical forear synthesizer. It used a system of wires and pulleys. Each pulley oscillated up and down at a frequency corresponding to a specific tidal constituent like the lunar semidernal cycle. A single wire ran over all the pulleys physically summing their vertical motions. The resulting movement drove a pen on a rotating drum, drawing the predicted tide curve for any port in the world
31:55
for a year in advance. This machine processed signals by literally adding them together with steel and wire. In the 1930s, Vanavar Bush at MIT constructed the differential analyzer, a roomsized mechanical computer designed to solve differential equations that were too complex for manual calculus. The heart of the machine was the ball and disc integrator. A horizontal disc rotated at a speed representing an independent variable like time. A heavy steel ball rested on
32:31
the disc driven by friction. The distance of the ball from the center of the disc represented the variable to be integrated. As the disc spun, it turned the ball which in turn rotated a cylinder. The total rotation of the cylinder represented the integral of the function. The machine's power came from its ability to chain these integrators together. The output of one integration could be the input to another, allowing for the solution of second order and third order differential
33:04
equations. The engineering miracle that made this possible was the torque amplifier invented by Henry Neman. The torque drive of the ball in bisque was incredibly weak. It couldn't turn the heavy gears of the next stage without slipping. The torque amplifier used a capston principle similar to a ship's winch. The weak output shaft tightened a string wrapped around a powerful rotating drum. The friction of the string against the drum grabbed the output shaft and turned it
33:37
with massive force. The differential analyzer was the supercomputer of its day. It was used to calculate ballistics tables for the army and radar propagation curves for the MIT radiation lab. Yet, it was a temperamental beast. Programming it meant using wrenches and screwdrivers to rearrange gear trains. A calculation could take days to set up. Its precision was limited by the mechanical slippage of the balls and the stretching of the torque amplifier strings. Yet,
34:10
it demonstrated that computation was a physical process that could simulate the differential equations of the universe. Pre-digital digital security and synthesis. In 1943, the Allies deployed Sig Sally, a secure voice system connecting Washington, London, and later other command centers. It was the first system to enable secure conversations between Winston Churchill and Franklin Roosevelt. Sig Sally was an analog machine that functioned digitally. It
34:45
occupied a large room, weighed 55 tons, and used thousands of vacuum tubes. To secure the voice, Sigali first used a vocoder or voice coder. It did not transmit the voice waveform directly. Instead, it analyzed the voice into 10 frequency bands and a pitch signal, measuring the energy in each band every 20 milliseconds. This was a form of signal compression. Crucially, Sigelli then quantized these energy levels. It rounded the continuous voltage of each band to one of six discrete
35:24
steps. This was the first practical use of pulse code modulation or PCM. To encrypt the signal, the system added a one-time pad key to the quantized values. This key was a stream of random noise generated by the thermal fluctuations of a vacuum tube, a truly random physical source. The noise was recorded onto phonograph records. Identical records were flown to both ends of the link. The transmitter added the noise to the voice modulo 6. The receiver playing the synchronized
36:02
record subtracted the noise to recover the voice. The system was nicknamed the green hornet because of the buzzing control tones heard on the line. Sig Sally proved that a continuous analog signal voice could be broken down into parameters, quantized into digits, encrypted with math, and reconstructed synthetically. It was the grandfather of every secured digital phone and voiceover IP system in use today. While Sigali secured the content of the message, another
36:36
invention sought to secure the transmission path itself. In 1941, Hollywood actress Hedi Lamar and avantgard composer George Antel received a patent for a secret communication system intended for radiog guided torpedoes. The problem was jamming. A simple radio guidance signal on a single frequency could be easily blocked by the enemy broadcasting noise on that channel. Lamar's idea was frequency hopping. She proposed that the transmitter and the torpedo should rapidly switch
37:13
frequencies in a synchronized pseudo random pattern. To an enemy listener, the signal would appear as fleeting, unpredictable bursts of noise scattered across the spectrum, making it impossible to jam. To achieve synchronization, Antel suggested using the technology of the player piano. Miniature paper rolls perforated with the hopping code would turn inside the torpedo and the controlling ship. The mechanism would use 88 frequencies matching the 88 keys of a piano.
37:48
Although the US Navy rejected the idea during the war as too mechanically cumbersome, fearing the paper rolls would jam in a torpedo. The concept of spread spectrum was later resurrected in the 1950s. With the advent of transistors, frequency hopping became feasible. It became the basis for secure military communications and eventually the underlying technology for modern consumer wireless standards like Bluetooth and Wi-Fi. Lamar's intuitive leap trading bandwidth for robustness
38:24
was a fundamental insight of modern signal theory. Digital revolution and modern architectures 1948 to present. The history of technological progress in the post-war era is at its core a history of translation. It is the story of humanity learning to translate the continuous infinite and messy fluctuations of the physical world. The vibration of a vocal cord, the scattering of light photons,
38:55
the propagation of radio waves into the discrete, orderly and finite language of mathematics. This translation, the paradigm shift from analog to digital was not merely a change in storage formats or transmission cables. It was a fundamental restructuring of how civilization processes reality. At the heart of this transformation lies the discipline of digital signal processing DSP. It is a field that sits at the uncomfortable intersection of abstract mathematics,
39:30
semiconductor physics, and information theory. For decades, it remained an arcane subd disccipline of electrical engineering, hitting inside military radar bunkers and university mainframes. Yet, as the decades passed, DSP migrated from the laboratory to the living room and eventually to the pocket of nearly every human being on the planet. This second part traces that evolutionary arc, exploring the theoretical origins in the 1940s, the algorithmic breakthroughs of the 1960s,
40:04
the silicon explosion of the 1980s, and the contemporary rise of artificial intelligence as the new engine of signal processing. It argues that we have passed through three distinct epics. The era of hardwaredefined processing where functions were wired into circuits. The era of softwaredefined processing where generalpurpose chips executed human written algorithms. And the emerging era of dataded processing where neural networks learn to interpret signals in ways
40:38
their creators never explicitly programmed. The information age begins. The digital world did not begin with a microchip. It began with a paper. In 1948, the United States was transitioning from a war economy to a consumer boom, and the telecommunications network was expanding rapidly. However, the systems of the day were fundamentally analog. Long-d distanceance phone calls were plagued by noise. The farther a signal traveled, the more it degraded, accumulating static that
41:12
could never be fully removed. Engineers viewed information as a vague qualitative concept much like meaning or intent. Shannon, a mathematician and engineer working at Bell Labs, changed this forever with the publication of a mathematical theory of communication. Shannon realized that information was not abstract. It was a physical quantity that could be measured, quantified, and manipulated much like mass or energy. He termed this quantity entropy,
41:45
a measure of the uncertainty or surprise contained in a message. Shannon's work laid the theoretical foundation for digitization by decoupling the message from the medium. In an analog system, the signal is the message. If the voltage representing a voice fluctuates, the voice distorts. Shannon demonstrated that if a message could be encoded into symbols, it could be transmitted error-free over a noisy channel, provided the transmission rate did not
42:15
exceed the channel capacity. This was a radical departure from the engineering wisdom of the time, which held that reducing noise required increasing power. Shannon showed that one could use coding. While Shannon provided the architecture for digital communication, a fundamental practical problem remained. The world is analog. Sound waves, light intensity, and temperature voltages are continuous functions of time. To process them digitally, they must be sampled,
42:48
measured at discrete intervals. Common intuition in the 1940s suggested that this chopping process must inevitably destroy information. If you measure a soundwave at 1201 and again at 1202, what happens to the sound that occurred at 1201.5? Is it lost forever? The answer provided by the Nyquist Shannon sampling theorem was a mathematical miracle that underpinned the entire digital revolution. Building on earlier insights by Harry Nyquist in 1928, Shannon proved that a
43:27
continuous signal could be perfectly reconstructed from its discrete samples without any loss of information provided one specific condition was met. The sampling rate must be strictly greater than twice the highest frequency component of the signal. This threshold known as the Nyquist rate implies that the spaces between samples do not contain hidden information assuming the signal is band limited. For example, the human ear can generally hear frequencies up to 20,000 hertz. In
44:02
accordance with the theorem, if an audio signal is sampled at more than 40,000 times per second, the resulting stream of numbers captures everything a human can physically hear. The discrete points are sufficient to mathematically recreate the infinite continuous curve of the original soundwave. This theorem gave engineers the mathematical confidence to abandon continuous analog storage media like vinyl records and magnetic tape in favor of digital formats. However,
44:36
the theorem also introduced a dangerous artifact known as aliasing. If a signal is undersampled, measured slower than the Nyquist rate, the highfrequency information does not simply vanish. It disguises itself as lower frequency information. A classic visual analogy is the wagon wheel effect seen in western films. A movie camera captures images at 24 frames per second. If a wagon wheel spins at a rate that aligns with this sampling frequency, the spokes may appear to
45:11
stand still or even rotate backward. The camera is under sampling the motion, creating a false or aliased low frequency representation of the high-speed rotation. In audio, this manifests as high-pitched signals folding over into the audible range as dissonant metallic distortion. To prevent this, every digital system must employ strict anti-aliasing filters to brutally sever any frequencies above the Nyquist limit before digitization begins. FFT Revolution,
45:48
the algorithm that sped up the world. For nearly two decades after Shannon's paper, digital signal processing was largely a theoretical exercise. The mathematics worked, but the economics did not. Processing a signal required transforming it from the time domain, a sequence of samples, to the frequency domain, a spectrum of sine waves, a task performed using the discrete forier transform, DFT. The DFT was computationally punishing. To analyze a signal with n data points, the
46:24
algorithm required n squared complex operations. For a modest data set of 1,000 points, a computer had to perform 1 million operations. In the era of vacuum tubes and punch cards, this exponential scaling made realtime signal processing a fantasy. The geopolitical climate of the 1960s changed the equation. The Cold War was at its height and the United States and the Soviet Union were engaged
46:53
in a nuclear arms race. Following the partial test ban treaty of 1963, which banned atmospheric nuclear tests, the superpowers moved their testing underground. The only way to monitor compliance was to listen to the Earth itself. Seismographs and acoustic sensors generated rivers of data that needed to be analyzed for the spectral signatures of nuclear detonations, which differed distinctly
47:21
from natural earthquakes. The urgency of national security demanded a faster way to compute the forier transform. The solution arrived in 1965 not from a weapons laboratory but from a collaboration between James Culie of IBM and John Tucky of Princeton. Tucky a brilliant statistician who was also credited with coining the term bit suggested a mathematical shortcut to Kulie during a brainstorming session. Kulie implemented the idea on an IBM7094 mainframe. The resulting paper, an
47:58
algorithm for the machine calculation of complex forier series, described what is now known as the fastfor transform, FFT. The FFT was not merely an optimization. It was an algorithmic paradigm shift. It reduced the computational complexity from n squared to n log n. For small data sets, the difference was negligible. But for the large data sets required in seismic monitoring or radar,
48:28
the difference was astronomical. For a signal with 4,96 data points, a standard DFT requires roughly 16 million operations. The FFT requires only about 49,000. This is a speed up factor of over 300. As n grows, the advantage becomes even more extreme. This logarithmic collapse of processing time meant that tasks which previously took hours of mainframe time could now be completed in seconds.
49:00
The mechanism behind the FFT is a strategy of divide and conquer. The algorithm exploits the symmetry of the complex roots of unity, the sign and cosine values used in the transformation. It recursively breaks a large DFT into smaller DFTs. If the number of samples is even, the data set is split into two halves, one containing the even indexed samples and one containing the odd indexed
49:29
samples. These halves are processed separately and then recombined using a butterfly operation, a data flow diagram that resembles a pair of wings. This splitting continues until the problem is reduced to the simplest possible unit, a twopoint transform. By reusing calculations rather than recomputing them, the FFT eliminates vast amounts of redundant arithmetic. Historical scholarship later revealed a fascinating twist. The FFT was not actually new. The great mathematician Carl
50:04
Friedrich Gaus had derived the same algorithm in 1805 predating Forier's own work on harmonic analysis to calculate the orbits of palace and Juno. Gaus wrote his findings in Latin in a notebook that was not widely analyzed until the late 20th century. Thus, the algorithm that powers the digital age is actually an artifact of the Napoleonic era rediscovered to police the atomic age. The immediate beneficiary of the FFT was the military. Radar systems operate by transmitting
50:41
radio pulses and analyzing the reflections, echoes, from targets. To determine the speed of an incoming aircraft or missile, radar relies on the Doppler effect, the slight shift in frequency of the reflected wave caused by the target's motion. In the analog era, distinguishing a moving jet from a stationary mountain, clutter was difficult and imprecise. With the FFT, radar systems could digitize the returning pulses and instantly transform them into the frequency domain. In
51:17
this domain, stationary objects appeared at the center frequency, zero Doppler shift, while moving targets appeared as distinct spikes shifted away from the center. This processing known as pulse Doppler radar allowed for look down, shoot down capabilities where an aircraft could detect low-flying targets against the clutter of the ground. The FFT allowed this analysis to happen in real time, scanning thousands of range bins and velocity gates simultaneously.
51:52
Similarly, in sonar, the FFT allowed submarines to analyze the acoustic spectrum of the ocean with high precision. Every ship and submarine emits a unique acoustic signature, a fingerprint created by the rotation of its propeller, the hum of its generator, and the vibration of its hull. By applying the FFT to hydrophone data, sonar operators could decompose the cacophony of the ocean into discrete frequencies, identifying the specific 60 Hz hum of a specific class
52:28
of enemy submarine miles away. The FFT effectively gave eyes to the blind, turning sound into a visual map of the environment. Hardware evolution, the rise of the DSP chip. While the FFT provided the software road map, the hardware of the 1960s and 70s was illequipped to travel it. Generalpurpose microprocessors, CPUs, were designed for logic, control flow, and data movement, the tasks needed to run an operating system or a word processor. They were not
53:06
designed for the relentless arithmetic of signal processing. DSP algorithms are fundamentally different from general computing. They are characterized by a single repetitive operation, the dot product. Whether filtering a signal, computing an FFT, or correlating two sequences, the processor must multiply a data sample by a coefficient and add the result to a running total. This is known as the multiply accumulate operation. On a standard microprocessor of the
53:41
late 1970s, a multiplication was a slow multicycle ordeal implemented in microode. Doing millions of them per second for realtime audio was impossible. The semiconductor industry responded by creating a new class of microprocessor, the digital signal processor, DSP. The race began in earnest in 1979 when Bell Labs introduced the DSP1. This pioneering chip was not sold to the public, but was crucial for AT&T's telecommunications network. It was designed to handle the complex
54:21
filtering required for the number one electronic switching system, 1 ESS, and its successors. The DSP1 allowed the phone network to transition from electromechanical relays to digital switching, significantly increasing reliability and call capacity. However, the chip that truly defined the era was the TMS 3210 released by Texas Instruments in 1982. Often considered the model T of the DSP world, the TMS 3210 brought high-speed signal processing to the mass market. Its success
55:02
lay in its architecture. Standard computers use a vonoman architecture where program instructions and data share the same memory and the same bus. This creates a bottleneck. the CPU cannot fetch the next instruction and the data it needs at the same time. TI engineers utilized a modified Harvard architecture for the TMS 3210. This design separates program memory from data memory providing each with its own bus. This allowed the chip to fetch an instruction eg multiply and the
55:40
data the numbers to be multiplied in a single clock cycle. Furthermore, the chip included a dedicated hardware multiplier, a massive circuit on the silicon die committed solely to math. This allowed the TMS 3210 to perform a multiply accumulate operation in just 200 nanose. This speed was revolutionary, enabling applications that were previously the domain of supercomputers to be performed by a $500 chip. The success of the TMS32 series triggered an arms race in the
56:17
semiconductor industry. Competitors like Motorola with the DSP 56000 series and analog devices with the ADSP 2100 and Shark families entered the fray. A major divide emerged between fixed point and floatingpoint processors. Fixed point DSPs, like the early TI chips, represent numbers as integers. They are fast, cheap, and power efficient, but they suffer from limited dynamic range. If a calculation results in a number larger than the register can hold, it overflows, causing
56:54
catastrophic distortion. Programmers had to painstakingly scale their numbers to avoid this. Floating point DSPs like analog devices shark chips represent numbers in scientific notation automatically handling the decimal point. This offers vast dynamic range and easier programming but requires more transistors costing more and consuming more power. The industry bifurcated. Fixed point chips went into cost-sensitive consumer devices like toys and mobile phones,
57:28
while floatingpoint chips dominated high-end professional audio, medical imaging, and military radar. The availability of cheap, high-speed DSP chips in the 1980s and '90s catalyzed a revolution in consumer electronics. The abstract theories of Shannon and the algorithms of Kulie and Tuki were finally etched into silicon. The compact disc CD launched in 1982 was the first digital product to reach the mass consumer. It was a joint development by Sony and Philips,
58:04
but the two giants had a technical disagreement. Sony with its advanced semiconductor capabilities had a 16bit digital to analog converter ready for production. Philips however only had a 14bit DAC, the TDA 1540. In the binary world, two bits is a massive difference. 16 bits offers four times the resolution of 14 bits. It seemed Philips would launch an inferior player. Philips engineers solved this hardware problem with DSP. They utilized a technique
58:41
called oversampling. Instead of feeding the 14bit DAC at the standard 44.1 kHz rate, they programmed a digital filter chip the SAA7030 to feed the DAC 4 times faster, 176.4 4 kHz. By distributing the quantization noise over a wider frequency bandwidth and then filtering out the ultrasonic noise, they achieved a signal to noise ratio in the audible band that was equivalent to a true 16bit system. This was a pivotal moment. It proved that smart signal processing could
59:20
compensate for inferior analog hardware. The Philips CD 100, despite its weaker DAC, was lauded for its sound quality, often described as warmer than the Sony counterparts, which used brick wall analog filters. Digital media is fragile. A scratch on a vinyl record causes a pop. A scratch on a CD destroys megabytes of data. To make the CD viable, engineers implemented cross interled read
59:51
Solomon coding circ. This is a heavy DSP workload. The data on a CD is not written sequentially. It is scattered interleved across the disk surface. When the laser reads the disk, the DSP deinterles the data and uses redundant parody bits to mathematically reconstruct any missing information. A CD player essentially solves a massive system of linear equations in real time as you listen to music. Without the DSP power to run Read Solomon algorithms, the optical disc
1:00:27
format would have been practically unusable due to its susceptibility to minor physical defects. The second great triumph of the DSP chip was the GSM mobile phone standard. In the analog cellular era, 1G, each user occupied a dedicated frequency channel. This severely limited the number of users a cell tower could support. Digital 2G GSM needed to fit more users into the same bandwidth. The
1:00:56
solution was compression. The human voice produces a lot of redundant data. The GSM standard utilized a DSP algorithm called linear predictive coding, LPC. Instead of transmitting the sound of the voice directly, LPC analyzes the voice to estimate the shape of the speaker's vocal tract, throat, and mouth, and the excitation signal, air from the lungs. The phone transmits these parameters rather
1:01:25
than the audio itself. The receiving phone's DSP then uses these parameters to synthesize a synthetic voice that sounds like the speaker. This reduced the data rate from 64 kilobits per second standard telephone quality to just 13 kilobits per second, allowing network operators to pack five times as many users onto the network. The DSP chip inside the Nokia and Motorola bricks of the 1990s was the engine of the mobile revolution. In the late 1990s, the internet arrived,
1:02:00
but bandwidth was scarce. Transmitting raw CD audio, 1.4 megabits per second, was impossible. The Moving Picture Experts Group, MPEG, developed the MP3 standard to solve this. Unlike zip compression which is lossless, MP3 is lossy. It throws information away, but it does so intelligently using a psycho acoustic model. The DSP running an MP3 encoder analyzes the audio using a modified discrete cosine transform MDCT. It looks for masking events. For example,
1:02:40
if a loud snare drum hit occurs, the human ear is temporarily deaf to softer sounds immediately following it. Temporal masking. Similarly, a loud tone at 1,000 hertz renders the ear insensitive to a quiet tone at 1,0 simultaneous masking. The DSP identifies these invisible sounds and deletes them from the data stream. It allocates bits only to the sounds the brain actually perceives. This
1:03:11
allowed MP3 to reduce file sizes by 90% while maintaining acceptable quality. The explosion of portable MP3 players and later the iPod was driven by the availability of low power DSPs capable of decoding these complex mathematical streams. Software and AI integration. the shift to defined radio and neural processing. As the 20th century dawned, the rigid coupling between hardware and function began to dissolve. In the 20th century, a radio was a collection of specific resistors,
1:03:49
capacitors, and inductors soldered together to resonate at a specific frequency. To change the radio from AM to FM or to a new military frequency, you had to physically rebuild the circuit. The new millennium brought the concept of virtualizing the hardware itself. The concept of softwaredefined radio or SDR proposes a radical simplification. Digitize the signal as close to the antenna as possible and do everything else in software. In an ideal SDR, the hardware is merely
1:04:25
a generic dumb pipe that captures the entire radio spectrum. And the tuner, filter, and de modulator are just lines of code running on a processor. The military, burdened by a logistical nightmare of incompatible radios for different branches and allies, pioneered this technology. The seinal project was speak easy, initiated by the US military in the early 1990s. The goal was to build a single radio unit that could emulate 10 different existing military waveforms from HF to
1:04:59
VHF to satellite links simply by loading different software modules. The processing requirements were immense. The Speak Easy Phase 1 demonstrator used banks of the fastest TI DSPS available, the TMS 320C40, and occupied a rack large enough to fill the back of a cargo truck. It was not portable, but it proved the point. The radio had become code. As bandwidths increased from kilohertz voice to megahertz video data, standard DSP chips could not keep up, the industry turned to field
1:05:37
programmable gate arrays or FPGAAS. An FPGA is a chameleon chip. Unlike a standard processor with a fixed instruction set, an FPGA consists of a sea of uncommitted logic gates that can be rewired electronically. An engineer can write code in languages like VHDL or VeraLogue that reconfigures the chip's internal circuitry to become a custom parallel processor. In modern 5G base stations, FPGAAS are the workh horses. They handle the massive parallel streams of data from myo antennas
1:06:16
performing billions of operations per second with a latency measured in micros secondsonds. They offer the speed of custom hardware with the flexibility of software updates. Simultaneously, the graphics processing unit GPU created for video games evolved into a scientific instrument. Nvidia's release of CUDA compute unified architecture in 2007 allowed engineers to access the thousands of parallel cores in a graphics card for non-graphical tasks. In fields
1:06:50
like synthetic aperture radar or S where an image is constructed from millions of radar pulses, GPUs proved superior to both DSPs and FPGAAS. They offered massive floatingpoint throughput and were easier to program than FPGAAS. Today, the signal processing backend of many advanced radar and sonar systems is essentially a gaming supercomput. We are currently witnessing the third and perhaps most disruptive phase of this history. We have moved from hardwaredefined analog
1:07:25
to softwaredefined algorithms written by humans to datad algorithms learned by machines. For decades, signal processing relied on hand-crafted features. In image processing, engineers spent years designing mathematical filters to detect edges, so bell filters, or identifying key points, SIFT and HOG algorithms to recognize objects. These algorithms were robust but rigid. They could only see what the engineer told them to look for. In 2012, a convolutional neural network named
1:08:03
AlexNet shattered this paradigm by winning the imageet competition. Alexet did not use hand-crafted filters. It learned its own filters by looking at millions of images. It discovered that edges are important in the first layer, shapes in the second, and textures in the third. The engineer was no longer designing the signal processing pipeline. The engineer was designing the architecture that learned the pipeline. This revolution has spread to audio and wireless
1:08:35
communications. In speech synthesis, traditionally systems relied on vocoders, parametric models of the human voice derived from the LPC research of the 1960s. These systems sounded intelligible but robotic because the mathematical model was a simplified approximation of reality. In 2016, DeepMind introduced WaveNet, a generative neural network that synthesizes audio sample by sample. WaveNet does not know what a vocal tract is. It does not know about sine waves or formants.
1:09:14
It uses a dilated causal convolution architecture to look at thousands of previous audio samples and predict the probability of the next sample. It learns the physics of air pressure waves directly from raw audio data. The result is a voice that captures the breaths, the lip smacks, and the subtle intonations of a human speaker. Nuances that no human engineer could ever write an equation for. While the original WaveNet was too slow for realtime use,
1:09:48
successor architectures like parallel wave GAN have optimized this process, allowing your smartphone assistant to speak with a neural voice generated in real time. In the wireless world, researchers are now proposing to replace the fundamental mathematical blocks of communication, the channel estimator and the equalizer with neural networks. The deep RX concept uses deep learning to interpret the distorted radio signals received by an antenna. In traditional
1:10:21
engineering, we model the channel mathematically, example fading, and design an inverter. A neural network, however, can handle the nonlinearities and chaos of realworld interference better than linear mathematical models. Some research suggests that 6G networks might use autoenccoders where the transmitter and receiver are two halves of a neural network that jointly learn a modulation scheme perfectly optimized for the specific environment they are in. Inventing a
1:10:56
new language of radio waves on the fly that no human designed. The journey from 1948 to the present is a narrative of increasing abstraction. Shannon gave us the bit detaching information from the physical world. Kulie and Tuki gave us the FFT detaching processing speed from the linear flow of time. The DSP chip designers gave us the hardware to decouple function from fixed circuitry. And now the pioneers of deep learning are detaching the algorithm from the human programmer. We
1:11:35
stand at a unique threshold. For 70 years, signal processing was a deductive science. We started with first principles, physics, math and deduced the solution. We are now entering an inductive era. We start with the data and induce the solution. The anti-aliasing filters, the forier transforms, and the error correcting codes that built the modern world are not disappearing, but they are being subsumed into a larger, more opaque, and infinitely more capable intelligence.
1:12:10
The digital revolution digitized the signal. The neural revolution is digitizing the intuition.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat