So the best way to begin explaining wavelets is through analogy to music. (I’m cheating a bit since this explanation is alluded to in the article 😆)
It is a nontrivial practical fact that you can express any reasonable sound as a sum of sine waves. Yes, by combining enough sine waves (which individually “move” for all time) in just the right weights, you can come up with “any” sound you want (edit: including sounds that “start” and “end”. Isn’t that wild?). And then, it turns out that if you give me just the weights, I can give you back the sound itself. And as a final physical fact, it turns out that we hear the weights of any given sound, averaged over some finite window of time (more on this window in a minute). Hence why we can pick out instruments from a band. And lastly, some phenomenon are easier to analyze by looking at the weights; music is an excellent example. In fact, when I mix music in my rapidly diminishing free time, I am often staring at a graph of the weights and seeing this these weights add together and make the instruments work together.
Formally, we use one of the Fourier transform frameworks. Each weight is associated to one unique sine with a given frequency. The size of the weight is called the frequency response at that frequency.
Now for many, many purposes, breaking up a signal in terms of sines is a perfectly appropriate choice. However, what you lose when you choose to look at just the weights is all timing information. (This is why I included the detail about the window in how you hear stuff. If you heard all frequencies over all time with no window, you would not be able to perceive rhythm.) The solution in music often is to simply impose a window on the signal and slide it as the play head moves.
However, we must now leave the realm of music to talk about wavelets in a domain where they are typically used. Now imagine you want to apply all your intuition about music [more accurately, theory of sound, not music theory] to seismic signals. Well… unfortunately, we really do care about the timing of these signals. So instead of ditching all the magical techniques of linear algebra and transform analysis, we can pick a new set of waves and decompose in terms of those. I.e., we use a transform “midway” between the Fourier transform and the identity transform (doing nothing, just working with the raw signal).
One way to do this is to start with a wavelet: any waveform with zero average and finite “length”. Then, you take this mother wavelet, and you create child wavelets by stretching and/or shifting the mother wavelet. Then, you break up your signals in terms of the wavelets. (I think you pick wavelets based on what you want to find. For example, if you want to find sharp changes, you can pick a Haar wavelet, which is basically a family of rectangles. And then, you can pick wavelets based on their statistics so that the variances and higher order statistics vanish.)
My favorite book on Wavelets, and one of my personal favorite books, is A Wavelet Tour of Signal Processing: The Sparse Way by Mallat. It’s a bit mathematically challenging, but it’s such a fun read. One of the few books I actually own in print. And it’s one those cool fields in math where you basically just start with like pure math and end up with some incredibly practical results and algorithms.
Research:
My background is in control theory. I work on analyzing dynamical systems, specifically large-scale, complicated (typically people use the word “complex”, but I really mean complicated, because all the systems I work on evolve in real spaces) systems that evolve in time according to differential equations (e.g. electronic circuits, mechanical systems, power systems) or difference equations (e.g. sampled versions of the above). The goal of my research is to make just enough assumptions and prove it using calculus so future generations don’t have to do so much calculus…because you have to do so much calculus that not even a supercomputer can solve it.
PM me for more details since I’m not quite ready to dox myself 😆
If I’m understanding your comment correctly, wavelets are a kind of discrete and/or finite quantization of the “full” infinite Fourier transform, by way of using more complex “basis vectors” than pure sine waves?
Very cool and thanks for the reading recommendation!
If I’m understanding your comment correctly, wavelets are a kind of discrete and/or finite quantization of the “full” infinite Fourier transform, by way of using more complex “basis vectors” than pure sine waves?
The second part is basically correct, but the first part needs a little bit of explanation.
So depending on what you need to do, you can actually use continuous-time (or continuous-space) wavelet transforms, or discrete-time (or discrete-space) wavelet transforms. The continuous-time wavelet transform is, practically, “just as exact” as the continuous-time Fourier transform. So instead of being “better” or “lesser” than the Fourier transform, it’s really a “different perspective” on the same space of signals by choosing a different set of “basis” vectors [1].
Also, wavelets often are “more complicated” than sine waves, but not necessarily. In fact, one of the first wavelets discovered was this, the Haar wavelet:
To be completely clear: this waveform is defined for any real number; it’s not sampled, and it is not a quantized version of some “better” wave! It is just 1 for any inputs between 0 and 1/2, -1 for any inputs between 1/2 and 1, and 0 everywhere else [2]. Although this wavelet happens to have a finite range (so not even countably infinite, you get {-1,0,1} and you don’t get upset), if you slap enough of these things together (possibly infinity of them), you can get back any “reasonable” waveform, where “reasonable” is precisely defined in Mallat’s book (it’s L2(R) if you’ve been exposed to Lp spaces).
Hope you enjoy the book as much as I have!
[1] “Basis” is in quotes because it depends on what you mean by “basis”. Typically, a “basis” in linear algebra means that you need to be able to exactly recover any element in the vector space with a weighted finite sum of the bases — a Hamel basis. In signal processing, in particular in Mallat’s book, we typically extend the notion of basis to allow for infinite linear combinations with limits. This means that we need to choose a topology, which is absolutely a reasonable requirement in signal processing, but not necessarily in “pure” linear algebra. I believe that the definition for “(orthonormal) basis” in Mallat’s book (in Appendix A) is called a(n orthonormal) Schauder basis in other parts of applied math.
By contrast, “vectors” is not in quotes above because, using the “abstract” definition of a vector space, the basis elements are indeed vectors, i.e. members of a space that you “cannot leave” by scaling or adding finite numbers of the basis elements.
Lastly, “signals” and “vectors” are mostly interchangeable within the signal processing discipline. In signal processing, we typically assume that signals have been given an inner product (“correlation”, “dot product”), therefore a norm (“length”) from the inner product, and therefore a topology (“abstract geometry”) from the norm. I.e., “signals” in signal processing usually have more structure than “vectors” in applied math. Waveform is not a mathematically precise term; here I just mean the plot of a signal.
[2] The values of the Haar wavelet at exactly {0,1/2,1} are indicate by the filled-in blue circles in the plot. However, since continuous wavelet transforms are integral transforms, the values at the points {0,1/2,1} can be changed to whatever you want as long as it’s finite. Rigorously, the Lebesgue integrals in the wavelet transform definitions are “blind to” a “small” (measure zero) set like {0,1/2,1}. From a signal processing perspective, changing the signal only on “small” sets like {0,1/2,1} is not enough to change the signal energy.
I think there is an “external” reason why the choice of values at {0,1/2,1} given in the plot makes sense. Mathematically, the choice makes the Haar mother wavelet right-continuous and upper semi-continuous, but I can’t remember off the top of my head why this is helpful for applications.
Okay here are the 🫘:
Wavelets:
So the best way to begin explaining wavelets is through analogy to music. (I’m cheating a bit since this explanation is alluded to in the article 😆)
It is a nontrivial practical fact that you can express any reasonable sound as a sum of sine waves. Yes, by combining enough sine waves (which individually “move” for all time) in just the right weights, you can come up with “any” sound you want (edit: including sounds that “start” and “end”. Isn’t that wild?). And then, it turns out that if you give me just the weights, I can give you back the sound itself. And as a final physical fact, it turns out that we hear the weights of any given sound, averaged over some finite window of time (more on this window in a minute). Hence why we can pick out instruments from a band. And lastly, some phenomenon are easier to analyze by looking at the weights; music is an excellent example. In fact, when I mix music in my rapidly diminishing free time, I am often staring at a graph of the weights and seeing this these weights add together and make the instruments work together.
Formally, we use one of the Fourier transform frameworks. Each weight is associated to one unique sine with a given frequency. The size of the weight is called the frequency response at that frequency.
Now for many, many purposes, breaking up a signal in terms of sines is a perfectly appropriate choice. However, what you lose when you choose to look at just the weights is all timing information. (This is why I included the detail about the window in how you hear stuff. If you heard all frequencies over all time with no window, you would not be able to perceive rhythm.) The solution in music often is to simply impose a window on the signal and slide it as the play head moves.
However, we must now leave the realm of music to talk about wavelets in a domain where they are typically used. Now imagine you want to apply all your intuition about music [more accurately, theory of sound, not music theory] to seismic signals. Well… unfortunately, we really do care about the timing of these signals. So instead of ditching all the magical techniques of linear algebra and transform analysis, we can pick a new set of waves and decompose in terms of those. I.e., we use a transform “midway” between the Fourier transform and the identity transform (doing nothing, just working with the raw signal).
One way to do this is to start with a wavelet: any waveform with zero average and finite “length”. Then, you take this mother wavelet, and you create child wavelets by stretching and/or shifting the mother wavelet. Then, you break up your signals in terms of the wavelets. (I think you pick wavelets based on what you want to find. For example, if you want to find sharp changes, you can pick a Haar wavelet, which is basically a family of rectangles. And then, you can pick wavelets based on their statistics so that the variances and higher order statistics vanish.)
My favorite book on Wavelets, and one of my personal favorite books, is A Wavelet Tour of Signal Processing: The Sparse Way by Mallat. It’s a bit mathematically challenging, but it’s such a fun read. One of the few books I actually own in print. And it’s one those cool fields in math where you basically just start with like pure math and end up with some incredibly practical results and algorithms.
Research:
My background is in control theory. I work on analyzing dynamical systems, specifically large-scale, complicated (typically people use the word “complex”, but I really mean complicated, because all the systems I work on evolve in real spaces) systems that evolve in time according to differential equations (e.g. electronic circuits, mechanical systems, power systems) or difference equations (e.g. sampled versions of the above). The goal of my research is to make just enough assumptions and prove it using calculus so future generations don’t have to do so much calculus…because you have to do so much calculus that not even a supercomputer can solve it.
PM me for more details since I’m not quite ready to dox myself 😆
If I’m understanding your comment correctly, wavelets are a kind of discrete and/or finite quantization of the “full” infinite Fourier transform, by way of using more complex “basis vectors” than pure sine waves?
Very cool and thanks for the reading recommendation!
The second part is basically correct, but the first part needs a little bit of explanation.
So depending on what you need to do, you can actually use continuous-time (or continuous-space) wavelet transforms, or discrete-time (or discrete-space) wavelet transforms. The continuous-time wavelet transform is, practically, “just as exact” as the continuous-time Fourier transform. So instead of being “better” or “lesser” than the Fourier transform, it’s really a “different perspective” on the same space of signals by choosing a different set of “basis” vectors [1].
Also, wavelets often are “more complicated” than sine waves, but not necessarily. In fact, one of the first wavelets discovered was this, the Haar wavelet:
To be completely clear: this waveform is defined for any real number; it’s not sampled, and it is not a quantized version of some “better” wave! It is just 1 for any inputs between 0 and 1/2, -1 for any inputs between 1/2 and 1, and 0 everywhere else [2]. Although this wavelet happens to have a finite range (so not even countably infinite, you get {-1,0,1} and you don’t get upset), if you slap enough of these things together (possibly infinity of them), you can get back any “reasonable” waveform, where “reasonable” is precisely defined in Mallat’s book (it’s L2(R) if you’ve been exposed to Lp spaces).
Hope you enjoy the book as much as I have!
[1] “Basis” is in quotes because it depends on what you mean by “basis”. Typically, a “basis” in linear algebra means that you need to be able to exactly recover any element in the vector space with a weighted finite sum of the bases — a Hamel basis. In signal processing, in particular in Mallat’s book, we typically extend the notion of basis to allow for infinite linear combinations with limits. This means that we need to choose a topology, which is absolutely a reasonable requirement in signal processing, but not necessarily in “pure” linear algebra. I believe that the definition for “(orthonormal) basis” in Mallat’s book (in Appendix A) is called a(n orthonormal) Schauder basis in other parts of applied math.
By contrast, “vectors” is not in quotes above because, using the “abstract” definition of a vector space, the basis elements are indeed vectors, i.e. members of a space that you “cannot leave” by scaling or adding finite numbers of the basis elements.
Lastly, “signals” and “vectors” are mostly interchangeable within the signal processing discipline. In signal processing, we typically assume that signals have been given an inner product (“correlation”, “dot product”), therefore a norm (“length”) from the inner product, and therefore a topology (“abstract geometry”) from the norm. I.e., “signals” in signal processing usually have more structure than “vectors” in applied math. Waveform is not a mathematically precise term; here I just mean the plot of a signal.
[2] The values of the Haar wavelet at exactly {0,1/2,1} are indicate by the filled-in blue circles in the plot. However, since continuous wavelet transforms are integral transforms, the values at the points {0,1/2,1} can be changed to whatever you want as long as it’s finite. Rigorously, the Lebesgue integrals in the wavelet transform definitions are “blind to” a “small” (measure zero) set like {0,1/2,1}. From a signal processing perspective, changing the signal only on “small” sets like {0,1/2,1} is not enough to change the signal energy.
I think there is an “external” reason why the choice of values at {0,1/2,1} given in the plot makes sense. Mathematically, the choice makes the Haar mother wavelet right-continuous and upper semi-continuous, but I can’t remember off the top of my head why this is helpful for applications.
Another book added to the list… I may never get to the Silmarillion. War and Peace certain is never happening.
Good luck with your research. May you find more answers than questions.