Linear change

"The shortest distance between two points is a straight line." That mathematical truism recalled from our geometry class is often quoted as a sort of proverb, reminding us that the most direct way to get somewhere is usually to head straight for it. The straight line stands for directness, and we can easily perceive its direction and predict where it will arrive if it continues. Thus, straight lines are useful for describing gradual-yet-direct change. If we draw a straight line from point A to point B, we're getting there by the most direct route, and we're also drawing every intermediate point that lies directly between A and B.

In the previous lesson, we provided the formula y=mx+b as a general way to describe any line. That's how it's given in arithmetic textbooks, but we'll need to rephrase it a bit for our purposes. For starters, how do we calculate the slope m? Well, for any two points--we'll call them A and B--we can calculate the slope of the line that runs through them (and thus the line segment that connects them) by dividing the vertical distance between them by the horizontal distance between them; that is to say by dividing the difference in their y values by the difference in their x values. So if the coordinates of point A are (xa, ya) and the coordinates of point B are (xb, yb), then the slope m is equal to (yb-ya)/(xb-xa). To refer to the last example from the previous lesson, the first point on the line segment is (0,36) and the last point is (100,96). so the slope of the line is (96-36)/(100-0), which is 60/100, which is 0.6. Thus, by knowing the slope m (0.6) and the offset b (36), we can calculate what y value will lie on the line for any x value we put into the formula.


In practical terms, for the purpose of programming linear change in sound, music, video, animation, etc., we'll need to know those values, or at least we'll need to be able to calculate them. (In the above example, we were able to calculate the slope because we knew the starting and ending x and y values.) Then, by starting at the desired value for x (a starting point in time) and proceeding to a desired destination value for x (a future point in time), we can calculate the values for y for as many intermediate x points as we want, to give the impression of a linear change in y. Before we look at an example, let's consider two other terms that are commonly used in digital media arts, which have direct relevance to this definition of a line.

-----

The term linear interpolation between two points A and B means finding an appropriate intermediate point (or points) that would exist if there were a straight line between A and B. To continue with the example we've been using, if we have the two points (0,36) and (100, 96), we can interpolate one or more points between them by calculating the y value at a hypothetical x value between 0 and 100. For example, just by using the formula and the known values for slope and offset, we can calculate that when x equals 20 y will equal 48, and when x equals 80 y will equal 84.
(0.6)20+36=48
(0.6)80+36=84
So, for any point between A and B, we can interpolate one or more additional points that will lie on a straight line segment between them. Another way to say this is that for any intermediate x value, we can find the appropriate corresponding y value.

Tangent: There's another way to think of linear interpolation, which is to think of "how far along" the intermediate x is, on a path from point A to point B. In other words, for the hypothetical x value, how far is it from its starting point xa, and how far is it from its destination point xb? We can actually calculate it as a fraction between 0 and 1 by calculating its distance from xa, which will be x-xa, and dividing that by the total distance between xa and xb, which will be xb-xa; so the fraction of the distance that a hypothetical x is on the path from xa to xb can be calculated with the expression (x-xa)/(xb-xa). So another way to think about calculating the y value for a hypothetical x value, is to multiply the destination y (yb) by that fraction because we're that fraction of the way there, and multiply the starting y (ya) by 1 minus that fraction. The equation for finding the y value that corresponds with a hypothetical x value is thus y=(yb(x-xa)/(xb-xa))+(ya(1-((x-xa)/(xb-xa))). Or, to put it a bit more simply, y = ((x-xa)(yb-ya))/(xb-xa)+yb. That's a valid formula for linear interpolation, or linear mapping of x to y.

This process of linear interpolation is thus very closely related to another term, linear mapping, which means making a direct correlation of an x value to its corresponding y value. If we have a given range of x values (say, from xa to xb), and a corresponding range of y values (say, from ya to yb), then for any value of x we can calculate the linearly corresponding y value. This is called mapping x values to y values. (In theory, there could be a wide variety of curved or non-linear "maps" by which we make these correlations between x values and y values, however, for now we'll stick to linear mapping.) The formula for linear mapping, if you needed to program it yourself, is shown in the preceding paragraph. Mercifully, Max provides many objects that can calculate mapping for you, such as zmap, scale, etc.), but it's important to understand mapping conceptually since it is key to all sorts of programming in digital media.

-----

Now let's look at a simple example of linear change. This program plays musical notes every 80 milliseconds (i.e., at the rate of 12.5 notes per second) for 2 seconds (2000 milliseconds). Over the course of those two seconds, the pitch of the notes changes linearly from MIDI 36 (low C) to MIDI 96 (high C) and the MIDI velocity (loudness) of the notes changes linearly from 124 (fortissimo) to 32 (piano). The result is a program that plays an ascending pentatonic scale, with a diminuendo.

Note that the program plays a major pentatonic scale, which is not a strictly linear configuration. (Some steps are 2 semitones, and some are 3 semitones.) Because we're stepping through five octaves of pitch in 25 increments, each step should be 1/5 of an octave. However, because MIDI does not allow for fractional values, the decimal part of the intermediary pitch values is truncated (chopped off), so ultimately not all the steps are exactly the same size. The fortuitous result of this truncation is the pattern that gives a major pentatonic scale. There's no magical mathematical relationship between truncation and this particular pitch pattern. It's just a fortunate (calculated) coincidence of the range, the number of notes being played within it, MIDI representation of pitch, and truncation of the fractional part of the number.

Let's look at a couple of details in the program. The clocker object reports at regular intervals the amount of time elapsed since it was turned on. This is similar to the timed counting demonstrated with the metro and counter objects, but in this case it is counting in increments of a certain number of milliseconds, corresponding exactly to the amount of time that has passed. This is handy because it allows us to check each report to see if a certain amount of time has passed--in this case 2000 milliseconds--and do something at that time (in this case, stop the process). Since we know the desired stopping time (that is to say, the destination x value) we can also use each reported time to calcuate how far along we are to the destination time, and use that fraction for linear mapping of time to the pitch and velocity values.

We can divide the elapsed time (x) by the total time (2000) to get a fraction between 0 and 1. We then multiply that by the range of the desired y values (yb-ya), and add the desired offset (ya) to it. (We calculated the range of desired y values by subtracting the starting value from the ending value, i.e., yb-ya.) That's what's happening in the two expr objects. In the case of pitch, ya equals 36 and yb equals 96 so the range is 96-36, which is 60. In the case of velocity, ya equals 124 and yb equals 32 so the range is 32-124, which is -92. That's where the range values 60 and -92 come from in the expr objects. The numbers 36 and 124 in the expr objects are the ya (offset) values. The number boxes (because they are integer number boxes) drop the fractional part of the output from the exprs.

This direct use of the y=mx+b formula inside an expr object is just one way to do linear mapping in Max. The line object does automated linear interpolation, kind of like this combination of clocker and expr shown here, and other objects such as zmap and scale calculate mapping of x and y values if you provide the points A and B.

Linear motion, linear change, linear interpolation, and linear mapping are frequently used in composition and digital media.

That's Why They Call Them Digital Media

The foregoing examples demonstrate that musical notes, colors, locations--and indeed anything else--can be described numerically. That's the essence of computational creativity, and the essence of digital intermedia/multimedia. Many of these lessons will be about exploring that way of systematically--and numerically--describing aesthetics and composition.

So far we've used numbers to describe and compare amounts of time and rates of speed, to count and enumerate, to index orders of events, and to indicate musical pitch, loudness, intensity of color, and position in a two-dimensional coordinate system. Composition and aesthetic appreciation are about the creation and recognition of patterns--sonic, temporal, visual, and rhetorical. Thus, to the extent that we can program a computer to generate interesting numerical patterns, we are on the path to computer-generated art.

Mathematical functions with two variables (such as x and y) can be graphed in two dimensions as a curve. If we consider one of the variables, x, to be the linear progression of time, then the shape of the curve displays the change in the other variable, y, over time. Almost any function with two variables is potentially useful for algorithmic composition of time-based media art (music, video, animation, etc.), if we just consider one variable to be the passage of time and the other variable to describe some characteristic of the art.

Let's look at some examples. Remember x will stand for the passage of time, and y will stand for the value of something else. Time is expressed in some arbitrary units; you can think of the units as seconds for now. Likewise y is some arbitrary thing we're measuring; for the sake of having both a sonic and a visual example, let's think of y as standing for the volume (loudness) of a tone or the brightness of a color.

What does this formula tell us?
y = 10
Well, there's no x in that formula. That means that y is independent of x. No matter what x is, the value of y will always be 10. This describes a constant value, not a variable one. y stays constant over time. If this were loudness or brightness, we would perceive no change.

Now how about this formula?
y = x
This means that y increases as time progresses. Since time progresses linearly, y will increase linearly. If we call the start of the time under consideration "time 0", meaning x = 0, then y will be 0 at that time. At time 10, y will be 10, and so on. So over the course of time going from 0 to 10, y will increase linearly from 0 to 10. The units we're using are arbitrary, but imagine that over the course of 10 seconds the volume of a sound fades in from 0 to 10, and a color fades from total darkness to a brightness of 10. As soon as we get to real examples, we'll have to concern ourselves with what the units actually are and what they mean, but for now we're just talking abstractly.

Here's a formula that gives a simple curve.
y = sin(2πx)
This causes y to vary up and down in a smooth, sinusoidal fashion. Of course, all of these formulae are rather oversimplified; we'd need to provide more numerical information to make x and y have the right values for the particular usage to which we're applying them. But these simple examples show that we can generate curves by tracking any two variables, and that by increasing x linearly to stand for the passage of time we can use the resulting y to control some aspect of digital media


We can add other constant numbers or variables into the formula to make adjustments that bring the y value into a desired range. Those additional constants or variables will usually do one of two things: either scale the x or y value by multiplication (multiply it by some amount to scale it to a larger or smaller range) or offset the x or y value by addition (add some amount to it to push it into a different region).

For example, a more complete and useful formula for generating a smooth sinusoidal change would look like this.
y = Asin(2πƒt+ø)+d
where A is the amplitude of the sinusoid, ƒ is its frequency, t is time, ø is its phase offset, and d is its DC offset. The variable t in this case is like x in the previous examples. ƒ scales the sinusoid on the horizontal axis, and A scales it on the vertical axis; ø allows us to offset the sinusoid on the horizontal axis, and d will offset it on the vertical axis. We'll use some version of this formula often in synthesizing and processing sound in future lessons.

Here's a simpler equation, that permits us to make any desired linear variation in y.
y = mx+b
where m is the slope (steepness of the angle) of the change in y, and b is the y intercept (the y value at which the line intersects the y axis), which we can call the (vertical) offset. Here's what that would look like with m = 0.6 and b = 36.

Since we will be starting things at time 0, and will be mostly dealing with non-negative y values for MIDI, color, coordinates, etc., it sometimes makes sense just to think about the upper-right quadrant of the graph, showing y values as x (time) increases from 0.

In the next lesson we'll put the y=mx+b equation into practice in a program.

Analysis of Some Patterns

Let's examine some of the decisions that were made in composing the content for the example of counting through a list. The contents of the table and coll objects determine what pattern of notes, colors, or positions we perceive when we read through them.

The process of counting through a numbered array is independent of the contents of the array. The array could be an array of anything, and we could re-order or change the contents of the array and still employ the same process to read through it. So the counting process is necessary, but it's fairly uninteresting on its own. (At least, it's fairly uninteresting until we give it a more interesting rhythm than just reading at a constant rate.) The contents of the array are what's more important in determining the aesthetic of what is produced.

The program displays a sequence of 16 colors, 16 musical notes, and 16 positions for a black dot. Let's look at the colors first.

The color of the background of the lcd object is described by three numbers from 0 to 255 that state the value of Red, Green, and Blue in the mix. This is called RGB, and is one common way of describing colors digitally. The color values in the coll object were chosen to progress more-or-less directly through the color spectrum from red to green to blue and back toward red. Since there are 16 steps in the array, this naturally entails a few intermediate colors along the way. However, sixteen steps is way too few to give an impression of smooth gradual change; rather, we get a pattern of distinct colors that traverse the spectrum. And, if the program's looping feature is turned on, the pattern of colors seems to return fairly naturally to where it started.


The pattern of sixteen musical pitches is a little less direct, and a bit more complex. The first eight notes are an upward arpeggiation of the pitches C,C,G,D,Eb,F#,A,Bb, which could be thought of as a Cm13#11 chord. The next four notes are an upward arpeggiation of the pitches F,C,G,Ab, which could be thought of as an Fmadd9 chord (and/or, if we include the high D that comes next, as an Fm69 chord), and the final four notes are a downward arpeggiation of the pitches D,F,B,G, which is a G7 chord. So the sixteen-note pattern can be thought of as a i-iv-V progression in C minor, or, if we group the last eight notes together as one harmony, a i-V progression in C minor (Cm13#11 and G7b9). In terms of countour, the pattern is one long upward motion, followed by a second shorter upward motion, followed by a short downward motion. And, as with the colors, if the program's looping feature is turned on, the pattern makes a logical return to its starting point.

The pattern of sixteen velocities, if thought of as 16th notes in a 4/4 measure, show accents on beats 1 and 3, with a lesser accent on beat 2, and also strong syncopated accents on the 16th notes immediately preceding beats 1 and 3. When the program is in looping mode, this gives a pattern of rhythmic accents that essentially supports the harmonic rhythm, but that has enough sense of syncopation and ambiguity to keep the rhythm from being too ponderous.

The pattern of positions for the black dot has a strong correlation with the pitch contour and the harmonic rhythm of the musical notes, in the sense that it makes one generally upward motion in the first eight steps, followed by a shorter upward motion (reaching the highest point at the same time as the pitch contour does), and ending with a short downward motion. The first eight steps are in the left side of the lcd, and the second eight steps are in the right side, corresponding to the tonic-dominant harmonic interpretation. And, again, when looped, the pattern makes a continuous curve back to its starting point.

Try playing the patterns back at different rates (by changing the time interval of the metro) to get a sense of the way that rate affects the way we group events perceptually. For instance, at the default rate of eight events per second (i.e., with a 125 ms time interval) the notes are at a humanly-performable rate and the harmonic implication of them is quite clear; the movement of the black dot is a bit too jerky to be perceived as smooth movement. At a much faster rate like 25 events per second (a period of 40 ms), the motion of the dots seems smoother because the "frame rate" is too fast for us to perceive the locations as separate events; at this rate the musical effect of the notes is now also blurred and more merged as a single harmonic sound, although the pitch contour is still audible. At a slower rate like 2.5 events per second (400 ms per event), we still perceive the longer-term pattern of sixteen events, but each event feels more like an individual step or beat. In the case of the colors, this gives us more time to register a comparison between successive colors, thus perceiving the progression more clearly.

Counting through a List

A counting program is useful for reading through a numbered list of information, such as a list of musical notes or chords, a list of colors, a list of position coordinates, a list of sound files, a list of image files, a list of movie files, a list of programs to run, etc.

Max has several ways of storing an indexed list, which programmers call an array.
table (or itable) - an array of integer numbers (with graphic display/editor)
buffer~ - an array of floating point numbers (designed for holding audio samples, but can be used for any array of floats)
coll - an array of arbitrary messages (which can be indexed with numbers or words in any order)
umenu - can be used as a popup menu or just as an array of messages

The table, buffer~, and umenu objects store their contents with numbered indexing, starting from 0, so to read through them you just need to count upward from 0. (In the coll object, you can choose the numbers and/or words you want to use for indexing.)

This program demonstrates the use of a counter to read through an array of stored data.

The metro and counter combination is just as demonstrated in the lesson on timed counting. But in this program we don't bother to show the numbers being produced by the counter. We just send them to other objects -- two table objects and two coll objects -- to look up the data that those objects contain.

The two table objects contain pitch and velocity information to play MIDI notes. (This is comparable to a 16-note pattern stored in a pattern generator in a program like Reason.) One of the coll objects contains RGB color data to describe sixteen different colors to change the background of an lcd object, and the other coll object contains position data (the left, top, bottom, and right coordinates) of a shape to be painted in the other lcd object. Double-click on the coll and table objects if you want to see their contents. The data contained in these objects was entered by hand by the programmer to describe the desired pattern of notes, colors, and positions.

This program provides a choice of whether to stop automatically when the end of the count is reached or to go immediately back to the beginning in a continuous loop. That's what's happening in the left side of the program. The gate 1 1 object is open initially (that's what the second argument means), so the counter's maximum flag (the number 1 that comes out of the third outlet when the counter reaches its maximum) will get through the gate, be detected by the select object, and turn off the metro. If we send a 0 into the left inlet of the gate, the outlet will close, the maximum flag will not get detected, the metro will continue to run, and the counter will loop back to its minimum value and continue.

To control this looping-or-stopping, we use a toggle labeled "loop". We'd like the toggle's on state to indicate "yes, looping is on", and its off state to indicate "no, looping is off". Turning looping on with the toggle will send out a 1, but what we want is a 0 to close the gate. The == 0 object makes that switch for us, because when it gets a 0 it outputs a 1, and when it gets a 1 it outputs a 0.

In this lesson we chose to use lists of note data, color data, and position data to demonstrate three different kinds of event that can be contained in an ordered array. It would also be quite easy to store an ordered list of sound files, or image files, or movie files, but we'll save that for another time.

It's even possible to store an array of arrays, or even an array of arrays of arrays. Why would you want to do that? Well, again, think of the Reason pattern generator, which stores a pattern--an array of MIDI data--in one of sixteen presets--which is an array of patterns--in one of eight banks--which are arrays of presets.

At a more advanced level, once you have written many programs that do many different things, you might want to store an ordered list of program names, to be run at the desired time, and then step through that list with a counting program.

So you can see once again that an "event" might be something as simple as a single note, or it might be something more complex like a video or a complete algorithmic process.