top of page

Composition with Sound and Light

by Barton McLean

(Leonardo Journal, Vol. 2, No. 1, pp. 19-22, 1992)

During the past seventeen years of touring and performing our electroacoustic music as the McLean Mix [1], our attention has increasingly gravitated toward the visual element. In 1990, while planning our new multimedia touring event Gods, Demons and the Earth, I invented, with the collaboration of Rodney Peck and Michael Rose from the Rensselaer Polytechnic Institute, a new visual instrument and attendant art form capable of being performed and perceived in fundamentally musical terms. Thus was born the Sparkling Light Console (SLC), a MIDI-controlled panel of c. 500 pulsed bulbs in five colors capable of projecting extremely complex, bright and sharply-focused light displays of linear and textural patterns in a large hall.

THE SPARKLING LIGHT CONSOLE

The SLC (refer to Figure1 photo) embodies two intertwined concepts, the instrument itself and the works produced on it. These sound-light compositions exist in two formats, one a concert work performed by myself as part of the Gods Demons and the Earth concert, and the other an audience- interactive sound-light installation, in which the visitors actually perform at the the SLC and three other audio stations. Both the performance and the installation have the title "Fireflies", inspired by the trance-like state one attains upon observing a field of fireflies [2] in June in the Northeast United States. Although the SLC was designed and constructed to fit my artistic purpose in specific works it nonetheless has broad potential for a variety of styles and concepts, due to its flexibility and ability to be customized to the artistic idea at hand. The software and hardware was designed and built locally in 1989-90 by myself and engineers Rodney Peck, Technical Coordinator, Image Processing Laboratory, and Michael Rose, Technical Director of the Integrated Electronic Arts (iEAR) Studios, both at Rensselaer Polytechnic Institute.

Through MIDI control the SLC can produce linear and textural patterns of pulsing in a matrix of 18 x 18 colored lights [3]. This instrument is versatile both in variety of patterns and means of modification. In the present software version, 128 patterns of up to 64 steps in varying rhythms can be initiated, each begun by pressing a key on the standard MIDI keyboard. These patterns, which are predetermined by previously being entered into a buffer, have strong linear, pulsative, and textural personalities. Their basic characteristics, chosen to fit the requirements of the two "Fireflies" works, fall into the following categories:

1.Full screen "random" pulses, regular or irregular, single color, multicolor, and color metamorphosis (from one color to another).

2. Specific area clusters, single color, color metamorphosis in one, two or three specific screen areas, or between two or three areas.

3. Various geometric patterns which sweep across the full screen, both single and multicolor.

4. Comet chases, which frolic about the screen and then "explode" at the end.
5. Duets and trios of single-color lights which seemingly "converse" with each other.

6. Washes of single and dual colors, which sweep over the screen from one area to another, gradually changing into other colors.

7. Any combinations of the above.

As the performance progresses, two basic visual responses are occurring: the obvious linear progressions of clearly-defined patterns as they bound and trace across the large 5' x 5' panel, and the more subtle textural play with combinations of density, color, and rhythm. The esthetics of the light composition lie in the interaction between these linear and textural forces.
 

 

Methods of Control

Any MIDI device, including an external sequencer, can be used to control the patterns. At present, with a standard MIDI keyboard employed, the following controlling options are available (refer to Figure 2):

  1. Eight banks of 16 patterns each. In order to initiate a pattern, one first selects a bank (black key), then presses a pattern key (white). While in the same bank, each subsequent pattern selection will instantly change the pattern. Various software versions accommodate up to eight simultaneous patterns.

  2. Speed of pattern is controlled by designated keys. Speed ranges from one pulse every several seconds to a blinding 15 pulses/second.

  3. Each pattern can be either in loop (run continuously until stopped) or one-shot mode. As with speed, the looping is now controlled by a designated pitch on the keyboard to achieve exact control.

  4. Gating. Banks are divided into left and right, with four banks each. It is possible to reverse gate (turn off) patterns of left, right, or both banks independently by pressing the left or right gate keys. Upon lifting the key, the pattern resumes.

  5. Off. The off key turns all patterns off. They will then reset.

  6. Pulse width. Relation of pulse width to speed is critical, as it controls the brightness of the image. PW is controlled by six user-selectable increments on six keys by first selecting the desired PW, then the pattern key. Typically, PW would be from 3% to 12 % with fairly slow patterns, and 9% to 30% for fast patterns.

Previous software version used MIDI key velocity to control speed, pulse width. This proved to be too crude for exact performance requirements.

ESTHETIC CONSIDERATIONS
As the development of the artistic concept and the actual instrument progressed, it gradually developed that we were creating an instrument fundamentally different from most other visual media which change over time (film, video, slides, computer graphics, and laser art) in that with the SLC, the primary elements, which are the individual points of light embedded in the panel, being static and constant, are completely discrete and lack any imageric or referential quality, and only acquire significance in combination with other points of light. It could be argued that pixels in a computer graphics or video work might also fit these criteria but they lack the one all-important characteristic -- they are not perceived nor are they meant to be perceived as individual, discrete elements in a work. Pixels in a video work would be more analogous to single cycles of a waveform in music, both of which operate below the level of conscious perception. The significance of a single point of light in the SLC, then, is bestowed through placement in time, spatial relationship with other points, and color. In this regard the analogy to musical perception is compelling, inviting direct analogies as follows:

Single discrete point of light; analogous to a single discrete musical note. Rhythm of light pattern; analogous to the rhythm of a musical phrase. Color of light; analogous to the timbre of a musical tone.

This does not imply that there should be a one-to-one relationship between a note and a point of light in the actual artistic work; only that both media share the rare quality of their primary constituents being able to exist independently as purely abstract quantities devoid of artistic significance, only acquiring significance when in combination and interaction with other equivalent primary constituents.

Although the human brain is far more acute visually than aurally, as evidenced by the far greater number of neurons dedicated to the visual as opposed to aural brain functions, the visual arts are curiously not as highly developed in employing the "musical" concept of utilizing basic discrete abstract components. It seems to me that the visual arts developed historically from purely referential images and only became abstract leading out from the reference. Traditional western music, on the other hand, always had a theoretical basis which centered on its prime abstract constituents (discrete pitches, harmonic relationships, etc.). In this music merely 12 pitches and octave equivalents are employed. Everything in at least the more traditional forms of western music develops from these few primary constituents. When a B-flat is heard in a work by Brahms, even though the context is different from a work by John Adams, the same B-flat in an Adams work will still sound like a B-flat [4]. In the visual arts basic elements such as line and primary colors certainly exist, but they do not endure as completely separate entities in a painting or an experimental computer graphics work as they do in a piece of music where their original properties are most often preserved intact, thereby providing a more fundamentally abstract foundation for their unceasingly varied combinations into new relationships. It is this same "musical" esthetic that I gradually uncovered in the SLC as the conception of the two "Firefly" works jelled. In my opinion these abstract absolute referential properties foreshadow the ability of the SLC to be creatively exploited in a variety of modes and styles in future light compositions, just as pitch, and to a lesser extent, rhythm and timbre have withstood centuries of adaptability in musical styles.

THE TWO "FIREFLIES"

Installation Version

The brilliant patterns of colored lights [5] combine with similar points of sound, over a sultry and continuous sonic pedal enticing the visitor into the space. Upon entering, the visitor hears a calm, ethereal wash of sound. The present three-station audio portion invites the visitor to approach either (1) a computer typewriter keyboard, where any alphabet letter may be typed, resulting in any one of 23 delicate "firefly-like" melodic passages, (2) an amplified autoharp which can be creatively explored (plucked, hit, etc.) and sent to several digital processors resulting in pitch change, delay, other processing relating to "points" of sound, and (3) a traditional synthesizer keyboard which provides "darker" FM synthesis [6] sounds to complement the other "lighter" stations. Then, upon approaching the SLC the visitor can, by pressing designated MIDI synthesizer keys, initiate any of these patterns of moving lights, stop them, change their speed, color, and pulse width (brightness), thereby being able to creatively explore and "sculpt" visual patterns of light in real time. The duration of the work depends only on the visitor's capacity and interest.

Performance Version

A preexisting 14-minute digital tape of the self-contained audio work Fireflies is played. A score exists for the SLC performer which resembles normal music notation. The light patterns begin with random pulses of blue and green, gradually gathering energy and color, changing into textural sweeps of masses of color, with some cluster and duet activity. Leading up to the climax c. 2/3 of the way through the work, the patterns gather momentum, eventually running at maximum speed, concentrating on virtuosic and rapidly-changing linear counterpoint often exploding into bursts of multicolored expanding tracers. At the climax an additional 100 bulbs on randomly-blinking circuits are activated, adding to the already dense activity, gradually diminishing in density until the end, which mirrors the beginning. Fireflies ends quietly with fewer and fewer lights pulsing at greater time intervals until music and lights finally end together. Fireflies is presently employed as the finale to our McLean Mix concert Gods, Demons and the Earth.

TECHNICAL DATA PERTAINING TO THE SPARKLING LIGHT CONSOLE [7] Theory of Operation (refer to Figure 4)

The information from a MIDI keyboard is sent to a Roland MPU 401 in its "dumb UART mode" and then to an IBM PC computer via the Roland Interface Card. The PC is running custom software which polls for MIDI events and continuously outputs, via its printer port, row-column data corresponding to lights to be lit, based on the PC software database of pre-programmed patterns. Connected to the PC's printer port is a custom-designed Interface Box which takes as its input the row-column logic level data and makes power connections to the light bulbs on the panel at the row-column locations.

The Interface Box: Printer Interface and Software Considerations (Refer to Figure 4)

In order to light a bulb, a row and a column must be simultaneously activated. Since 18 rows and 18 columns exist on the Light Panel, a minimum of ten data lines are needed, five designated as row and five as column numbers. Instead of providing an expensive binary I/O board for the PC, we designed an interface allowing us to use its standard parallel printer port. The port having only eight lines, two bytes need to be sent, one representing the row, the other the column. Again this only requires five bits. One more bit is needed to indicate whether this is a row or column byte. The remaining two bits are arbitrary. We added a debugging feature with the extra bits selected so that the data values fell into the lower and uppercase alphabet region. The row/column indicator bit was positioned such that row 1. is generated as a lowercase "a" and column 1. is an uppercase "A". The interface watches the data from the parallel port and enables one or the other latches to accept the incoming data based on the values of the upper three bits. To light a particular light, then, the program simply writes two bytes out the printer port. To turn all the lights off, a value is written to a nonexisting light.

The Interface Box: The Decoders

Each of the two large circuit boards in the interface box contain identical decoders. These decoders accept a five-bit binary number as their input and enable one of 18 output lines. The top board is the row decoder and connects one of 18 lines to ground. The bottom board is the column decoder and connects one of eighteen lines to the +24 volt power supply. The decoders themselves consist of three 74LS138 demultiplexers (one-of-eight decoder)in a way that three '138s are used to create a one-of- eighteen decoder.

The Interface Box: Data-to-Power Conversion

Here the two boards differ. Although they are enabled by identical circuitry, each makes a connection to a different side of the power supply rails as noted above. The first transistor stage of each acts as an inverter, the outputs of the '138s going low when enabled. The inverter then drives the second transistor, which is a saturated switch. The outputs from both sets of eighteen appear on the 37 pin connector on the back of the interface box. Pins 1 through 18 correspond to the outputs of columns 0 through 17. Pins 37 down to 20 correspond to the outputs of rows 0 through 17.

The Light Panel

The Light Panel itself is a simple diode row-column matrix, each bulb being connected from column to row by a diode, thereby ensuring that only one bulb will light when a row-column pair is enabled.

Entering New Light Patterns

By laying transparent paper over a colored diagram representing each colored light in its row-column number, it is possible to trace any pattern and then enter the data into a database via a PC editor, thereby building up a database of predetermined patterns from which to activate.

FUTURE DEVELOPMENT OF THE SPARKLING LIGHT CONSOLE

At present the Sparkling Light Console is undergoing significant conceptual, hardware, and software revision. Address inquiries to Barton McLean, R.D. #2, Box 33, Petersburg, NY, 12138, U.S.A. (518) 658 3595.

_____________________________________________________________

[1] The McLean Mix, our husband-wife electroacoustic music and media duo consisting of Priscilla McLean and myself, has performed full-time for the past nine years.

[2] Fireflies are insects with phosphorescent abdomens which form in huge clouds in June and blink in pulsating rhythms which form aggregate linear and textural light patterns.

[3] Additionally there are 100 special-effects lights whose completely random patterns can only be switched on and off at this point.

[4] It is instructive to observe how recent sound and visual art forms concerned with technology have developed to markedly different degrees. Lacking any discreet prime constituent that can exist as an independent entity out of context (such as a musical pitch), the abstract visual arts (computer graphics, video, film) are more free to rapidly develop their styles and language. In fact this becomes necessary, since the lack of discreet building blocks recognizable out of context (such as pitches in music) means that each new visual work must establish its own referential context anew, thereby promoting rapid development and change. Although computer music has undergone a technological explosion similar to video and computer graphics during the past twenty years, the evolution of the actual music content has been at best painstakingly slow. Most computer music today, to the extent that it utilizes the traditional 12 equally-tempered western pitches, sounds more conservative than the first experiments of the 1960's and early 1970's. This is because music is saddled or blessed, depending on how one views it, with pitches and all their attendant properties of referential meanings independent of the context of the musical work. Interestingly, to the extent that music, whether computer or acousticically-derived, departs from this traditional cradle, to that extent it is capable of a high degree of experimental evolution. To this same extent it runs the risk, as does abstract experimental video and computer graphics, of becoming rapidly outdated as new technologies and referential contexts dominate its direction and choice of style and language without the stabilizing characteristic of the non-contextual referential property of a music pitch or a point of light on a light panel. None of this is meant to be a value judgement.

[5] The light patterns consist of spirals, zig zag, random, comet chase, color metamorphosis, linear traces, etc., all being speed, pulse and color variable.

[6] "FM" stands for Frequency Modulation. In FM, one waveform modulates another, creating a new, complex waveform, achieving a timbral richness greater than the sum of the two original waves. By varying the level of the modulation wave, one can vary the amount of brightness heard in the new wave. FM is perhaps the most common method employed in basic sound synthesis today.

[7] Technical data was partially furnished by Rodney Peck and Michael Rose. For additional technical information contact them at their company, Mondale Designs, 12 Reid Ave., Troy, N.Y., 12180, U.S.A. (518) 273 1107.

Caption for Fig. 1:

The Sparkling Light Console seen counterclockwise as follows: Light Panel, Oberheim XK Keyboard, custom Interface Box, Roland MPU 401 MIDI Interface (just to the left of the computer), and IBM XT Computer showing the main window with the custom software.

Photo credit: Barton McLean C 1992 Barton McLean.

bottom of page