Place some notes on steps 1 through 4 of the sequencer below. Then press the “Generate” button, and the AI will complete your melody. Press “Play” to listen to your AI-assisted creation. Each time you press “Generate,” the AI will create a new pattern.


The AI is a neural network trained with thousands of monophonic melodies (monophonic means that only one note is played at a time). From this training, it learned the structure or shape of typical melodies. Neural networks can learn to do things by training with a large number of examples. This makes them ideal for applications where we cannot explain through “rules” what the task exactly is (either because there are too many rules or we don’t know them), but we can provide examples of what we want.

The neural network used in this chapter is very similar to the one used in the “Rhythm” chapter. You can read the explanation there to find out how it works.

How random are melodies?

There are no rules or scientific laws to say which sequences of notes are valid melodies and which aren’t. Music theory can explain many decisions behind compositions, but the boundaries of art can never be perfectly delineated.

Many different things can influence composers:

  • Biological and cultural factors (there’s melody in speech patterns).
  • Their chosen musical instrument (instruments determine which notes are available and which combinations are more natural to play).
  • How harmonic or dissonant two notes are.
  • Scales, modes, and other formal or cultural conventions.
  • Genres, styles, and fashions.

And within these frameworks, artists also use repetition, anticipation, surprise, and rhythm to engage listeners.

All these things generate structure: a web of relationships. Each note is related to the ones that came before (e.g., how likely or unlikely they are to be together, how similar or contrasting they are, etc.) and the ones that might come after.

The sequencer above does some very big simplifications: it only offers seven notes from the C major scale, and all notes have the same duration and loudness. This means that from the start, there’s a strong relationship between the notes we’ll use to build melodies: they’re close friends.

Still, in terms of structure, we’re only scratching the surface. Let’s see a sequencer with a new button that generates a melody randomly. At each step, it choses a note (or silence) as if by the roll of a die.


It’s reasonable to generate melodies randomly when working with a limited framework of related notes (such as a scale). If you generate many random melodies, you will come across some good ones and some that even seem to match your initial notes incredibly well (by pure chance). But you should notice differences if you generate many melodies using the AI and many with the random algorithm. The AI will generate more regular shapes, use a smaller selection of notes, and build on what you entered as prompt in the first steps. Also, you might notice that random melodies sound good only after you hear them in a loop several times (giving your brain a chance to establish patterns).

Before neural networks became popular, legacy AI systems often used a mix of rules and randomness. The sequencer below offers a third button that uses an algorithm based on a Markov chain. Try it several times.


We can explain the way this algorithm works through the following instructions:

  1. Write down the notes in the prompt (first four steps of the sequencer).
  2. Add arrows:
    1. … from each note to the one that came after
    2. … from every note to itself.
    3. … from the last note to the one that appears first.
  3. Starting on the note that appeared in the fourth step…
  4. … select an arrow at random and follow it to the next note.
  5. Repeat instruction 4, jumping from note to note by choosing arrows randomly, until the melody is complete.

The random algorithm can produce any possible combination of notes with equal probability. The results created by the Markov chain are hopefully more interesting, but they will all be limited by the possible combinations that these rules allow. There are a lot of interesting melodies it’s unable to produce.

We could create many more Markov chains, using different sets of rules, based on different ideas about composition and what makes melodies work. An algorithm could select one of these Markov chains at random and then use it to generate a melody. As we continue to add more chains, we would cover more and more interesting melodies.

Neural networks don’t require rules from us; They learn through a training process using examples. The potential melodies that a neural network will generate are those that follow similar structures or patterns to the ones used to train them. To build a neural network that can produce (almost) every interesting melody, we need to train it with a large amount of varied examples.

A brief history of sequencers

Sequencers, like the one used above, are a common tool in electronic music. They shorten the distance between writing and performing music. An artist can even “play the sequencer” and make writing music an artistic performance.

Drawing of the RCA Mark II Sound Synthesizer

The concept of sequencers goes back hundreds of years ago. Music boxes, pianolas, barrel organs and many other mechanical devices play music automatically through holes, dents or other physical markings on tapes, cylinders, discs, etc. The RCA Mark II Sound Synthesizer (1957), one of the earliest music synthesizers, could be controlled through paper tapes with punched holes.

Drawing of the Moog 960

The Moog 960 (1967) was an analog sequencer module for the Moog Modular Synthesizer. It could provide from 8 to 24 potentiometers that activated in sequence, at the desired tempo. These would power the synthesizer to play a looping melody. Bands, like Tangerine Dream, took the opportunity to manipulate its switches in real time to modify and evolve the sequences during live performance.

Drawing of the Romand MC-8 Microcomposer

The Roland MC-8 Microcomposer (1977) was one of the earliest sequencers driven by a microprocessor. It was a small stand-alone unit that connected to large complex modular synthesizers like the Roland System 700. Its memory could hold 5200 notes, and was even able to handle polyphonic sequences.

Drawing of the Roland MSQ-700

The Roland MSQ-700 (1983) was the first MIDI sequencer. MIDI is a standard that allows musical instruments and computers to be connected. A MIDI sequencer can be programmed using a piano-like keyboard. It can then play back the sequence through a synthesizer, sampler or some other sound-producing MIDI instrument. This established a new way of producing music, by manipulating note data instead of recorded sound.

Drawing of the Akai MPC-60

The Akai MPC-60 (1988) was a sampler and sequencer. It allowed artists to record sounds, manipulate, and trigger them through its 4x4 grid of pressure-sensitive pads, while also recording a sequence. It was a low-price unit that could be used by itself to produce whole tracks. It had a democratizing effect that revolutionized hip-hop and electronic music production.

Food for thought

Sequencers allow us to make music by “molding” it as it plays. We can change the notes of a sequence while it loops, its rhythm, or modulate the sound of the instruments smoothly transitioning to a different mood or color. The user interface of the sequencer (knobs, pads, mouse, keyboard, touch screen, etc.) limits how many simultaneous changes we can make while the music is playing, and sometimes it’s necessary to trigger pre-programmed variations.

How can AI enable new forms of interfacing with a sequencer to control the flow of an “elastic” live performance with a lot of expressive freedom?

Early sequencers required tedious note-by-note input. MIDI controllers, like keyboards and pressure-sensitive pads, allowed more expressive, fast, and natural input of sequences. What new mechanisms can AI introduce?


Based on the Google Magenta Melody RNN demo.

Copyright and license information in our GitHub repository.


  • Wikipedia contributors. “Music sequencer.” Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 2022-12-05. Retrieved 2022-12-06.
  • Wikipedia contributors. “Analog sequencer.” Wikipedia, The Free Encyclopedia. 2021-06-20. Retrieved 2022-12-06.
  • Mark Jenkins. “Behringer Beat: 960 Sequential Controller Review”. Synth and Software. 2020-07-29. Retrieved 2022-12-16.
  • Wikipedia contributors. “Roland MC-8 Microcomposer.” Wikipedia, The Free Encyclopedia. 2021-08-11. Retrieved 2022-12-16.
  • Steve Howell. “Roland MSQ700.” Sound on Sound Magazine. 1996-04. Retrieved 2022-12-06.
  • Wikipedia contributors. “Akai MPC.” Wikipedia, The Free Encyclopedia. 2022-10-24. Retrieved 2022-12-06.

Text is available under the Creative Commons Attribution License . Copyright © 2022 IMAGINARY gGmbH.