Difference between revisions of "Humdrum Lab 1"

Installing Humdrum tools

If you are using MacOS, first install Homebrew. If you are using Windows, install Windows Subsystem for Linux (version 2), or install a linux distribution inside of VirtualBox, or install cygwin.

Then, to use the following commands in a terminal (typically using the bash shell, using /Applications/Terminal.app in MacOS), install the command-line Humdrum tools.

For Windows users, I would recommend installing Linux Subsystem for Windows (LSW2). Your Windows 10/11 should be fairly up to date so that you can install version 2 of LSW. See https://docs.microsoft.com/en-us/windows/wsl for more information. I would also recommend installing Ubuntu linux from the Microsoft App store once LSW is enabled. You can also use a Stanford unix computer (see this network unix guide from CS 107)

The command sequence to install command-line Humdrum tools should look like this:

   git clone https://github.com/humdrum-tools/humdrum-tools
   cd humdrum-tools
   make update
   make
   make install

The installation can be located in any directory, and "make install" should insert the installation directory into the command search path.

To check for updates at a later date, type these commands in the humdrum-tools directory:

    make update
    make

Check that your installation is successful by re-logging into the terminal. Type this command to see if the Humdrum Tools are installed successfully:

    which key

If the command responds with a line of text, installation was successful. To see if the Humdrum Extras tools are installed try typing:

   which keycor

.

Bach Chorales

Download

A Humdrum edition of 4-Part Chorales by J.S. Bach is available at http://kern.ccarh.org/browse?l=chorales

The data files can be downloaded using the humcat and humsplit commands (first creating a new directory to store all of the downloaded files):

    cd ~/Desktop        # optional to display on Desktop in MacOS
    mkdir chorales
    cd chorales
    humsplit h://chorales 

This should create 370 files in the format chor001.krn, chor002.krn, chor003.krn, etc. You can type "ls -asCF" in a unix terminal (/Applications/Utilities/Terminal.app in MacOS) to see the list of files that were downloaded:

Alternate download method

If you want to down all Humdrum files available on GitHub, try these commands:

   git clone https://github.com/humdrum-tools/humdrum-data
   cd humdrum-data
   make

The Bach chorales will downloaded to "bach/chorales-270". You can browse through other repertories of Humdrum files that were downloaded.

Verovio Humdrum Viewer

The same chorales can be viewed on Verovio Humdrum Viewer from this link http://verovio.humdrum.org/?file=chorales&k=e

Clicking on a chorale title will load the work into the editor:

Copying data to VHV editor

You can also copy a downloaded score into the VHV editor by pasting the contents of the file into the editor. From the terminal in MacOS, you can use the pbcopy command:

    cat chor001.krn | pbcopy

And then type command-v after clicking in the text editor on the VHV website to paste the score into the editor.

Playback on the command-line

If humplay compiled successfully on your computer (in theory it will work in MacOS, Windows and Linux, but will depend on if the MIDI software can be compiled successfully), then try the humplay command to play a score from the command line:

    humplay chor146.krn

You can alternatively download a particular score from the web:

   humplay h://chorales/chor146.krn

See documentation of features for humplay at http://extras.humdrum.org/man/humplay

Some commands to try:

For example, type the characters "2m3m" to mute the 2nd (tenor) and 3rd (alto) voices to listen to only the outer voices. This sort of interactive playback is not (yet) available in VHV.

MIDI Rendering

Loading a score into the VHV editor will allow you to press the "play" button to listen to a MIDI rendering of the score; however, more control over the conversion to a MIDI file can be done by using the `hum2mid` command.

Convert a particular chorale into a MIDI file with this command:

    hum2mid chor001.krn -o chor001.mid

The data file can also be downloaded on demand:

    hum2mid h://371chorales/chor001.krn -o chor001.mid

Once the MIDI file has been created, you can open it by double-clicking in a file browser. On OS X computers, you can type:

   open chor001.mid

to do the equivalent of double-clicking. You must have a MIDI player associated with the file. For newer OS X computers, you will have to download Quicktime 7 in order to play the MIDI file (recent versions of Quicktime cannot play MIDI files). This no longer works in MacOS Catalina or later, so view the MIDI to MP3 conversion notes at the bottom of this page.

To pan the voices in the stereo field, try the --autopan option:

    hum2mid --autopan chor001.krn -o chor001pan.mid

To play the music with an organ sound try:

    hum2mid -f 19 chor001.krn -o chor001organ.mid

Converting all Humdrum files into MIDI files in a directory

This bash for-loop can convert all chorale scores into MIDI files:

  for file in *.krn
  do
      hum2mid $file -o $(basename $file .krn).mid
  done

To store the converted MIDI files in a separate directory:

  mkdir midi
  for file in *.krn
  do
      hum2mid $file -o midi/$(basename $file .krn).mid
  done

Key

Each chorale is hand-labeled with a musical key. To generate a histogram of key designations in the chorales:

  extractx -s 1 *.krn | egrep -i '^\*[a-g][#-]?:' | sortcount

The regular expression "\*[a-g][#-]?:" translates into English as: find lines of text starting with "*" followed by one of the characters "a, b, c, d, e, f, g, A, B, C, D, E, F, G" then optionally one of the characters "#" or "-" followed by a colon. This is the basic structure of a key interpretation in Humdrum **kern data. An optional three-letter modal category can also be added after the colon, such as "*d:dor" for D dorian.

Questions:

1. What is the most common key?
2. What is the most common major key?
3. What is the most common minor key?
4. What is the least common major key (other than zero counts)?
5. What is the least common minor key (other than zero counts)?
6. How many chorales are labeled as being in a modal key? Such as *G:mix for G mixolydian.

You can create a histogram plot by running this command pipeline:

  extractx -s 1 *.krn | egrep -i '^\*[a-g][#-]?:' | \
     sortcount -v -T "Key counts" > analysis.html

And then opening the file analysis.html in a webpage (such as by typing "open analysis.html" in MacOS):

Computational Key Identification

Use the keycor program to have the computer measure the key of the chorales:

   keycor chor001.krn

The computer should reply:

   The best is key G major

Compare this to the hand-labeled key:

    egrep -i '^\*[A-G][-#]?:' chor001.krn

Here is a bash for-loop which can be used to examine the key one chorale at a time:

  for i in *.krn
  do
     echo $i analysis:
     keycor $i
     extractx -s 1  $i | egrep -i '^\*[A-G][-#]?:'
     echo =====================
  done | less

Questions:

1. Find a chorale where the hand-labeled key does not match the automatically identified key.
2. Take that chorale, and run the following commands on it. Do these all give the same key analysis? Which ones give the correct answer:

  keycor  chorXXX.krn  --aa
  keycor  chorXXX.krn  --bb  
  keycor  chorXXX.krn  --kk
  keycor  chorXXX.krn  --kp
  keycor  chorXXX.krn  -s

Mkeyscape

The mkeyscape program can be used to view a detailed picture of the key throughout a piece of music. The online version of the program is more convenient, plus it has an interactive display of the color-to-key mappings: http://extras.humdrum.org/online?command=mkeyscape%20-ln%20h://370chorales/chor001.krn&run=true

The above link should produce this plot of the key structure, showing it is strongly in G major with no modulations to other keys, with the next strongest key region being C major for one measure (at bar 14):

To do this on the command-line with Imagemagick tools installed:

  mkeyscape -ln chor001.krn | convert - output.png

To install imagemagick tools in MacOS with Homebrew, type:

   brew install imagemagick

Most linux distributions should have imagemagick installed already; otherwise, install in some similar manner using "yum", "apt-get" or similar package manager for your version of unix.

Transposition

The Humdrum Extras program transpose can be used to transpose chorales to another key with the -k option. For example, tranpose the first chorale into F♯ major:

     transpose -k f# chor001.krn | less

Also notice that the transpose tool is available as a filter in the VHV editor. Here is the chorale without transposition:

And then adding the line:

    !!!filter: transpose -k f#

which will transpose the music down a half-step:

The Humdrum Toolkit program trans calculates transpositions as well, but using a different (2D) algorithm. Using trans, it is possible to change the mode of the music. For example, here is how to convert from a major to a minor key:

 trans -d -2 chor001.krn | trans -d 2 -c 3 -k '*k[b-e-]' 

The first transposition

   trans -d -2 chor001.krn

transposes the music from G major to its relative E minor. The second transposition moves the music from E minor up to G minor. the -k option is used to force the key signature to the standard one used for G minor.

The Humdrum Toolkit tools cannot be run from VHV, but to quickly copy an altered score, add the pbcopy command to the end of the pipeline:

      trans -d -2 chor001.krn | trans -d 2 -c 3 -k '*k[b-e-]'  | pbcopy

and then paste the score into the VHV editor.

Vocal Range

To count all of the notes by pitch-class for each vocal part, use the prange command. Here is a sample command which extracts the bass part data:

  extractx -s 1 *.krn | prange

or alternatively extracting by text pattern (Bass, Tenor, Alto, Soprano):

   extractx -g Bass *.krn | prange

For the bass part, the lowest note is CC (C2) and the highest note is e (E4), with a total vocal range of 28 semitones. The average (base-12) pitch is E- (E♭3), which is both the mean and median.

Do a similar vocal range analysis on the other three parts.

Questions:

1. Which voice has the widest range?
2. What is the highest and lowest note for each vocal part?

Graphic display of range using SCORE

Try the command:

    humcat *.krn | prange --score > ranges.txt

and then load the file "ranges.txt" into SCORE using the RE command. This should produce the following figure:

Tip: Paste the PMX data output from "prange --score" into http://score.sapp.org to generate notation online.

Also try

   humcat *.krn | prange --score --hover > ranges-hover.txt

to add SVG markup to show note counts when hovering over notes in vocal range histograms.

Try this command:

     transpose -k c *.krn | humcat | prange --score --hover > ranges-transpose.txt

What does it do?

Scale Degrees

The deg command can be used to calculate the scale degree of each note if the key has been indicated. Here is a command to count how often each scale degree occurs in the chorales:

  serialize *.krn | deg -a | ridx -H | grep -v r | sed 's/[^1-7]//g' | sort | uniq -c 

This should produce the result (or close enough depending on data and software updates):

 15695 1
 11267 2
 12500 3
 10723 4
 16225 5
  9052 6
 10604 7

The sortcount program can handle sorting and counting. Here is an example with the -p option showing the relative frequency of each scale degree in the data:

  serialize *.krn | deg -a | ridx -H | grep -v r | sed 's/[^1-7]//g' | sortcount -ph

**pcent	**data
18.85	5
18.24	1
14.53	3
13.08	2
12.46	4
12.32	7
10.51	6
*-	*-

The dominant (5) is the most common, and the submediant (6) is the least common.

Question:

1. Using the following template, what are the scale-degree frequencies in each part? Are there any differences?

    extractx -g Bass *krn | deg -a | ridx -H |  grep -v r | sed 's/[^1-7]//g'  | sortcount -ph

Melodic Intervals

Generate a list of the melodic intervals found in all voices, sorted by most common:

   serialize *.krn | mint | ridx -H | grep -v r | grep -v '[[]' | sortcount -p

Histogram plot of the same data (limiting to occurrences above 0.5%):

   serialize *.krn | mint | ridx -H | grep -v '[[]' | \
       sortcount -pv --min 0.01 -T "Melodic intervals in Bach chorales" > analysis.html

Sorting by interval size:

    serialize *.krn | mint | ridx -H | grep -v '[[]' | \
         sortcount -pv --min 0.01 -T "Melodic intervals in Bach chorales" \
              --sort interval > analysis.html 

Tritones can be notated as augmented 4ths (such as C-F♯) or diminished fifths (such as C–G♭).

    serialize *.krn | mint | ridx -H | grep A4 | wc -l
    serialize *.krn | mint | ridx -H | grep d5 | wc -l

Questions:

1. Which of these two spellings of the tritone are more common in the chorales?
2. How often do octaves (P8) occur in the chorales?
3. For the diminished fifth, what is the most common direction for the melodic interval, up or down? Here are commands to answer this question:

    serialize *.krn | mint | ridx -H | grep +d5 | wc -l
    serialize *.krn | mint | ridx -H | grep -- -d5 | wc -l

The double dash after grep in the second command indicates the end of options for grep, so that the string "-d5" is treated as the query rather than as the option -d5.

Harmony

Harmonic intervals

The hint command can be used to study harmonic intervals. "Hint" stands for "Harmonic INTervals". Here is an analysis of the number of intervals found in all of the chorales:

  hint -ac *.krn | serialize -c | ridx -H | sortcount -ph

**pcent	**data
15.88	m3
15.43	P5
14.2	M3
13.42	P1
9.51	P4
8.83	M6
8.74	m6
7.04	-
2.23	M2
1.36	m7
1.35	A4
0.97	d5
0.35	m2
0.22	M7
0.16	r
0.15	d7
0.06	A2
0.05	d4
0.04	A5
0	A6
0	d1
*-	*-

The most common interval between voices is a minor 3rd (or minor 3rd plus an octave transposition) at 15.88 of the harmonic intervals. Then 15.43% for perfect 5ths and 14.2 for major thirds. The most common "dissonant" interval is a major second at 2.23% of the intervals.

Chord prototypes

The tntype program can be used to examine harmonic sonorities in the music:

  tntype -a *.krn | tntype -fa | extractx -s '$1-$' | ridx -H | sortcount -ph

Here are the 13 most common harmonic sonorities that will be listed:

35.09	3-11B	{047}
17.26	3-11A	{037}
9.07	4-27B	{0368}
4.96	4-26	{0358}
4.36	3-10	{036}
4.26	3-9	{027}
2.8	4-20	{0158}
2.52	4-27A	{0258}
1.99	4-22A	{0247}
1.95	3-7A	{025}
1.73	4-23	{0257}
1.57	4-14A	{0237}
1.16	3-8A	{026}

35% of all sonorities in the chorales are major chords, which are called 3-11B ("Forte number") in set theory, with the semitone prototype {047} which would be CEG on C, or GBD on G. Minor triads are half as common at 17.26%. The most common 4-note sonority is 4-27B which would be (C, E♭, F♯, A) which is a fully-diminished seventh chord in tonal music theory.

Harmony/Melody

The cint program is a hybrid of the mint and hint programs. It examines 4-note counterpoint modules which contain two harmonic and two melodic intervals.

cint *.krn --raw -to | sortcount -ph | less

Here are all of the counterpoint modules which occur with a frequency of greater than 1%:

**pcent **data
2.33    5 -2 1 6
2.23    6 2 1 5
1.91    3 -2 1 4
1.89    4 -2 1 5
1.82    6 -2 -2 6
1.71    5 1 2 6
1.66    3 -2 -2 3
1.65    3 1 2 4
1.62    3 2 2 3
1.6     -6 2 2 -6
1.57    6 -2 2 8
1.47    10 2 -2 8
1.39    5 2 -2 3
1.37    4 2 1 3
1.36    6 1 -2 5
1.36    5 1 -2 4
1.34    -4 2 -2 -6
1.3     -7 -2 1 -6
1.28    4 1 -2 3
1.2     2 -2 1 3
1.2     -6 -2 1 -5
1.16    6 2 2 6
1.08    -3 2 1 -4
1.02    3 1 -2 2
1.02    8 1 -2 7

The module "5 -2 1 6" means that the two voices start together with a fifth, then move to a sixth, with the bottom voice going down a second while the top note stays on the same pitch.

Here is an example method to highlight the counterpoint module "5 -2 1 6" in graphical notation (note: webscore needs updating):

   cint -to --mark --search "5 -2 1 6" chor001.krn | satb2gs \
         | webscore > analysis.html

The cint tool is also implemented as a filter in VHV:

http://verovio.humdrum.org/?file=chorales/chor001.krn&k=e&filter=cint%20-to%20--search%20%225%20-2%201%206%22

Searching

This is the search method behind Themefinder.

Create a search index for the chorales:

    tindex *.krn > chorales.index

Search for the melodic sequence "C D E F G A B C", counting how many chorales it occurs within:

   themax -p "cdefgabc" chorales.index | wc -l

To locate the pattern within the music:

   themax -p "cdefgabc" chorales.index --loc | theloc

which should return the result:

  chor190.krn::1	58=10B2-65=11B3
  chor325.krn::1	17=5B1-24=6B4

This means that the melodic pattern "cdefgabc" occurs in two chorales (190 and 325). In both cases, the bass part (::1) has the pattern. The pattern occurs from note 58 to 65 in the bass part to chorale 190 which is from measure 10 beat 2 until measure 11 beat 3. Similarly, the pattern occurs in the bass part of chorale 325 in measure 5 to 6.

The search matches can be marked in the original data:

   themax -p "cdefgabc" chorales.index --loc | theloc -m | less

Each matched note will be prefixed with an "@" sign. Search for these marks in measure 10/11 and 5/6 in the two chorales.

To pull out the measures which contain the matched notes:

       themax -p "cdefgabc" chorales.index --loc | theloc -m | myank --marks

To display matches highlighted in graphical notation for the last chorale in the list:

   themax -p "cdefgabc" chorales.index --loc | tail -n 1 | theloc -m \
         | webscore --header > analysis.html

VHV search filter

VHV has a filter which does searching using another program called msearch:

Questions:

1. Search for another pattern, and report your findings.

The themax command can search for other melodic features, such as interval, contour, rhythm, meter.

Randomized Pitch

The pitchmix program can mix the order of notes in a score, while keeping the rhythm fixed. The default use of pitchmix will randomly move all notes in the score, keeping the rhythm in the original order:

   pitchmix chor001.krn > chor001.mixed

Adding the -v option will only mix notes within a voice:

   pitchmix -v chor001.krn > chor001.mixed

This can then be listened to on VHV or with MIDI and/or conversion to MP3 files:

   pitchmix chor001.krn | hum2mid -f 19 -o mixed.mid && open mixed.mid

The option -f 19 means to force the instrument to be General MIDI #19 (church organ).

For displaying notation, it is best to use the -n option which transposes the randomized notes to be within an interval of a fourth from the original note:

   pitchmix chor001.krn -n | webscore --header -s 25  -f satb2gs > output.html

The -d option specifies a randomization envelope over the duration of the music. For example, -d "0 0 1 1 2 0" is a list of time/weights: (0, 0), (1, 1), (2, 0) which means that the music starts with no randomization; in the middle of the music, there is an increase to 100% randomization, and by the end of the music, the is a gradual shift back to no randomization.

    hum2mid chor001.mixed -o chor001-mixed.mid

There are several options to control the type of pitch mixing.

Next lab

Next, you can view Humdrum Lab 2 for a review of the Essen Folksong Collection and some tools for dealing with it.