One of my motivations for working on chant is my fascination with the elusive nature of this music. Lacking defined rhythm, form, tonality or harmonic function, perception of structure in this music seems vague and subjective, yet endlessly compelling. I have been encouraged by several people to set down my observations on the musical structure of chant, and I will attempt a beginning here. Comments and observations are always welcome: please drop me a note at email@example.com.
On mode and English polyphony, see David Stern, ‘Thomas Morley and the Teaching of Modal Counterpoint in the Renaissance’, Theoria XVII (2010): 59-112.
Antiphon: Orietur in diebus p. 
Due to the amorphous nature of the music it is hard to know where to begin, so I will simply begin with the observation that in typical antiphons for the office it usually seems possible to reduce the neumatic style to a syllabic style by omitting ornamental notes. I find that this can help in discerning the underlying structure and at the same time helping to secure a satisfying performance. So, I shall address an antiphon that I happen to be looking at right now, Orietur in diebus, Antiphon 6 at Matins on Sundays in Advent. page . Here I would reduce ‘Orietur’ to F.D.C.F. To be specific, I would consider E a lower neighbour, and D of the third syllable an appoggiatura. This would be related to Schenker’s ‘Boundary-play’ idea. One could go into a more sophisticated analysis of the music of this first word, but my point here is really only to set up the idea of looking at the possibility of an underlying syllabic structure for antiphons. Moving along, I would see ‘diebus’ as F.G.G, ‘ejus’ as G.G, ‘pacis’ as A.G. ‘eum’ is particularly interesting; it could be understood as F.F. or G.F; or as two levels beyond the surface, an initial level of G.F. where G leans on F as an appoggiatura, and a deeper level of F.F, where the appoggiatura is removed. ‘omnes’ would be F.D; ‘reges’ F.C; ‘gentes’ simply F.F, and ‘servient’ would be G.G.G. (It is worthwhile to sing the antiphons reduced to one syllable per note. This can confirm one’s perceptions of the structure.)
Moving along we could then see the underlying pentatonic basis of the melody, C-D-F-G-A. and the basic contour would be movement through the third F-G-A-G-F. (April 17, 2020)
The two-fold function of a psalm-tone ending is give a satisfactory melodic close to each verse, and also to effect a convincing tonal return or link to the beginning of an antiphon. Besides this, a psalm-tone ending can effect a motivic ‘linkage’ to the beginning of an antiphon. So for example when the antiphon Dominus defensor vite mee (Noted Breviary A-04, p. ) is used at matins on Mondays, the gesture GAG of the opening is foreshadowed in the ending of the psalm-tone. We should be aware, however, that this sort of thing is not found consistently. The antiphon Alleluya iii. for sext on ferias in Eastertide (Noted Breviary B-33, p. 1347) contains the same (F.)GAG gesture. Here the connection is to be again with the end of the psalm-tone, F.GA.G.(F), but the gesture seems to be more of an echo than a direct repetition.
Heinrich Schenker, who greatly admired this technique in the works of the nineteenth century masters, might have been surprised to find it also here and there in the simple chant that he generally dismissed.
Linear progressions and tonal structure
I am not prepared to make bold generalizations concerning linear progressions and tonal structure in Gregorian chant. But I will point out that these concepts are certainly relevant. The invitatory antiphon Alleluya iii. for matins on Mondays after the octave of Easter (Noted Breviary B-33, p. 1338) displays very clearly an initial ascent F.G.A. over the course of the first two alleluyas; the final alleluya begins with a rapid retracing of this rising third (a ‘slide’ or double appoggiatura), and concludes with A-G-F, a ‘3-2-1’ descent. G is decorated with a turn, and F with an appoggiatura.
In larger terms the Venite itself (Noted Breviary D-1, page 35*.) is based on the upper neighbour, Bb and its accompanying G, which resolve back to A and F respectively at the end of the verse. The final gesture of the verse repeats the initial ascent found in the antiphon. That is, the verses of the Venite are based on the upper neighbour, while the antiphon is based on the principal note, A.
Antiphon: Veniet fortior me p. 116
This mode VIII antiphon is unusual in that it is full of B-flats, except for the first two Bs. The B-naturals are lower neighbours to C; the B-flats are upper neighbours to A. To our thinking the profusion of B-flats seems to deny the mode. On the other hand, the ‘structural tones of the antiphon are C, A, and G, which confidently assert mode VIII. Compositionally, the piece seems to be based on CDCA and its sequenced repetition, AB-flatAF, after which the A falls to the final G. The sequencing of C-A to A-F forces the music into an F hexachordal structure. It would seem that the conflict, such as it is, is between the G mode and the F hexachord. There may be conflict in the theoretical ideas of mode and hexachord, but there is no conflict in the music of this chant. Further, the antiphon is entirely in accord with the psalm tone: GAC-DC : C-ACDC. (In contrast, tone VIII.i. includes B natural.)
Antiphon: Accipite Spiritum Sanctum p. 1531
In most Sarum sources this mode VII antiphon is transposed to a C finalis, giving a ‘major’ scale instead of the putative mixolydian scale. Nevertheless, the psalm-tone retains its usual shape, with B-flat in its transposed form. (This does not appear in the cue for the psalm-tone, but it may be observed in the Sarum Psalm-Tones. p. 60*.) Most of the continental sources available give this antiphon in the normal transposition, and thus in a ‘pure’ mixolydian mode.
Antiphon: Hora est jam nos p. 
This chant exhibits a ‘twist’ at the end. The first two phrases, full of B-flats, sound very much like mode II transposed; the final phrase makes the shift, turning the music to mode VIII. Of course, when sung in context, following the tone VIII-psalm, the first part of the antiphon provides a strong tonal contrast and the final phrase returns the music clearly to mode VIII.
Multiple forms of the same piece
Antiphon: Justus Dominus p. 
In the sources this mode VII antiphon appears in various transpositions that reflect variations of detail in the setting of one word, ‘Dominus’.
Worcester F-160:273r. and 277r. (the most important English monastic source) gives what is a simple and presumably older form: DE.D.D, a simple inflection above the reciting note on the accented syllable. This form also appears in many other monastic sources. Sarum ms. BL-52359:472v. give a more ornamented form, DEFD.CD.C, a more developed figure that touches the characteristic flat-seventh of mode VII. This is found in a wide variety of sources and is adopted in the Nocturnale Romanum. We find, however, in Sarum ms. AS:637 a transposition of the chant down a whole tone, with a key signature of B-flat. Here the figure appears (untransposed) as DEF#D.CD.D. This version gives the raised-seventh, which seems out of place in mode VII, but is in fact found in a number of different chants. Most interesting among them is the Sarum version of Alma Redemptoris Mater, which (transposed to a finalis of C) includes both the raised-seventh in the upper register and the lowered seventh–the so called ‘Gallican cadence’ in the lower register (at ‘Genitorem’). (This example is particularly telling as it reverses the expected figures, which would more normally be the flattened seventh in the upper register and the raised seventh in the lower register.) Next we come to the version found in the printed Sarum antiphonale, 1519-C:15r. Here the music is transposed to a C finalis with no flats indicated; this is simply a transposition of the version found in AS:637–and yet the transposition may reflect a growing awareness of tonality, as this version, dating from 1519, notionally mode VII ending on C, uses only the notes of the C major scale. On the other hand, this version already appears in the early 12th c. P-Pnm lat. 12044:299v, with B-natural clearly marked! To complete this survey, we see in some German sources, such as A-Gu 29:336r. another variation, where ‘Dominus’ is set DEGD.CD.D. This characteristic German style avoids the instability implied by the F/F-sharp (or B/B-flat), and instead omits the seventh note above the finalis entirely, taking the octave instead. While this short survey displays some of the variety of forms that appear in chant, its main purpose is to support the underlying notion of principle tones (in this case D) and added ornaments that are fundamentally decorative, not structural.
Responsory. Angelus Domini (Ferial Responsory for Quinquagesima)
This mode II responsory is unusual and remarkable for its high B-natural–normally in mode II responsories the highest note is B-flat. On the poetic side, the employment of B-natural seems to reflect the idea ‘extendas’, literally a stretching of the pitch. B-natural is accommodated by its local relationship to G as a sustained and stable pitch. In typical mode I and II contexts the sustained pitch is A, with an upper neighbour B-flat.
Melodic pattern and mode
There is some degree of coordination between melodic pattern and mode. For instance, the opening gesture D-A-Bb-A is strongly associated with mode I. The cadence pattern EGFF.FE. is strongly associated with modes III and IV. However, there are exceptions: antiphon ‘Sancti qui sperant’ (matins, many martyrs), begins DA.ABb A, and yet is mode IV. Responsory ‘Absterget Deus’ (matins, many martyrs) has a cadence EGFF.FE. Cases such as this bring to mind the notion of temporary change of modal focus within a single chant–in analogy with modulation in tonal music. So we can think of the antiphon ‘Sancti qui sperant’, for example, as beginning with a focus on Mode I, but concluding with an expression of mode IV. The responsory ‘Absterget Deus’ could be construed as having a temporary shift to Mode IV at the aforementioned cadence. Another example would be the responsory ‘Corde et animo’ (responsory 8, feast of the Nativity of S. Mary and the Conception of St. Mary), where the beginning, up to ‘solennitate’ would seem to be in transposed mode I, while the remainder is clearly in mode VII.
Chains of Thirds
Professor William Mahrt, Stanford University, discusses the organization of Gregorian melodies through chains of thirds, based on A-C-D-F-a-c-e in Sacred Music 136-2 (2009). This article is reprinted in William Mahrt, The Musical Shape of the Liturgy (Richmond, Virginia: Church Music Association of America, 2012): 217-221.
Transposition of motives
Transposition of motives occurs from time to time in the repertoire. I believe that it is a very important compositional feature, not only in terms of developing motivic unity, but also in its latent potential to imply ‘modulation’ and imitation, fundamental aspects of the polyphonic style that developed later.
The Officium for the fourth Sunday after Trinity contains an excellent example. The opening word, ‘Dominus’, set ACD.D.DED, is subsequently repeated with a transposed version of the same melody: DFG.G.GAG. (Incidentally, this appears to be within a broad arpeggiation of the pentatonic set: ACD at ‘Dominus’; CDF at ‘illuminatio’; CDFG at ‘et salus; DFG at ‘Domino’; DFGA at ‘a quo trepidabo; and capping the series at ‘tribulant’ with ‘fa supra la’, DFGAB-flat.)