Motifs: Art and Mathematics

Sorin Mitran (author@unc.edu)

University of North Carolina at Chapel Hill

SetDirectory[NotebookDirectory[]];

Introduction

Recurring musical phrases, repetition of brush strokes, a predilection for proof by contradiction are examples of motifs for the composer, painter,  mathematician. Such idiosyncracies are contributory to a distinctive style, but to what extent are these peculiarities automatisms as opposed to flashes of inspiration? Could one disentangle the inspirational from the mechanical in some universal way? Composition, tone, space and shape are concepts within art, mathematics and music, albeit with a different meaning in each field. Could one find some universal way to distinguish between these elements in a symphony, painting or theorem? 

The theory and practice of artificial neural networks (ANNs), sequences of mathematical operations inspired by observations of synapses in the brain, allows one to consider the above questions. When furnished with a large amount of data ANNs can be trained to mimic what human intelligence perceives as a motif. The processes is however somewhat haphazard, and the much more advanced neural networks of the musician, artist or mathematician are ultimately required to guide the training. Nonetheless, remarkable results can be achieved to the point that one can consider ANNs as a tool to produce new works of music, art or mathematics much in the way of an instrument, brush or axiomatic system is used.

Here is an example of transferring Van Gogh’s distinctive brush strokes to one of Mondrian’s pieces. First, some operations are defined to read a database of images, invoke the training of an ANN to mimic the style of Van Gogh, and apply it to another image.

TransferStyle[p1_,p2_,w_,F1_,F2_]:=Module[{im,im1,im2},
im1=Import[F1[[p1]]]; im2=Import[F2[[p2]]];
im=ImageRestyle[im1,w->im2];
Return[{im1,im2,im}]]

SetDirectory["~/courses/MATH547ML/data/paintings"];
MondrianPaintings=FileNames["Piet_Mondrian*"];
VanGoghPaintings=FileNames["Vincent*"];
nMondrianPaintings = Length[MondrianPaintings];
nVanGoghPaintings= Length[VanGoghPaintings];
{nMondrianPaintings,nVanGoghPaintings}

Invoking the above definitions leads to the following results

TransferStyle[9,288,.5,MondrianPaintings,VanGoghPaintings]

There is clearly a resemblance in the composite image to the original sources. It also clearly has the air of something mechanical in the transfer of color and tone to the underlying composition of the Mondrian original.

The neural network should be guided to produce a more convincing synthesis. One option is to choose the data sets to distinctly focus on overall composition or style. For example, this transfer of Miro’s style to a photograph.

ImageRestyle[, ]

Another, much more interesting option, is to use the expert knowledge in mathematics, music and visual art to guide the training process, and bring out specific elements of interest. This is the main interest of this course which combines quantitative reasoning (QR general education requirement) with the approaches of the visual and performing arts (VP general education requirement).