MATH590 INF: Information theory

In this module homework we apply the methods of information theory to induce a structure upon the point clouds of probability distributions determined in the previous STC module.

Viscoelastic flow in a convergent channel

Consider the flow of polymers in a convergent channel -1<x<1. The x-velocity of the background flow in the channel is assumed to be known:

u[x_]=(2/π)ArcTan[20x]+2;
Plot[u[x],{x,-1,1},Axes->False,Frame->True,FrameLabel->{"x","u(x)"},GridLines->Automatic,PlotLabel->"Background flow velocity"]

Assuming a dilute aqueous solution, the polymers respond to changes in the liquid velocity by changing their conformation. This process is used, for instance, to stretch DNA molecules prior to identifying base pairs. The simplest model for the complicated internal conformation is a vector Q from one end of the molecule to the other end, the so-called dumbbell model. 

We are interested in extracting the probability distribution functions for Q from data. In practice, this would be obtained by measurements. Here, however, we generate synthetic data from a stochastic process that models the stretching of the polymer by shear within the flow according to the stochastic differential equation

dQ = (η u’(x) Q - k Q) dt + σ dW 

with η,k,σ, model constants that correspond to viscosity, dumbbell stiffness, temperature, respectively. The dW term is a Wiener process modeling the Brownian motion of the dumbbell ends. Note that Q=0 is taken to signify that the “spring” between the two ends of the dumbbell is at its equilibrium, zero-force length

Thermal effects

Taking η=0 leads to a model that is only influenced by temperature dQ = - k Q dt +  σ dW, in which thermal effects force the dumbbell away from its Q=0 equilibrium length (expressed through the mean value μ=0). This is known as an Ornstein-Uhlenbeck process. Generate the synthetic data for 5 instances (i.e., 5 polymers) over 100 time steps of 0.01s.

η=0; k=1; σ=10; μ=0; x=0; nPoly=5;
proc=OrnsteinUhlenbeckProcess[μ,σ,η u'[x]+ k];
db=Table[RandomFunction[proc,{0,1,0.01},2]["ValueList"]\[Transpose],{nPoly}];

Represent the time-evolution of the two components of Q=(Q1,Q2)

ListPlot[db,Joined->True,AspectRatio->Automatic,Frame->True,Axes->False,FrameLabel->{"Q1","Q2"}]

Also construct a direct representation of the evolution of the dumbbell vector itself

For[i=1; dbVec={}, i≤nPoly, i++,AppendTo[dbVec, Graphics[{ColorData[i,"ColorList"][[i]],Map[Line[{{0,0},#}]& ,db[[i]]]}]] ];

Show[dbVec,Frame->True,GridLines->Automatic]

The above can be interpreted as depicting the evolution over 100 time increments of the end-to-end (dumbbell) vector for nPoly=5 different polymers.

Flow effects

Now turn on viscosity, by setting η>0

η=.1; k=1.; σ=10.; μ=0; x=0; nPoly=5;
proc=OrnsteinUhlenbeckProcess[μ,σ,η u'[x]+ k];
db=Table[RandomFunction[proc,{0,1,0.01},2]["ValueList"]\[Transpose],{nPoly}];

Represent the time-evolution of the two components of Q=(Q1,Q2)

ListPlot[db,Joined->True,AspectRatio->Automatic,Frame->True,Axes->False,FrameLabel->{"Q1","Q2"}]

Karhunen-Loeve, SVD

In the previous section, a simplified physical model was used to generate synthetic data as opposed to measured data (e.g., accelerometer measurements, images). Consider now the main problem of interest in this module of extracting an approximation of the PDF of the data. The PDF itself is unknown beforehand, but can be approximated as a multivariate Gaussian distribution through the singular value decomposition (SVD). The goal in this module is to build a point cloud of such multivariate Gaussian approximations. In subsequent modules, the point cloud is used to construct non-Gaussian PDF approximants.

Generate data

η=.1; k=1.; σ=10.; μ=0; x=0; nPoly=50;
proc=OrnsteinUhlenbeckProcess[μ,σ,η u'[x]+ k];
data=Flatten[Table[RandomFunction[proc,{0,1,0.01},2]["ValueList"]\[Transpose],{nPoly}],1];

m=Mean[data];
X=Map[Plus[#,-m]&,data];
{Ns,Nd}=Dimensions[X]

Compute the SVD, limited to two modes. The singular values specify the data variance

{U,S,V}=SingularValueDecomposition[X,2];
MatrixForm[N[(S^2)/Ns]]

Homework questions

Question 1

Use data from different viscosities η=10^(-k), k=1,2,3,4,5. Plot the evolution of the information I for Qx,Qy (the two components of the dumbbell vector) with axial coordinate x for each viscosity.

Question 2

As above, plot the average information, e.g., the entropy S for Qx,Qy.

Question 3

As above, plot the joint entropy of the two components (Qx,Qy).

Question 4

As above, plot the mutual information of Qx and Qy

Question 5

Form a differential equation relating the slope of the channel to estimates of the mean of Qx,Qy. Solve the ODE and compare to the imposed channel shape.