MATH590 STC: Stochastic processes

Our goal in this module is to find concise descriptions of data through probability distributions of the observations. Rather than cluster the observed data points, these are interpreted as arising from a stochastic process, and the goal is to find the probability density that has generated the points.

Brownian motion

Examples of Brownian motion in 2D

A single path

μ=1; σ=10;
proc=WienerProcess[μ,σ];
rw=Table[RandomFunction[proc,{0,1,0.001},2]["ValueList"]\[Transpose],{1}];
ListPlot[rw,Joined->True,AspectRatio->Automatic,Frame->True,Axes->False]

Multiple paths

μ=1; σ=1;
proc=WienerProcess[μ,σ];
rw=Table[RandomFunction[proc,{0,1,0.001},2]["ValueList"]\[Transpose],{20}];
ListPlot[rw,Joined->True,AspectRatio->Automatic,Frame->True,Axes->False]

Karhunen-Loeve, SVD

Generate normally distributed data

Ns=100; (* Number of samples *)
μ={0,0}; (* Means *)
Σ={{1,0},{0,1}}; (* Covariance matrix *)
data=RandomVariate[MultinormalDistribution[μ,Σ],Ns];
ListPlot[data,AspectRatio->Automatic,Frame->True]

Could standard clustering (i.e., set partitioning methods) efficiently describe the data?

clusters=FindClusters[data];
Nclusters=Length[clusters]

ListPlot[clusters]

Switch now to description in PDF space. Seek a fit to a normal distribution

fit=FindDistributionParameters[data,MultinormalDistribution[{m1,m2},{{S11,S12},{S21,S22}}]]

Notice that the fit, even knowing that the distribution is Gaussian, does not recover the zero mean initially imposed.

m=Mean[data]

Center the data

X=Map[Plus[#,-m]&,data];
Dimensions[X]

Compute the SVD, limited to two modes. The singular values specify the data variance

{U,S,V}=SingularValueDecomposition[X,2];
MatrixForm[N[(S^2)/Ns]]

The covariance matrix obtained by the fit to the data is

CG={{S11,S12},{S21,S22}} /. fit;
MatrixForm[CG]

Reduction to diagonal form recovers the variances

Eigenvalues[CG]

Explore the convergence rate of the fit to the imposed mean and variances through a function

GaussianDataFit[μ_,Σ_,Ns_]:=Module[{data,fit,m,n,U,S,V,Λ,ΣS},
data=RandomVariate[MultinormalDistribution[μ,Σ],Ns];
fit=FindDistributionParameters[data,MultinormalDistribution[{m1,m2},{{S11,S12},{S21,S22}}]];
m=Mean[data];
X=Map[Plus[#,-m]&,data];
n=Length[μ];
{U,S,V}=SingularValueDecomposition[X,n];
ΣS=Diagonal[N[(S^2)/Ns]];
CG={{S11,S12},{S21,S22}} /. fit;
Λ=Eigenvalues[CG];
Return[{μ,m,ΣS,Λ}]
]

MatrixForm@GaussianDataFit[{0,0},{{1,0},{0,1}},100]