Linear Discriminant Analysis Matlab Example

Linear Discriminant Analysis(LDA)

 LDA는 PCA와 비슷하지만 class의 개념이 도입되었다는 점에서 다르다. PCA가 모든 data를 scatter시키는 것이 목적이라면 LDA는 between-class variance를 키우고, within-class variance를 최소화하는데 그 목적이 있다. 

1. LDA 함수 

% LDA - MATLAB subroutine to perform linear discriminant analysis

% by Will Dwinnell and Deniz Sevis

function W = LDA(Input,Target,Priors)

% Determine size of input data

[n m] = size(Input);

% Discover and count unique class labels

ClassLabel = unique(Target);

k = length(ClassLabel);

% Initialize

nGroup     = NaN(k,1);     % Group counts

GroupMean  = NaN(k,m);     % Group sample means

PooledCov  = zeros(m,m);   % Pooled covariance

W          = NaN(k,m+1);   % model coefficients

if  (nargin >= 3)  PriorProb = Priors;  end

% Loop over classes to perform intermediate calculations

for i = 1:k,

    % Establish location and size of each class

    Group      = (Target == ClassLabel(i));

    nGroup(i)  = sum(double(Group));

    

    % Calculate group mean vectors

    GroupMean(i,:) = mean(Input(Group,:));

    

    % Accumulate pooled covariance information

    PooledCov = PooledCov + ((nGroup(i) - 1) / (n - k) ).* cov(Input(Group,:));

end

% Assign prior probabilities

if  (nargin >= 3)

    % Use the user-supplied priors

    PriorProb = Priors;

else

    % Use the sample probabilities

    PriorProb = nGroup / n;

end

% Loop over classes to calculate linear discriminant coefficients

for i = 1:k,

    % Intermediate calculation for efficiency

    % This replaces:  GroupMean(g,:) * inv(PooledCov)

    Temp = GroupMean(i,:) / PooledCov;

    

    % Constant

    W(i,1) = -0.5 * Temp * GroupMean(i,:)' + log(PriorProb(i));

    

    % Linear

    W(i,2:end) = Temp;

end

% Housekeeping

clear Temp

end

% EOF

2. LDA 테스트 

%%

clc;

clear all

close all;

%%

% Generate example data: 2 groups, of 10 and 15, respectively

X = [randn(10,2); randn(15,2) + 1.5];  Y = [zeros(10,1); ones(15,1)];

% Data set 1

X1size = 10;

X1dimension = 2;

X1classIndex = 1;

X1Data = randn(X1size, X1dimension);  %zero mean and unit covariance;

X1Label = X1classIndex*ones(X1size, 1);

% Data set 2

X2size = 15;

X2dimension = 2;

X2classIndex = 2;

X2Data = 1.5 + randn(X2size, X2dimension);

X2Label = X2classIndex*ones(X2size, 1);

% Data set 3

X3size = 15;

X3dimension = 2;

X3classIndex = 3;

X3Data = [4.5*ones(X3size, 1) -2*ones(X3size, 1)] + randn(X3size, X3dimension);

X3Label = X3classIndex*ones(X3size, 1);

% Calculate linear discriminant coefficients

numClass = 3;

Xtotal = [X1Data ; X2Data; X3Data];

Labels = [X1Label ; X2Label; X3Label];

W = LDA(Xtotal, Labels);

% Calulcate linear scores for training data

L = [ones(X1size+X2size+X3size,1) Xtotal] * W';

% Calculate class probabilities

P = exp(L) ./ repmat( sum(exp(L),2), [1 numClass] )

% plot data

hold on;

plot(X1Data(:,1), X1Data(:,2), 'ro');

plot(X2Data(:,1), X2Data(:,2), 'bo');

plot(X3Data(:,1), X3Data(:,2), 'go');

for i = 1:size(Xtotal, 1)

    if i <= X1size

        P1(i) = P(i, 1);

        text(X1Data(i,1), X1Data(i,2), num2str(P1(i)) );

    elseif i<= X1size+X2size

        P2(i-X1size) = P(i, 2);

        text(X2Data(i-X1size,1), X2Data(i-X1size,2), num2str(P2(i-X1size)) );

    elseif i<= X1size+X2size+X3size

        P3(i-X1size-X2size) = P(i, 3);

        text(X3Data(i-X1size-X2size,1), X3Data(i-X1size-X2size,2), num2str(P3(i-X1size-X2size)) );

    end

end

title('LDA data set');

legend('Class 1', 'Class 2', 'Class 3');

hold off;

3. 수행 결과 

 - 각 포인트 옆에 점은 LDA에서 해당 class라고 인식할 확률이다.