# Linear Discriminant Analysis Matlab Example

Posted 2012. 6. 9. 17:19

Linear Discriminant Analysis(LDA)

LDA는 PCA와 비슷하지만 class의 개념이 도입되었다는 점에서 다르다. PCA가 모든 data를 scatter시키는 것이 목적이라면 LDA는 between-class variance를 키우고, within-class variance를 최소화하는데 그 목적이 있다.

1. LDA 함수

% LDA - MATLAB subroutine to perform linear discriminant analysis

% by Will Dwinnell and Deniz Sevis

function W = LDA(Input,Target,Priors)

% Determine size of input data

[n m] = size(Input);

% Discover and count unique class labels

ClassLabel = unique(Target);

k = length(ClassLabel);

% Initialize

nGroup     = NaN(k,1);     % Group counts

GroupMean  = NaN(k,m);     % Group sample means

PooledCov  = zeros(m,m);   % Pooled covariance

W          = NaN(k,m+1);   % model coefficients

if  (nargin >= 3)  PriorProb = Priors;  end

% Loop over classes to perform intermediate calculations

for i = 1:k,

% Establish location and size of each class

Group      = (Target == ClassLabel(i));

nGroup(i)  = sum(double(Group));

% Calculate group mean vectors

GroupMean(i,:) = mean(Input(Group,:));

% Accumulate pooled covariance information

PooledCov = PooledCov + ((nGroup(i) - 1) / (n - k) ).* cov(Input(Group,:));

end

% Assign prior probabilities

if  (nargin >= 3)

% Use the user-supplied priors

PriorProb = Priors;

else

% Use the sample probabilities

PriorProb = nGroup / n;

end

% Loop over classes to calculate linear discriminant coefficients

for i = 1:k,

% Intermediate calculation for efficiency

% This replaces:  GroupMean(g,:) * inv(PooledCov)

Temp = GroupMean(i,:) / PooledCov;

% Constant

W(i,1) = -0.5 * Temp * GroupMean(i,:)' + log(PriorProb(i));

% Linear

W(i,2:end) = Temp;

end

% Housekeeping

clear Temp

end

% EOF

2. LDA 테스트

%%

clc;

clear all

close all;

%%

% Generate example data: 2 groups, of 10 and 15, respectively

X = [randn(10,2); randn(15,2) + 1.5];  Y = [zeros(10,1); ones(15,1)];

% Data set 1

X1size = 10;

X1dimension = 2;

X1classIndex = 1;

X1Data = randn(X1size, X1dimension);  %zero mean and unit covariance;

X1Label = X1classIndex*ones(X1size, 1);

% Data set 2

X2size = 15;

X2dimension = 2;

X2classIndex = 2;

X2Data = 1.5 + randn(X2size, X2dimension);

X2Label = X2classIndex*ones(X2size, 1);

% Data set 3

X3size = 15;

X3dimension = 2;

X3classIndex = 3;

X3Data = [4.5*ones(X3size, 1) -2*ones(X3size, 1)] + randn(X3size, X3dimension);

X3Label = X3classIndex*ones(X3size, 1);

% Calculate linear discriminant coefficients

numClass = 3;

Xtotal = [X1Data ; X2Data; X3Data];

Labels = [X1Label ; X2Label; X3Label];

W = LDA(Xtotal, Labels);

% Calulcate linear scores for training data

L = [ones(X1size+X2size+X3size,1) Xtotal] * W';

% Calculate class probabilities

P = exp(L) ./ repmat( sum(exp(L),2), [1 numClass] )

% plot data

hold on;

plot(X1Data(:,1), X1Data(:,2), 'ro');

plot(X2Data(:,1), X2Data(:,2), 'bo');

plot(X3Data(:,1), X3Data(:,2), 'go');

for i = 1:size(Xtotal, 1)

if i <= X1size

P1(i) = P(i, 1);

text(X1Data(i,1), X1Data(i,2), num2str(P1(i)) );

elseif i<= X1size+X2size

P2(i-X1size) = P(i, 2);

text(X2Data(i-X1size,1), X2Data(i-X1size,2), num2str(P2(i-X1size)) );

elseif i<= X1size+X2size+X3size

P3(i-X1size-X2size) = P(i, 3);

text(X3Data(i-X1size-X2size,1), X3Data(i-X1size-X2size,2), num2str(P3(i-X1size-X2size)) );

end

end

title('LDA data set');

legend('Class 1', 'Class 2', 'Class 3');

hold off;

3. 수행 결과

- 각 포인트 옆에 점은 LDA에서 해당 class라고 인식할 확률이다.

#### 'Enginius > Machine Learning' 카테고리의 다른 글

 내가 만든 LDA in Matlab  (2) 2012.06.10 2012.06.09 2012.06.09 2012.06.07 2012.06.05 2012.05.31