# Python机器学习（四）：PCA 主成分分析

Jacob的 Python机器学习系列：
Python机器学习（一）：kNN算法
Python机器学习（二）：线性回归算法
Python机器学习（三）：梯度下降法
Python机器学习（四）：PCA 主成分分析
Python机器学习（五）：SVM 支撑向量机

### 求解目标

1. 对样本进行demean处理（使所有样本的均值为0）
2. 取一个轴的方向 w = （w1，w2...，wn），使我们的样本，映射到w之后，使下式最大

均方差

w为单位向量，则有

### 编程实现

``````"""
Created by 杨帮杰 on 11/4/2018
Right to use this code in any way you want without
warranty, support or any guarantee of it working
E-mail: yangbangjie1998@qq.com
Association: SCAU 华南农业大学
"""
import numpy as np

class PCA:

def __init__(self, n_components):
"""初始化PCA"""
assert n_components >= 1, "n_components must be valid"
self.n_components = n_components
self.components_ = None

def fit(self, X, eta=0.01, n_iters=1e4):
"""获得数据集的前n个主成分"""
assert self.n_components <= X.shape[1], \
"n_components must not be greater than the feature number of X"

def demean(X):
return X - np.mean(X, axis=0)

def f(w, X):
return np.sum((X.dot(w) ** 2)) / len(X)

def direction(w):
return w / np.linalg.norm(w)

def first_components(X, initial_w, eta=0.01, n_iters=1e4, epsilon=1e-8):
w = direction(initial_w)
cur_iter = 0

while cur_iter < n_iters:
last_w = w
w = direction(w)
if(abs(f(w, X) - f(last_w, X)) < epsilon):
break
cur_iter += 1

return w

X_pca = demean(X)
self.components_ = np.empty(shape=(self.n_components, X.shape[1]))
for i in range(self.n_components):
initial_w = np.random.random(X_pca.shape[1])
w = first_components(X_pca, initial_w, eta, n_iters)
self.components_[i,:] = w
X_pca = X_pca - X_pca.dot(w).reshape(-1,1) * w

return self

def transform(self, X):
"""将给定的X，映射到各个主成分分量中"""
assert X.shape[1] == self.components_.shape[1]

return X.dot(self.components_.T)

def inverse_transform(self, X):
"""将给定的X，反向映射回原来的特征空间"""
assert X.shape[1] == self.components_.shape[0]

return X.dot(self.components_)

def __repr__(self):
return "PCA(n_components = %d" % self.n_components

``````

References:
Python3 入门机器学习 经典算法与应用 —— liuyubobobo