Tensorflow (version 2 code) - Universal Approximation Theorem & Statsmodels

Universal Approximation Theorem

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import SGD
from tensorflow.keras import optimizers
import math

Target function

np.linspace(0, 10, 100) # 최소, 최대, 균등한 간격

x = np.linspace(-10, 10, 100)
y = 3 * np.sin(x) * np.cos(x) * (6 * x**2 + 3 * x**3 + x**1) * np.tan(x)

plt.plot(x, y)
plt.show()

함수가 아무리 복잡해도 인공신경망은 어느 정도 함수를 모사할 수 있다.

 

Approximation with 4-layered NN

model = Sequential()
# model.add(Dense(10, input_shape=(1,), activation='relu'))
# model.add(Dense(10, input_shape=(1,), activation='relu'))
# model.add(Dense(10, input_shape=(1,), activation='relu'))
# model.add(Dense(10, input_shape=(1,), activation='relu'))
# model.add(Dense(1, activation='linear'))

model.add(Dense(10, input_dim=1, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation=None))

rms = optimizers.RMSprop(learning_rate=0.01, rho=0.9, epsilon=1e-08, decay=0.0)
model.compile(loss='mean_squared_error', optimizer=rms, metrics=['accuracy'])
model.fit(x, y, verbose=0, epochs=3000)

prediction = model.predict(x)
plt.plot(x, prediction)
plt.show()

Approximation with 1-layered NN

model = Sequential()
model.add(Dense(10000, input_dim=1, activation='relu'))
model.add(Dense(1, activation=None))

rms = optimizers.RMSprop(learning_rate=0.06, rho=0.9, epsilon=1e-08, decay=0.0)
model.compile(loss='mean_squared_error', optimizer=rms, metrics=['accuracy'])
model.fit(x, y, verbose=0, epochs=5000)

prediction = model.predict(x)
plt.plot(x, prediction)
plt.show()

---

9. (Appendix 2) Linear Regression & Logistic Regression with TF.keras & Statsmodels
추가 예정