%md
(raaz) Thus, the perceptron is defined by:

$$
f(1, x\_1,x\_2,\ldots , x\_n \, ; \, w\_0,w\_1,w\_2,\ldots , w\_n) =
\begin{cases}
1 & \text{if} \quad \sum\_{i=0}^n w\_i x\_i > 0 \\\\
0 & \text{otherwise}
\end{cases}
$$
and implementable with the following arithmetical and logical unit (ALU) operations in a machine:

 * n inputs from one \\(n\\)-dimensional data point: \\( x_1,x_2,\ldots x_n \, \in \, \mathbb{R}^n\\)
 * arithmetic operations
   * n+1 multiplications
   * n additions
* boolean operations
  * one if-then on an inequality
* one output \\(o \in \\{0,1\\}\\), i.e., \\(o\\) belongs to the set containing \\(0\\) and \\(1\\)
* n+1 parameters of interest

This is just a hyperplane given by a dot product of \\(n+1\\) known inputs and \\(n+1\\) unknown parameters that can be estimated. This hyperplane can be used to define a hyperplane that partitions \\(\mathbb{R}^{n+1}\\), the real Euclidean space, into two parts labelled by the outputs \\(0\\) and \\(1\\).

The problem of finding estimates of the parameters, \\( (\hat{w}\_0,\hat{w}\_1,\hat{w}\_2,\ldots \hat{w}\_n) \in \mathbb{R}^{(n+1)} \\), in some statistically meaningful manner for a predicting task by using the training data given by, say \\(k\\) *labelled points*, where you know both the input and output:
$$
 \left( ( \, 1, x\_1^{(1)},x\_2^{(1)}, \ldots x\_n^{(1)}), (o^{(1)}) \, ), \, ( \, 1, x\_1^{(2)},x\_2^{(2)}, \ldots x\_n^{(2)}), (o^{(2)}) \, ), \, \ldots \, , ( \, 1, x\_1^{(k)},x\_2^{(k)}, \ldots x\_n^{(k)}), (o^{(k)}) \, ) \right) \, \in \, (\mathbb{R}^{n+1} \times \\{ 0,1 \\} )^k
$$
is the machine learning problem here. 

Succinctly, we are after a random mapping, denoted below by \\( \mapsto\_{\rightsquigarrow} \\), called the *estimator*:
$$
(\mathbb{R}^{n+1} \times \\{0,1\\})^k \mapsto_{\rightsquigarrow} \, \left( \, \mathtt{model}( (1,x\_1,x\_2,\ldots,x\_n) \,;\, (\hat{w}\_0,\hat{w}\_1,\hat{w}\_2,\ldots \hat{w}\_n)) : \mathbb{R}^{n+1} \to \\{0,1\\} \,  \right)
$$
which takes *random* labelled dataset (to understand random here think of two scientists doing independent experiments to get their own training datasets) of size \\(k\\) and returns a *model*. These mathematical notions correspond exactly to the `estimator` and `model` (which is a `transformer`) in the language of Apache Spark's Machine Learning Pipleines we have seen before.

We can use this `transformer` for *prediction* of *unlabelled data* where we only observe the input and what to know the output under some reasonable assumptions.  

Of course we want to be able to generalize so we don't overfit to the training data using some *empirical risk minisation rule* such as cross-validation. Again, we have seen these in Apache Spark for other ML methods like linear regression and decision trees.

(raaz) Thus, the perceptron is defined by:

$f(1, x_1,x_2,\ldots , x_n \, ; \, w_0,w_1,w_2,\ldots , w_n) = \begin{cases} 1 & \text{if} \quad \sum_{i=0}^n w_i x_i > 0 \\ 0 & \text{otherwise} \end{cases}$ and implementable with the following arithmetical and logical unit (ALU) operations in a machine:

n inputs from one $n$ -dimensional data point: $x_1,x_2,\ldots x_n \, \in \, \mathbb{R}^n$
arithmetic operations
- n+1 multiplications
- n additions
- boolean operations
- one if-then on an inequality
- one output $o \in \{0,1\}$ , i.e., $o$ belongs to the set containing $0$ and $1$
- n+1 parameters of interest

This is just a hyperplane given by a dot product of $n+1$ known inputs and $n+1$ unknown parameters that can be estimated. This hyperplane can be used to define a hyperplane that partitions $\mathbb{R}^{n+1}$ , the real Euclidean space, into two parts labelled by the outputs $0$ and $1$ .

The problem of finding estimates of the parameters, $(\hat{w}_0,\hat{w}_1,\hat{w}_2,\ldots \hat{w}_n) \in \mathbb{R}^{(n+1)}$ , in some statistically meaningful manner for a predicting task by using the training data given by, say $k$ labelled points, where you know both the input and output: $\left( ( \, 1, x_1^{(1)},x_2^{(1)}, \ldots x_n^{(1)}), (o^{(1)}) \, ), \, ( \, 1, x_1^{(2)},x_2^{(2)}, \ldots x_n^{(2)}), (o^{(2)}) \, ), \, \ldots \, , ( \, 1, x_1^{(k)},x_2^{(k)}, \ldots x_n^{(k)}), (o^{(k)}) \, ) \right) \, \in \, (\mathbb{R}^{n+1} \times \{ 0,1 \} )^k$ is the machine learning problem here.

Succinctly, we are after a random mapping, denoted below by $\mapsto_{\rightsquigarrow}$ , called the estimator: $(\mathbb{R}^{n+1} \times \{0,1\})^k \mapsto_{\rightsquigarrow} \, \left( \, \mathtt{model}( (1,x_1,x_2,\ldots,x_n) \,;\, (\hat{w}_0,\hat{w}_1,\hat{w}_2,\ldots \hat{w}_n)) : \mathbb{R}^{n+1} \to \{0,1\} \, \right)$ which takes random labelled dataset (to understand random here think of two scientists doing independent experiments to get their own training datasets) of size $k$ and returns a model. These mathematical notions correspond exactly to the estimator and model (which is a transformer) in the language of Apache Spark's Machine Learning Pipleines we have seen before.

We can use this transformer for prediction of unlabelled data where we only observe the input and what to know the output under some reasonable assumptions.

Of course we want to be able to generalize so we don't overfit to the training data using some empirical risk minisation rule such as cross-validation. Again, we have seen these in Apache Spark for other ML methods like linear regression and decision trees.

If the output isn't right, we can adjust the weights, threshold, or bias ( $x_0$ above)

The model was inspired by discoveries about the neurons of animals, so hopes were quite high that it could lead to a sophisticated machine. This model can be extended by adding multiple neurons in parallel. And we can use linear output instead of a threshold if we like for the output.

If we were to do so, the output would look like ${x \cdot w} + w_0$ (this is where the vector multiplication and, eventually, matrix multiplication, comes in)

When we look at the math this way, we see that despite this being an interesting model, it's really just a fancy linear calculation.

And, in fact, the proof that this model -- being linear -- could not solve any problems whose solution was nonlinear ... led to the first of several "AI / neural net winters" when the excitement was quickly replaced by disappointment, and most research was abandoned.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_squared_error

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)

import IPython.display as disp
pd.set_option('display.width', 200)
disp.display(df[:10])

Unnamed: 0 carat cut color clarity depth table price x y z 0 1 0.23 Ideal E SI2 61.5 55.0 326 3.95 3.98 2.43 1 2 0.21 Premium E SI1 59.8 61.0 326 3.89 3.84 2.31 2 3 0.23 Good E VS1 56.9 65.0 327 4.05 4.07 2.31 3 4 0.29 Premium I VS2 62.4 58.0 334 4.20 4.23 2.63 4 5 0.31 Good J SI2 63.3 58.0 335 4.34 4.35 2.75 5 6 0.24 Very Good J VVS2 62.8 57.0 336 3.94 3.96 2.48 6 7 0.24 Very Good I VVS1 62.3 57.0 336 3.95 3.98 2.47 7 8 0.26 Very Good H SI1 61.9 55.0 337 4.07 4.11 2.53 8 9 0.22 Fair E VS2 65.1 61.0 337 3.87 3.78 2.49 9 10 0.23 Very Good H VS1 59.4 61.0 338 4.00 4.05 2.39

df2 = df.drop(df.columns[0], axis=1)

disp.display(df2[:3])

carat cut color clarity depth table price x y z 0 0.23 Ideal E SI2 61.5 55.0 326 3.95 3.98 2.43 1 0.21 Premium E SI1 59.8 61.0 326 3.89 3.84 2.31 2 0.23 Good E VS1 56.9 65.0 327 4.05 4.07 2.31

df3 = pd.get_dummies(df2) # this gives a one-hot encoding of categorial variables

disp.display(df3[range(7,18)][:3])

cut_Fair cut_Good cut_Ideal cut_Premium cut_Very Good color_D color_E color_F color_G color_H color_I 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 2 0 1 0 0 0 0 1 0 0 0 0

# pre-process to get y
y = df3.iloc[:,3:4].as_matrix().flatten()
y.flatten()

# preprocess and reshape X as a matrix
X = df3.drop(df3.columns[3], axis=1).as_matrix()
np.shape(X)

# break the dataset into training and test set with a 75% and 25% split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Define a decisoin tree model with max depth 10
dt = DecisionTreeRegressor(random_state=0, max_depth=10)

# fit the decision tree to the training data to get a fitted model
model = dt.fit(X_train, y_train)

# predict the features or X values of the test data using the fitted model
y_pred = model.predict(X_test)

# print the MSE performance measure of the fit by comparing the predicted versus the observed values of y 
print("RMSE %f" % np.sqrt(mean_squared_error(y_test, y_pred)) )

RMSE 726.921870

from sklearn import linear_model

# Do the same with linear regression and not a worse MSE
lr = linear_model.LinearRegression()
linear_model = lr.fit(X_train, y_train)

y_pred = linear_model.predict(X_test)
print("RMSE %f" % np.sqrt(mean_squared_error(y_test, y_pred)) )

RMSE 1124.086095

Neural Network with Keras

Keras is a High-Level API for Neural Networks and Deep Learning

"Being able to go from idea to result with the least possible delay is key to doing good research."

Maintained by Francois Chollet at Google, it provides

High level APIs
Pluggable backends for Theano, TensorFlow, CNTK, MXNet
CPU/GPU support
The now-officially-endorsed high-level wrapper for TensorFlow; a version ships in TF
Model persistence and other niceties
JavaScript, iOS, etc. deployment
Interop with further frameworks, like DeepLearning4J, Spark DL Pipelines ...

Well, with all this, why would you ever not use Keras?

As an API/Facade, Keras doesn't directly expose all of the internals you might need for something custom and low-level ... so you might need to implement at a lower level first, and then perhaps wrap it to make it easily usable in Keras.

Mr. Chollet compiles stats (roughly quarterly) on "[t]he state of the deep learning landscape: GitHub activity of major libraries over the past quarter (tickets, forks, and contributors)."

(October 2017: https://twitter.com/fchollet/status/915366704401719296; https://twitter.com/fchollet/status/915626952408436736)

GitHub

Research

Keras has wide adoption in industry

from keras.models import Sequential
from keras.layers import Dense

# we are going to add layers sequentially one after the other (feed-forward) to our neural network model
model = Sequential()

# the first layer has 30 nodes (or neurons) with input dimension 26 for our diamonds data
# we will use Nomal or Guassian kernel to initialise the weights we want to estimate
# our activation function is linear (to mimic linear regression)
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='linear'))
# the next layer is for the response y and has only one node
model.add(Dense(1, kernel_initializer='normal', activation='linear'))
# compile the model with other specifications for loss and type of gradient descent optimisation routine
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
# fit the model to the training data using stochastic gradient descent with a batch-size of 200 and 10% of data held out for validation
history = model.fit(X_train, y_train, epochs=10, batch_size=200, validation_split=0.1)

scores = model.evaluate(X_test, y_test)
print()
print("test set RMSE: %f" % np.sqrt(scores[1]))

Train on 36409 samples, validate on 4046 samples Epoch 1/10 200/36409 [..............................] - ETA: 38s - loss: 28101556.0000 - mean_squared_error: 28101556.0000 9800/36409 [=======>......................] - ETA: 0s - loss: 31413143.2653 - mean_squared_error: 31413143.2653 19200/36409 [==============>...............] - ETA: 0s - loss: 30940913.6250 - mean_squared_error: 30940913.6250 29000/36409 [======================>.......] - ETA: 0s - loss: 31018350.3862 - mean_squared_error: 31018350.3862 36409/36409 [==============================] - 0s 12us/step - loss: 30697648.5569 - mean_squared_error: 30697648.5569 - val_loss: 30116203.9614 - val_mean_squared_error: 30116203.9614 Epoch 2/10 200/36409 [..............................] - ETA: 0s - loss: 37737276.0000 - mean_squared_error: 37737276.0000 10000/36409 [=======>......................] - ETA: 0s - loss: 29111447.6400 - mean_squared_error: 29111447.6400 19600/36409 [===============>..............] - ETA: 0s - loss: 27750291.1429 - mean_squared_error: 27750291.1429 29400/36409 [=======================>......] - ETA: 0s - loss: 26848646.5850 - mean_squared_error: 26848646.5850 36409/36409 [==============================] - 0s 6us/step - loss: 26212353.8018 - mean_squared_error: 26212353.8018 - val_loss: 23821286.3945 - val_mean_squared_error: 23821286.3945 Epoch 3/10 200/36409 [..............................] - ETA: 0s - loss: 25278296.0000 - mean_squared_error: 25278296.0000 10000/36409 [=======>......................] - ETA: 0s - loss: 21888158.5600 - mean_squared_error: 21888158.5600 20000/36409 [===============>..............] - ETA: 0s - loss: 21141107.9400 - mean_squared_error: 21141107.9400 29400/36409 [=======================>......] - ETA: 0s - loss: 20615190.4082 - mean_squared_error: 20615190.4082 36409/36409 [==============================] - 0s 5us/step - loss: 20059579.3258 - mean_squared_error: 20059579.3258 - val_loss: 18364091.3015 - val_mean_squared_error: 18364091.3015 Epoch 4/10 200/36409 [..............................] - ETA: 0s - loss: 18400482.0000 - mean_squared_error: 18400482.0000 9800/36409 [=======>......................] - ETA: 0s - loss: 17198346.4490 - mean_squared_error: 17198346.4490 19800/36409 [===============>..............] - ETA: 0s - loss: 16747464.6162 - mean_squared_error: 16747464.6162 29800/36409 [=======================>......] - ETA: 0s - loss: 16521444.1812 - mean_squared_error: 16521444.1812 36409/36409 [==============================] - 0s 5us/step - loss: 16322453.1544 - mean_squared_error: 16322453.1544 - val_loss: 16167590.7874 - val_mean_squared_error: 16167590.7874 Epoch 5/10 200/36409 [..............................] - ETA: 0s - loss: 18818984.0000 - mean_squared_error: 18818984.0000 9800/36409 [=======>......................] - ETA: 0s - loss: 15261798.2653 - mean_squared_error: 15261798.2653 19200/36409 [==============>...............] - ETA: 0s - loss: 15163629.7708 - mean_squared_error: 15163629.7708 28800/36409 [======================>.......] - ETA: 0s - loss: 15220626.7222 - mean_squared_error: 15220626.7222 36409/36409 [==============================] - 0s 6us/step - loss: 15258819.0238 - mean_squared_error: 15258819.0238 - val_loss: 15734077.6787 - val_mean_squared_error: 15734077.6787 Epoch 6/10 200/36409 [..............................] - ETA: 0s - loss: 16899202.0000 - mean_squared_error: 16899202.0000 10000/36409 [=======>......................] - ETA: 0s - loss: 15390067.6200 - mean_squared_error: 15390067.6200 19800/36409 [===============>..............] - ETA: 0s - loss: 15157950.5758 - mean_squared_error: 15157950.5758 29400/36409 [=======================>......] - ETA: 0s - loss: 15025312.4014 - mean_squared_error: 15025312.4014 36409/36409 [==============================] - 0s 5us/step - loss: 15069842.6453 - mean_squared_error: 15069842.6453 - val_loss: 15626440.4513 - val_mean_squared_error: 15626440.4513 Epoch 7/10 200/36409 [..............................] - ETA: 0s - loss: 14186056.0000 - mean_squared_error: 14186056.0000 9600/36409 [======>.......................] - ETA: 0s - loss: 15303948.8750 - mean_squared_error: 15303948.8750 19400/36409 [==============>...............] - ETA: 0s - loss: 15229250.0412 - mean_squared_error: 15229250.0412 29200/36409 [=======================>......] - ETA: 0s - loss: 14990177.5753 - mean_squared_error: 14990177.5753 36409/36409 [==============================] - 0s 6us/step - loss: 14985261.6362 - mean_squared_error: 14985261.6362 - val_loss: 15538706.0652 - val_mean_squared_error: 15538706.0652 Epoch 8/10 200/36409 [..............................] - ETA: 0s - loss: 10689206.0000 - mean_squared_error: 10689206.0000 10200/36409 [=======>......................] - ETA: 0s - loss: 15100902.6667 - mean_squared_error: 15100902.6667 20000/36409 [===============>..............] - ETA: 0s - loss: 14800498.9400 - mean_squared_error: 14800498.9400 29800/36409 [=======================>......] - ETA: 0s - loss: 14935705.7517 - mean_squared_error: 14935705.7517 36409/36409 [==============================] - 0s 5us/step - loss: 14898895.5659 - mean_squared_error: 14898895.5659 - val_loss: 15443912.3772 - val_mean_squared_error: 15443912.3772 Epoch 9/10 200/36409 [..............................] - ETA: 0s - loss: 20342628.0000 - mean_squared_error: 20342628.0000 10000/36409 [=======>......................] - ETA: 0s - loss: 14904159.1200 - mean_squared_error: 14904159.1200 20000/36409 [===============>..............] - ETA: 0s - loss: 14663626.4700 - mean_squared_error: 14663626.4700 29600/36409 [=======================>......] - ETA: 0s - loss: 14824327.7297 - mean_squared_error: 14824327.7297 36409/36409 [==============================] - 0s 5us/step - loss: 14804130.4193 - mean_squared_error: 14804130.4193 - val_loss: 15341353.5571 - val_mean_squared_error: 15341353.5571 Epoch 10/10 200/36409 [..............................] - ETA: 0s - loss: 15346465.0000 - mean_squared_error: 15346465.0000 9800/36409 [=======>......................] - ETA: 0s - loss: 14485484.7347 - mean_squared_error: 14485484.7347 19400/36409 [==============>...............] - ETA: 0s - loss: 14679624.3711 - mean_squared_error: 14679624.3711 29200/36409 [=======================>......] - ETA: 0s - loss: 14811040.3562 - mean_squared_error: 14811040.3562 36409/36409 [==============================] - 0s 5us/step - loss: 14701344.4206 - mean_squared_error: 14701344.4206 - val_loss: 15231073.3737 - val_mean_squared_error: 15231073.3737 32/13485 [..............................] - ETA: 0s 4544/13485 [=========>....................] - ETA: 0s 9024/13485 [===================>..........] - ETA: 0s 13485/13485 [==============================] - 0s 11us/step () test set RMSE: 3800.703217

model.summary() # do you understand why the number of parameters in layer 1 is 810? 26*30+30=810

_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_9 (Dense) (None, 30) 810 _________________________________________________________________ dense_10 (Dense) (None, 1) 31 ================================================================= Total params: 841 Trainable params: 841 Non-trainable params: 0 _________________________________________________________________

y.std()

Out[22]: 3989.4027576288736

import matplotlib.pyplot as plt

fig, ax = plt.subplots()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper left')

display(fig)

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].as_matrix().flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).as_matrix()
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='linear'))
model.add(Dense(1, kernel_initializer='normal', activation='linear'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=250, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples Epoch 1/250 - 1s - loss: 28491219.3217 - mean_squared_error: 28491219.3217 - val_loss: 23885769.4177 - val_mean_squared_error: 23885769.4177 Epoch 2/250 - 0s - loss: 18262129.7388 - mean_squared_error: 18262129.7388 - val_loss: 16212928.3584 - val_mean_squared_error: 16212928.3584 Epoch 3/250 - 0s - loss: 15174516.9295 - mean_squared_error: 15174516.9295 - val_loss: 15626628.9160 - val_mean_squared_error: 15626628.9160 Epoch 4/250 - 0s - loss: 14944525.2839 - mean_squared_error: 14944525.2839 - val_loss: 15449083.0371 - val_mean_squared_error: 15449083.0371 Epoch 5/250 - 0s - loss: 14764672.5734 - mean_squared_error: 14764672.5734 - val_loss: 15251654.9273 - val_mean_squared_error: 15251654.9273 Epoch 6/250 - 0s - loss: 14556759.2726 - mean_squared_error: 14556759.2726 - val_loss: 15016743.5522 - val_mean_squared_error: 15016743.5522 Epoch 7/250 - 0s - loss: 14311102.1074 - mean_squared_error: 14311102.1074 - val_loss: 14741243.2620 - val_mean_squared_error: 14741243.2620 Epoch 8/250 - 0s - loss: 14017543.1143 - mean_squared_error: 14017543.1143 - val_loss: 14406070.2224 - val_mean_squared_error: 14406070.2224 Epoch 9/250 - 0s - loss: 13657812.0115 - mean_squared_error: 13657812.0115 - val_loss: 13995799.2556 - val_mean_squared_error: 13995799.2556 Epoch 10/250 - 0s - loss: 13217894.3215 - mean_squared_error: 13217894.3215 - val_loss: 13491442.5660 - val_mean_squared_error: 13491442.5660 Epoch 11/250 - 0s - loss: 12674428.2858 - mean_squared_error: 12674428.2858 - val_loss: 12880994.0761 - val_mean_squared_error: 12880994.0761 Epoch 12/250 - 0s - loss: 12016912.5950 - mean_squared_error: 12016912.5950 - val_loss: 12131363.7029 - val_mean_squared_error: 12131363.7029 Epoch 13/250 - 0s - loss: 11239141.9037 - mean_squared_error: 11239141.9037 - val_loss: 11271771.9145 - val_mean_squared_error: 11271771.9145 Epoch 14/250 - 0s - loss: 10360925.0985 - mean_squared_error: 10360925.0985 - val_loss: 10324658.8547 - val_mean_squared_error: 10324658.8547 Epoch 15/250 - 0s - loss: 9429777.5382 - mean_squared_error: 9429777.5382 - val_loss: 9341338.7192 - val_mean_squared_error: 9341338.7192 Epoch 16/250 - 0s - loss: 8479088.1341 - mean_squared_error: 8479088.1341 - val_loss: 8354023.1498 - val_mean_squared_error: 8354023.1498 Epoch 17/250 - 0s - loss: 7568391.0748 - mean_squared_error: 7568391.0748 - val_loss: 7442124.6740 - val_mean_squared_error: 7442124.6740 Epoch 18/250 - 0s - loss: 6721418.7344 - mean_squared_error: 6721418.7344 - val_loss: 6586103.3364 - val_mean_squared_error: 6586103.3364 Epoch 19/250 - 0s - loss: 5972789.8510 - mean_squared_error: 5972789.8510 - val_loss: 5847909.5638 - val_mean_squared_error: 5847909.5638 Epoch 20/250 - 0s - loss: 5335677.0147 - mean_squared_error: 5335677.0147 - val_loss: 5227564.1179 - val_mean_squared_error: 5227564.1179 Epoch 21/250 - 0s - loss: 4813880.8706 - mean_squared_error: 4813880.8706 - val_loss: 4722315.5530 - val_mean_squared_error: 4722315.5530 Epoch 22/250 - 0s - loss: 4393803.8884 - mean_squared_error: 4393803.8884 - val_loss: 4320488.9071 - val_mean_squared_error: 4320488.9071 Epoch 23/250 - 0s - loss: 4064228.0331 - mean_squared_error: 4064228.0331 - val_loss: 3989324.3641 - val_mean_squared_error: 3989324.3641 Epoch 24/250 - 0s - loss: 3807723.9155 - mean_squared_error: 3807723.9155 - val_loss: 3736172.2453 - val_mean_squared_error: 3736172.2453 Epoch 25/250 - 0s - loss: 3608517.3634 - mean_squared_error: 3608517.3634 - val_loss: 3534118.7984 - val_mean_squared_error: 3534118.7984 Epoch 26/250 - 0s - loss: 3452243.3615 - mean_squared_error: 3452243.3615 - val_loss: 3384725.5243 - val_mean_squared_error: 3384725.5243 Epoch 27/250 - 0s - loss: 3327891.4224 - mean_squared_error: 3327891.4224 - val_loss: 3252092.4262 - val_mean_squared_error: 3252092.4262 Epoch 28/250 - 0s - loss: 3228602.8184 - mean_squared_error: 3228602.8184 - val_loss: 3148144.5572 - val_mean_squared_error: 3148144.5572 Epoch 29/250 - 0s - loss: 3143797.7878 - mean_squared_error: 3143797.7878 - val_loss: 3057926.2773 - val_mean_squared_error: 3057926.2773 Epoch 30/250 - 0s - loss: 3069881.2861 - mean_squared_error: 3069881.2861 - val_loss: 2980950.3217 - val_mean_squared_error: 2980950.3217 Epoch 31/250 - 0s - loss: 3003853.4743 - mean_squared_error: 3003853.4743 - val_loss: 2917935.9723 - val_mean_squared_error: 2917935.9723 Epoch 32/250 - 0s - loss: 2945749.0039 - mean_squared_error: 2945749.0039 - val_loss: 2850592.1838 - val_mean_squared_error: 2850592.1838 Epoch 33/250 - 0s - loss: 2892105.9780 - mean_squared_error: 2892105.9780 - val_loss: 2792872.0755 - val_mean_squared_error: 2792872.0755 Epoch 34/250 - 0s - loss: 2842148.0772 - mean_squared_error: 2842148.0772 - val_loss: 2741323.9650 - val_mean_squared_error: 2741323.9650 Epoch 35/250 - 0s - loss: 2796612.5332 - mean_squared_error: 2796612.5332 - val_loss: 2704994.5147 - val_mean_squared_error: 2704994.5147 Epoch 36/250 - 0s - loss: 2755158.5985 - mean_squared_error: 2755158.5985 - val_loss: 2661656.8867 - val_mean_squared_error: 2661656.8867 Epoch 37/250 - 0s - loss: 2718158.8410 - mean_squared_error: 2718158.8410 - val_loss: 2609774.1920 - val_mean_squared_error: 2609774.1920 Epoch 38/250 - 0s - loss: 2679836.7112 - mean_squared_error: 2679836.7112 - val_loss: 2573426.7727 - val_mean_squared_error: 2573426.7727 Epoch 39/250 - 0s - loss: 2646180.1379 - mean_squared_error: 2646180.1379 - val_loss: 2535727.3375 - val_mean_squared_error: 2535727.3375 Epoch 40/250 - 0s - loss: 2615183.2112 - mean_squared_error: 2615183.2112 - val_loss: 2503947.4630 - val_mean_squared_error: 2503947.4630 Epoch 41/250 - 0s - loss: 2585675.0433 - mean_squared_error: 2585675.0433 - val_loss: 2473989.3857 - val_mean_squared_error: 2473989.3857 Epoch 42/250 - 0s - loss: 2558225.3623 - mean_squared_error: 2558225.3623 - val_loss: 2449015.9460 - val_mean_squared_error: 2449015.9460 Epoch 43/250 - 0s - loss: 2533288.5659 - mean_squared_error: 2533288.5659 - val_loss: 2417341.7216 - val_mean_squared_error: 2417341.7216 Epoch 44/250 - 0s - loss: 2509064.7028 - mean_squared_error: 2509064.7028 - val_loss: 2392832.8228 - val_mean_squared_error: 2392832.8228 Epoch 45/250 - 0s - loss: 2487459.6374 - mean_squared_error: 2487459.6374 - val_loss: 2372149.2280 - val_mean_squared_error: 2372149.2280 Epoch 46/250 - 0s - loss: 2468291.8221 - mean_squared_error: 2468291.8221 - val_loss: 2349463.8397 - val_mean_squared_error: 2349463.8397 Epoch 47/250 - 0s - loss: 2448348.3060 - mean_squared_error: 2448348.3060 - val_loss: 2329858.0893 - val_mean_squared_error: 2329858.0893 Epoch 48/250 - 0s - loss: 2430979.3135 - mean_squared_error: 2430979.3135 - val_loss: 2311943.3117 - val_mean_squared_error: 2311943.3117 Epoch 49/250 - 0s - loss: 2414875.6873 - mean_squared_error: 2414875.6873 - val_loss: 2298119.6149 - val_mean_squared_error: 2298119.6149 Epoch 50/250 - 0s - loss: 2400211.5722 - mean_squared_error: 2400211.5722 - val_loss: 2287963.9946 - val_mean_squared_error: 2287963.9946 Epoch 51/250 - 0s - loss: 2386321.2078 - mean_squared_error: 2386321.2078 - val_loss: 2272429.4482 - val_mean_squared_error: 2272429.4482 Epoch 52/250 - 0s - loss: 2373529.5114 - mean_squared_error: 2373529.5114 - val_loss: 2253672.8613 - val_mean_squared_error: 2253672.8613 Epoch 53/250 - 0s - loss: 2362138.4809 - mean_squared_error: 2362138.4809 - val_loss: 2241564.3501 - val_mean_squared_error: 2241564.3501 Epoch 54/250 - 0s - loss: 2350962.5079 - mean_squared_error: 2350962.5079 - val_loss: 2228854.2856 - val_mean_squared_error: 2228854.2856 Epoch 55/250 - 0s - loss: 2339201.8977 - mean_squared_error: 2339201.8977 - val_loss: 2219977.7907 - val_mean_squared_error: 2219977.7907 Epoch 56/250 - 0s - loss: 2328787.0270 - mean_squared_error: 2328787.0270 - val_loss: 2209364.8874 - val_mean_squared_error: 2209364.8874 Epoch 57/250 - 0s - loss: 2318957.3597 - mean_squared_error: 2318957.3597 - val_loss: 2199942.2882 - val_mean_squared_error: 2199942.2882 Epoch 58/250 - 0s - loss: 2310680.1843 - mean_squared_error: 2310680.1843 - val_loss: 2188976.2991 - val_mean_squared_error: 2188976.2991 Epoch 59/250 - 0s - loss: 2303175.6789 - mean_squared_error: 2303175.6789 - val_loss: 2182051.3485 - val_mean_squared_error: 2182051.3485 Epoch 60/250 - 0s - loss: 2293841.7191 - mean_squared_error: 2293841.7191 - val_loss: 2186315.6409 - val_mean_squared_error: 2186315.6409 Epoch 61/250 - 0s - loss: 2286020.3573 - mean_squared_error: 2286020.3573 - val_loss: 2165740.7252 - val_mean_squared_error: 2165740.7252 Epoch 62/250 - 0s - loss: 2279051.4943 - mean_squared_error: 2279051.4943 - val_loss: 2158763.2060 - val_mean_squared_error: 2158763.2060 Epoch 63/250 - 0s - loss: 2271475.3495 - mean_squared_error: 2271475.3495 - val_loss: 2152770.6583 - val_mean_squared_error: 2152770.6583 Epoch 64/250 - 0s - loss: 2265085.5156 - mean_squared_error: 2265085.5156 - val_loss: 2144353.0341 - val_mean_squared_error: 2144353.0341 Epoch 65/250 - 0s - loss: 2257484.7807 - mean_squared_error: 2257484.7807 - val_loss: 2137699.1556 - val_mean_squared_error: 2137699.1556 Epoch 66/250 - 0s - loss: 2249552.2248 - mean_squared_error: 2249552.2248 - val_loss: 2132078.8452 - val_mean_squared_error: 2132078.8452 Epoch 67/250 - 0s - loss: 2244073.1922 - mean_squared_error: 2244073.1922 - val_loss: 2127342.1857 - val_mean_squared_error: 2127342.1857 Epoch 68/250 - 0s - loss: 2238638.8097 - mean_squared_error: 2238638.8097 - val_loss: 2120465.6718 - val_mean_squared_error: 2120465.6718 Epoch 69/250 - 0s - loss: 2230916.4960 - mean_squared_error: 2230916.4960 - val_loss: 2117704.5874 - val_mean_squared_error: 2117704.5874 Epoch 70/250 - 0s - loss: 2226334.2649 - mean_squared_error: 2226334.2649 - val_loss: 2109176.2234 - val_mean_squared_error: 2109176.2234 Epoch 71/250 - 0s - loss: 2219205.3223 - mean_squared_error: 2219205.3223 - val_loss: 2112386.9385 - val_mean_squared_error: 2112386.9385 Epoch 72/250 - 0s - loss: 2214646.4007 - mean_squared_error: 2214646.4007 - val_loss: 2096550.7317 - val_mean_squared_error: 2096550.7317 Epoch 73/250 - 0s - loss: 2208030.9046 - mean_squared_error: 2208030.9046 - val_loss: 2093388.0783 - val_mean_squared_error: 2093388.0783 Epoch 74/250 - 0s - loss: 2202061.2657 - mean_squared_error: 2202061.2657 - val_loss: 2086613.5556 - val_mean_squared_error: 2086613.5556 Epoch 75/250 - 0s - loss: 2196172.7749 - mean_squared_error: 2196172.7749 - val_loss: 2080788.6867 - val_mean_squared_error: 2080788.6867 Epoch 76/250 - 0s - loss: 2191272.2086 - mean_squared_error: 2191272.2086 - val_loss: 2091689.5903 - val_mean_squared_error: 2091689.5903 Epoch 77/250 - 0s - loss: 2184674.9965 - mean_squared_error: 2184674.9965 - val_loss: 2071263.0217 - val_mean_squared_error: 2071263.0217 Epoch 78/250 - 0s - loss: 2179264.1785 - mean_squared_error: 2179264.1785 - val_loss: 2065878.1463 - val_mean_squared_error: 2065878.1463 Epoch 79/250 - 0s - loss: 2173635.1442 - mean_squared_error: 2173635.1442 - val_loss: 2060655.8856 - val_mean_squared_error: 2060655.8856 Epoch 80/250 - 0s - loss: 2168273.2437 - mean_squared_error: 2168273.2437 - val_loss: 2060632.1855 - val_mean_squared_error: 2060632.1855 Epoch 81/250 - 0s - loss: 2163022.3020 - mean_squared_error: 2163022.3020 - val_loss: 2051177.3330 - val_mean_squared_error: 2051177.3330 Epoch 82/250 - 0s - loss: 2157621.1025 - mean_squared_error: 2157621.1025 - val_loss: 2046531.8500 - val_mean_squared_error: 2046531.8500 Epoch 83/250 - 0s - loss: 2153347.7397 - mean_squared_error: 2153347.7397 - val_loss: 2041649.5164 - val_mean_squared_error: 2041649.5164 Epoch 84/250 - 0s - loss: 2147905.2811 - mean_squared_error: 2147905.2811 - val_loss: 2036815.1064 - val_mean_squared_error: 2036815.1064 Epoch 85/250 - 0s - loss: 2142554.4027 - mean_squared_error: 2142554.4027 - val_loss: 2032796.2310 - val_mean_squared_error: 2032796.2310 Epoch 86/250 - 0s - loss: 2137829.3583 - mean_squared_error: 2137829.3583 - val_loss: 2031853.6131 - val_mean_squared_error: 2031853.6131 Epoch 87/250 - 0s - loss: 2132519.2008 - mean_squared_error: 2132519.2008 - val_loss: 2030490.3039 - val_mean_squared_error: 2030490.3039 Epoch 88/250 - 0s - loss: 2127016.3250 - mean_squared_error: 2127016.3250 - val_loss: 2020684.9947 - val_mean_squared_error: 2020684.9947 Epoch 89/250 - 0s - loss: 2121851.6609 - mean_squared_error: 2121851.6609 - val_loss: 2016022.5633 - val_mean_squared_error: 2016022.5633 Epoch 90/250 - 0s - loss: 2117662.5043 - mean_squared_error: 2117662.5043 - val_loss: 2010337.9569 - val_mean_squared_error: 2010337.9569 Epoch 91/250 - 0s - loss: 2112542.9926 - mean_squared_error: 2112542.9926 - val_loss: 2020211.4308 - val_mean_squared_error: 2020211.4308 Epoch 92/250 - 0s - loss: 2107865.1081 - mean_squared_error: 2107865.1081 - val_loss: 2004868.1403 - val_mean_squared_error: 2004868.1403 Epoch 93/250 - 0s - loss: 2103107.2448 - mean_squared_error: 2103107.2448 - val_loss: 2004523.8293 - val_mean_squared_error: 2004523.8293 Epoch 94/250 - 0s - loss: 2098785.2728 - mean_squared_error: 2098785.2728 - val_loss: 2006746.1995 - val_mean_squared_error: 2006746.1995 Epoch 95/250 - 0s - loss: 2094392.0981 - mean_squared_error: 2094392.0981 - val_loss: 2015872.2060 - val_mean_squared_error: 2015872.2060 Epoch 96/250 - 0s - loss: 2088351.5520 - mean_squared_error: 2088351.5520 - val_loss: 1987963.1605 - val_mean_squared_error: 1987963.1605 Epoch 97/250 - 0s - loss: 2083281.3127 - mean_squared_error: 2083281.3127 - val_loss: 1982317.8391 - val_mean_squared_error: 1982317.8391 Epoch 98/250 - 0s - loss: 2079334.3246 - mean_squared_error: 2079334.3246 - val_loss: 1986385.0095 - val_mean_squared_error: 1986385.0095 Epoch 99/250 - 0s - loss: 2074639.2967 - mean_squared_error: 2074639.2967 - val_loss: 1972916.8085 - val_mean_squared_error: 1972916.8085 Epoch 100/250 - 0s - loss: 2069718.0345 - mean_squared_error: 2069718.0345 - val_loss: 1970312.4277 - val_mean_squared_error: 1970312.4277 Epoch 101/250 - 0s - loss: 2065881.9258 - mean_squared_error: 2065881.9258 - val_loss: 1977506.7564 - val_mean_squared_error: 1977506.7564 Epoch 102/250 - 0s - loss: 2061375.6670 - mean_squared_error: 2061375.6670 - val_loss: 1972389.1459 - val_mean_squared_error: 1972389.1459 Epoch 103/250 - 0s - loss: 2055962.7156 - mean_squared_error: 2055962.7156 - val_loss: 1960733.0200 - val_mean_squared_error: 1960733.0200 Epoch 104/250 - 0s - loss: 2052171.3932 - mean_squared_error: 2052171.3932 - val_loss: 1955574.4043 - val_mean_squared_error: 1955574.4043 Epoch 105/250 - 0s - loss: 2046835.8604 - mean_squared_error: 2046835.8604 - val_loss: 1950047.3523 - val_mean_squared_error: 1950047.3523 Epoch 106/250 - 0s - loss: 2041886.3047 - mean_squared_error: 2041886.3047 - val_loss: 1946093.0289 - val_mean_squared_error: 1946093.0289 Epoch 107/250 - 0s - loss: 2037196.2222 - mean_squared_error: 2037196.2222 - val_loss: 1949554.4843 - val_mean_squared_error: 1949554.4843 Epoch 108/250 - 0s - loss: 2032640.8686 - mean_squared_error: 2032640.8686 - val_loss: 1946834.4377 - val_mean_squared_error: 1946834.4377 Epoch 109/250 - 0s - loss: 2029532.5345 - mean_squared_error: 2029532.5345 - val_loss: 1935923.5203 - val_mean_squared_error: 1935923.5203 Epoch 110/250 - 0s - loss: 2024320.1388 - mean_squared_error: 2024320.1388 - val_loss: 1931483.9789 - val_mean_squared_error: 1931483.9789 Epoch 111/250 - 0s - loss: 2019146.0171 - mean_squared_error: 2019146.0171 - val_loss: 1924558.2290 - val_mean_squared_error: 1924558.2290 Epoch 112/250 - 0s - loss: 2014698.1843 - mean_squared_error: 2014698.1843 - val_loss: 1930959.4795 - val_mean_squared_error: 1930959.4795 Epoch 113/250 - 0s - loss: 2010283.3501 - mean_squared_error: 2010283.3501 - val_loss: 1916585.2233 - val_mean_squared_error: 1916585.2233 Epoch 114/250 - 0s - loss: 2006106.9084 - mean_squared_error: 2006106.9084 - val_loss: 1912377.0285 - val_mean_squared_error: 1912377.0285 Epoch 115/250 - 0s - loss: 2001727.7207 - mean_squared_error: 2001727.7207 - val_loss: 1912506.8241 - val_mean_squared_error: 1912506.8241 Epoch 116/250 - 0s - loss: 1996867.5454 - mean_squared_error: 1996867.5454 - val_loss: 1905103.1087 - val_mean_squared_error: 1905103.1087 Epoch 117/250 - 0s - loss: 1992362.4066 - mean_squared_error: 1992362.4066 - val_loss: 1909744.2719 - val_mean_squared_error: 1909744.2719 Epoch 118/250 - 0s - loss: 1988649.1120 - mean_squared_error: 1988649.1120 - val_loss: 1897487.2244 - val_mean_squared_error: 1897487.2244 Epoch 119/250 - 0s - loss: 1984399.7788 - mean_squared_error: 1984399.7788 - val_loss: 1894243.5131 - val_mean_squared_error: 1894243.5131 Epoch 120/250 - 0s - loss: 1979863.9807 - mean_squared_error: 1979863.9807 - val_loss: 1890351.2316 - val_mean_squared_error: 1890351.2316 Epoch 121/250 - 0s - loss: 1974992.6992 - mean_squared_error: 1974992.6992 - val_loss: 1887232.6038 - val_mean_squared_error: 1887232.6038 Epoch 122/250 - 0s - loss: 1971293.2739 - mean_squared_error: 1971293.2739 - val_loss: 1883094.7196 - val_mean_squared_error: 1883094.7196 Epoch 123/250 - 0s - loss: 1967652.8666 - mean_squared_error: 1967652.8666 - val_loss: 1880776.2502 - val_mean_squared_error: 1880776.2502 Epoch 124/250 - 0s - loss: 1962337.3258 - mean_squared_error: 1962337.3258 - val_loss: 1875115.5960 - val_mean_squared_error: 1875115.5960 Epoch 125/250 - 0s - loss: 1957642.2209 - mean_squared_error: 1957642.2209 - val_loss: 1885500.7002 - val_mean_squared_error: 1885500.7002 Epoch 126/250 - 0s - loss: 1953632.1903 - mean_squared_error: 1953632.1903 - val_loss: 1867913.3843 - val_mean_squared_error: 1867913.3843 Epoch 127/250 - 0s - loss: 1949615.8090 - mean_squared_error: 1949615.8090 - val_loss: 1863340.9736 - val_mean_squared_error: 1863340.9736 Epoch 128/250 - 0s - loss: 1945531.0725 - mean_squared_error: 1945531.0725 - val_loss: 1859356.5073 - val_mean_squared_error: 1859356.5073 Epoch 129/250 - 0s - loss: 1940890.7186 - mean_squared_error: 1940890.7186 - val_loss: 1859058.7449 - val_mean_squared_error: 1859058.7449 Epoch 130/250 - 0s - loss: 1937136.4930 - mean_squared_error: 1937136.4930 - val_loss: 1851884.3197 - val_mean_squared_error: 1851884.3197 Epoch 131/250 - 0s - loss: 1932133.3433 - mean_squared_error: 1932133.3433 - val_loss: 1851862.2305 - val_mean_squared_error: 1851862.2305 Epoch 132/250 - 0s - loss: 1927362.7782 - mean_squared_error: 1927362.7782 - val_loss: 1844729.4241 - val_mean_squared_error: 1844729.4241 Epoch 133/250 - 0s - loss: 1922927.4253 - mean_squared_error: 1922927.4253 - val_loss: 1843720.2533 - val_mean_squared_error: 1843720.2533 Epoch 134/250 - 0s - loss: 1919534.0592 - mean_squared_error: 1919534.0592 - val_loss: 1837458.4233 - val_mean_squared_error: 1837458.4233 Epoch 135/250 - 0s - loss: 1915310.5604 - mean_squared_error: 1915310.5604 - val_loss: 1836560.7027 - val_mean_squared_error: 1836560.7027 Epoch 136/250 - 0s - loss: 1910561.5297 - mean_squared_error: 1910561.5297 - val_loss: 1829169.3662 - val_mean_squared_error: 1829169.3662 Epoch 137/250 - 0s - loss: 1906927.9989 - mean_squared_error: 1906927.9989 - val_loss: 1828976.3169 - val_mean_squared_error: 1828976.3169 Epoch 138/250 - 0s - loss: 1901777.5216 - mean_squared_error: 1901777.5216 - val_loss: 1821902.8355 - val_mean_squared_error: 1821902.8355 Epoch 139/250 - 0s - loss: 1898193.0952 - mean_squared_error: 1898193.0952 - val_loss: 1820055.3003 - val_mean_squared_error: 1820055.3003 Epoch 140/250 - 0s - loss: 1894638.0243 - mean_squared_error: 1894638.0243 - val_loss: 1824675.2797 - val_mean_squared_error: 1824675.2797 Epoch 141/250 - 0s - loss: 1889652.2684 - mean_squared_error: 1889652.2684 - val_loss: 1810856.4123 - val_mean_squared_error: 1810856.4123 Epoch 142/250 - 0s - loss: 1885172.4209 - mean_squared_error: 1885172.4209 - val_loss: 1811507.5374 - val_mean_squared_error: 1811507.5374 Epoch 143/250 - 0s - loss: 1880849.9645 - mean_squared_error: 1880849.9645 - val_loss: 1804455.1621 - val_mean_squared_error: 1804455.1621 Epoch 144/250 - 0s - loss: 1878023.9024 - mean_squared_error: 1878023.9024 - val_loss: 1809492.8590 - val_mean_squared_error: 1809492.8590 Epoch 145/250 - 0s - loss: 1872457.6617 - mean_squared_error: 1872457.6617 - val_loss: 1795905.5589 - val_mean_squared_error: 1795905.5589 Epoch 146/250 - 0s - loss: 1867696.2444 - mean_squared_error: 1867696.2444 - val_loss: 1804266.8772 - val_mean_squared_error: 1804266.8772 Epoch 147/250 - 0s - loss: 1864600.9269 - mean_squared_error: 1864600.9269 - val_loss: 1788362.1574 - val_mean_squared_error: 1788362.1574 Epoch 148/250 - 0s - loss: 1860130.6490 - mean_squared_error: 1860130.6490 - val_loss: 1784222.2456 - val_mean_squared_error: 1784222.2456 Epoch 149/250 - 0s - loss: 1856007.0796 - mean_squared_error: 1856007.0796 - val_loss: 1780728.0488 - val_mean_squared_error: 1780728.0488 Epoch 150/250 - 0s - loss: 1852311.6258 - mean_squared_error: 1852311.6258 - val_loss: 1780861.1786 - val_mean_squared_error: 1780861.1786 Epoch 151/250 - 0s - loss: 1847320.6135 - mean_squared_error: 1847320.6135 - val_loss: 1773283.1322 - val_mean_squared_error: 1773283.1322 Epoch 152/250 - 0s - loss: 1843755.1188 - mean_squared_error: 1843755.1188 - val_loss: 1772574.1925 - val_mean_squared_error: 1772574.1925 Epoch 153/250 - 0s - loss: 1839902.7247 - mean_squared_error: 1839902.7247 - val_loss: 1766250.9261 - val_mean_squared_error: 1766250.9261 Epoch 154/250 - 0s - loss: 1834495.1264 - mean_squared_error: 1834495.1264 - val_loss: 1764381.0849 - val_mean_squared_error: 1764381.0849 Epoch 155/250 - 0s - loss: 1832261.5215 - mean_squared_error: 1832261.5215 - val_loss: 1760187.0743 - val_mean_squared_error: 1760187.0743 Epoch 156/250 - 0s - loss: 1827288.4371 - mean_squared_error: 1827288.4371 - val_loss: 1762313.1666 - val_mean_squared_error: 1762313.1666 Epoch 157/250 - 0s - loss: 1823718.1723 - mean_squared_error: 1823718.1723 - val_loss: 1753117.2768 - val_mean_squared_error: 1753117.2768 Epoch 158/250 - 0s - loss: 1819285.1757 - mean_squared_error: 1819285.1757 - val_loss: 1750069.2710 - val_mean_squared_error: 1750069.2710 Epoch 159/250 - 0s - loss: 1814956.1455 - mean_squared_error: 1814956.1455 - val_loss: 1750201.8942 - val_mean_squared_error: 1750201.8942 Epoch 160/250 - 0s - loss: 1811288.9790 - mean_squared_error: 1811288.9790 - val_loss: 1742480.3644 - val_mean_squared_error: 1742480.3644 Epoch 161/250 - 0s - loss: 1806832.4167 - mean_squared_error: 1806832.4167 - val_loss: 1740825.5788 - val_mean_squared_error: 1740825.5788 Epoch 162/250 - 0s - loss: 1803009.5503 - mean_squared_error: 1803009.5503 - val_loss: 1734655.2090 - val_mean_squared_error: 1734655.2090 Epoch 163/250 - 0s - loss: 1799302.5109 - mean_squared_error: 1799302.5109 - val_loss: 1731850.9553 - val_mean_squared_error: 1731850.9553 Epoch 164/250 - 0s - loss: 1795306.4234 - mean_squared_error: 1795306.4234 - val_loss: 1731392.5811 - val_mean_squared_error: 1731392.5811 Epoch 165/250 - 0s - loss: 1791507.7240 - mean_squared_error: 1791507.7240 - val_loss: 1727721.0402 - val_mean_squared_error: 1727721.0402 Epoch 166/250 - 0s - loss: 1787176.1448 - mean_squared_error: 1787176.1448 - val_loss: 1720911.0697 - val_mean_squared_error: 1720911.0697 Epoch 167/250 - 0s - loss: 1781905.6211 - mean_squared_error: 1781905.6211 - val_loss: 1719351.7794 - val_mean_squared_error: 1719351.7794 Epoch 168/250 - 0s - loss: 1779861.0857 - mean_squared_error: 1779861.0857 - val_loss: 1719003.7058 - val_mean_squared_error: 1719003.7058 Epoch 169/250 - 0s - loss: 1775434.3415 - mean_squared_error: 1775434.3415 - val_loss: 1710018.3915 - val_mean_squared_error: 1710018.3915 Epoch 170/250 - 0s - loss: 1770058.1794 - mean_squared_error: 1770058.1794 - val_loss: 1706083.8167 - val_mean_squared_error: 1706083.8167 Epoch 171/250 - 0s - loss: 1766499.7814 - mean_squared_error: 1766499.7814 - val_loss: 1707662.4464 - val_mean_squared_error: 1707662.4464 Epoch 172/250 - 0s - loss: 1763422.4239 - mean_squared_error: 1763422.4239 - val_loss: 1699525.6440 - val_mean_squared_error: 1699525.6440 Epoch 173/250 - 0s - loss: 1759100.8694 - mean_squared_error: 1759100.8694 - val_loss: 1702088.4232 - val_mean_squared_error: 1702088.4232 Epoch 174/250 - 0s - loss: 1754727.7591 - mean_squared_error: 1754727.7591 - val_loss: 1691336.3602 - val_mean_squared_error: 1691336.3602 Epoch 175/250 - 0s - loss: 1750976.0382 - mean_squared_error: 1750976.0382 - val_loss: 1689726.2166 - val_mean_squared_error: 1689726.2166 Epoch 176/250 - 0s - loss: 1745855.5479 - mean_squared_error: 1745855.5479 - val_loss: 1684441.2240 - val_mean_squared_error: 1684441.2240 Epoch 177/250 - 0s - loss: 1742120.0573 - mean_squared_error: 1742120.0573 - val_loss: 1682025.0566 - val_mean_squared_error: 1682025.0566 Epoch 178/250 - 0s - loss: 1738722.1015 - mean_squared_error: 1738722.1015 - val_loss: 1680803.8638 - val_mean_squared_error: 1680803.8638 Epoch 179/250 - 0s - loss: 1735046.0498 - mean_squared_error: 1735046.0498 - val_loss: 1673423.6456 - val_mean_squared_error: 1673423.6456 Epoch 180/250 - 0s - loss: 1730597.3073 - mean_squared_error: 1730597.3073 - val_loss: 1673260.8376 - val_mean_squared_error: 1673260.8376 Epoch 181/250 - 0s - loss: 1726893.0606 - mean_squared_error: 1726893.0606 - val_loss: 1666612.4174 - val_mean_squared_error: 1666612.4174 Epoch 182/250 - 0s - loss: 1723595.6851 - mean_squared_error: 1723595.6851 - val_loss: 1663159.8190 - val_mean_squared_error: 1663159.8190 Epoch 183/250 - 0s - loss: 1718215.3418 - mean_squared_error: 1718215.3418 - val_loss: 1682753.7991 - val_mean_squared_error: 1682753.7991 Epoch 184/250 - 0s - loss: 1715240.0257 - mean_squared_error: 1715240.0257 - val_loss: 1655998.7817 - val_mean_squared_error: 1655998.7817 Epoch 185/250 - 0s - loss: 1710310.8813 - mean_squared_error: 1710310.8813 - val_loss: 1652499.6405 - val_mean_squared_error: 1652499.6405 Epoch 186/250 - 0s - loss: 1706194.4504 - mean_squared_error: 1706194.4504 - val_loss: 1652202.3520 - val_mean_squared_error: 1652202.3520 Epoch 187/250 - 0s - loss: 1703143.5678 - mean_squared_error: 1703143.5678 - val_loss: 1646046.9575 - val_mean_squared_error: 1646046.9575 Epoch 188/250 - 0s - loss: 1699053.5930 - mean_squared_error: 1699053.5930 - val_loss: 1642285.1786 - val_mean_squared_error: 1642285.1786 Epoch 189/250 - 0s - loss: 1694139.1135 - mean_squared_error: 1694139.1135 - val_loss: 1645611.0360 - val_mean_squared_error: 1645611.0360 Epoch 190/250 - 0s - loss: 1690011.1303 - mean_squared_error: 1690011.1303 - val_loss: 1635355.5027 - val_mean_squared_error: 1635355.5027 Epoch 191/250 - 0s - loss: 1687186.1279 - mean_squared_error: 1687186.1279 - val_loss: 1631430.2996 - val_mean_squared_error: 1631430.2996 Epoch 192/250 - 0s - loss: 1682407.9588 - mean_squared_error: 1682407.9588 - val_loss: 1628006.1327 - val_mean_squared_error: 1628006.1327 Epoch 193/250 - 0s - loss: 1679966.4123 - mean_squared_error: 1679966.4123 - val_loss: 1625461.3485 - val_mean_squared_error: 1625461.3485 Epoch 194/250 - 0s - loss: 1674929.3425 - mean_squared_error: 1674929.3425 - val_loss: 1627828.1645 - val_mean_squared_error: 1627828.1645 Epoch 195/250 - 0s - loss: 1671375.5653 - mean_squared_error: 1671375.5653 - val_loss: 1618734.5428 - val_mean_squared_error: 1618734.5428 Epoch 196/250 - 0s - loss: 1666804.9442 - mean_squared_error: 1666804.9442 - val_loss: 1615558.5876 - val_mean_squared_error: 1615558.5876 Epoch 197/250 - 0s - loss: 1663388.1087 - mean_squared_error: 1663388.1087 - val_loss: 1611048.3662 - val_mean_squared_error: 1611048.3662 Epoch 198/250 - 0s - loss: 1658909.2213 - mean_squared_error: 1658909.2213 - val_loss: 1607997.1087 - val_mean_squared_error: 1607997.1087 Epoch 199/250 - 0s - loss: 1656002.0048 - mean_squared_error: 1656002.0048 - val_loss: 1604547.2686 - val_mean_squared_error: 1604547.2686 Epoch 200/250 - 0s - loss: 1651028.4805 - mean_squared_error: 1651028.4805 - val_loss: 1617295.4973 - val_mean_squared_error: 1617295.4973 Epoch 201/250 - 0s - loss: 1648101.1830 - mean_squared_error: 1648101.1830 - val_loss: 1596742.9183 - val_mean_squared_error: 1596742.9183 Epoch 202/250 - 0s - loss: 1645068.8276 - mean_squared_error: 1645068.8276 - val_loss: 1597667.4695 - val_mean_squared_error: 1597667.4695 Epoch 203/250 - 0s - loss: 1639275.9104 - mean_squared_error: 1639275.9104 - val_loss: 1590165.4120 - val_mean_squared_error: 1590165.4120 Epoch 204/250 - 0s - loss: 1636734.3399 - mean_squared_error: 1636734.3399 - val_loss: 1594571.5751 - val_mean_squared_error: 1594571.5751 Epoch 205/250 - 0s - loss: 1632550.5110 - mean_squared_error: 1632550.5110 - val_loss: 1582428.1987 - val_mean_squared_error: 1582428.1987 Epoch 206/250 - 0s - loss: 1627740.2648 - mean_squared_error: 1627740.2648 - val_loss: 1582794.0982 - val_mean_squared_error: 1582794.0982 Epoch 207/250 - 0s - loss: 1624661.7697 - mean_squared_error: 1624661.7697 - val_loss: 1575813.0507 - val_mean_squared_error: 1575813.0507 Epoch 208/250 - 0s - loss: 1621405.3260 - mean_squared_error: 1621405.3260 - val_loss: 1578353.3074 - val_mean_squared_error: 1578353.3074 Epoch 209/250 - 0s - loss: 1617610.8753 - mean_squared_error: 1617610.8753 - val_loss: 1571793.6565 - val_mean_squared_error: 1571793.6565 Epoch 210/250 - 0s - loss: 1613287.7830 - mean_squared_error: 1613287.7830 - val_loss: 1565228.1964 - val_mean_squared_error: 1565228.1964 Epoch 211/250 - 0s - loss: 1609531.9519 - mean_squared_error: 1609531.9519 - val_loss: 1574323.6174 - val_mean_squared_error: 1574323.6174 Epoch 212/250 - 0s - loss: 1606760.7368 - mean_squared_error: 1606760.7368 - val_loss: 1562812.2576 - val_mean_squared_error: 1562812.2576 Epoch 213/250 - 0s - loss: 1602434.8987 - mean_squared_error: 1602434.8987 - val_loss: 1574046.5179 - val_mean_squared_error: 1574046.5179 Epoch 214/250 - 0s - loss: 1598833.0059 - mean_squared_error: 1598833.0059 - val_loss: 1551716.8104 - val_mean_squared_error: 1551716.8104 Epoch 215/250 - 0s - loss: 1594174.6572 - mean_squared_error: 1594174.6572 - val_loss: 1549087.0527 - val_mean_squared_error: 1549087.0527 Epoch 216/250 - 0s - loss: 1591720.1553 - mean_squared_error: 1591720.1553 - val_loss: 1544962.2633 - val_mean_squared_error: 1544962.2633 Epoch 217/250 - 0s - loss: 1586981.5828 - mean_squared_error: 1586981.5828 - val_loss: 1542278.7089 - val_mean_squared_error: 1542278.7089 Epoch 218/250 - 0s - loss: 1583475.2432 - mean_squared_error: 1583475.2432 - val_loss: 1541631.3228 - val_mean_squared_error: 1541631.3228 Epoch 219/250 - 0s - loss: 1580077.1593 - mean_squared_error: 1580077.1593 - val_loss: 1535941.0057 - val_mean_squared_error: 1535941.0057 Epoch 220/250 - 0s - loss: 1576142.0043 - mean_squared_error: 1576142.0043 - val_loss: 1535944.0277 - val_mean_squared_error: 1535944.0277 Epoch 221/250 - 0s - loss: 1573398.5700 - mean_squared_error: 1573398.5700 - val_loss: 1529586.8494 - val_mean_squared_error: 1529586.8494 Epoch 222/250 - 0s - loss: 1568685.8879 - mean_squared_error: 1568685.8879 - val_loss: 1524862.3149 - val_mean_squared_error: 1524862.3149 Epoch 223/250 - 0s - loss: 1565756.8390 - mean_squared_error: 1565756.8390 - val_loss: 1526767.2780 - val_mean_squared_error: 1526767.2780 Epoch 224/250 - 0s - loss: 1561886.2023 - mean_squared_error: 1561886.2023 - val_loss: 1519050.4504 - val_mean_squared_error: 1519050.4504 Epoch 225/250 - 0s - loss: 1558566.3714 - mean_squared_error: 1558566.3714 - val_loss: 1522032.5675 - val_mean_squared_error: 1522032.5675 Epoch 226/250 - 0s - loss: 1554783.3866 - mean_squared_error: 1554783.3866 - val_loss: 1513231.8030 - val_mean_squared_error: 1513231.8030 Epoch 227/250 - 0s - loss: 1550797.1188 - mean_squared_error: 1550797.1188 - val_loss: 1509315.3206 - val_mean_squared_error: 1509315.3206 Epoch 228/250 - 0s - loss: 1548495.3307 - mean_squared_error: 1548495.3307 - val_loss: 1505629.4250 - val_mean_squared_error: 1505629.4250 Epoch 229/250 - 0s - loss: 1544194.7336 - mean_squared_error: 1544194.7336 - val_loss: 1510430.9881 - val_mean_squared_error: 1510430.9881 Epoch 230/250 - 0s - loss: 1540820.6817 - mean_squared_error: 1540820.6817 - val_loss: 1503159.9626 - val_mean_squared_error: 1503159.9626 Epoch 231/250 - 0s - loss: 1537021.6385 - mean_squared_error: 1537021.6385 - val_loss: 1505039.2447 - val_mean_squared_error: 1505039.2447 Epoch 232/250 - 0s - loss: 1533289.4870 - mean_squared_error: 1533289.4870 - val_loss: 1504286.8382 - val_mean_squared_error: 1504286.8382 Epoch 233/250 - 0s - loss: 1530604.8589 - mean_squared_error: 1530604.8589 - val_loss: 1489318.1259 - val_mean_squared_error: 1489318.1259 Epoch 234/250 - 0s - loss: 1527521.2587 - mean_squared_error: 1527521.2587 - val_loss: 1486197.9848 - val_mean_squared_error: 1486197.9848 Epoch 235/250 - 0s - loss: 1524169.4817 - mean_squared_error: 1524169.4817 - val_loss: 1485331.1503 - val_mean_squared_error: 1485331.1503 Epoch 236/250 - 0s - loss: 1520607.7352 - mean_squared_error: 1520607.7352 - val_loss: 1491311.6805 - val_mean_squared_error: 1491311.6805 Epoch 237/250 - 0s - loss: 1518421.0148 - mean_squared_error: 1518421.0148 - val_loss: 1478969.0401 - val_mean_squared_error: 1478969.0401 Epoch 238/250 - 0s - loss: 1514360.0578 - mean_squared_error: 1514360.0578 - val_loss: 1478182.5436 - val_mean_squared_error: 1478182.5436 Epoch 239/250 - 0s - loss: 1511329.5684 - mean_squared_error: 1511329.5684 - val_loss: 1476309.6793 - val_mean_squared_error: 1476309.6793 Epoch 240/250 - 0s - loss: 1507556.3408 - mean_squared_error: 1507556.3408 - val_loss: 1480174.3921 - val_mean_squared_error: 1480174.3921 Epoch 241/250 - 0s - loss: 1504283.9552 - mean_squared_error: 1504283.9552 - val_loss: 1466960.2980 - val_mean_squared_error: 1466960.2980 Epoch 242/250 - 0s - loss: 1501980.5718 - mean_squared_error: 1501980.5718 - val_loss: 1462204.9121 - val_mean_squared_error: 1462204.9121 Epoch 243/250 - 0s - loss: 1497907.3774 - mean_squared_error: 1497907.3774 - val_loss: 1459351.3592 - val_mean_squared_error: 1459351.3592 Epoch 244/250 - 0s - loss: 1495000.1418 - mean_squared_error: 1495000.1418 - val_loss: 1456332.0101 - val_mean_squared_error: 1456332.0101 Epoch 245/250 - 0s - loss: 1493069.4832 - mean_squared_error: 1493069.4832 - val_loss: 1453457.5532 - val_mean_squared_error: 1453457.5532 Epoch 246/250 - 0s - loss: 1488667.8436 - mean_squared_error: 1488667.8436 - val_loss: 1450555.4977 - val_mean_squared_error: 1450555.4977 Epoch 247/250 - 0s - loss: 1485844.4110 - mean_squared_error: 1485844.4110 - val_loss: 1447594.3666 - val_mean_squared_error: 1447594.3666 Epoch 248/250 - 0s - loss: 1482453.0225 - mean_squared_error: 1482453.0225 - val_loss: 1445203.0277 - val_mean_squared_error: 1445203.0277 Epoch 249/250 - 0s - loss: 1481065.1817 - mean_squared_error: 1481065.1817 - val_loss: 1443750.0622 - val_mean_squared_error: 1443750.0622 Epoch 250/250 - 0s - loss: 1477818.0102 - mean_squared_error: 1477818.0102 - val_loss: 1443422.5918 - val_mean_squared_error: 1443422.5918 32/13485 [..............................] - ETA: 0s 4512/13485 [=========>....................] - ETA: 0s 8864/13485 [==================>...........] - ETA: 0s 13280/13485 [============================>.] - ETA: 0s 13485/13485 [==============================] - 0s 11us/step root mean_squared_error: 1204.425877

%md 
### Training: Gradient Descent

A family of numeric optimization techniques, where we solve a problem with the following pattern:

1. Describe the error in the model output: this is usually some difference between the the true values and the model's predicted values, as a function of the model parameters (weights)

2. Compute the gradient, or directional derivative, of the error -- the "slope toward lower error"

4. Adjust the parameters of the model variables in the indicated direction

5. Repeat

<img src="https://i.imgur.com/HOYViqN.png" width=500>

#### Some ideas to help build your intuition

* What happens if the variables (imagine just 2, to keep the mental picture simple) are on wildly different scales ... like one ranges from -1 to 1 while another from -1e6 to +1e6?

* What if some of the variables are correlated? I.e., a change in one corresponds to, say, a linear change in another?

* Other things being equal, an approximate solution with fewer variables is easier to work with than one with more -- how could we get rid of some less valuable parameters? (e.g., L1 penalty)

* How do we know how far to "adjust" our parameters with each step?

<img src="http://i.imgur.com/AvM2TN6.png" width=600>

What if we have billions of data points? Does it makes sense to use all of them for each update? Is there a shortcut?

Yes: *Stochastic Gradient Descent*

Stochastic gradient descent is an iterative learning algorithm that uses a training dataset to update a model.
- The batch size is a hyperparameter of gradient descent that controls the number of training samples to work through before the model's internal parameters are updated.
- The number of epochs is a hyperparameter of gradient descent that controls the number of complete passes through the training dataset.

See [https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9](https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9).

But SGD has some shortcomings, so we typically use a "smarter" version of SGD, which has rules for adjusting the learning rate and even direction in order to avoid common problems.

What about that "Adam" optimizer? Adam is short for "adaptive moment" and is a variant of SGD that includes momentum calculations that change over time. For more detail on optimizers, see the chapter "Training Deep Neural Nets" in Aurélien Géron's book: *Hands-On Machine Learning with Scikit-Learn and TensorFlow* (http://shop.oreilly.com/product/0636920052289.do)

See [https://keras.io/optimizers/](https://keras.io/optimizers/) and references therein.

Training: Gradient Descent

A family of numeric optimization techniques, where we solve a problem with the following pattern:

Describe the error in the model output: this is usually some difference between the the true values and the model's predicted values, as a function of the model parameters (weights)
Compute the gradient, or directional derivative, of the error -- the "slope toward lower error"
Adjust the parameters of the model variables in the indicated direction
Repeat

Some ideas to help build your intuition

What happens if the variables (imagine just 2, to keep the mental picture simple) are on wildly different scales ... like one ranges from -1 to 1 while another from -1e6 to +1e6?
What if some of the variables are correlated? I.e., a change in one corresponds to, say, a linear change in another?
Other things being equal, an approximate solution with fewer variables is easier to work with than one with more -- how could we get rid of some less valuable parameters? (e.g., L1 penalty)
How do we know how far to "adjust" our parameters with each step?

What if we have billions of data points? Does it makes sense to use all of them for each update? Is there a shortcut?

Yes: Stochastic Gradient Descent

Stochastic gradient descent is an iterative learning algorithm that uses a training dataset to update a model.

The batch size is a hyperparameter of gradient descent that controls the number of training samples to work through before the model's internal parameters are updated.
The number of epochs is a hyperparameter of gradient descent that controls the number of complete passes through the training dataset.

See https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9.

But SGD has some shortcomings, so we typically use a "smarter" version of SGD, which has rules for adjusting the learning rate and even direction in order to avoid common problems.

What about that "Adam" optimizer? Adam is short for "adaptive moment" and is a variant of SGD that includes momentum calculations that change over time. For more detail on optimizers, see the chapter "Training Deep Neural Nets" in Aurélien Géron's book: Hands-On Machine Learning with Scikit-Learn and TensorFlow (http://shop.oreilly.com/product/0636920052289.do)

See https://keras.io/optimizers/ and references therein.

Training: Backpropagation

With a simple, flat model, we could use SGD or a related algorithm to derive the weights, since the error depends directly on those weights.

With a deeper network, we have a couple of challenges:

The error is computed from the final layer, so the gradient of the error doesn't tell us immediately about problems in other-layer weights
Our tiny diamonds model has almost a thousand weights. Bigger models can easily have millions of weights. Each of those weights may need to move a little at a time, and we have to watch out for underflow or undersignificance situations.

The insight is to iteratively calculate errors, one layer at a time, starting at the output. This is called backpropagation. It is neither magical nor surprising. The challenge is just doing it fast and not losing information.

Where does the non-linearity fit in?

We start with the inputs to a perceptron -- these could be from source data, for example.
We multiply each input by its respective weight, which gets us the $x \cdot w$
Then add the "bias" -- an extra learnable parameter, to get ${x \cdot w} + b$
- This value (so far) is sometimes called the "pre-activation"
Now, apply a non-linear "activation function" to this value, such as the logistic sigmoid

Now the network can "learn" non-linear functions

To gain some intuition, consider that where the sigmoid is close to 1, we can think of that neuron as being "on" or activated, giving a specific output. When close to zero, it is "off."

So each neuron is a bit like a switch. If we have enough of them, we can theoretically express arbitrarily many different signals.

In some ways this is like the original artificial neuron, with the thresholding output -- the main difference is that the sigmoid gives us a smooth (arbitrarily differentiable) output that we can optimize over using gradient descent to learn the weights.

Where does the signal "go" from these neurons?

In a regression problem, like the diamonds dataset, the activations from the hidden layer can feed into a single output neuron, with a simple linear activation representing the final output of the calculation.
Frequently we want a classification output instead -- e.g., with MNIST digits, where we need to choose from 10 classes. In that case, we can feed the outputs from these hidden neurons forward into a final layer of 10 neurons, and compare those final neurons' activation levels.

Ok, before we talk any more theory, let's run it and see if we can do better on our diamonds dataset adding this "sigmoid activation."

While that's running, let's look at the code:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].as_matrix().flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).as_matrix()
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='sigmoid')) # <- change to nonlinear activation
model.add(Dense(1, kernel_initializer='normal', activation='linear')) # <- activation is linear in output layer for this regression

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=2000, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples Epoch 1/2000 - 1s - loss: 31380780.6047 - mean_squared_error: 31380780.6047 - val_loss: 32358332.3638 - val_mean_squared_error: 32358332.3638 Epoch 2/2000 - 0s - loss: 31298065.4203 - mean_squared_error: 31298065.4203 - val_loss: 32276571.6461 - val_mean_squared_error: 32276571.6461 Epoch 3/2000 - 0s - loss: 31216004.9920 - mean_squared_error: 31216004.9920 - val_loss: 32191762.6911 - val_mean_squared_error: 32191762.6911 Epoch 4/2000 - 0s - loss: 31133911.1605 - mean_squared_error: 31133911.1605 - val_loss: 32109959.1181 - val_mean_squared_error: 32109959.1181 Epoch 5/2000 - 0s - loss: 31053780.8829 - mean_squared_error: 31053780.8829 - val_loss: 32029314.2709 - val_mean_squared_error: 32029314.2709 Epoch 6/2000 - 0s - loss: 30974261.1565 - mean_squared_error: 30974261.1565 - val_loss: 31949091.0203 - val_mean_squared_error: 31949091.0203 Epoch 7/2000 - 0s - loss: 30895358.6525 - mean_squared_error: 30895358.6525 - val_loss: 31869483.1992 - val_mean_squared_error: 31869483.1992 Epoch 8/2000 - 0s - loss: 30816933.3015 - mean_squared_error: 30816933.3015 - val_loss: 31790248.7850 - val_mean_squared_error: 31790248.7850 Epoch 9/2000 - 0s - loss: 30735031.6336 - mean_squared_error: 30735031.6336 - val_loss: 31703530.1018 - val_mean_squared_error: 31703530.1018 Epoch 10/2000 - 0s - loss: 30650979.2737 - mean_squared_error: 30650979.2737 - val_loss: 31620223.0746 - val_mean_squared_error: 31620223.0746 Epoch 11/2000 - 0s - loss: 30564208.1914 - mean_squared_error: 30564208.1914 - val_loss: 31527235.8487 - val_mean_squared_error: 31527235.8487 Epoch 12/2000 - 0s - loss: 30471671.6893 - mean_squared_error: 30471671.6893 - val_loss: 31434505.1438 - val_mean_squared_error: 31434505.1438 Epoch 13/2000 - 0s - loss: 30382774.6087 - mean_squared_error: 30382774.6087 - val_loss: 31345867.3376 - val_mean_squared_error: 31345867.3376 Epoch 14/2000 - 0s - loss: 30295878.0809 - mean_squared_error: 30295878.0809 - val_loss: 31258539.8992 - val_mean_squared_error: 31258539.8992 Epoch 15/2000 - 0s - loss: 30210245.2780 - mean_squared_error: 30210245.2780 - val_loss: 31172163.3673 - val_mean_squared_error: 31172163.3673 Epoch 16/2000 - 0s - loss: 30125167.6951 - mean_squared_error: 30125167.6951 - val_loss: 31086263.9644 - val_mean_squared_error: 31086263.9644 Epoch 17/2000 - 0s - loss: 30040693.1683 - mean_squared_error: 30040693.1683 - val_loss: 31000947.0559 - val_mean_squared_error: 31000947.0559 Epoch 18/2000 - 0s - loss: 29956545.2932 - mean_squared_error: 29956545.2932 - val_loss: 30915863.5709 - val_mean_squared_error: 30915863.5709 Epoch 19/2000 - 0s - loss: 29872689.6088 - mean_squared_error: 29872689.6088 - val_loss: 30831129.4968 - val_mean_squared_error: 30831129.4968 Epoch 20/2000 - 0s - loss: 29789302.8007 - mean_squared_error: 29789302.8007 - val_loss: 30746769.3732 - val_mean_squared_error: 30746769.3732 Epoch 21/2000 - 0s - loss: 29706222.5647 - mean_squared_error: 29706222.5647 - val_loss: 30662787.0954 - val_mean_squared_error: 30662787.0954 Epoch 22/2000 - 0s - loss: 29623479.2155 - mean_squared_error: 29623479.2155 - val_loss: 30579097.6026 - val_mean_squared_error: 30579097.6026 Epoch 23/2000 - 0s - loss: 29540912.2036 - mean_squared_error: 29540912.2036 - val_loss: 30495587.1537 - val_mean_squared_error: 30495587.1537 Epoch 24/2000 - 0s - loss: 29458598.1043 - mean_squared_error: 29458598.1043 - val_loss: 30412313.0509 - val_mean_squared_error: 30412313.0509 Epoch 25/2000 - 0s - loss: 29376574.1866 - mean_squared_error: 29376574.1866 - val_loss: 30329317.4652 - val_mean_squared_error: 30329317.4652 Epoch 26/2000 - 0s - loss: 29294745.7942 - mean_squared_error: 29294745.7942 - val_loss: 30246553.0677 - val_mean_squared_error: 30246553.0677 Epoch 27/2000 - 0s - loss: 29213295.1197 - mean_squared_error: 29213295.1197 - val_loss: 30164191.6738 - val_mean_squared_error: 30164191.6738 Epoch 28/2000 - 0s - loss: 29132082.6000 - mean_squared_error: 29132082.6000 - val_loss: 30082031.7647 - val_mean_squared_error: 30082031.7647 Epoch 29/2000 - 0s - loss: 29051069.7344 - mean_squared_error: 29051069.7344 - val_loss: 30000055.4869 - val_mean_squared_error: 30000055.4869 Epoch 30/2000 - 0s - loss: 28970459.9729 - mean_squared_error: 28970459.9729 - val_loss: 29918478.6387 - val_mean_squared_error: 29918478.6387 Epoch 31/2000 - 0s - loss: 28889962.9339 - mean_squared_error: 28889962.9339 - val_loss: 29836927.4503 - val_mean_squared_error: 29836927.4503 Epoch 32/2000 - 0s - loss: 28809790.9063 - mean_squared_error: 28809790.9063 - val_loss: 29755820.0830 - val_mean_squared_error: 29755820.0830 Epoch 33/2000 - 0s - loss: 28729963.0352 - mean_squared_error: 28729963.0352 - val_loss: 29675137.9496 - val_mean_squared_error: 29675137.9496 Epoch 34/2000 - 0s - loss: 28650360.5442 - mean_squared_error: 28650360.5442 - val_loss: 29594480.1928 - val_mean_squared_error: 29594480.1928 Epoch 35/2000 - 0s - loss: 28570924.5415 - mean_squared_error: 28570924.5415 - val_loss: 29514072.8730 - val_mean_squared_error: 29514072.8730 Epoch 36/2000 - 0s - loss: 28491709.2935 - mean_squared_error: 28491709.2935 - val_loss: 29433960.4943 - val_mean_squared_error: 29433960.4943 Epoch 37/2000 - 0s - loss: 28412798.1367 - mean_squared_error: 28412798.1367 - val_loss: 29354021.7696 - val_mean_squared_error: 29354021.7696 Epoch 38/2000 - 0s - loss: 28334104.4454 - mean_squared_error: 28334104.4454 - val_loss: 29274362.7504 - val_mean_squared_error: 29274362.7504 Epoch 39/2000 - 0s - loss: 28255585.5671 - mean_squared_error: 28255585.5671 - val_loss: 29194935.7578 - val_mean_squared_error: 29194935.7578 Epoch 40/2000 - 0s - loss: 28177311.3649 - mean_squared_error: 28177311.3649 - val_loss: 29115711.6283 - val_mean_squared_error: 29115711.6283 Epoch 41/2000 - 0s - loss: 28099405.9583 - mean_squared_error: 28099405.9583 - val_loss: 29036816.6812 - val_mean_squared_error: 29036816.6812 Epoch 42/2000 - 0s - loss: 28021745.1748 - mean_squared_error: 28021745.1748 - val_loss: 28958206.8423 - val_mean_squared_error: 28958206.8423 Epoch 43/2000 - 0s - loss: 27944321.2553 - mean_squared_error: 27944321.2553 - val_loss: 28879830.6901 - val_mean_squared_error: 28879830.6901 Epoch 44/2000 - 0s - loss: 27867160.2867 - mean_squared_error: 27867160.2867 - val_loss: 28801703.4691 - val_mean_squared_error: 28801703.4691 Epoch 45/2000 - 0s - loss: 27790252.0029 - mean_squared_error: 27790252.0029 - val_loss: 28723817.4879 - val_mean_squared_error: 28723817.4879 Epoch 46/2000 - 0s - loss: 27713554.5320 - mean_squared_error: 27713554.5320 - val_loss: 28646179.9268 - val_mean_squared_error: 28646179.9268 Epoch 47/2000 - 0s - loss: 27637164.8281 - mean_squared_error: 27637164.8281 - val_loss: 28568974.5803 - val_mean_squared_error: 28568974.5803 Epoch 48/2000 - 0s - loss: 27560923.5064 - mean_squared_error: 27560923.5064 - val_loss: 28491688.7721 - val_mean_squared_error: 28491688.7721 Epoch 49/2000 - 0s - loss: 27485063.1059 - mean_squared_error: 27485063.1059 - val_loss: 28414863.0114 - val_mean_squared_error: 28414863.0114 Epoch 50/2000 - 0s - loss: 27409439.2144 - mean_squared_error: 27409439.2144 - val_loss: 28338173.7677 - val_mean_squared_error: 28338173.7677 Epoch 51/2000 - 0s - loss: 27334131.0234 - mean_squared_error: 27334131.0234 - val_loss: 28262001.8013 - val_mean_squared_error: 28262001.8013 Epoch 52/2000 - 0s - loss: 27258972.5527 - mean_squared_error: 27258972.5527 - val_loss: 28185839.0064 - val_mean_squared_error: 28185839.0064 Epoch 53/2000 - 0s - loss: 27184161.5334 - mean_squared_error: 27184161.5334 - val_loss: 28110077.3406 - val_mean_squared_error: 28110077.3406 Epoch 54/2000 - 0s - loss: 27109518.9422 - mean_squared_error: 27109518.9422 - val_loss: 28034423.0133 - val_mean_squared_error: 28034423.0133 Epoch 55/2000 - 0s - loss: 27035128.6361 - mean_squared_error: 27035128.6361 - val_loss: 27959151.0756 - val_mean_squared_error: 27959151.0756 Epoch 56/2000 - 0s - loss: 26960971.6158 - mean_squared_error: 26960971.6158 - val_loss: 27884048.2521 - val_mean_squared_error: 27884048.2521 Epoch 57/2000 - 0s - loss: 26887081.3519 - mean_squared_error: 26887081.3519 - val_loss: 27809200.8601 - val_mean_squared_error: 27809200.8601 Epoch 58/2000 - 0s - loss: 26813503.5389 - mean_squared_error: 26813503.5389 - val_loss: 27734627.8863 - val_mean_squared_error: 27734627.8863 Epoch 59/2000 - 0s - loss: 26740185.8467 - mean_squared_error: 26740185.8467 - val_loss: 27660315.3821 - val_mean_squared_error: 27660315.3821 Epoch 60/2000 - 0s - loss: 26667075.5801 - mean_squared_error: 26667075.5801 - val_loss: 27586281.0489 - val_mean_squared_error: 27586281.0489 Epoch 61/2000 - 0s - loss: 26594314.0441 - mean_squared_error: 26594314.0441 - val_loss: 27512567.3356 - val_mean_squared_error: 27512567.3356 Epoch 62/2000 - 0s - loss: 26521670.2177 - mean_squared_error: 26521670.2177 - val_loss: 27438897.1527 - val_mean_squared_error: 27438897.1527 Epoch 63/2000 - 0s - loss: 26449391.6573 - mean_squared_error: 26449391.6573 - val_loss: 27365727.1676 - val_mean_squared_error: 27365727.1676 Epoch 64/2000 - 0s - loss: 26377206.7494 - mean_squared_error: 26377206.7494 - val_loss: 27292557.8497 - val_mean_squared_error: 27292557.8497 Epoch 65/2000 - 0s - loss: 26305377.2261 - mean_squared_error: 26305377.2261 - val_loss: 27219727.1705 - val_mean_squared_error: 27219727.1705 Epoch 66/2000 - 0s - loss: 26233774.5640 - mean_squared_error: 26233774.5640 - val_loss: 27147248.8393 - val_mean_squared_error: 27147248.8393 Epoch 67/2000 - 0s - loss: 26162426.6621 - mean_squared_error: 26162426.6621 - val_loss: 27074894.4360 - val_mean_squared_error: 27074894.4360 Epoch 68/2000 - 0s - loss: 26091317.9390 - mean_squared_error: 26091317.9390 - val_loss: 27002882.7682 - val_mean_squared_error: 27002882.7682 Epoch 69/2000 - 0s - loss: 26020459.9902 - mean_squared_error: 26020459.9902 - val_loss: 26931139.3267 - val_mean_squared_error: 26931139.3267 Epoch 70/2000 - 0s - loss: 25949811.0884 - mean_squared_error: 25949811.0884 - val_loss: 26859526.5754 - val_mean_squared_error: 26859526.5754 Epoch 71/2000 - 0s - loss: 25879355.5384 - mean_squared_error: 25879355.5384 - val_loss: 26788026.7138 - val_mean_squared_error: 26788026.7138 Epoch 72/2000 - 0s - loss: 25809385.0737 - mean_squared_error: 25809385.0737 - val_loss: 26717113.4187 - val_mean_squared_error: 26717113.4187 Epoch 73/2000 - 0s - loss: 25739624.7902 - mean_squared_error: 25739624.7902 - val_loss: 26646419.4276 - val_mean_squared_error: 26646419.4276 Epoch 74/2000 - 0s - loss: 25670110.1360 - mean_squared_error: 25670110.1360 - val_loss: 26576000.0336 - val_mean_squared_error: 26576000.0336 Epoch 75/2000 - 0s - loss: 25600809.1711 - mean_squared_error: 25600809.1711 - val_loss: 26505692.0465 - val_mean_squared_error: 26505692.0465 Epoch 76/2000 - 0s - loss: 25531733.8437 - mean_squared_error: 25531733.8437 - val_loss: 26435641.7113 - val_mean_squared_error: 26435641.7113 Epoch 77/2000 - 0s - loss: 25462972.3005 - mean_squared_error: 25462972.3005 - val_loss: 26365985.2813 - val_mean_squared_error: 26365985.2813 Epoch 78/2000 - 0s - loss: 25394526.8229 - mean_squared_error: 25394526.8229 - val_loss: 26296592.8700 - val_mean_squared_error: 26296592.8700 Epoch 79/2000 - 0s - loss: 25326216.6557 - mean_squared_error: 25326216.6557 - val_loss: 26227286.5368 - val_mean_squared_error: 26227286.5368 Epoch 80/2000 - 0s - loss: 25258160.7971 - mean_squared_error: 25258160.7971 - val_loss: 26158228.9204 - val_mean_squared_error: 26158228.9204 Epoch 81/2000 - 0s - loss: 25190418.3716 - mean_squared_error: 25190418.3716 - val_loss: 26089568.0910 - val_mean_squared_error: 26089568.0910 Epoch 82/2000 - 0s - loss: 25122887.6684 - mean_squared_error: 25122887.6684 - val_loss: 26021011.0905 - val_mean_squared_error: 26021011.0905 Epoch 83/2000 - 0s - loss: 25055646.1286 - mean_squared_error: 25055646.1286 - val_loss: 25952877.7914 - val_mean_squared_error: 25952877.7914 Epoch 84/2000 - 0s - loss: 24988565.3453 - mean_squared_error: 24988565.3453 - val_loss: 25884860.6960 - val_mean_squared_error: 25884860.6960 Epoch 85/2000 - 0s - loss: 24921778.5702 - mean_squared_error: 24921778.5702 - val_loss: 25817128.1157 - val_mean_squared_error: 25817128.1157 Epoch 86/2000 - 0s - loss: 24855278.4745 - mean_squared_error: 24855278.4745 - val_loss: 25749717.6303 - val_mean_squared_error: 25749717.6303 Epoch 87/2000 - 0s - loss: 24789066.9677 - mean_squared_error: 24789066.9677 - val_loss: 25682459.5175 - val_mean_squared_error: 25682459.5175 Epoch 88/2000 - 0s - loss: 24723090.3912 - mean_squared_error: 24723090.3912 - val_loss: 25615506.6901 - val_mean_squared_error: 25615506.6901 Epoch 89/2000 - 0s - loss: 24657334.3180 - mean_squared_error: 24657334.3180 - val_loss: 25548882.4024 - val_mean_squared_error: 25548882.4024 Epoch 90/2000 - 0s - loss: 24591838.8394 - mean_squared_error: 24591838.8394 - val_loss: 25482345.4513 - val_mean_squared_error: 25482345.4513 Epoch 91/2000 - 0s - loss: 24526544.8664 - mean_squared_error: 24526544.8664 - val_loss: 25416164.0761 - val_mean_squared_error: 25416164.0761 Epoch 92/2000 - 0s - loss: 24461648.8232 - mean_squared_error: 24461648.8232 - val_loss: 25350287.8626 - val_mean_squared_error: 25350287.8626 Epoch 93/2000 - 0s - loss: 24396903.4064 - mean_squared_error: 24396903.4064 - val_loss: 25284579.1340 - val_mean_squared_error: 25284579.1340 Epoch 94/2000 - 0s - loss: 24332436.7249 - mean_squared_error: 24332436.7249 - val_loss: 25219231.1409 - val_mean_squared_error: 25219231.1409 Epoch 95/2000 - 0s - loss: 24268330.4633 - mean_squared_error: 24268330.4633 - val_loss: 25154138.4330 - val_mean_squared_error: 25154138.4330 Epoch 96/2000 - 0s - loss: 24204372.0587 - mean_squared_error: 24204372.0587 - val_loss: 25089203.5255 - val_mean_squared_error: 25089203.5255 Epoch 97/2000 - 0s - loss: 24140610.1780 - mean_squared_error: 24140610.1780 - val_loss: 25024465.2664 - val_mean_squared_error: 25024465.2664 Epoch 98/2000 - 0s - loss: 24077252.2966 - mean_squared_error: 24077252.2966 - val_loss: 24960163.7430 - val_mean_squared_error: 24960163.7430 Epoch 99/2000 - 0s - loss: 24013917.8359 - mean_squared_error: 24013917.8359 - val_loss: 24895891.1112 - val_mean_squared_error: 24895891.1112 Epoch 100/2000 - 0s - loss: 23951020.9714 - mean_squared_error: 23951020.9714 - val_loss: 24831932.6505 - val_mean_squared_error: 24831932.6505 Epoch 101/2000 - 0s - loss: 23888220.1202 - mean_squared_error: 23888220.1202 - val_loss: 24768160.1671 - val_mean_squared_error: 24768160.1671 Epoch 102/2000 - 0s - loss: 23825687.9442 - mean_squared_error: 23825687.9442 - val_loss: 24704775.5087 - val_mean_squared_error: 24704775.5087 Epoch 103/2000 - 0s - loss: 23763591.7315 - mean_squared_error: 23763591.7315 - val_loss: 24641768.4014 - val_mean_squared_error: 24641768.4014 Epoch 104/2000 - 0s - loss: 23701595.4344 - mean_squared_error: 23701595.4344 - val_loss: 24578754.7355 - val_mean_squared_error: 24578754.7355 Epoch 105/2000 - 0s - loss: 23639821.5487 - mean_squared_error: 23639821.5487 - val_loss: 24516050.8166 - val_mean_squared_error: 24516050.8166 Epoch 106/2000 - 0s - loss: 23578443.7971 - mean_squared_error: 23578443.7971 - val_loss: 24453735.3900 - val_mean_squared_error: 24453735.3900 Epoch 107/2000 - 0s - loss: 23517176.9031 - mean_squared_error: 23517176.9031 - val_loss: 24391556.2126 - val_mean_squared_error: 24391556.2126 Epoch 108/2000 - 0s - loss: 23456172.7204 - mean_squared_error: 23456172.7204 - val_loss: 24329568.1493 - val_mean_squared_error: 24329568.1493 Epoch 109/2000 - 0s - loss: 23395525.0800 - mean_squared_error: 23395525.0800 - val_loss: 24267975.9090 - val_mean_squared_error: 24267975.9090 Epoch 110/2000 - 0s - loss: 23335126.7265 - mean_squared_error: 23335126.7265 - val_loss: 24206602.6041 - val_mean_squared_error: 24206602.6041 Epoch 111/2000 - 0s - loss: 23274969.8814 - mean_squared_error: 23274969.8814 - val_loss: 24145492.8828 - val_mean_squared_error: 24145492.8828 Epoch 112/2000 - 0s - loss: 23215052.1725 - mean_squared_error: 23215052.1725 - val_loss: 24084635.8517 - val_mean_squared_error: 24084635.8517 Epoch 113/2000 - 0s - loss: 23155380.1623 - mean_squared_error: 23155380.1623 - val_loss: 24023868.4518 - val_mean_squared_error: 24023868.4518 Epoch 114/2000 - 0s - loss: 23095990.8764 - mean_squared_error: 23095990.8764 - val_loss: 23963653.1221 - val_mean_squared_error: 23963653.1221 Epoch 115/2000 - 0s - loss: 23036852.2296 - mean_squared_error: 23036852.2296 - val_loss: 23903552.8304 - val_mean_squared_error: 23903552.8304 Epoch 116/2000 - 0s - loss: 22977854.2557 - mean_squared_error: 22977854.2557 - val_loss: 23843577.2842 - val_mean_squared_error: 23843577.2842 Epoch 117/2000 - 0s - loss: 22919132.7115 - mean_squared_error: 22919132.7115 - val_loss: 23783941.1903 - val_mean_squared_error: 23783941.1903 Epoch 118/2000 - 0s - loss: 22860675.2993 - mean_squared_error: 22860675.2993 - val_loss: 23724452.7316 - val_mean_squared_error: 23724452.7316 Epoch 119/2000 - 0s - loss: 22802480.1820 - mean_squared_error: 22802480.1820 - val_loss: 23665374.7593 - val_mean_squared_error: 23665374.7593 Epoch 120/2000 - 0s - loss: 22744486.7348 - mean_squared_error: 22744486.7348 - val_loss: 23606429.2111 - val_mean_squared_error: 23606429.2111 Epoch 121/2000 - 0s - loss: 22686857.5650 - mean_squared_error: 22686857.5650 - val_loss: 23547899.4839 - val_mean_squared_error: 23547899.4839 Epoch 122/2000 - 0s - loss: 22629439.9110 - mean_squared_error: 22629439.9110 - val_loss: 23489489.6391 - val_mean_squared_error: 23489489.6391 Epoch 123/2000 - 0s - loss: 22572283.1380 - mean_squared_error: 22572283.1380 - val_loss: 23431363.3495 - val_mean_squared_error: 23431363.3495 Epoch 124/2000 - 0s - loss: 22515383.5186 - mean_squared_error: 22515383.5186 - val_loss: 23373465.8695 - val_mean_squared_error: 23373465.8695 Epoch 125/2000 - 0s - loss: 22458715.0694 - mean_squared_error: 22458715.0694 - val_loss: 23315892.6387 - val_mean_squared_error: 23315892.6387 Epoch 126/2000 - 0s - loss: 22402235.9658 - mean_squared_error: 22402235.9658 - val_loss: 23258538.3223 - val_mean_squared_error: 23258538.3223 Epoch 127/2000 - 0s - loss: 22346055.1642 - mean_squared_error: 22346055.1642 - val_loss: 23201325.3880 - val_mean_squared_error: 23201325.3880 Epoch 128/2000 - 0s - loss: 22290144.3171 - mean_squared_error: 22290144.3171 - val_loss: 23144563.7865 - val_mean_squared_error: 23144563.7865 Epoch 129/2000 - 0s - loss: 22234343.6437 - mean_squared_error: 22234343.6437 - val_loss: 23087770.5348 - val_mean_squared_error: 23087770.5348 Epoch 130/2000 - 0s - loss: 22179189.1702 - mean_squared_error: 22179189.1702 - val_loss: 23031742.3094 - val_mean_squared_error: 23031742.3094 Epoch 131/2000 - 0s - loss: 22124210.8962 - mean_squared_error: 22124210.8962 - val_loss: 22975737.6411 - val_mean_squared_error: 22975737.6411 Epoch 132/2000 - 0s - loss: 22069474.0278 - mean_squared_error: 22069474.0278 - val_loss: 22920065.3208 - val_mean_squared_error: 22920065.3208 Epoch 133/2000 - 0s - loss: 22014981.8085 - mean_squared_error: 22014981.8085 - val_loss: 22864551.8389 - val_mean_squared_error: 22864551.8389 Epoch 134/2000 - 0s - loss: 21960659.8293 - mean_squared_error: 21960659.8293 - val_loss: 22809443.7350 - val_mean_squared_error: 22809443.7350 Epoch 135/2000 - 0s - loss: 21906667.1327 - mean_squared_error: 21906667.1327 - val_loss: 22754352.2323 - val_mean_squared_error: 22754352.2323 Epoch 136/2000 - 0s - loss: 21852773.1706 - mean_squared_error: 21852773.1706 - val_loss: 22699528.5279 - val_mean_squared_error: 22699528.5279 Epoch 137/2000 - 0s - loss: 21799218.2557 - mean_squared_error: 21799218.2557 - val_loss: 22645053.6391 - val_mean_squared_error: 22645053.6391 Epoch 138/2000 - 0s - loss: 21745988.5180 - mean_squared_error: 21745988.5180 - val_loss: 22590829.6362 - val_mean_squared_error: 22590829.6362 Epoch 139/2000 - 0s - loss: 21692884.7601 - mean_squared_error: 21692884.7601 - val_loss: 22536811.3139 - val_mean_squared_error: 22536811.3139 Epoch 140/2000 - 0s - loss: 21639922.0363 - mean_squared_error: 21639922.0363 - val_loss: 22482902.6268 - val_mean_squared_error: 22482902.6268 Epoch 141/2000 - 0s - loss: 21587277.8681 - mean_squared_error: 21587277.8681 - val_loss: 22429230.5803 - val_mean_squared_error: 22429230.5803 Epoch 142/2000 - 0s - loss: 21534855.1150 - mean_squared_error: 21534855.1150 - val_loss: 22375962.8720 - val_mean_squared_error: 22375962.8720 Epoch 143/2000 - 0s - loss: 21482826.3633 - mean_squared_error: 21482826.3633 - val_loss: 22323030.2056 - val_mean_squared_error: 22323030.2056 Epoch 144/2000 - 0s - loss: 21431066.9901 - mean_squared_error: 21431066.9901 - val_loss: 22270324.7177 - val_mean_squared_error: 22270324.7177 Epoch 145/2000 - 0s - loss: 21379457.4848 - mean_squared_error: 21379457.4848 - val_loss: 22217833.6293 - val_mean_squared_error: 22217833.6293 Epoch 146/2000 - 0s - loss: 21328267.0717 - mean_squared_error: 21328267.0717 - val_loss: 22165618.7365 - val_mean_squared_error: 22165618.7365 Epoch 147/2000 - 0s - loss: 21277279.8207 - mean_squared_error: 21277279.8207 - val_loss: 22113582.5645 - val_mean_squared_error: 22113582.5645 Epoch 148/2000 - 0s - loss: 21226358.6830 - mean_squared_error: 21226358.6830 - val_loss: 22061819.5957 - val_mean_squared_error: 22061819.5957 Epoch 149/2000 - 0s - loss: 21175828.1236 - mean_squared_error: 21175828.1236 - val_loss: 22010352.4785 - val_mean_squared_error: 22010352.4785 Epoch 150/2000 - 0s - loss: 21125455.0652 - mean_squared_error: 21125455.0652 - val_loss: 21958971.6164 - val_mean_squared_error: 21958971.6164 Epoch 151/2000 - 0s - loss: 21075471.0393 - mean_squared_error: 21075471.0393 - val_loss: 21908054.8433 - val_mean_squared_error: 21908054.8433 Epoch 152/2000 - 0s - loss: 21025684.3461 - mean_squared_error: 21025684.3461 - val_loss: 21857321.4810 - val_mean_squared_error: 21857321.4810 Epoch 153/2000 - 0s - loss: 20976130.8501 - mean_squared_error: 20976130.8501 - val_loss: 21806772.0712 - val_mean_squared_error: 21806772.0712 Epoch 154/2000 - 0s - loss: 20926800.1894 - mean_squared_error: 20926800.1894 - val_loss: 21756469.2773 - val_mean_squared_error: 21756469.2773 Epoch 155/2000 - 0s - loss: 20877648.5055 - mean_squared_error: 20877648.5055 - val_loss: 21706429.8903 - val_mean_squared_error: 21706429.8903 Epoch 156/2000 - 0s - loss: 20828803.0014 - mean_squared_error: 20828803.0014 - val_loss: 21656640.7276 - val_mean_squared_error: 21656640.7276 Epoch 157/2000 - 0s - loss: 20780237.4358 - mean_squared_error: 20780237.4358 - val_loss: 21607164.1770 - val_mean_squared_error: 21607164.1770 Epoch 158/2000 - 0s - loss: 20731964.5111 - mean_squared_error: 20731964.5111 - val_loss: 21557928.7899 - val_mean_squared_error: 21557928.7899 Epoch 159/2000 - 0s - loss: 20683983.8124 - mean_squared_error: 20683983.8124 - val_loss: 21508942.7563 - val_mean_squared_error: 21508942.7563 Epoch 160/2000 - 0s - loss: 20636215.6980 - mean_squared_error: 20636215.6980 - val_loss: 21460266.9946 - val_mean_squared_error: 21460266.9946 Epoch 161/2000 - 0s - loss: 20588576.9063 - mean_squared_error: 20588576.9063 - val_loss: 21411659.6817 - val_mean_squared_error: 21411659.6817 Epoch 162/2000 - 0s - loss: 20541391.1964 - mean_squared_error: 20541391.1964 - val_loss: 21363572.7583 - val_mean_squared_error: 21363572.7583 Epoch 163/2000 - 0s - loss: 20494382.0617 - mean_squared_error: 20494382.0617 - val_loss: 21315584.2096 - val_mean_squared_error: 21315584.2096 Epoch 164/2000 - 0s - loss: 20447582.2063 - mean_squared_error: 20447582.2063 - val_loss: 21267848.2007 - val_mean_squared_error: 21267848.2007 Epoch 165/2000 - 0s - loss: 20401091.7874 - mean_squared_error: 20401091.7874 - val_loss: 21220484.2867 - val_mean_squared_error: 21220484.2867 Epoch 166/2000 - 0s - loss: 20354782.9350 - mean_squared_error: 20354782.9350 - val_loss: 21173170.9224 - val_mean_squared_error: 21173170.9224 Epoch 167/2000 - 0s - loss: 20308683.8885 - mean_squared_error: 20308683.8885 - val_loss: 21126166.0732 - val_mean_squared_error: 21126166.0732 Epoch 168/2000 - 0s - loss: 20262880.0498 - mean_squared_error: 20262880.0498 - val_loss: 21079453.9427 - val_mean_squared_error: 21079453.9427 Epoch 169/2000 - 0s - loss: 20217302.2948 - mean_squared_error: 20217302.2948 - val_loss: 21032954.3272 - val_mean_squared_error: 21032954.3272 Epoch 170/2000 - 0s - loss: 20172039.3799 - mean_squared_error: 20172039.3799 - val_loss: 20986661.4800 - val_mean_squared_error: 20986661.4800 Epoch 171/2000 - 0s - loss: 20127028.1829 - mean_squared_error: 20127028.1829 - val_loss: 20940699.0222 - val_mean_squared_error: 20940699.0222 Epoch 172/2000 - 0s - loss: 20082127.6462 - mean_squared_error: 20082127.6462 - val_loss: 20894755.4909 - val_mean_squared_error: 20894755.4909 Epoch 173/2000 - 0s - loss: 20037422.4802 - mean_squared_error: 20037422.4802 - val_loss: 20849197.8527 - val_mean_squared_error: 20849197.8527 *** WARNING: skipped 241242 bytes of output *** Epoch 1832/2000 - 0s - loss: 15919839.0633 - mean_squared_error: 15919839.0633 - val_loss: 16556644.7647 - val_mean_squared_error: 16556644.7647 Epoch 1833/2000 - 0s - loss: 15919840.3255 - mean_squared_error: 15919840.3255 - val_loss: 16556635.6149 - val_mean_squared_error: 16556635.6149 Epoch 1834/2000 - 0s - loss: 15919841.6200 - mean_squared_error: 15919841.6200 - val_loss: 16556647.3777 - val_mean_squared_error: 16556647.3777 Epoch 1835/2000 - 0s - loss: 15919840.0938 - mean_squared_error: 15919840.0938 - val_loss: 16556649.3386 - val_mean_squared_error: 16556649.3386 Epoch 1836/2000 - 0s - loss: 15919840.6451 - mean_squared_error: 15919840.6451 - val_loss: 16556650.5734 - val_mean_squared_error: 16556650.5734 Epoch 1837/2000 - 0s - loss: 15919842.1420 - mean_squared_error: 15919842.1420 - val_loss: 16556663.7143 - val_mean_squared_error: 16556663.7143 Epoch 1838/2000 - 0s - loss: 15919842.1427 - mean_squared_error: 15919842.1427 - val_loss: 16556685.6446 - val_mean_squared_error: 16556685.6446 Epoch 1839/2000 - 0s - loss: 15919841.2475 - mean_squared_error: 15919841.2475 - val_loss: 16556689.8868 - val_mean_squared_error: 16556689.8868 Epoch 1840/2000 - 0s - loss: 15919837.4733 - mean_squared_error: 15919837.4733 - val_loss: 16556697.9367 - val_mean_squared_error: 16556697.9367 Epoch 1841/2000 - 0s - loss: 15919838.4133 - mean_squared_error: 15919838.4133 - val_loss: 16556688.7870 - val_mean_squared_error: 16556688.7870 Epoch 1842/2000 - 0s - loss: 15919840.9300 - mean_squared_error: 15919840.9300 - val_loss: 16556710.4123 - val_mean_squared_error: 16556710.4123 Epoch 1843/2000 - 0s - loss: 15919842.6381 - mean_squared_error: 15919842.6381 - val_loss: 16556705.8774 - val_mean_squared_error: 16556705.8774 Epoch 1844/2000 - 0s - loss: 15919837.4598 - mean_squared_error: 15919837.4598 - val_loss: 16556709.6480 - val_mean_squared_error: 16556709.6480 Epoch 1845/2000 - 0s - loss: 15919838.9912 - mean_squared_error: 15919838.9912 - val_loss: 16556699.7350 - val_mean_squared_error: 16556699.7350 Epoch 1846/2000 - 0s - loss: 15919842.1577 - mean_squared_error: 15919842.1577 - val_loss: 16556698.5793 - val_mean_squared_error: 16556698.5793 Epoch 1847/2000 - 0s - loss: 15919842.4977 - mean_squared_error: 15919842.4977 - val_loss: 16556697.8873 - val_mean_squared_error: 16556697.8873 Epoch 1848/2000 - 0s - loss: 15919841.3402 - mean_squared_error: 15919841.3402 - val_loss: 16556698.8255 - val_mean_squared_error: 16556698.8255 Epoch 1849/2000 - 0s - loss: 15919838.7708 - mean_squared_error: 15919838.7708 - val_loss: 16556703.6095 - val_mean_squared_error: 16556703.6095 Epoch 1850/2000 - 0s - loss: 15919837.3867 - mean_squared_error: 15919837.3867 - val_loss: 16556710.7069 - val_mean_squared_error: 16556710.7069 Epoch 1851/2000 - 0s - loss: 15919837.8497 - mean_squared_error: 15919837.8497 - val_loss: 16556712.7182 - val_mean_squared_error: 16556712.7182 Epoch 1852/2000 - 0s - loss: 15919836.2385 - mean_squared_error: 15919836.2385 - val_loss: 16556729.8883 - val_mean_squared_error: 16556729.8883 Epoch 1853/2000 - 0s - loss: 15919839.4165 - mean_squared_error: 15919839.4165 - val_loss: 16556707.3915 - val_mean_squared_error: 16556707.3915 Epoch 1854/2000 - 0s - loss: 15919839.0620 - mean_squared_error: 15919839.0620 - val_loss: 16556693.9283 - val_mean_squared_error: 16556693.9283 Epoch 1855/2000 - 0s - loss: 15919840.7434 - mean_squared_error: 15919840.7434 - val_loss: 16556700.7958 - val_mean_squared_error: 16556700.7958 Epoch 1856/2000 - 0s - loss: 15919839.6519 - mean_squared_error: 15919839.6519 - val_loss: 16556705.2462 - val_mean_squared_error: 16556705.2462 Epoch 1857/2000 - 0s - loss: 15919839.2747 - mean_squared_error: 15919839.2747 - val_loss: 16556678.6411 - val_mean_squared_error: 16556678.6411 Epoch 1858/2000 - 0s - loss: 15919837.8284 - mean_squared_error: 15919837.8284 - val_loss: 16556697.2106 - val_mean_squared_error: 16556697.2106 Epoch 1859/2000 - 0s - loss: 15919842.1135 - mean_squared_error: 15919842.1135 - val_loss: 16556700.1913 - val_mean_squared_error: 16556700.1913 Epoch 1860/2000 - 0s - loss: 15919840.7381 - mean_squared_error: 15919840.7381 - val_loss: 16556703.0657 - val_mean_squared_error: 16556703.0657 Epoch 1861/2000 - 0s - loss: 15919837.5396 - mean_squared_error: 15919837.5396 - val_loss: 16556699.3920 - val_mean_squared_error: 16556699.3920 Epoch 1862/2000 - 0s - loss: 15919838.4874 - mean_squared_error: 15919838.4874 - val_loss: 16556716.9224 - val_mean_squared_error: 16556716.9224 Epoch 1863/2000 - 0s - loss: 15919837.8046 - mean_squared_error: 15919837.8046 - val_loss: 16556725.6139 - val_mean_squared_error: 16556725.6139 Epoch 1864/2000 - 0s - loss: 15919840.9139 - mean_squared_error: 15919840.9139 - val_loss: 16556720.3747 - val_mean_squared_error: 16556720.3747 Epoch 1865/2000 - 0s - loss: 15919840.1795 - mean_squared_error: 15919840.1795 - val_loss: 16556722.3109 - val_mean_squared_error: 16556722.3109 Epoch 1866/2000 - 0s - loss: 15919843.0618 - mean_squared_error: 15919843.0618 - val_loss: 16556721.0391 - val_mean_squared_error: 16556721.0391 Epoch 1867/2000 - 0s - loss: 15919841.2889 - mean_squared_error: 15919841.2889 - val_loss: 16556744.9086 - val_mean_squared_error: 16556744.9086 Epoch 1868/2000 - 0s - loss: 15919839.5626 - mean_squared_error: 15919839.5626 - val_loss: 16556722.7899 - val_mean_squared_error: 16556722.7899 Epoch 1869/2000 - 0s - loss: 15919845.4279 - mean_squared_error: 15919845.4279 - val_loss: 16556720.3470 - val_mean_squared_error: 16556720.3470 Epoch 1870/2000 - 0s - loss: 15919837.2327 - mean_squared_error: 15919837.2327 - val_loss: 16556730.4474 - val_mean_squared_error: 16556730.4474 Epoch 1871/2000 - 0s - loss: 15919836.4915 - mean_squared_error: 15919836.4915 - val_loss: 16556728.8285 - val_mean_squared_error: 16556728.8285 Epoch 1872/2000 - 0s - loss: 15919848.0744 - mean_squared_error: 15919848.0744 - val_loss: 16556742.1928 - val_mean_squared_error: 16556742.1928 Epoch 1873/2000 - 0s - loss: 15919837.7825 - mean_squared_error: 15919837.7825 - val_loss: 16556739.9011 - val_mean_squared_error: 16556739.9011 Epoch 1874/2000 - 0s - loss: 15919839.3969 - mean_squared_error: 15919839.3969 - val_loss: 16556747.4587 - val_mean_squared_error: 16556747.4587 Epoch 1875/2000 - 0s - loss: 15919839.6557 - mean_squared_error: 15919839.6557 - val_loss: 16556750.2323 - val_mean_squared_error: 16556750.2323 Epoch 1876/2000 - 0s - loss: 15919835.1444 - mean_squared_error: 15919835.1444 - val_loss: 16556749.6050 - val_mean_squared_error: 16556749.6050 Epoch 1877/2000 - 0s - loss: 15919836.5018 - mean_squared_error: 15919836.5018 - val_loss: 16556753.1676 - val_mean_squared_error: 16556753.1676 Epoch 1878/2000 - 0s - loss: 15919839.9839 - mean_squared_error: 15919839.9839 - val_loss: 16556745.1547 - val_mean_squared_error: 16556745.1547 Epoch 1879/2000 - 0s - loss: 15919842.4734 - mean_squared_error: 15919842.4734 - val_loss: 16556743.0104 - val_mean_squared_error: 16556743.0104 Epoch 1880/2000 - 0s - loss: 15919841.0024 - mean_squared_error: 15919841.0024 - val_loss: 16556740.5314 - val_mean_squared_error: 16556740.5314 Epoch 1881/2000 - 0s - loss: 15919838.0398 - mean_squared_error: 15919838.0398 - val_loss: 16556748.0262 - val_mean_squared_error: 16556748.0262 Epoch 1882/2000 - 0s - loss: 15919840.2566 - mean_squared_error: 15919840.2566 - val_loss: 16556742.1710 - val_mean_squared_error: 16556742.1710 Epoch 1883/2000 - 0s - loss: 15919839.9691 - mean_squared_error: 15919839.9691 - val_loss: 16556754.4973 - val_mean_squared_error: 16556754.4973 Epoch 1884/2000 - 0s - loss: 15919842.2178 - mean_squared_error: 15919842.2178 - val_loss: 16556768.8739 - val_mean_squared_error: 16556768.8739 Epoch 1885/2000 - 0s - loss: 15919836.8102 - mean_squared_error: 15919836.8102 - val_loss: 16556762.7469 - val_mean_squared_error: 16556762.7469 Epoch 1886/2000 - 0s - loss: 15919842.2946 - mean_squared_error: 15919842.2946 - val_loss: 16556747.3361 - val_mean_squared_error: 16556747.3361 Epoch 1887/2000 - 0s - loss: 15919845.0852 - mean_squared_error: 15919845.0852 - val_loss: 16556757.1236 - val_mean_squared_error: 16556757.1236 Epoch 1888/2000 - 0s - loss: 15919842.2618 - mean_squared_error: 15919842.2618 - val_loss: 16556758.4439 - val_mean_squared_error: 16556758.4439 Epoch 1889/2000 - 0s - loss: 15919836.4589 - mean_squared_error: 15919836.4589 - val_loss: 16556775.5561 - val_mean_squared_error: 16556775.5561 Epoch 1890/2000 - 0s - loss: 15919837.9622 - mean_squared_error: 15919837.9622 - val_loss: 16556770.6382 - val_mean_squared_error: 16556770.6382 Epoch 1891/2000 - 0s - loss: 15919837.7282 - mean_squared_error: 15919837.7282 - val_loss: 16556775.9486 - val_mean_squared_error: 16556775.9486 Epoch 1892/2000 - 0s - loss: 15919838.4250 - mean_squared_error: 15919838.4250 - val_loss: 16556779.2289 - val_mean_squared_error: 16556779.2289 Epoch 1893/2000 - 0s - loss: 15919843.4984 - mean_squared_error: 15919843.4984 - val_loss: 16556777.0732 - val_mean_squared_error: 16556777.0732 Epoch 1894/2000 - 0s - loss: 15919843.3910 - mean_squared_error: 15919843.3910 - val_loss: 16556771.6846 - val_mean_squared_error: 16556771.6846 Epoch 1895/2000 - 0s - loss: 15919844.0152 - mean_squared_error: 15919844.0152 - val_loss: 16556793.1028 - val_mean_squared_error: 16556793.1028 Epoch 1896/2000 - 0s - loss: 15919838.8479 - mean_squared_error: 15919838.8479 - val_loss: 16556779.3030 - val_mean_squared_error: 16556779.3030 Epoch 1897/2000 - 0s - loss: 15919841.8588 - mean_squared_error: 15919841.8588 - val_loss: 16556779.0929 - val_mean_squared_error: 16556779.0929 Epoch 1898/2000 - 0s - loss: 15919841.9264 - mean_squared_error: 15919841.9264 - val_loss: 16556775.5067 - val_mean_squared_error: 16556775.5067 Epoch 1899/2000 - 0s - loss: 15919837.4295 - mean_squared_error: 15919837.4295 - val_loss: 16556756.5571 - val_mean_squared_error: 16556756.5571 Epoch 1900/2000 - 0s - loss: 15919842.6663 - mean_squared_error: 15919842.6663 - val_loss: 16556770.0306 - val_mean_squared_error: 16556770.0306 Epoch 1901/2000 - 0s - loss: 15919840.8584 - mean_squared_error: 15919840.8584 - val_loss: 16556772.1295 - val_mean_squared_error: 16556772.1295 Epoch 1902/2000 - 0s - loss: 15919838.3398 - mean_squared_error: 15919838.3398 - val_loss: 16556772.9174 - val_mean_squared_error: 16556772.9174 Epoch 1903/2000 - 0s - loss: 15919840.8919 - mean_squared_error: 15919840.8919 - val_loss: 16556779.0826 - val_mean_squared_error: 16556779.0826 Epoch 1904/2000 - 0s - loss: 15919840.6112 - mean_squared_error: 15919840.6112 - val_loss: 16556777.8384 - val_mean_squared_error: 16556777.8384 Epoch 1905/2000 - 0s - loss: 15919841.8981 - mean_squared_error: 15919841.8981 - val_loss: 16556751.7761 - val_mean_squared_error: 16556751.7761 Epoch 1906/2000 - 0s - loss: 15919839.4216 - mean_squared_error: 15919839.4216 - val_loss: 16556758.8156 - val_mean_squared_error: 16556758.8156 Epoch 1907/2000 - 0s - loss: 15919839.1625 - mean_squared_error: 15919839.1625 - val_loss: 16556743.3915 - val_mean_squared_error: 16556743.3915 Epoch 1908/2000 - 0s - loss: 15919844.9876 - mean_squared_error: 15919844.9876 - val_loss: 16556745.4009 - val_mean_squared_error: 16556745.4009 Epoch 1909/2000 - 0s - loss: 15919838.2083 - mean_squared_error: 15919838.2083 - val_loss: 16556738.0514 - val_mean_squared_error: 16556738.0514 Epoch 1910/2000 - 0s - loss: 15919838.9869 - mean_squared_error: 15919838.9869 - val_loss: 16556746.2289 - val_mean_squared_error: 16556746.2289 Epoch 1911/2000 - 0s - loss: 15919837.5660 - mean_squared_error: 15919837.5660 - val_loss: 16556758.8403 - val_mean_squared_error: 16556758.8403 Epoch 1912/2000 - 0s - loss: 15919839.5910 - mean_squared_error: 15919839.5910 - val_loss: 16556757.6569 - val_mean_squared_error: 16556757.6569 Epoch 1913/2000 - 0s - loss: 15919841.6379 - mean_squared_error: 15919841.6379 - val_loss: 16556757.2595 - val_mean_squared_error: 16556757.2595 Epoch 1914/2000 - 0s - loss: 15919838.3454 - mean_squared_error: 15919838.3454 - val_loss: 16556737.4622 - val_mean_squared_error: 16556737.4622 Epoch 1915/2000 - 0s - loss: 15919836.9584 - mean_squared_error: 15919836.9584 - val_loss: 16556750.6288 - val_mean_squared_error: 16556750.6288 Epoch 1916/2000 - 0s - loss: 15919841.6337 - mean_squared_error: 15919841.6337 - val_loss: 16556742.0465 - val_mean_squared_error: 16556742.0465 Epoch 1917/2000 - 0s - loss: 15919837.1959 - mean_squared_error: 15919837.1959 - val_loss: 16556732.6154 - val_mean_squared_error: 16556732.6154 Epoch 1918/2000 - 0s - loss: 15919839.7069 - mean_squared_error: 15919839.7069 - val_loss: 16556722.4315 - val_mean_squared_error: 16556722.4315 Epoch 1919/2000 - 0s - loss: 15919841.5924 - mean_squared_error: 15919841.5924 - val_loss: 16556710.8077 - val_mean_squared_error: 16556710.8077 Epoch 1920/2000 - 0s - loss: 15919838.2981 - mean_squared_error: 15919838.2981 - val_loss: 16556726.9560 - val_mean_squared_error: 16556726.9560 Epoch 1921/2000 - 0s - loss: 15919842.2630 - mean_squared_error: 15919842.2630 - val_loss: 16556718.1676 - val_mean_squared_error: 16556718.1676 Epoch 1922/2000 - 0s - loss: 15919841.9537 - mean_squared_error: 15919841.9537 - val_loss: 16556696.9728 - val_mean_squared_error: 16556696.9728 Epoch 1923/2000 - 0s - loss: 15919837.5424 - mean_squared_error: 15919837.5424 - val_loss: 16556700.0440 - val_mean_squared_error: 16556700.0440 Epoch 1924/2000 - 0s - loss: 15919838.7335 - mean_squared_error: 15919838.7335 - val_loss: 16556691.4266 - val_mean_squared_error: 16556691.4266 Epoch 1925/2000 - 0s - loss: 15919841.1641 - mean_squared_error: 15919841.1641 - val_loss: 16556709.9333 - val_mean_squared_error: 16556709.9333 Epoch 1926/2000 - 0s - loss: 15919839.5108 - mean_squared_error: 15919839.5108 - val_loss: 16556718.7815 - val_mean_squared_error: 16556718.7815 Epoch 1927/2000 - 0s - loss: 15919843.1280 - mean_squared_error: 15919843.1280 - val_loss: 16556725.6747 - val_mean_squared_error: 16556725.6747 Epoch 1928/2000 - 0s - loss: 15919839.0706 - mean_squared_error: 15919839.0706 - val_loss: 16556712.9263 - val_mean_squared_error: 16556712.9263 Epoch 1929/2000 - 0s - loss: 15919838.6063 - mean_squared_error: 15919838.6063 - val_loss: 16556722.3450 - val_mean_squared_error: 16556722.3450 Epoch 1930/2000 - 0s - loss: 15919834.3925 - mean_squared_error: 15919834.3925 - val_loss: 16556733.3292 - val_mean_squared_error: 16556733.3292 Epoch 1931/2000 - 0s - loss: 15919839.9176 - mean_squared_error: 15919839.9176 - val_loss: 16556740.5798 - val_mean_squared_error: 16556740.5798 Epoch 1932/2000 - 0s - loss: 15919837.0179 - mean_squared_error: 15919837.0179 - val_loss: 16556745.3638 - val_mean_squared_error: 16556745.3638 Epoch 1933/2000 - 0s - loss: 15919848.6947 - mean_squared_error: 15919848.6947 - val_loss: 16556746.5329 - val_mean_squared_error: 16556746.5329 Epoch 1934/2000 - 0s - loss: 15919844.9725 - mean_squared_error: 15919844.9725 - val_loss: 16556730.0910 - val_mean_squared_error: 16556730.0910 Epoch 1935/2000 - 0s - loss: 15919841.7440 - mean_squared_error: 15919841.7440 - val_loss: 16556734.9565 - val_mean_squared_error: 16556734.9565 Epoch 1936/2000 - 0s - loss: 15919842.5312 - mean_squared_error: 15919842.5312 - val_loss: 16556726.4142 - val_mean_squared_error: 16556726.4142 Epoch 1937/2000 - 0s - loss: 15919840.9410 - mean_squared_error: 15919840.9410 - val_loss: 16556731.3485 - val_mean_squared_error: 16556731.3485 Epoch 1938/2000 - 0s - loss: 15919840.0616 - mean_squared_error: 15919840.0616 - val_loss: 16556749.6298 - val_mean_squared_error: 16556749.6298 Epoch 1939/2000 - 0s - loss: 15919847.3593 - mean_squared_error: 15919847.3593 - val_loss: 16556732.9338 - val_mean_squared_error: 16556732.9338 Epoch 1940/2000 - 0s - loss: 15919842.1849 - mean_squared_error: 15919842.1849 - val_loss: 16556737.9916 - val_mean_squared_error: 16556737.9916 Epoch 1941/2000 - 0s - loss: 15919838.7040 - mean_squared_error: 15919838.7040 - val_loss: 16556734.5022 - val_mean_squared_error: 16556734.5022 Epoch 1942/2000 - 0s - loss: 15919838.1714 - mean_squared_error: 15919838.1714 - val_loss: 16556738.9278 - val_mean_squared_error: 16556738.9278 Epoch 1943/2000 - 0s - loss: 15919840.9966 - mean_squared_error: 15919840.9966 - val_loss: 16556731.4320 - val_mean_squared_error: 16556731.4320 Epoch 1944/2000 - 0s - loss: 15919839.2703 - mean_squared_error: 15919839.2703 - val_loss: 16556749.2343 - val_mean_squared_error: 16556749.2343 Epoch 1945/2000 - 0s - loss: 15919835.8730 - mean_squared_error: 15919835.8730 - val_loss: 16556757.6569 - val_mean_squared_error: 16556757.6569 Epoch 1946/2000 - 0s - loss: 15919843.3970 - mean_squared_error: 15919843.3970 - val_loss: 16556749.2126 - val_mean_squared_error: 16556749.2126 Epoch 1947/2000 - 0s - loss: 15919836.3736 - mean_squared_error: 15919836.3736 - val_loss: 16556777.0475 - val_mean_squared_error: 16556777.0475 Epoch 1948/2000 - 0s - loss: 15919838.4017 - mean_squared_error: 15919838.4017 - val_loss: 16556770.5734 - val_mean_squared_error: 16556770.5734 Epoch 1949/2000 - 0s - loss: 15919841.0001 - mean_squared_error: 15919841.0001 - val_loss: 16556777.0475 - val_mean_squared_error: 16556777.0475 Epoch 1950/2000 - 0s - loss: 15919840.6887 - mean_squared_error: 15919840.6887 - val_loss: 16556797.0094 - val_mean_squared_error: 16556797.0094 Epoch 1951/2000 - 0s - loss: 15919837.1316 - mean_squared_error: 15919837.1316 - val_loss: 16556769.6401 - val_mean_squared_error: 16556769.6401 Epoch 1952/2000 - 0s - loss: 15919839.7709 - mean_squared_error: 15919839.7709 - val_loss: 16556784.5185 - val_mean_squared_error: 16556784.5185 Epoch 1953/2000 - 0s - loss: 15919842.3838 - mean_squared_error: 15919842.3838 - val_loss: 16556778.0722 - val_mean_squared_error: 16556778.0722 Epoch 1954/2000 - 0s - loss: 15919838.7597 - mean_squared_error: 15919838.7597 - val_loss: 16556758.4439 - val_mean_squared_error: 16556758.4439 Epoch 1955/2000 - 0s - loss: 15919837.6232 - mean_squared_error: 15919837.6232 - val_loss: 16556769.7741 - val_mean_squared_error: 16556769.7741 Epoch 1956/2000 - 0s - loss: 15919841.1736 - mean_squared_error: 15919841.1736 - val_loss: 16556783.8428 - val_mean_squared_error: 16556783.8428 Epoch 1957/2000 - 0s - loss: 15919838.4189 - mean_squared_error: 15919838.4189 - val_loss: 16556789.1789 - val_mean_squared_error: 16556789.1789 Epoch 1958/2000 - 0s - loss: 15919844.6613 - mean_squared_error: 15919844.6613 - val_loss: 16556774.1008 - val_mean_squared_error: 16556774.1008 Epoch 1959/2000 - 0s - loss: 15919839.5645 - mean_squared_error: 15919839.5645 - val_loss: 16556772.2037 - val_mean_squared_error: 16556772.2037 Epoch 1960/2000 - 0s - loss: 15919837.8679 - mean_squared_error: 15919837.8679 - val_loss: 16556784.0519 - val_mean_squared_error: 16556784.0519 Epoch 1961/2000 - 0s - loss: 15919845.2541 - mean_squared_error: 15919845.2541 - val_loss: 16556773.5858 - val_mean_squared_error: 16556773.5858 Epoch 1962/2000 - 0s - loss: 15919843.2494 - mean_squared_error: 15919843.2494 - val_loss: 16556760.4770 - val_mean_squared_error: 16556760.4770 Epoch 1963/2000 - 0s - loss: 15919841.6358 - mean_squared_error: 15919841.6358 - val_loss: 16556767.1972 - val_mean_squared_error: 16556767.1972 Epoch 1964/2000 - 0s - loss: 15919845.4326 - mean_squared_error: 15919845.4326 - val_loss: 16556776.0381 - val_mean_squared_error: 16556776.0381 Epoch 1965/2000 - 0s - loss: 15919841.3805 - mean_squared_error: 15919841.3805 - val_loss: 16556780.6110 - val_mean_squared_error: 16556780.6110 Epoch 1966/2000 - 0s - loss: 15919840.0666 - mean_squared_error: 15919840.0666 - val_loss: 16556780.5225 - val_mean_squared_error: 16556780.5225 Epoch 1967/2000 - 0s - loss: 15919842.7959 - mean_squared_error: 15919842.7959 - val_loss: 16556774.6332 - val_mean_squared_error: 16556774.6332 Epoch 1968/2000 - 0s - loss: 15919840.4225 - mean_squared_error: 15919840.4225 - val_loss: 16556779.5625 - val_mean_squared_error: 16556779.5625 Epoch 1969/2000 - 0s - loss: 15919840.7085 - mean_squared_error: 15919840.7085 - val_loss: 16556764.1537 - val_mean_squared_error: 16556764.1537 Epoch 1970/2000 - 0s - loss: 15919842.7566 - mean_squared_error: 15919842.7566 - val_loss: 16556765.7059 - val_mean_squared_error: 16556765.7059 Epoch 1971/2000 - 0s - loss: 15919838.0659 - mean_squared_error: 15919838.0659 - val_loss: 16556773.7187 - val_mean_squared_error: 16556773.7187 Epoch 1972/2000 - 0s - loss: 15919840.6388 - mean_squared_error: 15919840.6388 - val_loss: 16556791.3376 - val_mean_squared_error: 16556791.3376 Epoch 1973/2000 - 0s - loss: 15919840.7910 - mean_squared_error: 15919840.7910 - val_loss: 16556788.7588 - val_mean_squared_error: 16556788.7588 Epoch 1974/2000 - 0s - loss: 15919847.2463 - mean_squared_error: 15919847.2463 - val_loss: 16556776.7909 - val_mean_squared_error: 16556776.7909 Epoch 1975/2000 - 0s - loss: 15919839.2122 - mean_squared_error: 15919839.2122 - val_loss: 16556773.5858 - val_mean_squared_error: 16556773.5858 Epoch 1976/2000 - 0s - loss: 15919842.3183 - mean_squared_error: 15919842.3183 - val_loss: 16556770.4745 - val_mean_squared_error: 16556770.4745 Epoch 1977/2000 - 0s - loss: 15919842.5022 - mean_squared_error: 15919842.5022 - val_loss: 16556780.1186 - val_mean_squared_error: 16556780.1186 Epoch 1978/2000 - 0s - loss: 15919842.7939 - mean_squared_error: 15919842.7939 - val_loss: 16556785.3589 - val_mean_squared_error: 16556785.3589 Epoch 1979/2000 - 0s - loss: 15919842.2156 - mean_squared_error: 15919842.2156 - val_loss: 16556791.3109 - val_mean_squared_error: 16556791.3109 Epoch 1980/2000 - 0s - loss: 15919837.1162 - mean_squared_error: 15919837.1162 - val_loss: 16556786.1241 - val_mean_squared_error: 16556786.1241 Epoch 1981/2000 - 0s - loss: 15919843.6045 - mean_squared_error: 15919843.6045 - val_loss: 16556763.5976 - val_mean_squared_error: 16556763.5976 Epoch 1982/2000 - 0s - loss: 15919840.3795 - mean_squared_error: 15919840.3795 - val_loss: 16556779.0465 - val_mean_squared_error: 16556779.0465 Epoch 1983/2000 - 0s - loss: 15919840.6743 - mean_squared_error: 15919840.6743 - val_loss: 16556755.6426 - val_mean_squared_error: 16556755.6426 Epoch 1984/2000 - 0s - loss: 15919839.0763 - mean_squared_error: 15919839.0763 - val_loss: 16556746.8809 - val_mean_squared_error: 16556746.8809 Epoch 1985/2000 - 0s - loss: 15919843.7900 - mean_squared_error: 15919843.7900 - val_loss: 16556765.8789 - val_mean_squared_error: 16556765.8789 Epoch 1986/2000 - 0s - loss: 15919839.8720 - mean_squared_error: 15919839.8720 - val_loss: 16556766.1518 - val_mean_squared_error: 16556766.1518 Epoch 1987/2000 - 0s - loss: 15919841.0103 - mean_squared_error: 15919841.0103 - val_loss: 16556761.3658 - val_mean_squared_error: 16556761.3658 Epoch 1988/2000 - 0s - loss: 15919844.0040 - mean_squared_error: 15919844.0040 - val_loss: 16556743.2175 - val_mean_squared_error: 16556743.2175 Epoch 1989/2000 - 0s - loss: 15919840.8275 - mean_squared_error: 15919840.8275 - val_loss: 16556755.9051 - val_mean_squared_error: 16556755.9051 Epoch 1990/2000 - 0s - loss: 15919839.0663 - mean_squared_error: 15919839.0663 - val_loss: 16556755.2264 - val_mean_squared_error: 16556755.2264 Epoch 1991/2000 - 0s - loss: 15919841.7517 - mean_squared_error: 15919841.7517 - val_loss: 16556744.1176 - val_mean_squared_error: 16556744.1176 Epoch 1992/2000 - 0s - loss: 15919839.2471 - mean_squared_error: 15919839.2471 - val_loss: 16556753.8700 - val_mean_squared_error: 16556753.8700 Epoch 1993/2000 - 0s - loss: 15919839.7289 - mean_squared_error: 15919839.7289 - val_loss: 16556742.0484 - val_mean_squared_error: 16556742.0484 Epoch 1994/2000 - 0s - loss: 15919842.0739 - mean_squared_error: 15919842.0739 - val_loss: 16556749.2343 - val_mean_squared_error: 16556749.2343 Epoch 1995/2000 - 0s - loss: 15919844.3030 - mean_squared_error: 15919844.3030 - val_loss: 16556739.4686 - val_mean_squared_error: 16556739.4686 Epoch 1996/2000 - 0s - loss: 15919837.5737 - mean_squared_error: 15919837.5737 - val_loss: 16556745.3144 - val_mean_squared_error: 16556745.3144 Epoch 1997/2000 - 0s - loss: 15919836.2656 - mean_squared_error: 15919836.2656 - val_loss: 16556755.5467 - val_mean_squared_error: 16556755.5467 Epoch 1998/2000 - 0s - loss: 15919835.8301 - mean_squared_error: 15919835.8301 - val_loss: 16556757.5057 - val_mean_squared_error: 16556757.5057 Epoch 1999/2000 - 0s - loss: 15919842.5290 - mean_squared_error: 15919842.5290 - val_loss: 16556768.8245 - val_mean_squared_error: 16556768.8245 Epoch 2000/2000 - 0s - loss: 15919839.9017 - mean_squared_error: 15919839.9017 - val_loss: 16556769.6495 - val_mean_squared_error: 16556769.6495 32/13485 [..............................] - ETA: 0s 4384/13485 [========>.....................] - ETA: 0s 8640/13485 [==================>...........] - ETA: 0s 12960/13485 [===========================>..] - ETA: 0s 13485/13485 [==============================] - 0s 12us/step root mean_squared_error: 3963.688024

What is different here?

We've changed the activation in the hidden layer to "sigmoid" per our discussion.
Next, notice that we're running 2000 training epochs!

Even so, it takes a looooong time to converge. If you experiment a lot, you'll find that ... it still takes a long time to converge. Around the early part of the most recent deep learning renaissance, researchers started experimenting with other non-linearities.

(Remember, we're talking about non-linear activations in the hidden layer. The output here is still using "linear" rather than "softmax" because we're performing regression, not classification.)

In theory, any non-linearity should allow learning, and maybe we can use one that "works better"

By "works better" we mean

Simpler gradient - faster to compute
Less prone to "saturation" -- where the neuron ends up way off in the 0 or 1 territory of the sigmoid and can't easily learn anything
Keeps gradients "big" -- avoiding the large, flat, near-zero gradient areas of the sigmoid

Turns out that a big breakthrough and popular solution is a very simple hack:

Rectified Linear Unit (ReLU)

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].as_matrix().flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).as_matrix()
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='relu')) # <--- CHANGE IS HERE
model.add(Dense(1, kernel_initializer='normal', activation='linear'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=2000, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples Epoch 1/2000 - 1s - loss: 29938863.4237 - mean_squared_error: 29938863.4237 - val_loss: 27832422.8473 - val_mean_squared_error: 27832422.8473 Epoch 2/2000 - 0s - loss: 22388635.5017 - mean_squared_error: 22388635.5017 - val_loss: 19167540.5645 - val_mean_squared_error: 19167540.5645 Epoch 3/2000 - 0s - loss: 16431872.0007 - mean_squared_error: 16431872.0007 - val_loss: 16017428.7776 - val_mean_squared_error: 16017428.7776 Epoch 4/2000 - 0s - loss: 15169837.3108 - mean_squared_error: 15169837.3108 - val_loss: 15670100.1587 - val_mean_squared_error: 15670100.1587 Epoch 5/2000 - 0s - loss: 15005770.5474 - mean_squared_error: 15005770.5474 - val_loss: 15541303.1453 - val_mean_squared_error: 15541303.1453 Epoch 6/2000 - 0s - loss: 14886009.9103 - mean_squared_error: 14886009.9103 - val_loss: 15410182.4088 - val_mean_squared_error: 15410182.4088 Epoch 7/2000 - 0s - loss: 14750829.7501 - mean_squared_error: 14750829.7501 - val_loss: 15262493.8393 - val_mean_squared_error: 15262493.8393 Epoch 8/2000 - 0s - loss: 14598241.2764 - mean_squared_error: 14598241.2764 - val_loss: 15092539.8774 - val_mean_squared_error: 15092539.8774 Epoch 9/2000 - 0s - loss: 14426493.4177 - mean_squared_error: 14426493.4177 - val_loss: 14905501.4780 - val_mean_squared_error: 14905501.4780 Epoch 10/2000 - 0s - loss: 14228848.9103 - mean_squared_error: 14228848.9103 - val_loss: 14688015.9975 - val_mean_squared_error: 14688015.9975 Epoch 11/2000 - 1s - loss: 14001212.6608 - mean_squared_error: 14001212.6608 - val_loss: 14434330.9461 - val_mean_squared_error: 14434330.9461 Epoch 12/2000 - 0s - loss: 13736570.0820 - mean_squared_error: 13736570.0820 - val_loss: 14139074.0257 - val_mean_squared_error: 14139074.0257 Epoch 13/2000 - 0s - loss: 13428367.8534 - mean_squared_error: 13428367.8534 - val_loss: 13795523.7548 - val_mean_squared_error: 13795523.7548 Epoch 14/2000 - 0s - loss: 13070743.7831 - mean_squared_error: 13070743.7831 - val_loss: 13397306.3450 - val_mean_squared_error: 13397306.3450 Epoch 15/2000 - 0s - loss: 12656768.5956 - mean_squared_error: 12656768.5956 - val_loss: 12945219.0559 - val_mean_squared_error: 12945219.0559 Epoch 16/2000 - 0s - loss: 12185512.7982 - mean_squared_error: 12185512.7982 - val_loss: 12423358.2358 - val_mean_squared_error: 12423358.2358 Epoch 17/2000 - 0s - loss: 11655525.9851 - mean_squared_error: 11655525.9851 - val_loss: 11847318.5447 - val_mean_squared_error: 11847318.5447 Epoch 18/2000 - 0s - loss: 11071839.4945 - mean_squared_error: 11071839.4945 - val_loss: 11221099.8136 - val_mean_squared_error: 11221099.8136 Epoch 19/2000 - 0s - loss: 10444863.5801 - mean_squared_error: 10444863.5801 - val_loss: 10549720.4142 - val_mean_squared_error: 10549720.4142 Epoch 20/2000 - 0s - loss: 9788272.2497 - mean_squared_error: 9788272.2497 - val_loss: 9859712.9110 - val_mean_squared_error: 9859712.9110 Epoch 21/2000 - 0s - loss: 9115201.0716 - mean_squared_error: 9115201.0716 - val_loss: 9161731.1725 - val_mean_squared_error: 9161731.1725 Epoch 22/2000 - 0s - loss: 8446740.0727 - mean_squared_error: 8446740.0727 - val_loss: 8473166.5581 - val_mean_squared_error: 8473166.5581 Epoch 23/2000 - 0s - loss: 7793792.6583 - mean_squared_error: 7793792.6583 - val_loss: 7801613.9909 - val_mean_squared_error: 7801613.9909 Epoch 24/2000 - 0s - loss: 7166955.5462 - mean_squared_error: 7166955.5462 - val_loss: 7164038.4654 - val_mean_squared_error: 7164038.4654 Epoch 25/2000 - 0s - loss: 6582194.8524 - mean_squared_error: 6582194.8524 - val_loss: 6574832.5309 - val_mean_squared_error: 6574832.5309 Epoch 26/2000 - 0s - loss: 6047351.9016 - mean_squared_error: 6047351.9016 - val_loss: 6035904.0623 - val_mean_squared_error: 6035904.0623 Epoch 27/2000 - 0s - loss: 5566922.6957 - mean_squared_error: 5566922.6957 - val_loss: 5558418.2491 - val_mean_squared_error: 5558418.2491 Epoch 28/2000 - 0s - loss: 5143339.4477 - mean_squared_error: 5143339.4477 - val_loss: 5132038.7902 - val_mean_squared_error: 5132038.7902 Epoch 29/2000 - 0s - loss: 4767790.8131 - mean_squared_error: 4767790.8131 - val_loss: 4757215.1453 - val_mean_squared_error: 4757215.1453 Epoch 30/2000 - 0s - loss: 4439649.8817 - mean_squared_error: 4439649.8817 - val_loss: 4431028.5476 - val_mean_squared_error: 4431028.5476 Epoch 31/2000 - 0s - loss: 4152965.2396 - mean_squared_error: 4152965.2396 - val_loss: 4137767.6985 - val_mean_squared_error: 4137767.6985 Epoch 32/2000 - 0s - loss: 3901266.2654 - mean_squared_error: 3901266.2654 - val_loss: 3882356.7435 - val_mean_squared_error: 3882356.7435 Epoch 33/2000 - 0s - loss: 3677236.9365 - mean_squared_error: 3677236.9365 - val_loss: 3646491.6930 - val_mean_squared_error: 3646491.6930 Epoch 34/2000 - 0s - loss: 3477286.9178 - mean_squared_error: 3477286.9178 - val_loss: 3443949.0140 - val_mean_squared_error: 3443949.0140 Epoch 35/2000 - 0s - loss: 3302099.7013 - mean_squared_error: 3302099.7013 - val_loss: 3258397.3565 - val_mean_squared_error: 3258397.3565 Epoch 36/2000 - 0s - loss: 3144469.7298 - mean_squared_error: 3144469.7298 - val_loss: 3091707.3880 - val_mean_squared_error: 3091707.3880 Epoch 37/2000 - 0s - loss: 3001370.1311 - mean_squared_error: 3001370.1311 - val_loss: 2935768.6719 - val_mean_squared_error: 2935768.6719 Epoch 38/2000 - 0s - loss: 2869674.9387 - mean_squared_error: 2869674.9387 - val_loss: 2804373.1770 - val_mean_squared_error: 2804373.1770 Epoch 39/2000 - 0s - loss: 2750813.5957 - mean_squared_error: 2750813.5957 - val_loss: 2669017.5266 - val_mean_squared_error: 2669017.5266 Epoch 40/2000 - 0s - loss: 2639596.0706 - mean_squared_error: 2639596.0706 - val_loss: 2549130.5860 - val_mean_squared_error: 2549130.5860 Epoch 41/2000 - 0s - loss: 2539679.8545 - mean_squared_error: 2539679.8545 - val_loss: 2439497.7667 - val_mean_squared_error: 2439497.7667 Epoch 42/2000 - 0s - loss: 2446811.9564 - mean_squared_error: 2446811.9564 - val_loss: 2338626.5081 - val_mean_squared_error: 2338626.5081 Epoch 43/2000 - 0s - loss: 2362560.7893 - mean_squared_error: 2362560.7893 - val_loss: 2244730.6283 - val_mean_squared_error: 2244730.6283 Epoch 44/2000 - 0s - loss: 2283878.5586 - mean_squared_error: 2283878.5586 - val_loss: 2158127.9470 - val_mean_squared_error: 2158127.9470 Epoch 45/2000 - 0s - loss: 2210463.4408 - mean_squared_error: 2210463.4408 - val_loss: 2076556.6046 - val_mean_squared_error: 2076556.6046 Epoch 46/2000 - 0s - loss: 2141897.1993 - mean_squared_error: 2141897.1993 - val_loss: 2000310.3009 - val_mean_squared_error: 2000310.3009 Epoch 47/2000 - 0s - loss: 2078454.0809 - mean_squared_error: 2078454.0809 - val_loss: 1931987.7154 - val_mean_squared_error: 1931987.7154 Epoch 48/2000 - 0s - loss: 2021185.5238 - mean_squared_error: 2021185.5238 - val_loss: 1865060.8356 - val_mean_squared_error: 1865060.8356 Epoch 49/2000 - 0s - loss: 1966803.1820 - mean_squared_error: 1966803.1820 - val_loss: 1808340.0919 - val_mean_squared_error: 1808340.0919 Epoch 50/2000 - 0s - loss: 1915811.9821 - mean_squared_error: 1915811.9821 - val_loss: 1747630.3844 - val_mean_squared_error: 1747630.3844 Epoch 51/2000 - 0s - loss: 1866521.3634 - mean_squared_error: 1866521.3634 - val_loss: 1693315.2384 - val_mean_squared_error: 1693315.2384 Epoch 52/2000 - 0s - loss: 1821326.0610 - mean_squared_error: 1821326.0610 - val_loss: 1642927.1448 - val_mean_squared_error: 1642927.1448 Epoch 53/2000 - 0s - loss: 1778010.7790 - mean_squared_error: 1778010.7790 - val_loss: 1593469.0986 - val_mean_squared_error: 1593469.0986 Epoch 54/2000 - 0s - loss: 1737470.6897 - mean_squared_error: 1737470.6897 - val_loss: 1549439.4479 - val_mean_squared_error: 1549439.4479 Epoch 55/2000 - 0s - loss: 1700140.8353 - mean_squared_error: 1700140.8353 - val_loss: 1512758.0407 - val_mean_squared_error: 1512758.0407 Epoch 56/2000 - 0s - loss: 1664932.0321 - mean_squared_error: 1664932.0321 - val_loss: 1468464.1324 - val_mean_squared_error: 1468464.1324 Epoch 57/2000 - 0s - loss: 1631250.8124 - mean_squared_error: 1631250.8124 - val_loss: 1430306.0396 - val_mean_squared_error: 1430306.0396 Epoch 58/2000 - 0s - loss: 1599123.9408 - mean_squared_error: 1599123.9408 - val_loss: 1402570.5938 - val_mean_squared_error: 1402570.5938 Epoch 59/2000 - 0s - loss: 1569169.3397 - mean_squared_error: 1569169.3397 - val_loss: 1361781.7992 - val_mean_squared_error: 1361781.7992 Epoch 60/2000 - 0s - loss: 1540956.2998 - mean_squared_error: 1540956.2998 - val_loss: 1329689.5006 - val_mean_squared_error: 1329689.5006 Epoch 61/2000 - 0s - loss: 1513870.4650 - mean_squared_error: 1513870.4650 - val_loss: 1299788.0126 - val_mean_squared_error: 1299788.0126 Epoch 62/2000 - 0s - loss: 1488514.5449 - mean_squared_error: 1488514.5449 - val_loss: 1271596.4853 - val_mean_squared_error: 1271596.4853 Epoch 63/2000 - 0s - loss: 1464872.0181 - mean_squared_error: 1464872.0181 - val_loss: 1245587.1696 - val_mean_squared_error: 1245587.1696 Epoch 64/2000 - 0s - loss: 1441970.8339 - mean_squared_error: 1441970.8339 - val_loss: 1220000.9017 - val_mean_squared_error: 1220000.9017 Epoch 65/2000 - 0s - loss: 1419747.2820 - mean_squared_error: 1419747.2820 - val_loss: 1200149.5500 - val_mean_squared_error: 1200149.5500 Epoch 66/2000 - 0s - loss: 1399810.3150 - mean_squared_error: 1399810.3150 - val_loss: 1174179.1523 - val_mean_squared_error: 1174179.1523 Epoch 67/2000 - 0s - loss: 1379762.8659 - mean_squared_error: 1379762.8659 - val_loss: 1155421.8361 - val_mean_squared_error: 1155421.8361 Epoch 68/2000 - 0s - loss: 1361613.9023 - mean_squared_error: 1361613.9023 - val_loss: 1131801.8392 - val_mean_squared_error: 1131801.8392 Epoch 69/2000 - 0s - loss: 1344540.7886 - mean_squared_error: 1344540.7886 - val_loss: 1112408.5979 - val_mean_squared_error: 1112408.5979 Epoch 70/2000 - 0s - loss: 1327972.0930 - mean_squared_error: 1327972.0930 - val_loss: 1095669.5814 - val_mean_squared_error: 1095669.5814 Epoch 71/2000 - 0s - loss: 1311907.4601 - mean_squared_error: 1311907.4601 - val_loss: 1080797.0534 - val_mean_squared_error: 1080797.0534 Epoch 72/2000 - 0s - loss: 1297300.1515 - mean_squared_error: 1297300.1515 - val_loss: 1063274.4691 - val_mean_squared_error: 1063274.4691 Epoch 73/2000 - 0s - loss: 1282702.7680 - mean_squared_error: 1282702.7680 - val_loss: 1047914.8447 - val_mean_squared_error: 1047914.8447 Epoch 74/2000 - 0s - loss: 1269224.1888 - mean_squared_error: 1269224.1888 - val_loss: 1035664.7105 - val_mean_squared_error: 1035664.7105 Epoch 75/2000 - 0s - loss: 1255846.9290 - mean_squared_error: 1255846.9290 - val_loss: 1022400.9963 - val_mean_squared_error: 1022400.9963 Epoch 76/2000 - 0s - loss: 1243903.4871 - mean_squared_error: 1243903.4871 - val_loss: 1003828.2156 - val_mean_squared_error: 1003828.2156 Epoch 77/2000 - 0s - loss: 1232750.9959 - mean_squared_error: 1232750.9959 - val_loss: 991562.2552 - val_mean_squared_error: 991562.2552 Epoch 78/2000 - 0s - loss: 1221784.7819 - mean_squared_error: 1221784.7819 - val_loss: 988014.4801 - val_mean_squared_error: 988014.4801 Epoch 79/2000 - 0s - loss: 1210924.4235 - mean_squared_error: 1210924.4235 - val_loss: 966999.6418 - val_mean_squared_error: 966999.6418 Epoch 80/2000 - 0s - loss: 1201281.6524 - mean_squared_error: 1201281.6524 - val_loss: 957178.3798 - val_mean_squared_error: 957178.3798 Epoch 81/2000 - 0s - loss: 1191342.2884 - mean_squared_error: 1191342.2884 - val_loss: 945778.2848 - val_mean_squared_error: 945778.2848 Epoch 82/2000 - 0s - loss: 1182401.2386 - mean_squared_error: 1182401.2386 - val_loss: 941735.3018 - val_mean_squared_error: 941735.3018 Epoch 83/2000 - 0s - loss: 1173785.2414 - mean_squared_error: 1173785.2414 - val_loss: 926331.6598 - val_mean_squared_error: 926331.6598 Epoch 84/2000 - 0s - loss: 1165100.9749 - mean_squared_error: 1165100.9749 - val_loss: 922599.2534 - val_mean_squared_error: 922599.2534 Epoch 85/2000 - 0s - loss: 1157599.6469 - mean_squared_error: 1157599.6469 - val_loss: 909060.7537 - val_mean_squared_error: 909060.7537 Epoch 86/2000 - 0s - loss: 1149606.7023 - mean_squared_error: 1149606.7023 - val_loss: 905816.1376 - val_mean_squared_error: 905816.1376 Epoch 87/2000 - 0s - loss: 1142876.2513 - mean_squared_error: 1142876.2513 - val_loss: 894544.3443 - val_mean_squared_error: 894544.3443 Epoch 88/2000 - 0s - loss: 1135421.7432 - mean_squared_error: 1135421.7432 - val_loss: 885002.8672 - val_mean_squared_error: 885002.8672 Epoch 89/2000 - 0s - loss: 1128510.0997 - mean_squared_error: 1128510.0997 - val_loss: 885705.4161 - val_mean_squared_error: 885705.4161 Epoch 90/2000 - 0s - loss: 1122707.7963 - mean_squared_error: 1122707.7963 - val_loss: 869758.4723 - val_mean_squared_error: 869758.4723 Epoch 91/2000 - 0s - loss: 1115116.7629 - mean_squared_error: 1115116.7629 - val_loss: 870980.7851 - val_mean_squared_error: 870980.7851 Epoch 92/2000 - 0s - loss: 1108894.0348 - mean_squared_error: 1108894.0348 - val_loss: 855680.1749 - val_mean_squared_error: 855680.1749 Epoch 93/2000 - 0s - loss: 1103963.9562 - mean_squared_error: 1103963.9562 - val_loss: 852032.6139 - val_mean_squared_error: 852032.6139 Epoch 94/2000 - 0s - loss: 1098281.1880 - mean_squared_error: 1098281.1880 - val_loss: 845218.1591 - val_mean_squared_error: 845218.1591 Epoch 95/2000 - 0s - loss: 1092883.6761 - mean_squared_error: 1092883.6761 - val_loss: 836951.9199 - val_mean_squared_error: 836951.9199 Epoch 96/2000 - 0s - loss: 1087001.8999 - mean_squared_error: 1087001.8999 - val_loss: 831424.8668 - val_mean_squared_error: 831424.8668 Epoch 97/2000 - 0s - loss: 1081821.6931 - mean_squared_error: 1081821.6931 - val_loss: 826467.4629 - val_mean_squared_error: 826467.4629 Epoch 98/2000 - 0s - loss: 1076217.9142 - mean_squared_error: 1076217.9142 - val_loss: 819017.7581 - val_mean_squared_error: 819017.7581 Epoch 99/2000 - 0s - loss: 1072431.6157 - mean_squared_error: 1072431.6157 - val_loss: 817342.3566 - val_mean_squared_error: 817342.3566 Epoch 100/2000 - 0s - loss: 1066940.7825 - mean_squared_error: 1066940.7825 - val_loss: 808668.5299 - val_mean_squared_error: 808668.5299 Epoch 101/2000 - 0s - loss: 1062883.5691 - mean_squared_error: 1062883.5691 - val_loss: 804621.1654 - val_mean_squared_error: 804621.1654 Epoch 102/2000 - 0s - loss: 1058172.6794 - mean_squared_error: 1058172.6794 - val_loss: 798353.0681 - val_mean_squared_error: 798353.0681 Epoch 103/2000 - 0s - loss: 1053867.6362 - mean_squared_error: 1053867.6362 - val_loss: 804793.9548 - val_mean_squared_error: 804793.9548 Epoch 104/2000 - 0s - loss: 1049076.8368 - mean_squared_error: 1049076.8368 - val_loss: 789712.6155 - val_mean_squared_error: 789712.6155 Epoch 105/2000 - 0s - loss: 1045631.3630 - mean_squared_error: 1045631.3630 - val_loss: 785457.5816 - val_mean_squared_error: 785457.5816 Epoch 106/2000 - 0s - loss: 1040585.5933 - mean_squared_error: 1040585.5933 - val_loss: 780374.5012 - val_mean_squared_error: 780374.5012 Epoch 107/2000 - 0s - loss: 1037087.4549 - mean_squared_error: 1037087.4549 - val_loss: 783961.0112 - val_mean_squared_error: 783961.0112 Epoch 108/2000 - 0s - loss: 1033408.3128 - mean_squared_error: 1033408.3128 - val_loss: 773750.7693 - val_mean_squared_error: 773750.7693 Epoch 109/2000 - 0s - loss: 1029350.9100 - mean_squared_error: 1029350.9100 - val_loss: 771727.1558 - val_mean_squared_error: 771727.1558 Epoch 110/2000 - 0s - loss: 1025883.1703 - mean_squared_error: 1025883.1703 - val_loss: 768502.3288 - val_mean_squared_error: 768502.3288 Epoch 111/2000 - 0s - loss: 1021833.5502 - mean_squared_error: 1021833.5502 - val_loss: 760970.9669 - val_mean_squared_error: 760970.9669 Epoch 112/2000 - 0s - loss: 1018940.0217 - mean_squared_error: 1018940.0217 - val_loss: 760039.7225 - val_mean_squared_error: 760039.7225 Epoch 113/2000 - 0s - loss: 1014862.8041 - mean_squared_error: 1014862.8041 - val_loss: 764375.0141 - val_mean_squared_error: 764375.0141 Epoch 114/2000 - 0s - loss: 1011251.5824 - mean_squared_error: 1011251.5824 - val_loss: 754512.3164 - val_mean_squared_error: 754512.3164 Epoch 115/2000 - 0s - loss: 1008575.3374 - mean_squared_error: 1008575.3374 - val_loss: 748945.7377 - val_mean_squared_error: 748945.7377 Epoch 116/2000 - 0s - loss: 1005124.7456 - mean_squared_error: 1005124.7456 - val_loss: 749031.7273 - val_mean_squared_error: 749031.7273 Epoch 117/2000 - 0s - loss: 1001222.4575 - mean_squared_error: 1001222.4575 - val_loss: 739509.4394 - val_mean_squared_error: 739509.4394 Epoch 118/2000 - 0s - loss: 998861.1399 - mean_squared_error: 998861.1399 - val_loss: 745729.3119 - val_mean_squared_error: 745729.3119 Epoch 119/2000 - 0s - loss: 996182.4343 - mean_squared_error: 996182.4343 - val_loss: 757251.6217 - val_mean_squared_error: 757251.6217 Epoch 120/2000 - 0s - loss: 992934.4757 - mean_squared_error: 992934.4757 - val_loss: 733761.8970 - val_mean_squared_error: 733761.8970 Epoch 121/2000 - 0s - loss: 989464.3495 - mean_squared_error: 989464.3495 - val_loss: 728592.8324 - val_mean_squared_error: 728592.8324 Epoch 122/2000 - 0s - loss: 986923.2727 - mean_squared_error: 986923.2727 - val_loss: 731263.4120 - val_mean_squared_error: 731263.4120 Epoch 123/2000 - 0s - loss: 984284.1567 - mean_squared_error: 984284.1567 - val_loss: 734552.3397 - val_mean_squared_error: 734552.3397 Epoch 124/2000 - 0s - loss: 981964.4697 - mean_squared_error: 981964.4697 - val_loss: 721918.1604 - val_mean_squared_error: 721918.1604 Epoch 125/2000 - 0s - loss: 978881.0860 - mean_squared_error: 978881.0860 - val_loss: 718240.2593 - val_mean_squared_error: 718240.2593 Epoch 126/2000 - 0s - loss: 976116.4305 - mean_squared_error: 976116.4305 - val_loss: 716768.1848 - val_mean_squared_error: 716768.1848 Epoch 127/2000 - 0s - loss: 973458.0178 - mean_squared_error: 973458.0178 - val_loss: 715422.3851 - val_mean_squared_error: 715422.3851 Epoch 128/2000 - 0s - loss: 970484.2357 - mean_squared_error: 970484.2357 - val_loss: 709885.0327 - val_mean_squared_error: 709885.0327 Epoch 129/2000 - 0s - loss: 967710.4944 - mean_squared_error: 967710.4944 - val_loss: 706953.8875 - val_mean_squared_error: 706953.8875 Epoch 130/2000 - 0s - loss: 964926.7113 - mean_squared_error: 964926.7113 - val_loss: 704451.7275 - val_mean_squared_error: 704451.7275 Epoch 131/2000 - 0s - loss: 962205.7342 - mean_squared_error: 962205.7342 - val_loss: 703047.8709 - val_mean_squared_error: 703047.8709 Epoch 132/2000 - 0s - loss: 959855.5042 - mean_squared_error: 959855.5042 - val_loss: 699065.4193 - val_mean_squared_error: 699065.4193 Epoch 133/2000 - 0s - loss: 957806.8261 - mean_squared_error: 957806.8261 - val_loss: 709171.6166 - val_mean_squared_error: 709171.6166 Epoch 134/2000 - 0s - loss: 954564.9092 - mean_squared_error: 954564.9092 - val_loss: 696548.4695 - val_mean_squared_error: 696548.4695 Epoch 135/2000 - 0s - loss: 952602.2452 - mean_squared_error: 952602.2452 - val_loss: 697575.4079 - val_mean_squared_error: 697575.4079 Epoch 136/2000 - 0s - loss: 950352.9306 - mean_squared_error: 950352.9306 - val_loss: 689600.8201 - val_mean_squared_error: 689600.8201 Epoch 137/2000 - 0s - loss: 947697.6631 - mean_squared_error: 947697.6631 - val_loss: 689256.6092 - val_mean_squared_error: 689256.6092 Epoch 138/2000 - 0s - loss: 944707.3380 - mean_squared_error: 944707.3380 - val_loss: 697876.2123 - val_mean_squared_error: 697876.2123 Epoch 139/2000 - 0s - loss: 942931.2569 - mean_squared_error: 942931.2569 - val_loss: 688838.1186 - val_mean_squared_error: 688838.1186 Epoch 140/2000 - 0s - loss: 940727.0246 - mean_squared_error: 940727.0246 - val_loss: 683651.2944 - val_mean_squared_error: 683651.2944 Epoch 141/2000 - 0s - loss: 937611.2467 - mean_squared_error: 937611.2467 - val_loss: 679610.9674 - val_mean_squared_error: 679610.9674 Epoch 142/2000 - 0s - loss: 936145.2758 - mean_squared_error: 936145.2758 - val_loss: 679409.5040 - val_mean_squared_error: 679409.5040 Epoch 143/2000 - 0s - loss: 933791.8269 - mean_squared_error: 933791.8269 - val_loss: 674969.9004 - val_mean_squared_error: 674969.9004 Epoch 144/2000 - 0s - loss: 931934.5522 - mean_squared_error: 931934.5522 - val_loss: 673677.1087 - val_mean_squared_error: 673677.1087 Epoch 145/2000 - 0s - loss: 929419.1329 - mean_squared_error: 929419.1329 - val_loss: 672486.2596 - val_mean_squared_error: 672486.2596 Epoch 146/2000 - 0s - loss: 928259.8743 - mean_squared_error: 928259.8743 - val_loss: 679053.0771 - val_mean_squared_error: 679053.0771 Epoch 147/2000 - 0s - loss: 925840.3659 - mean_squared_error: 925840.3659 - val_loss: 670425.6315 - val_mean_squared_error: 670425.6315 Epoch 148/2000 - 0s - loss: 923053.3375 - mean_squared_error: 923053.3375 - val_loss: 666447.0378 - val_mean_squared_error: 666447.0378 Epoch 149/2000 - 0s - loss: 921570.8007 - mean_squared_error: 921570.8007 - val_loss: 671207.0466 - val_mean_squared_error: 671207.0466 Epoch 150/2000 - 0s - loss: 919778.3405 - mean_squared_error: 919778.3405 - val_loss: 663078.1736 - val_mean_squared_error: 663078.1736 Epoch 151/2000 - 0s - loss: 916954.4063 - mean_squared_error: 916954.4063 - val_loss: 661748.4101 - val_mean_squared_error: 661748.4101 Epoch 152/2000 - 0s - loss: 915275.2257 - mean_squared_error: 915275.2257 - val_loss: 660109.1333 - val_mean_squared_error: 660109.1333 Epoch 153/2000 - 0s - loss: 913935.4979 - mean_squared_error: 913935.4979 - val_loss: 660320.5198 - val_mean_squared_error: 660320.5198 Epoch 154/2000 - 0s - loss: 912403.0531 - mean_squared_error: 912403.0531 - val_loss: 658724.5849 - val_mean_squared_error: 658724.5849 Epoch 155/2000 - 0s - loss: 909915.3601 - mean_squared_error: 909915.3601 - val_loss: 655034.8768 - val_mean_squared_error: 655034.8768 Epoch 156/2000 - 0s - loss: 908297.1211 - mean_squared_error: 908297.1211 - val_loss: 654181.0143 - val_mean_squared_error: 654181.0143 Epoch 157/2000 - 0s - loss: 905624.7004 - mean_squared_error: 905624.7004 - val_loss: 656986.1485 - val_mean_squared_error: 656986.1485 Epoch 158/2000 - 0s - loss: 905111.6589 - mean_squared_error: 905111.6589 - val_loss: 655903.3115 - val_mean_squared_error: 655903.3115 Epoch 159/2000 - 0s - loss: 902944.8210 - mean_squared_error: 902944.8210 - val_loss: 654524.7966 - val_mean_squared_error: 654524.7966 Epoch 160/2000 - 0s - loss: 901144.1101 - mean_squared_error: 901144.1101 - val_loss: 651841.0205 - val_mean_squared_error: 651841.0205 Epoch 161/2000 - 0s - loss: 899314.5646 - mean_squared_error: 899314.5646 - val_loss: 646358.7536 - val_mean_squared_error: 646358.7536 Epoch 162/2000 - 0s - loss: 897382.2375 - mean_squared_error: 897382.2375 - val_loss: 646032.7151 - val_mean_squared_error: 646032.7151 Epoch 163/2000 - 0s - loss: 895482.0745 - mean_squared_error: 895482.0745 - val_loss: 647151.8484 - val_mean_squared_error: 647151.8484 Epoch 164/2000 - 0s - loss: 894577.8852 - mean_squared_error: 894577.8852 - val_loss: 643899.3744 - val_mean_squared_error: 643899.3744 Epoch 165/2000 - 0s - loss: 892187.8241 - mean_squared_error: 892187.8241 - val_loss: 641453.4793 - val_mean_squared_error: 641453.4793 Epoch 166/2000 - 0s - loss: 890396.7710 - mean_squared_error: 890396.7710 - val_loss: 639900.2524 - val_mean_squared_error: 639900.2524 Epoch 167/2000 - 0s - loss: 889097.9396 - mean_squared_error: 889097.9396 - val_loss: 640526.9566 - val_mean_squared_error: 640526.9566 Epoch 168/2000 - 0s - loss: 887303.5403 - mean_squared_error: 887303.5403 - val_loss: 637232.9263 - val_mean_squared_error: 637232.9263 Epoch 169/2000 - 0s - loss: 885714.2098 - mean_squared_error: 885714.2098 - val_loss: 638746.7358 - val_mean_squared_error: 638746.7358 Epoch 170/2000 - 0s - loss: 884497.2334 - mean_squared_error: 884497.2334 - val_loss: 634734.1812 - val_mean_squared_error: 634734.1812 Epoch 171/2000 - 0s - loss: 882821.3801 - mean_squared_error: 882821.3801 - val_loss: 634757.9038 - val_mean_squared_error: 634757.9038 Epoch 172/2000 - 0s - loss: 880668.8433 - mean_squared_error: 880668.8433 - val_loss: 631159.9473 - val_mean_squared_error: 631159.9473 Epoch 173/2000 - 0s - loss: 879786.0717 - mean_squared_error: 879786.0717 - val_loss: 632434.0280 - val_mean_squared_error: 632434.0280 Epoch 174/2000 - 0s - loss: 877921.8627 - mean_squared_error: 877921.8627 - val_loss: 629854.0848 - val_mean_squared_error: 629854.0848 Epoch 175/2000 - 0s - loss: 876936.1627 - mean_squared_error: 876936.1627 - val_loss: 639762.5728 - val_mean_squared_error: 639762.5728 Epoch 176/2000 - 0s - loss: 874990.3023 - mean_squared_error: 874990.3023 - val_loss: 627614.5739 - val_mean_squared_error: 627614.5739 Epoch 177/2000 - 0s - loss: 873407.4634 - mean_squared_error: 873407.4634 - val_loss: 625926.6324 - val_mean_squared_error: 625926.6324 Epoch 178/2000 - 0s - loss: 871298.3328 - mean_squared_error: 871298.3328 - val_loss: 654647.0849 - val_mean_squared_error: 654647.0849 Epoch 179/2000 - 0s - loss: 871422.8988 - mean_squared_error: 871422.8988 - val_loss: 624784.2562 - val_mean_squared_error: 624784.2562 Epoch 180/2000 - 0s - loss: 868830.5286 - mean_squared_error: 868830.5286 - val_loss: 623332.0319 - val_mean_squared_error: 623332.0319 *** WARNING: skipped 225655 bytes of output *** Epoch 1823/2000 - 0s - loss: 397390.2962 - mean_squared_error: 397390.2962 - val_loss: 353209.6269 - val_mean_squared_error: 353209.6269 Epoch 1824/2000 - 0s - loss: 397583.4764 - mean_squared_error: 397583.4764 - val_loss: 353313.4139 - val_mean_squared_error: 353313.4139 Epoch 1825/2000 - 0s - loss: 396889.4295 - mean_squared_error: 396889.4295 - val_loss: 353050.2891 - val_mean_squared_error: 353050.2891 Epoch 1826/2000 - 0s - loss: 396688.1741 - mean_squared_error: 396688.1741 - val_loss: 352968.9833 - val_mean_squared_error: 352968.9833 Epoch 1827/2000 - 0s - loss: 396879.9892 - mean_squared_error: 396879.9892 - val_loss: 353596.4543 - val_mean_squared_error: 353596.4543 Epoch 1828/2000 - 0s - loss: 397163.0979 - mean_squared_error: 397163.0979 - val_loss: 355002.6447 - val_mean_squared_error: 355002.6447 Epoch 1829/2000 - 0s - loss: 397371.9557 - mean_squared_error: 397371.9557 - val_loss: 360991.2768 - val_mean_squared_error: 360991.2768 Epoch 1830/2000 - 0s - loss: 397330.9307 - mean_squared_error: 397330.9307 - val_loss: 352858.5714 - val_mean_squared_error: 352858.5714 Epoch 1831/2000 - 0s - loss: 397069.0619 - mean_squared_error: 397069.0619 - val_loss: 356316.7071 - val_mean_squared_error: 356316.7071 Epoch 1832/2000 - 0s - loss: 397248.6073 - mean_squared_error: 397248.6073 - val_loss: 352828.9623 - val_mean_squared_error: 352828.9623 Epoch 1833/2000 - 0s - loss: 396082.9898 - mean_squared_error: 396082.9898 - val_loss: 352698.9154 - val_mean_squared_error: 352698.9154 Epoch 1834/2000 - 0s - loss: 397319.9301 - mean_squared_error: 397319.9301 - val_loss: 352547.3161 - val_mean_squared_error: 352547.3161 Epoch 1835/2000 - 0s - loss: 396171.3512 - mean_squared_error: 396171.3512 - val_loss: 352890.7365 - val_mean_squared_error: 352890.7365 Epoch 1836/2000 - 0s - loss: 397558.3849 - mean_squared_error: 397558.3849 - val_loss: 352797.1587 - val_mean_squared_error: 352797.1587 Epoch 1837/2000 - 0s - loss: 396650.8045 - mean_squared_error: 396650.8045 - val_loss: 352551.9260 - val_mean_squared_error: 352551.9260 Epoch 1838/2000 - 0s - loss: 396322.6335 - mean_squared_error: 396322.6335 - val_loss: 352522.2167 - val_mean_squared_error: 352522.2167 Epoch 1839/2000 - 0s - loss: 396564.7714 - mean_squared_error: 396564.7714 - val_loss: 356043.7555 - val_mean_squared_error: 356043.7555 Epoch 1840/2000 - 0s - loss: 396659.8503 - mean_squared_error: 396659.8503 - val_loss: 353330.3860 - val_mean_squared_error: 353330.3860 Epoch 1841/2000 - 0s - loss: 396947.4321 - mean_squared_error: 396947.4321 - val_loss: 352941.9958 - val_mean_squared_error: 352941.9958 Epoch 1842/2000 - 0s - loss: 396686.2913 - mean_squared_error: 396686.2913 - val_loss: 353136.8284 - val_mean_squared_error: 353136.8284 Epoch 1843/2000 - 0s - loss: 397013.0222 - mean_squared_error: 397013.0222 - val_loss: 357842.3895 - val_mean_squared_error: 357842.3895 Epoch 1844/2000 - 0s - loss: 396760.1421 - mean_squared_error: 396760.1421 - val_loss: 353699.3172 - val_mean_squared_error: 353699.3172 Epoch 1845/2000 - 0s - loss: 396413.3329 - mean_squared_error: 396413.3329 - val_loss: 353588.4062 - val_mean_squared_error: 353588.4062 Epoch 1846/2000 - 0s - loss: 396626.7565 - mean_squared_error: 396626.7565 - val_loss: 352466.3480 - val_mean_squared_error: 352466.3480 Epoch 1847/2000 - 0s - loss: 396830.8680 - mean_squared_error: 396830.8680 - val_loss: 353063.0210 - val_mean_squared_error: 353063.0210 Epoch 1848/2000 - 0s - loss: 396382.9396 - mean_squared_error: 396382.9396 - val_loss: 352376.7488 - val_mean_squared_error: 352376.7488 Epoch 1849/2000 - 0s - loss: 396371.6686 - mean_squared_error: 396371.6686 - val_loss: 353424.5889 - val_mean_squared_error: 353424.5889 Epoch 1850/2000 - 0s - loss: 395971.4266 - mean_squared_error: 395971.4266 - val_loss: 353351.6123 - val_mean_squared_error: 353351.6123 Epoch 1851/2000 - 0s - loss: 396122.5219 - mean_squared_error: 396122.5219 - val_loss: 352835.1150 - val_mean_squared_error: 352835.1150 Epoch 1852/2000 - 0s - loss: 395927.3674 - mean_squared_error: 395927.3674 - val_loss: 358838.3104 - val_mean_squared_error: 358838.3104 Epoch 1853/2000 - 0s - loss: 396239.7226 - mean_squared_error: 396239.7226 - val_loss: 353089.9473 - val_mean_squared_error: 353089.9473 Epoch 1854/2000 - 0s - loss: 396131.3741 - mean_squared_error: 396131.3741 - val_loss: 352670.2379 - val_mean_squared_error: 352670.2379 Epoch 1855/2000 - 0s - loss: 396213.9961 - mean_squared_error: 396213.9961 - val_loss: 355889.2487 - val_mean_squared_error: 355889.2487 Epoch 1856/2000 - 0s - loss: 395907.4523 - mean_squared_error: 395907.4523 - val_loss: 353334.4587 - val_mean_squared_error: 353334.4587 Epoch 1857/2000 - 0s - loss: 396093.9572 - mean_squared_error: 396093.9572 - val_loss: 355890.8466 - val_mean_squared_error: 355890.8466 Epoch 1858/2000 - 0s - loss: 396148.8290 - mean_squared_error: 396148.8290 - val_loss: 352772.3574 - val_mean_squared_error: 352772.3574 Epoch 1859/2000 - 0s - loss: 396374.1615 - mean_squared_error: 396374.1615 - val_loss: 352408.7239 - val_mean_squared_error: 352408.7239 Epoch 1860/2000 - 0s - loss: 396224.3341 - mean_squared_error: 396224.3341 - val_loss: 352462.1340 - val_mean_squared_error: 352462.1340 Epoch 1861/2000 - 0s - loss: 395775.2315 - mean_squared_error: 395775.2315 - val_loss: 354116.2042 - val_mean_squared_error: 354116.2042 Epoch 1862/2000 - 0s - loss: 396024.3581 - mean_squared_error: 396024.3581 - val_loss: 352205.7111 - val_mean_squared_error: 352205.7111 Epoch 1863/2000 - 0s - loss: 395613.7328 - mean_squared_error: 395613.7328 - val_loss: 352385.5322 - val_mean_squared_error: 352385.5322 Epoch 1864/2000 - 0s - loss: 396227.5467 - mean_squared_error: 396227.5467 - val_loss: 352645.4930 - val_mean_squared_error: 352645.4930 Epoch 1865/2000 - 0s - loss: 396004.5046 - mean_squared_error: 396004.5046 - val_loss: 353968.2366 - val_mean_squared_error: 353968.2366 Epoch 1866/2000 - 0s - loss: 396020.5718 - mean_squared_error: 396020.5718 - val_loss: 354020.5083 - val_mean_squared_error: 354020.5083 Epoch 1867/2000 - 0s - loss: 395672.2817 - mean_squared_error: 395672.2817 - val_loss: 353314.8828 - val_mean_squared_error: 353314.8828 Epoch 1868/2000 - 0s - loss: 395795.3118 - mean_squared_error: 395795.3118 - val_loss: 352232.4819 - val_mean_squared_error: 352232.4819 Epoch 1869/2000 - 0s - loss: 395839.6442 - mean_squared_error: 395839.6442 - val_loss: 352895.0211 - val_mean_squared_error: 352895.0211 Epoch 1870/2000 - 0s - loss: 396112.3882 - mean_squared_error: 396112.3882 - val_loss: 353443.6297 - val_mean_squared_error: 353443.6297 Epoch 1871/2000 - 0s - loss: 395928.2209 - mean_squared_error: 395928.2209 - val_loss: 352084.1819 - val_mean_squared_error: 352084.1819 Epoch 1872/2000 - 0s - loss: 395532.7622 - mean_squared_error: 395532.7622 - val_loss: 351922.8059 - val_mean_squared_error: 351922.8059 Epoch 1873/2000 - 0s - loss: 394991.3888 - mean_squared_error: 394991.3888 - val_loss: 352354.4650 - val_mean_squared_error: 352354.4650 Epoch 1874/2000 - 0s - loss: 395868.8979 - mean_squared_error: 395868.8979 - val_loss: 353307.8581 - val_mean_squared_error: 353307.8581 Epoch 1875/2000 - 0s - loss: 395731.7098 - mean_squared_error: 395731.7098 - val_loss: 351873.0492 - val_mean_squared_error: 351873.0492 Epoch 1876/2000 - 0s - loss: 395132.8965 - mean_squared_error: 395132.8965 - val_loss: 351852.0628 - val_mean_squared_error: 351852.0628 Epoch 1877/2000 - 0s - loss: 395518.6568 - mean_squared_error: 395518.6568 - val_loss: 351841.3919 - val_mean_squared_error: 351841.3919 Epoch 1878/2000 - 0s - loss: 395575.1640 - mean_squared_error: 395575.1640 - val_loss: 351899.8707 - val_mean_squared_error: 351899.8707 Epoch 1879/2000 - 0s - loss: 395166.7193 - mean_squared_error: 395166.7193 - val_loss: 352113.9540 - val_mean_squared_error: 352113.9540 Epoch 1880/2000 - 0s - loss: 394905.8654 - mean_squared_error: 394905.8654 - val_loss: 354147.1539 - val_mean_squared_error: 354147.1539 Epoch 1881/2000 - 0s - loss: 395029.2658 - mean_squared_error: 395029.2658 - val_loss: 354190.9711 - val_mean_squared_error: 354190.9711 Epoch 1882/2000 - 0s - loss: 395449.4634 - mean_squared_error: 395449.4634 - val_loss: 352587.5978 - val_mean_squared_error: 352587.5978 Epoch 1883/2000 - 0s - loss: 395276.2936 - mean_squared_error: 395276.2936 - val_loss: 352548.3871 - val_mean_squared_error: 352548.3871 Epoch 1884/2000 - 0s - loss: 394877.4458 - mean_squared_error: 394877.4458 - val_loss: 352523.8919 - val_mean_squared_error: 352523.8919 Epoch 1885/2000 - 0s - loss: 395268.0316 - mean_squared_error: 395268.0316 - val_loss: 353429.1411 - val_mean_squared_error: 353429.1411 Epoch 1886/2000 - 0s - loss: 395224.4547 - mean_squared_error: 395224.4547 - val_loss: 351589.5054 - val_mean_squared_error: 351589.5054 Epoch 1887/2000 - 0s - loss: 394930.0051 - mean_squared_error: 394930.0051 - val_loss: 351562.0556 - val_mean_squared_error: 351562.0556 Epoch 1888/2000 - 0s - loss: 395516.0361 - mean_squared_error: 395516.0361 - val_loss: 355380.2538 - val_mean_squared_error: 355380.2538 Epoch 1889/2000 - 0s - loss: 395076.7389 - mean_squared_error: 395076.7389 - val_loss: 351812.1470 - val_mean_squared_error: 351812.1470 Epoch 1890/2000 - 0s - loss: 394752.7494 - mean_squared_error: 394752.7494 - val_loss: 352221.3691 - val_mean_squared_error: 352221.3691 Epoch 1891/2000 - 0s - loss: 395093.2258 - mean_squared_error: 395093.2258 - val_loss: 351600.1716 - val_mean_squared_error: 351600.1716 Epoch 1892/2000 - 0s - loss: 395046.2364 - mean_squared_error: 395046.2364 - val_loss: 352985.1909 - val_mean_squared_error: 352985.1909 Epoch 1893/2000 - 0s - loss: 394981.4052 - mean_squared_error: 394981.4052 - val_loss: 355643.5620 - val_mean_squared_error: 355643.5620 Epoch 1894/2000 - 0s - loss: 394896.0592 - mean_squared_error: 394896.0592 - val_loss: 352217.3145 - val_mean_squared_error: 352217.3145 Epoch 1895/2000 - 0s - loss: 394552.8456 - mean_squared_error: 394552.8456 - val_loss: 354756.0459 - val_mean_squared_error: 354756.0459 Epoch 1896/2000 - 0s - loss: 395051.9754 - mean_squared_error: 395051.9754 - val_loss: 353615.7922 - val_mean_squared_error: 353615.7922 Epoch 1897/2000 - 0s - loss: 394813.7603 - mean_squared_error: 394813.7603 - val_loss: 351627.1015 - val_mean_squared_error: 351627.1015 Epoch 1898/2000 - 0s - loss: 394762.4370 - mean_squared_error: 394762.4370 - val_loss: 351538.6374 - val_mean_squared_error: 351538.6374 Epoch 1899/2000 - 0s - loss: 394675.4384 - mean_squared_error: 394675.4384 - val_loss: 354470.4357 - val_mean_squared_error: 354470.4357 Epoch 1900/2000 - 0s - loss: 395040.6355 - mean_squared_error: 395040.6355 - val_loss: 351296.9343 - val_mean_squared_error: 351296.9343 Epoch 1901/2000 - 0s - loss: 394279.4941 - mean_squared_error: 394279.4941 - val_loss: 351612.3258 - val_mean_squared_error: 351612.3258 Epoch 1902/2000 - 0s - loss: 394407.4887 - mean_squared_error: 394407.4887 - val_loss: 353701.2704 - val_mean_squared_error: 353701.2704 Epoch 1903/2000 - 0s - loss: 394554.4996 - mean_squared_error: 394554.4996 - val_loss: 357333.1414 - val_mean_squared_error: 357333.1414 Epoch 1904/2000 - 0s - loss: 394564.6796 - mean_squared_error: 394564.6796 - val_loss: 352524.9584 - val_mean_squared_error: 352524.9584 Epoch 1905/2000 - 0s - loss: 394802.8757 - mean_squared_error: 394802.8757 - val_loss: 351848.0285 - val_mean_squared_error: 351848.0285 Epoch 1906/2000 - 0s - loss: 394504.0665 - mean_squared_error: 394504.0665 - val_loss: 352623.8804 - val_mean_squared_error: 352623.8804 Epoch 1907/2000 - 0s - loss: 394084.1085 - mean_squared_error: 394084.1085 - val_loss: 352172.3271 - val_mean_squared_error: 352172.3271 Epoch 1908/2000 - 0s - loss: 394395.0822 - mean_squared_error: 394395.0822 - val_loss: 353779.5877 - val_mean_squared_error: 353779.5877 Epoch 1909/2000 - 0s - loss: 394750.7526 - mean_squared_error: 394750.7526 - val_loss: 351798.7503 - val_mean_squared_error: 351798.7503 Epoch 1910/2000 - 0s - loss: 394402.4697 - mean_squared_error: 394402.4697 - val_loss: 353139.7217 - val_mean_squared_error: 353139.7217 Epoch 1911/2000 - 0s - loss: 395000.8836 - mean_squared_error: 395000.8836 - val_loss: 351335.7235 - val_mean_squared_error: 351335.7235 Epoch 1912/2000 - 0s - loss: 394422.7909 - mean_squared_error: 394422.7909 - val_loss: 351607.3362 - val_mean_squared_error: 351607.3362 Epoch 1913/2000 - 0s - loss: 394186.5859 - mean_squared_error: 394186.5859 - val_loss: 351026.2007 - val_mean_squared_error: 351026.2007 Epoch 1914/2000 - 0s - loss: 394105.2153 - mean_squared_error: 394105.2153 - val_loss: 351883.1954 - val_mean_squared_error: 351883.1954 Epoch 1915/2000 - 0s - loss: 394142.5565 - mean_squared_error: 394142.5565 - val_loss: 351992.0573 - val_mean_squared_error: 351992.0573 Epoch 1916/2000 - 0s - loss: 394209.5957 - mean_squared_error: 394209.5957 - val_loss: 352018.4708 - val_mean_squared_error: 352018.4708 Epoch 1917/2000 - 0s - loss: 393973.0199 - mean_squared_error: 393973.0199 - val_loss: 352887.3921 - val_mean_squared_error: 352887.3921 Epoch 1918/2000 - 0s - loss: 394573.7631 - mean_squared_error: 394573.7631 - val_loss: 351478.7113 - val_mean_squared_error: 351478.7113 Epoch 1919/2000 - 0s - loss: 394233.9014 - mean_squared_error: 394233.9014 - val_loss: 351782.2848 - val_mean_squared_error: 351782.2848 Epoch 1920/2000 - 0s - loss: 393708.4802 - mean_squared_error: 393708.4802 - val_loss: 350981.9167 - val_mean_squared_error: 350981.9167 Epoch 1921/2000 - 0s - loss: 394551.6854 - mean_squared_error: 394551.6854 - val_loss: 351259.1100 - val_mean_squared_error: 351259.1100 Epoch 1922/2000 - 0s - loss: 394300.5748 - mean_squared_error: 394300.5748 - val_loss: 352688.1585 - val_mean_squared_error: 352688.1585 Epoch 1923/2000 - 0s - loss: 394623.9563 - mean_squared_error: 394623.9563 - val_loss: 354501.3561 - val_mean_squared_error: 354501.3561 Epoch 1924/2000 - 0s - loss: 394134.7472 - mean_squared_error: 394134.7472 - val_loss: 351192.2701 - val_mean_squared_error: 351192.2701 Epoch 1925/2000 - 0s - loss: 394189.8784 - mean_squared_error: 394189.8784 - val_loss: 351355.5768 - val_mean_squared_error: 351355.5768 Epoch 1926/2000 - 0s - loss: 393910.6115 - mean_squared_error: 393910.6115 - val_loss: 350905.4286 - val_mean_squared_error: 350905.4286 Epoch 1927/2000 - 0s - loss: 394152.6381 - mean_squared_error: 394152.6381 - val_loss: 351523.0727 - val_mean_squared_error: 351523.0727 Epoch 1928/2000 - 0s - loss: 394294.2439 - mean_squared_error: 394294.2439 - val_loss: 355596.1872 - val_mean_squared_error: 355596.1872 Epoch 1929/2000 - 0s - loss: 394115.4967 - mean_squared_error: 394115.4967 - val_loss: 355812.5067 - val_mean_squared_error: 355812.5067 Epoch 1930/2000 - 0s - loss: 393797.4838 - mean_squared_error: 393797.4838 - val_loss: 351328.4472 - val_mean_squared_error: 351328.4472 Epoch 1931/2000 - 0s - loss: 394132.0738 - mean_squared_error: 394132.0738 - val_loss: 351409.6900 - val_mean_squared_error: 351409.6900 Epoch 1932/2000 - 0s - loss: 393748.0739 - mean_squared_error: 393748.0739 - val_loss: 351585.4802 - val_mean_squared_error: 351585.4802 Epoch 1933/2000 - 0s - loss: 393903.1115 - mean_squared_error: 393903.1115 - val_loss: 353536.1962 - val_mean_squared_error: 353536.1962 Epoch 1934/2000 - 0s - loss: 393744.9373 - mean_squared_error: 393744.9373 - val_loss: 351350.9460 - val_mean_squared_error: 351350.9460 Epoch 1935/2000 - 0s - loss: 393876.7852 - mean_squared_error: 393876.7852 - val_loss: 351608.3271 - val_mean_squared_error: 351608.3271 Epoch 1936/2000 - 0s - loss: 394187.8004 - mean_squared_error: 394187.8004 - val_loss: 351772.0344 - val_mean_squared_error: 351772.0344 Epoch 1937/2000 - 0s - loss: 393540.9881 - mean_squared_error: 393540.9881 - val_loss: 353472.4013 - val_mean_squared_error: 353472.4013 Epoch 1938/2000 - 0s - loss: 393782.9584 - mean_squared_error: 393782.9584 - val_loss: 355712.6185 - val_mean_squared_error: 355712.6185 Epoch 1939/2000 - 0s - loss: 393449.4543 - mean_squared_error: 393449.4543 - val_loss: 352500.5792 - val_mean_squared_error: 352500.5792 Epoch 1940/2000 - 0s - loss: 393696.1645 - mean_squared_error: 393696.1645 - val_loss: 350963.5166 - val_mean_squared_error: 350963.5166 Epoch 1941/2000 - 0s - loss: 393587.2240 - mean_squared_error: 393587.2240 - val_loss: 351407.7796 - val_mean_squared_error: 351407.7796 Epoch 1942/2000 - 0s - loss: 394481.9304 - mean_squared_error: 394481.9304 - val_loss: 351683.8144 - val_mean_squared_error: 351683.8144 Epoch 1943/2000 - 0s - loss: 393125.3010 - mean_squared_error: 393125.3010 - val_loss: 351086.1665 - val_mean_squared_error: 351086.1665 Epoch 1944/2000 - 0s - loss: 393339.4577 - mean_squared_error: 393339.4577 - val_loss: 351448.7661 - val_mean_squared_error: 351448.7661 Epoch 1945/2000 - 0s - loss: 393831.5202 - mean_squared_error: 393831.5202 - val_loss: 350619.9688 - val_mean_squared_error: 350619.9688 Epoch 1946/2000 - 0s - loss: 393500.0341 - mean_squared_error: 393500.0341 - val_loss: 350897.6872 - val_mean_squared_error: 350897.6872 Epoch 1947/2000 - 0s - loss: 393520.8761 - mean_squared_error: 393520.8761 - val_loss: 350720.5475 - val_mean_squared_error: 350720.5475 Epoch 1948/2000 - 0s - loss: 393573.5040 - mean_squared_error: 393573.5040 - val_loss: 350986.0485 - val_mean_squared_error: 350986.0485 Epoch 1949/2000 - 0s - loss: 393692.4018 - mean_squared_error: 393692.4018 - val_loss: 351082.4832 - val_mean_squared_error: 351082.4832 Epoch 1950/2000 - 0s - loss: 393331.1218 - mean_squared_error: 393331.1218 - val_loss: 351067.0518 - val_mean_squared_error: 351067.0518 Epoch 1951/2000 - 0s - loss: 394190.3323 - mean_squared_error: 394190.3323 - val_loss: 351539.0065 - val_mean_squared_error: 351539.0065 Epoch 1952/2000 - 0s - loss: 393509.6535 - mean_squared_error: 393509.6535 - val_loss: 350568.9273 - val_mean_squared_error: 350568.9273 Epoch 1953/2000 - 0s - loss: 393606.4257 - mean_squared_error: 393606.4257 - val_loss: 350894.8561 - val_mean_squared_error: 350894.8561 Epoch 1954/2000 - 0s - loss: 392888.9072 - mean_squared_error: 392888.9072 - val_loss: 354381.4223 - val_mean_squared_error: 354381.4223 Epoch 1955/2000 - 0s - loss: 393230.7114 - mean_squared_error: 393230.7114 - val_loss: 351739.0441 - val_mean_squared_error: 351739.0441 Epoch 1956/2000 - 0s - loss: 393856.4126 - mean_squared_error: 393856.4126 - val_loss: 350882.9445 - val_mean_squared_error: 350882.9445 Epoch 1957/2000 - 0s - loss: 393106.8604 - mean_squared_error: 393106.8604 - val_loss: 350710.3429 - val_mean_squared_error: 350710.3429 Epoch 1958/2000 - 0s - loss: 393181.0624 - mean_squared_error: 393181.0624 - val_loss: 352552.5458 - val_mean_squared_error: 352552.5458 Epoch 1959/2000 - 0s - loss: 393266.2823 - mean_squared_error: 393266.2823 - val_loss: 351487.5482 - val_mean_squared_error: 351487.5482 Epoch 1960/2000 - 0s - loss: 393295.5574 - mean_squared_error: 393295.5574 - val_loss: 350987.4455 - val_mean_squared_error: 350987.4455 Epoch 1961/2000 - 0s - loss: 393692.0431 - mean_squared_error: 393692.0431 - val_loss: 350439.3957 - val_mean_squared_error: 350439.3957 Epoch 1962/2000 - 0s - loss: 393166.8479 - mean_squared_error: 393166.8479 - val_loss: 350663.6741 - val_mean_squared_error: 350663.6741 Epoch 1963/2000 - 0s - loss: 393099.1775 - mean_squared_error: 393099.1775 - val_loss: 350883.2008 - val_mean_squared_error: 350883.2008 Epoch 1964/2000 - 0s - loss: 393037.7546 - mean_squared_error: 393037.7546 - val_loss: 353033.8329 - val_mean_squared_error: 353033.8329 Epoch 1965/2000 - 0s - loss: 393300.0515 - mean_squared_error: 393300.0515 - val_loss: 350703.8575 - val_mean_squared_error: 350703.8575 Epoch 1966/2000 - 0s - loss: 392879.9891 - mean_squared_error: 392879.9891 - val_loss: 350489.7307 - val_mean_squared_error: 350489.7307 Epoch 1967/2000 - 0s - loss: 393060.9444 - mean_squared_error: 393060.9444 - val_loss: 350217.0181 - val_mean_squared_error: 350217.0181 Epoch 1968/2000 - 0s - loss: 392965.4809 - mean_squared_error: 392965.4809 - val_loss: 351338.5765 - val_mean_squared_error: 351338.5765 Epoch 1969/2000 - 0s - loss: 393158.6968 - mean_squared_error: 393158.6968 - val_loss: 350496.7281 - val_mean_squared_error: 350496.7281 Epoch 1970/2000 - 0s - loss: 393017.0785 - mean_squared_error: 393017.0785 - val_loss: 351737.2013 - val_mean_squared_error: 351737.2013 Epoch 1971/2000 - 0s - loss: 392864.5516 - mean_squared_error: 392864.5516 - val_loss: 351419.1868 - val_mean_squared_error: 351419.1868 Epoch 1972/2000 - 0s - loss: 392802.1848 - mean_squared_error: 392802.1848 - val_loss: 350596.9279 - val_mean_squared_error: 350596.9279 Epoch 1973/2000 - 0s - loss: 393229.4714 - mean_squared_error: 393229.4714 - val_loss: 352161.0665 - val_mean_squared_error: 352161.0665 Epoch 1974/2000 - 0s - loss: 393905.6125 - mean_squared_error: 393905.6125 - val_loss: 351650.0413 - val_mean_squared_error: 351650.0413 Epoch 1975/2000 - 0s - loss: 393232.5136 - mean_squared_error: 393232.5136 - val_loss: 350512.8810 - val_mean_squared_error: 350512.8810 Epoch 1976/2000 - 0s - loss: 392996.3353 - mean_squared_error: 392996.3353 - val_loss: 350257.6995 - val_mean_squared_error: 350257.6995 Epoch 1977/2000 - 0s - loss: 392572.7001 - mean_squared_error: 392572.7001 - val_loss: 351083.8819 - val_mean_squared_error: 351083.8819 Epoch 1978/2000 - 0s - loss: 393093.1763 - mean_squared_error: 393093.1763 - val_loss: 350253.7080 - val_mean_squared_error: 350253.7080 Epoch 1979/2000 - 0s - loss: 392868.2256 - mean_squared_error: 392868.2256 - val_loss: 353122.8400 - val_mean_squared_error: 353122.8400 Epoch 1980/2000 - 0s - loss: 392905.3495 - mean_squared_error: 392905.3495 - val_loss: 350134.9386 - val_mean_squared_error: 350134.9386 Epoch 1981/2000 - 0s - loss: 392815.0487 - mean_squared_error: 392815.0487 - val_loss: 350536.9625 - val_mean_squared_error: 350536.9625 Epoch 1982/2000 - 0s - loss: 392264.1909 - mean_squared_error: 392264.1909 - val_loss: 351010.5383 - val_mean_squared_error: 351010.5383 Epoch 1983/2000 - 0s - loss: 392464.8971 - mean_squared_error: 392464.8971 - val_loss: 352511.0336 - val_mean_squared_error: 352511.0336 Epoch 1984/2000 - 0s - loss: 392728.0114 - mean_squared_error: 392728.0114 - val_loss: 350361.5722 - val_mean_squared_error: 350361.5722 Epoch 1985/2000 - 0s - loss: 392867.1127 - mean_squared_error: 392867.1127 - val_loss: 350956.7653 - val_mean_squared_error: 350956.7653 Epoch 1986/2000 - 0s - loss: 393184.4572 - mean_squared_error: 393184.4572 - val_loss: 350048.0675 - val_mean_squared_error: 350048.0675 Epoch 1987/2000 - 0s - loss: 392299.4790 - mean_squared_error: 392299.4790 - val_loss: 350371.2309 - val_mean_squared_error: 350371.2309 Epoch 1988/2000 - 0s - loss: 392543.2559 - mean_squared_error: 392543.2559 - val_loss: 349955.0816 - val_mean_squared_error: 349955.0816 Epoch 1989/2000 - 0s - loss: 392207.9921 - mean_squared_error: 392207.9921 - val_loss: 350564.9174 - val_mean_squared_error: 350564.9174 Epoch 1990/2000 - 0s - loss: 392585.1471 - mean_squared_error: 392585.1471 - val_loss: 351939.7183 - val_mean_squared_error: 351939.7183 Epoch 1991/2000 - 0s - loss: 392636.6855 - mean_squared_error: 392636.6855 - val_loss: 350742.1343 - val_mean_squared_error: 350742.1343 Epoch 1992/2000 - 0s - loss: 392685.5213 - mean_squared_error: 392685.5213 - val_loss: 350790.6480 - val_mean_squared_error: 350790.6480 Epoch 1993/2000 - 0s - loss: 392365.0846 - mean_squared_error: 392365.0846 - val_loss: 350923.8519 - val_mean_squared_error: 350923.8519 Epoch 1994/2000 - 0s - loss: 392777.8883 - mean_squared_error: 392777.8883 - val_loss: 352346.1241 - val_mean_squared_error: 352346.1241 Epoch 1995/2000 - 0s - loss: 392558.7267 - mean_squared_error: 392558.7267 - val_loss: 350589.9146 - val_mean_squared_error: 350589.9146 Epoch 1996/2000 - 0s - loss: 392280.0797 - mean_squared_error: 392280.0797 - val_loss: 350947.3711 - val_mean_squared_error: 350947.3711 Epoch 1997/2000 - 0s - loss: 392389.0843 - mean_squared_error: 392389.0843 - val_loss: 351116.2743 - val_mean_squared_error: 351116.2743 Epoch 1998/2000 - 0s - loss: 392736.0409 - mean_squared_error: 392736.0409 - val_loss: 350774.5216 - val_mean_squared_error: 350774.5216 Epoch 1999/2000 - 0s - loss: 392251.0854 - mean_squared_error: 392251.0854 - val_loss: 349920.3269 - val_mean_squared_error: 349920.3269 Epoch 2000/2000 - 0s - loss: 392027.3671 - mean_squared_error: 392027.3671 - val_loss: 349922.0056 - val_mean_squared_error: 349922.0056 32/13485 [..............................] - ETA: 0s 4544/13485 [=========>....................] - ETA: 0s 8928/13485 [==================>...........] - ETA: 0s 13472/13485 [============================>.] - ETA: 0s 13485/13485 [==============================] - 0s 11us/step root mean_squared_error: 619.680206

Multilayer Networks

If a single-layer perceptron network learns the importance of different combinations of features in the data...

What would another network learn if it had a second (hidden) layer of neurons?

It depends on how we train the network. We'll talk in the next section about how this training works, but the general idea is that we still work backward from the error gradient.

That is, the last layer learns from error in the output; the second-to-last layer learns from error transmitted through that last layer, etc. It's a touch hand-wavy for now, but we'll make it more concrete later.

Given this approach, we can say that:

The second (hidden) layer is learning features composed of activations in the first (hidden) layer
The first (hidden) layer is learning feature weights that enable the second layer to perform best
- Why? Earlier, the first hidden layer just learned feature weights because that's how it was judged
- Now, the first hidden layer is judged on the error in the second layer, so it learns to contribute to that second layer
The second layer is learning new features that aren't explicit in the data, and is teaching the first layer to supply it with the necessary information to compose these new features

So instead of just feature weighting and combining, we have new feature learning!

This concept is the foundation of the "Deep Feed-Forward Network"

Let's try it!

Add a layer to your Keras network, perhaps another 20 neurons, and see how the training goes.

if you get stuck, there is a solution in the Keras-DFFN notebook

I'm getting RMSE < $1000 by epoch 35 or so

< $800 by epoch 90

In this configuration, mine makes progress to around 700 epochs or so and then stalls with RMSE around $560

SDS-2.x, Scalable Data Engineering Science

Artificial Neural Network - Perceptron

Linear Perceptron

Neural Network with Keras

Keras is a High-Level API for Neural Networks and Deep Learning

"Being able to go from idea to result with the least possible delay is key to doing good research."

Keras has wide adoption in industry

We'll build a "Dense Feed-Forward Shallow" Network:

Training: Gradient Descent

Some ideas to help build your intuition

Training: Backpropagation

Ok so we've come up with a very slow way to perform a linear regression.

Welcome to Neural Networks in the 1960s!

Watch closely now because this is where the magic happens...

Non-Linearity + Perceptron = Universal Approximation