ScaDaMaLe Course site and book

This is a 2019-2021 augmentation and update of Adam Breindel's initial notebooks.

Thanks to Christian von Koch and William Anzén for their contributions towards making these materials Spark 3.0.1 and Python 3+ compliant.

Artificial Neural Network - Perceptron

The field of artificial neural networks started out with an electromechanical binary unit called a perceptron.

The perceptron took a weighted set of input signals and chose an ouput state (on/off or high/low) based on a threshold.

(raaz) Thus, the perceptron is defined by:

\[ f(1, x_1,x_2,\ldots , x_n , ; , w_0,w_1,w_2,\ldots , w_n) = \begin{cases} 1 & \text{if} \quad \sum_{i=0}^n w_i x_i > 0 \ 0 & \text{otherwise} \end{cases} \] and implementable with the following arithmetical and logical unit (ALU) operations in a machine:

n inputs from one $n$-dimensional data point: $x_1,x_2,\ldots x_n , \in , \mathbb{R}^n$
arithmetic operations
- n+1 multiplications
- n additions
boolean operations
- one if-then on an inequality
one output $o \in {0,1}$, i.e., $o$ belongs to the set containing $0$ and $1$
n+1 parameters of interest

This is just a hyperplane given by a dot product of $n+1$ known inputs and $n+1$ unknown parameters that can be estimated. This hyperplane can be used to define a hyperplane that partitions $\mathbb{R}^{n+1}$, the real Euclidean space, into two parts labelled by the outputs $0$ and $1$.

The problem of finding estimates of the parameters, $(\hat{w}_0,\hat{w}_1,\hat{w}_2,\ldots \hat{w}_n) \in \mathbb{R}^{(n+1)}$, in some statistically meaningful manner for a predicting task by using the training data given by, say $k$ labelled points, where you know both the input and output: \[ \left( ( , 1, x_1^{(1)},x_2^{(1)}, \ldots x_n^{(1)}), (o^{(1)}) , ), , ( , 1, x_1^{(2)},x_2^{(2)}, \ldots x_n^{(2)}), (o^{(2)}) , ), , \ldots , , ( , 1, x_1^{(k)},x_2^{(k)}, \ldots x_n^{(k)}), (o^{(k)}) , ) \right) , \in , (\mathbb{R}^{n+1} \times { 0,1 } )^k \] is the machine learning problem here.

Succinctly, we are after a random mapping, denoted below by $\mapsto_{\rightsquigarrow}$, called the estimator: \[ (\mathbb{R}^{n+1} \times {0,1})^k \mapsto_{\rightsquigarrow} , \left( , \mathtt{model}( (1,x_1,x_2,\ldots,x_n) ,;, (\hat{w}_0,\hat{w}_1,\hat{w}_2,\ldots \hat{w}_n)) : \mathbb{R}^{n+1} \to {0,1} , \right) \] which takes random labelled dataset (to understand random here think of two scientists doing independent experiments to get their own training datasets) of size $k$ and returns a model. These mathematical notions correspond exactly to the estimator and model (which is a transformer) in the language of Apache Spark's Machine Learning Pipleines we have seen before.

We can use this transformer for prediction of unlabelled data where we only observe the input and what to know the output under some reasonable assumptions.

Of course we want to be able to generalize so we don't overfit to the training data using some empirical risk minisation rule such as cross-validation. Again, we have seen these in Apache Spark for other ML methods like linear regression and decision trees.

If the output isn't right, we can adjust the weights, threshold, or bias ($x_0$ above)

The model was inspired by discoveries about the neurons of animals, so hopes were quite high that it could lead to a sophisticated machine. This model can be extended by adding multiple neurons in parallel. And we can use linear output instead of a threshold if we like for the output.

If we were to do so, the output would look like ${x \cdot w} + w_0$ (this is where the vector multiplication and, eventually, matrix multiplication, comes in)

When we look at the math this way, we see that despite this being an interesting model, it's really just a fancy linear calculation.

And, in fact, the proof that this model -- being linear -- could not solve any problems whose solution was nonlinear ... led to the first of several "AI / neural net winters" when the excitement was quickly replaced by disappointment, and most research was abandoned.

Linear Perceptron

We'll get to the non-linear part, but the linear perceptron model is a great way to warm up and bridge the gap from traditional linear regression to the neural-net flavor.

Let's look at a problem -- the diamonds dataset from R -- and analyze it using two traditional methods in Scikit-Learn, and then we'll start attacking it with neural networks!

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_squared_error

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)

import IPython.display as disp
pd.set_option('display.width', 200)
disp.display(df[:10])

   Unnamed: 0  carat        cut color clarity  ...  table  price     x     y     z
0           1   0.23      Ideal     E     SI2  ...   55.0    326  3.95  3.98  2.43
1           2   0.21    Premium     E     SI1  ...   61.0    326  3.89  3.84  2.31
2           3   0.23       Good     E     VS1  ...   65.0    327  4.05  4.07  2.31
3           4   0.29    Premium     I     VS2  ...   58.0    334  4.20  4.23  2.63
4           5   0.31       Good     J     SI2  ...   58.0    335  4.34  4.35  2.75
5           6   0.24  Very Good     J    VVS2  ...   57.0    336  3.94  3.96  2.48
6           7   0.24  Very Good     I    VVS1  ...   57.0    336  3.95  3.98  2.47
7           8   0.26  Very Good     H     SI1  ...   55.0    337  4.07  4.11  2.53
8           9   0.22       Fair     E     VS2  ...   61.0    337  3.87  3.78  2.49
9          10   0.23  Very Good     H     VS1  ...   61.0    338  4.00  4.05  2.39

[10 rows x 11 columns]

df2 = df.drop(df.columns[0], axis=1)

disp.display(df2[:3])

   carat      cut color clarity  depth  table  price     x     y     z
0   0.23    Ideal     E     SI2   61.5   55.0    326  3.95  3.98  2.43
1   0.21  Premium     E     SI1   59.8   61.0    326  3.89  3.84  2.31
2   0.23     Good     E     VS1   56.9   65.0    327  4.05  4.07  2.31

df3 = pd.get_dummies(df2) # this gives a one-hot encoding of categorial variables

disp.display(df3.iloc[:3, 7:18])

   cut_Fair  cut_Good  cut_Ideal  ...  color_G  color_H  color_I
0         0         0          1  ...        0        0        0
1         0         0          0  ...        0        0        0
2         0         1          0  ...        0        0        0

[3 rows x 11 columns]

# pre-process to get y
y = df3.iloc[:,3:4].values.flatten()
y.flatten()

# preprocess and reshape X as a matrix
X = df3.drop(df3.columns[3], axis=1).values
np.shape(X)

# break the dataset into training and test set with a 75% and 25% split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Define a decisoin tree model with max depth 10
dt = DecisionTreeRegressor(random_state=0, max_depth=10)

# fit the decision tree to the training data to get a fitted model
model = dt.fit(X_train, y_train)

# predict the features or X values of the test data using the fitted model
y_pred = model.predict(X_test)

# print the MSE performance measure of the fit by comparing the predicted versus the observed values of y 
print("RMSE %f" % np.sqrt(mean_squared_error(y_test, y_pred)) )

RMSE 727.042036

from sklearn import linear_model

# Do the same with linear regression and not a worse MSE
lr = linear_model.LinearRegression()
linear_model = lr.fit(X_train, y_train)

y_pred = linear_model.predict(X_test)
print("RMSE %f" % np.sqrt(mean_squared_error(y_test, y_pred)) )

RMSE 1124.086095

Now that we have a baseline, let's build a neural network -- linear at first -- and go further.

Neural Network with Keras

Keras is a High-Level API for Neural Networks and Deep Learning

"Being able to go from idea to result with the least possible delay is key to doing good research."

Maintained by Francois Chollet at Google, it provides

High level APIs
Pluggable backends for Theano, TensorFlow, CNTK, MXNet
CPU/GPU support
The now-officially-endorsed high-level wrapper for TensorFlow; a version ships in TF
Model persistence and other niceties
JavaScript, iOS, etc. deployment
Interop with further frameworks, like DeepLearning4J, Spark DL Pipelines ...

Well, with all this, why would you ever not use Keras?

As an API/Facade, Keras doesn't directly expose all of the internals you might need for something custom and low-level ... so you might need to implement at a lower level first, and then perhaps wrap it to make it easily usable in Keras.

Mr. Chollet compiles stats (roughly quarterly) on "[t]he state of the deep learning landscape: GitHub activity of major libraries over the past quarter (tickets, forks, and contributors)."

(October 2017: https://twitter.com/fchollet/status/915366704401719296; https://twitter.com/fchollet/status/915626952408436736)

GitHub

Research

Keras has wide adoption in industry

We'll build a "Dense Feed-Forward Shallow" Network:

(the number of units in the following diagram does not exactly match ours)

Grab a Keras API cheat sheet from https://s3.amazonaws.com/assets.datacamp.com/blogassets/KerasCheatSheetPython.pdf

from keras.models import Sequential
from keras.layers import Dense

# we are going to add layers sequentially one after the other (feed-forward) to our neural network model
model = Sequential()

# the first layer has 30 nodes (or neurons) with input dimension 26 for our diamonds data
# we will use Nomal or Guassian kernel to initialise the weights we want to estimate
# our activation function is linear (to mimic linear regression)
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='linear'))
# the next layer is for the response y and has only one node
model.add(Dense(1, kernel_initializer='normal', activation='linear'))
# compile the model with other specifications for loss and type of gradient descent optimisation routine
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
# fit the model to the training data using stochastic gradient descent with a batch-size of 200 and 10% of data held out for validation
history = model.fit(X_train, y_train, epochs=10, batch_size=200, validation_split=0.1)

scores = model.evaluate(X_test, y_test)
print()
print("test set RMSE: %f" % np.sqrt(scores[1]))

Using TensorFlow backend.
WARNING:tensorflow:From /databricks/python/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /databricks/python/lib/python3.7/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Train on 36409 samples, validate on 4046 samples
Epoch 1/10

  200/36409 [..............................] - ETA: 30s - loss: 24515974.0000 - mean_squared_error: 24515974.0000
 8600/36409 [======>.......................] - ETA: 0s - loss: 30972551.0233 - mean_squared_error: 30972551.0233 
16400/36409 [============>.................] - ETA: 0s - loss: 31092082.1707 - mean_squared_error: 31092082.1707
24400/36409 [===================>..........] - ETA: 0s - loss: 30821887.3934 - mean_squared_error: 30821887.3934
33200/36409 [==========================>...] - ETA: 0s - loss: 30772256.5783 - mean_squared_error: 30772256.5783
36409/36409 [==============================] - 0s 12us/step - loss: 30662987.2877 - mean_squared_error: 30662987.2877 - val_loss: 30057002.5457 - val_mean_squared_error: 30057002.5457
Epoch 2/10

  200/36409 [..............................] - ETA: 0s - loss: 27102510.0000 - mean_squared_error: 27102510.0000
 9000/36409 [======>.......................] - ETA: 0s - loss: 28506919.5556 - mean_squared_error: 28506919.5556
17600/36409 [=============>................] - ETA: 0s - loss: 27940962.7727 - mean_squared_error: 27940962.7727
26200/36409 [====================>.........] - ETA: 0s - loss: 27211811.0076 - mean_squared_error: 27211811.0076
34800/36409 [===========================>..] - ETA: 0s - loss: 26426392.1034 - mean_squared_error: 26426392.1034
36409/36409 [==============================] - 0s 6us/step - loss: 26167957.1320 - mean_squared_error: 26167957.1320 - val_loss: 23785895.3554 - val_mean_squared_error: 23785895.3554
Epoch 3/10

  200/36409 [..............................] - ETA: 0s - loss: 17365908.0000 - mean_squared_error: 17365908.0000
 7200/36409 [====>.........................] - ETA: 0s - loss: 22444847.5556 - mean_squared_error: 22444847.5556
16000/36409 [============>.................] - ETA: 0s - loss: 21373980.3000 - mean_squared_error: 21373980.3000
24000/36409 [==================>...........] - ETA: 0s - loss: 21005080.0500 - mean_squared_error: 21005080.0500
31800/36409 [=========================>....] - ETA: 0s - loss: 20236407.7170 - mean_squared_error: 20236407.7170
36409/36409 [==============================] - 0s 7us/step - loss: 20020372.1159 - mean_squared_error: 20020372.1159 - val_loss: 18351806.8710 - val_mean_squared_error: 18351806.8710
Epoch 4/10

  200/36409 [..............................] - ETA: 0s - loss: 17831442.0000 - mean_squared_error: 17831442.0000
 8400/36409 [=====>........................] - ETA: 0s - loss: 17513972.2857 - mean_squared_error: 17513972.2857
17000/36409 [=============>................] - ETA: 0s - loss: 16829699.6941 - mean_squared_error: 16829699.6941
25600/36409 [====================>.........] - ETA: 0s - loss: 16673756.4375 - mean_squared_error: 16673756.4375
33800/36409 [==========================>...] - ETA: 0s - loss: 16443651.8225 - mean_squared_error: 16443651.8225
36409/36409 [==============================] - 0s 6us/step - loss: 16317392.6930 - mean_squared_error: 16317392.6930 - val_loss: 16164358.2887 - val_mean_squared_error: 16164358.2887
Epoch 5/10

  200/36409 [..............................] - ETA: 0s - loss: 20413018.0000 - mean_squared_error: 20413018.0000
 7400/36409 [=====>........................] - ETA: 0s - loss: 15570987.0000 - mean_squared_error: 15570987.0000
14800/36409 [===========>..................] - ETA: 0s - loss: 15013196.5405 - mean_squared_error: 15013196.5405
23400/36409 [==================>...........] - ETA: 0s - loss: 15246935.8034 - mean_squared_error: 15246935.8034
32000/36409 [=========================>....] - ETA: 0s - loss: 15250803.7375 - mean_squared_error: 15250803.7375
36409/36409 [==============================] - 0s 7us/step - loss: 15255931.8414 - mean_squared_error: 15255931.8414 - val_loss: 15730755.8908 - val_mean_squared_error: 15730755.8908
Epoch 6/10

  200/36409 [..............................] - ETA: 0s - loss: 18564152.0000 - mean_squared_error: 18564152.0000
 7600/36409 [=====>........................] - ETA: 0s - loss: 15086204.8421 - mean_squared_error: 15086204.8421
16200/36409 [============>.................] - ETA: 0s - loss: 15104538.2593 - mean_squared_error: 15104538.2593
24600/36409 [===================>..........] - ETA: 0s - loss: 15172120.3008 - mean_squared_error: 15172120.3008
32600/36409 [=========================>....] - ETA: 0s - loss: 15123702.1043 - mean_squared_error: 15123702.1043
36409/36409 [==============================] - 0s 7us/step - loss: 15066138.8398 - mean_squared_error: 15066138.8398 - val_loss: 15621212.2521 - val_mean_squared_error: 15621212.2521
Epoch 7/10

  200/36409 [..............................] - ETA: 0s - loss: 12937932.0000 - mean_squared_error: 12937932.0000
 8400/36409 [=====>........................] - ETA: 0s - loss: 15215220.5238 - mean_squared_error: 15215220.5238
16400/36409 [============>.................] - ETA: 0s - loss: 15116822.9268 - mean_squared_error: 15116822.9268
24600/36409 [===================>..........] - ETA: 0s - loss: 14993875.2439 - mean_squared_error: 14993875.2439
32600/36409 [=========================>....] - ETA: 0s - loss: 14956622.0184 - mean_squared_error: 14956622.0184
36409/36409 [==============================] - 0s 7us/step - loss: 14981999.6368 - mean_squared_error: 14981999.6368 - val_loss: 15533945.2353 - val_mean_squared_error: 15533945.2353
Epoch 8/10

  200/36409 [..............................] - ETA: 0s - loss: 17393156.0000 - mean_squared_error: 17393156.0000
 7800/36409 [=====>........................] - ETA: 0s - loss: 15290136.5128 - mean_squared_error: 15290136.5128
16200/36409 [============>.................] - ETA: 0s - loss: 15074332.1235 - mean_squared_error: 15074332.1235
24600/36409 [===================>..........] - ETA: 0s - loss: 14987445.0488 - mean_squared_error: 14987445.0488
33000/36409 [==========================>...] - ETA: 0s - loss: 14853941.5394 - mean_squared_error: 14853941.5394
36409/36409 [==============================] - 0s 7us/step - loss: 14896132.4141 - mean_squared_error: 14896132.4141 - val_loss: 15441119.5566 - val_mean_squared_error: 15441119.5566
Epoch 9/10

  200/36409 [..............................] - ETA: 0s - loss: 12659630.0000 - mean_squared_error: 12659630.0000
 8600/36409 [======>.......................] - ETA: 0s - loss: 14682766.8605 - mean_squared_error: 14682766.8605
17000/36409 [=============>................] - ETA: 0s - loss: 14851612.5882 - mean_squared_error: 14851612.5882
25600/36409 [====================>.........] - ETA: 0s - loss: 14755020.0234 - mean_squared_error: 14755020.0234
34200/36409 [===========================>..] - ETA: 0s - loss: 14854599.4737 - mean_squared_error: 14854599.4737
36409/36409 [==============================] - 0s 6us/step - loss: 14802259.4853 - mean_squared_error: 14802259.4853 - val_loss: 15339340.7177 - val_mean_squared_error: 15339340.7177
Epoch 10/10

  200/36409 [..............................] - ETA: 0s - loss: 14473119.0000 - mean_squared_error: 14473119.0000
 8600/36409 [======>.......................] - ETA: 0s - loss: 14292346.6512 - mean_squared_error: 14292346.6512
16200/36409 [============>.................] - ETA: 0s - loss: 14621621.4938 - mean_squared_error: 14621621.4938
24600/36409 [===================>..........] - ETA: 0s - loss: 14648206.4228 - mean_squared_error: 14648206.4228
33200/36409 [==========================>...] - ETA: 0s - loss: 14746160.4398 - mean_squared_error: 14746160.4398
36409/36409 [==============================] - 0s 7us/step - loss: 14699508.8054 - mean_squared_error: 14699508.8054 - val_loss: 15226518.1542 - val_mean_squared_error: 15226518.1542

   32/13485 [..............................] - ETA: 0s
 4096/13485 [========>.....................] - ETA: 0s
 7040/13485 [==============>...............] - ETA: 0s
 9920/13485 [=====================>........] - ETA: 0s
13485/13485 [==============================] - 0s 15us/step

test set RMSE: 3800.812819

model.summary() # do you understand why the number of parameters in layer 1 is 810? 26*30+30=810

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 30)                810       
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 31        
=================================================================
Total params: 841
Trainable params: 841
Non-trainable params: 0
_________________________________________________________________

Notes:

We didn't have to explicitly write the "input" layer, courtesy of the Keras API. We just said input_dim=26 on the first (and only) hidden layer.
kernel_initializer='normal' is a simple (though not always optimal) weight initialization
Epoch: 1 pass over all of the training data
Batch: Records processes together in a single training pass

How is our RMSE vs. the std dev of the response?

y.std()

Let's look at the error ...

import matplotlib.pyplot as plt

fig, ax = plt.subplots()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper left')

display(fig)

Let's set up a "long-running" training. This will take a few minutes to converge to the same performance we got more or less instantly with our sklearn linear regression :)

While it's running, we can talk about the training.

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].values.flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).values
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='linear'))
model.add(Dense(1, kernel_initializer='normal', activation='linear'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=250, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples
Epoch 1/250
 - 1s - loss: 28387336.4384 - mean_squared_error: 28387336.4384 - val_loss: 23739923.0816 - val_mean_squared_error: 23739923.0816
Epoch 2/250
 - 0s - loss: 18190573.9659 - mean_squared_error: 18190573.9659 - val_loss: 16213271.2121 - val_mean_squared_error: 16213271.2121
Epoch 3/250
 - 0s - loss: 15172630.1573 - mean_squared_error: 15172630.1573 - val_loss: 15625113.6841 - val_mean_squared_error: 15625113.6841
Epoch 4/250
 - 0s - loss: 14945745.1874 - mean_squared_error: 14945745.1874 - val_loss: 15453493.5042 - val_mean_squared_error: 15453493.5042
Epoch 5/250
 - 0s - loss: 14767640.2591 - mean_squared_error: 14767640.2591 - val_loss: 15254403.9486 - val_mean_squared_error: 15254403.9486
Epoch 6/250
 - 0s - loss: 14557887.6100 - mean_squared_error: 14557887.6100 - val_loss: 15019167.1374 - val_mean_squared_error: 15019167.1374
Epoch 7/250
 - 0s - loss: 14309593.5448 - mean_squared_error: 14309593.5448 - val_loss: 14737183.8052 - val_mean_squared_error: 14737183.8052
Epoch 8/250
 - 0s - loss: 14013285.0941 - mean_squared_error: 14013285.0941 - val_loss: 14407560.3356 - val_mean_squared_error: 14407560.3356
Epoch 9/250
 - 0s - loss: 13656069.2042 - mean_squared_error: 13656069.2042 - val_loss: 13997285.5230 - val_mean_squared_error: 13997285.5230
Epoch 10/250
 - 0s - loss: 13216458.2848 - mean_squared_error: 13216458.2848 - val_loss: 13489102.3277 - val_mean_squared_error: 13489102.3277
Epoch 11/250
 - 0s - loss: 12677035.7927 - mean_squared_error: 12677035.7927 - val_loss: 12879791.1409 - val_mean_squared_error: 12879791.1409
Epoch 12/250
 - 0s - loss: 12026548.9956 - mean_squared_error: 12026548.9956 - val_loss: 12144681.5783 - val_mean_squared_error: 12144681.5783
Epoch 13/250
 - 0s - loss: 11261201.3992 - mean_squared_error: 11261201.3992 - val_loss: 11298467.2511 - val_mean_squared_error: 11298467.2511
Epoch 14/250
 - 0s - loss: 10394848.5657 - mean_squared_error: 10394848.5657 - val_loss: 10359981.5892 - val_mean_squared_error: 10359981.5892
Epoch 15/250
 - 0s - loss: 9464690.1582 - mean_squared_error: 9464690.1582 - val_loss: 9377032.8176 - val_mean_squared_error: 9377032.8176
Epoch 16/250
 - 0s - loss: 8522523.9198 - mean_squared_error: 8522523.9198 - val_loss: 8407744.1396 - val_mean_squared_error: 8407744.1396
Epoch 17/250
 - 0s - loss: 7614509.1284 - mean_squared_error: 7614509.1284 - val_loss: 7483504.2244 - val_mean_squared_error: 7483504.2244
Epoch 18/250
 - 0s - loss: 6773874.9790 - mean_squared_error: 6773874.9790 - val_loss: 6642935.4123 - val_mean_squared_error: 6642935.4123
Epoch 19/250
 - 0s - loss: 6029308.8020 - mean_squared_error: 6029308.8020 - val_loss: 5907274.1873 - val_mean_squared_error: 5907274.1873
Epoch 20/250
 - 0s - loss: 5389141.6630 - mean_squared_error: 5389141.6630 - val_loss: 5281166.3648 - val_mean_squared_error: 5281166.3648
Epoch 21/250
 - 0s - loss: 4860038.6980 - mean_squared_error: 4860038.6980 - val_loss: 4764607.7313 - val_mean_squared_error: 4764607.7313
Epoch 22/250
 - 0s - loss: 4431739.1137 - mean_squared_error: 4431739.1137 - val_loss: 4351510.2142 - val_mean_squared_error: 4351510.2142
Epoch 23/250
 - 0s - loss: 4094045.5128 - mean_squared_error: 4094045.5128 - val_loss: 4021251.9317 - val_mean_squared_error: 4021251.9317
Epoch 24/250
 - 0s - loss: 3829986.6989 - mean_squared_error: 3829986.6989 - val_loss: 3757295.1629 - val_mean_squared_error: 3757295.1629
Epoch 25/250
 - 0s - loss: 3623823.6510 - mean_squared_error: 3623823.6510 - val_loss: 3552794.5538 - val_mean_squared_error: 3552794.5538
Epoch 26/250
 - 0s - loss: 3465635.4962 - mean_squared_error: 3465635.4962 - val_loss: 3393235.2467 - val_mean_squared_error: 3393235.2467
Epoch 27/250
 - 0s - loss: 3340038.8348 - mean_squared_error: 3340038.8348 - val_loss: 3269279.0370 - val_mean_squared_error: 3269279.0370
Epoch 28/250
 - 0s - loss: 3236200.2134 - mean_squared_error: 3236200.2134 - val_loss: 3156617.4010 - val_mean_squared_error: 3156617.4010
Epoch 29/250
 - 0s - loss: 3150650.9059 - mean_squared_error: 3150650.9059 - val_loss: 3065790.8180 - val_mean_squared_error: 3065790.8180
Epoch 30/250
 - 0s - loss: 3075868.5099 - mean_squared_error: 3075868.5099 - val_loss: 2992714.7820 - val_mean_squared_error: 2992714.7820
Epoch 31/250
 - 0s - loss: 3011537.2513 - mean_squared_error: 3011537.2513 - val_loss: 2916240.5889 - val_mean_squared_error: 2916240.5889
Epoch 32/250
 - 0s - loss: 2950834.9374 - mean_squared_error: 2950834.9374 - val_loss: 2853476.3283 - val_mean_squared_error: 2853476.3283
Epoch 33/250
 - 0s - loss: 2896146.2107 - mean_squared_error: 2896146.2107 - val_loss: 2797956.3931 - val_mean_squared_error: 2797956.3931
Epoch 34/250
 - 0s - loss: 2845359.1844 - mean_squared_error: 2845359.1844 - val_loss: 2756268.6799 - val_mean_squared_error: 2756268.6799
Epoch 35/250
 - 0s - loss: 2799651.4802 - mean_squared_error: 2799651.4802 - val_loss: 2696497.8046 - val_mean_squared_error: 2696497.8046
Epoch 36/250
 - 0s - loss: 2756932.6045 - mean_squared_error: 2756932.6045 - val_loss: 2652229.5122 - val_mean_squared_error: 2652229.5122
Epoch 37/250
 - 0s - loss: 2718146.0056 - mean_squared_error: 2718146.0056 - val_loss: 2610052.9367 - val_mean_squared_error: 2610052.9367
Epoch 38/250
 - 0s - loss: 2681826.8783 - mean_squared_error: 2681826.8783 - val_loss: 2576333.5379 - val_mean_squared_error: 2576333.5379
Epoch 39/250
 - 0s - loss: 2646768.1726 - mean_squared_error: 2646768.1726 - val_loss: 2535503.5141 - val_mean_squared_error: 2535503.5141
Epoch 40/250
 - 0s - loss: 2614777.8154 - mean_squared_error: 2614777.8154 - val_loss: 2503085.7453 - val_mean_squared_error: 2503085.7453
Epoch 41/250
 - 0s - loss: 2585534.7258 - mean_squared_error: 2585534.7258 - val_loss: 2473390.2650 - val_mean_squared_error: 2473390.2650
Epoch 42/250
 - 0s - loss: 2560146.6899 - mean_squared_error: 2560146.6899 - val_loss: 2444374.3726 - val_mean_squared_error: 2444374.3726
Epoch 43/250
 - 0s - loss: 2532669.1185 - mean_squared_error: 2532669.1185 - val_loss: 2418029.5618 - val_mean_squared_error: 2418029.5618
Epoch 44/250
 - 0s - loss: 2509591.7315 - mean_squared_error: 2509591.7315 - val_loss: 2393902.2946 - val_mean_squared_error: 2393902.2946
Epoch 45/250
 - 0s - loss: 2485903.8244 - mean_squared_error: 2485903.8244 - val_loss: 2374926.1074 - val_mean_squared_error: 2374926.1074
Epoch 46/250
 - 0s - loss: 2468145.9276 - mean_squared_error: 2468145.9276 - val_loss: 2353132.2925 - val_mean_squared_error: 2353132.2925
Epoch 47/250
 - 0s - loss: 2449389.6913 - mean_squared_error: 2449389.6913 - val_loss: 2330850.7967 - val_mean_squared_error: 2330850.7967
Epoch 48/250
 - 0s - loss: 2430694.4924 - mean_squared_error: 2430694.4924 - val_loss: 2315977.1396 - val_mean_squared_error: 2315977.1396
Epoch 49/250
 - 0s - loss: 2416348.0670 - mean_squared_error: 2416348.0670 - val_loss: 2295317.3459 - val_mean_squared_error: 2295317.3459
Epoch 50/250
 - 0s - loss: 2400174.9707 - mean_squared_error: 2400174.9707 - val_loss: 2280247.3585 - val_mean_squared_error: 2280247.3585
Epoch 51/250
 - 0s - loss: 2386847.7805 - mean_squared_error: 2386847.7805 - val_loss: 2269282.1988 - val_mean_squared_error: 2269282.1988
Epoch 52/250
 - 0s - loss: 2373865.5490 - mean_squared_error: 2373865.5490 - val_loss: 2253546.8710 - val_mean_squared_error: 2253546.8710
Epoch 53/250
 - 0s - loss: 2362687.3404 - mean_squared_error: 2362687.3404 - val_loss: 2241699.1739 - val_mean_squared_error: 2241699.1739
Epoch 54/250
 - 0s - loss: 2350158.3027 - mean_squared_error: 2350158.3027 - val_loss: 2229586.8293 - val_mean_squared_error: 2229586.8293
Epoch 55/250
 - 0s - loss: 2340199.4378 - mean_squared_error: 2340199.4378 - val_loss: 2223216.2266 - val_mean_squared_error: 2223216.2266
Epoch 56/250
 - 0s - loss: 2328815.8881 - mean_squared_error: 2328815.8881 - val_loss: 2215892.9633 - val_mean_squared_error: 2215892.9633
Epoch 57/250
 - 0s - loss: 2319769.2307 - mean_squared_error: 2319769.2307 - val_loss: 2202649.4680 - val_mean_squared_error: 2202649.4680
Epoch 58/250
 - 0s - loss: 2311106.1876 - mean_squared_error: 2311106.1876 - val_loss: 2190911.9708 - val_mean_squared_error: 2190911.9708
Epoch 59/250
 - 0s - loss: 2303140.6940 - mean_squared_error: 2303140.6940 - val_loss: 2190453.6531 - val_mean_squared_error: 2190453.6531
Epoch 60/250
 - 0s - loss: 2295309.2408 - mean_squared_error: 2295309.2408 - val_loss: 2173813.5528 - val_mean_squared_error: 2173813.5528
Epoch 61/250
 - 0s - loss: 2286456.1451 - mean_squared_error: 2286456.1451 - val_loss: 2167401.4068 - val_mean_squared_error: 2167401.4068
Epoch 62/250
 - 0s - loss: 2278925.0642 - mean_squared_error: 2278925.0642 - val_loss: 2158559.3234 - val_mean_squared_error: 2158559.3234
Epoch 63/250
 - 0s - loss: 2272597.3724 - mean_squared_error: 2272597.3724 - val_loss: 2151993.7547 - val_mean_squared_error: 2151993.7547
Epoch 64/250
 - 0s - loss: 2265358.6924 - mean_squared_error: 2265358.6924 - val_loss: 2146581.5500 - val_mean_squared_error: 2146581.5500
Epoch 65/250
 - 0s - loss: 2259204.1708 - mean_squared_error: 2259204.1708 - val_loss: 2138775.9063 - val_mean_squared_error: 2138775.9063
Epoch 66/250
 - 0s - loss: 2251698.1355 - mean_squared_error: 2251698.1355 - val_loss: 2132688.4142 - val_mean_squared_error: 2132688.4142
Epoch 67/250
 - 0s - loss: 2245842.9347 - mean_squared_error: 2245842.9347 - val_loss: 2127250.3298 - val_mean_squared_error: 2127250.3298
Epoch 68/250
 - 0s - loss: 2239787.1670 - mean_squared_error: 2239787.1670 - val_loss: 2128124.5670 - val_mean_squared_error: 2128124.5670
Epoch 69/250
 - 0s - loss: 2233091.2950 - mean_squared_error: 2233091.2950 - val_loss: 2120944.2249 - val_mean_squared_error: 2120944.2249
Epoch 70/250
 - 0s - loss: 2227085.7098 - mean_squared_error: 2227085.7098 - val_loss: 2114163.3953 - val_mean_squared_error: 2114163.3953
Epoch 71/250
 - 0s - loss: 2220383.1575 - mean_squared_error: 2220383.1575 - val_loss: 2119813.9272 - val_mean_squared_error: 2119813.9272
Epoch 72/250
 - 0s - loss: 2215016.5886 - mean_squared_error: 2215016.5886 - val_loss: 2098265.5178 - val_mean_squared_error: 2098265.5178
Epoch 73/250
 - 0s - loss: 2209031.0828 - mean_squared_error: 2209031.0828 - val_loss: 2093349.4299 - val_mean_squared_error: 2093349.4299
Epoch 74/250
 - 0s - loss: 2203458.1824 - mean_squared_error: 2203458.1824 - val_loss: 2087530.1435 - val_mean_squared_error: 2087530.1435
Epoch 75/250
 - 0s - loss: 2197507.4423 - mean_squared_error: 2197507.4423 - val_loss: 2084310.7636 - val_mean_squared_error: 2084310.7636
Epoch 76/250
 - 0s - loss: 2191870.9516 - mean_squared_error: 2191870.9516 - val_loss: 2078751.1230 - val_mean_squared_error: 2078751.1230
Epoch 77/250
 - 0s - loss: 2186590.2370 - mean_squared_error: 2186590.2370 - val_loss: 2073632.4209 - val_mean_squared_error: 2073632.4209
Epoch 78/250
 - 0s - loss: 2180675.0755 - mean_squared_error: 2180675.0755 - val_loss: 2067455.0566 - val_mean_squared_error: 2067455.0566
Epoch 79/250
 - 0s - loss: 2175871.8791 - mean_squared_error: 2175871.8791 - val_loss: 2062080.2746 - val_mean_squared_error: 2062080.2746
Epoch 80/250
 - 0s - loss: 2170960.8270 - mean_squared_error: 2170960.8270 - val_loss: 2076134.0571 - val_mean_squared_error: 2076134.0571
Epoch 81/250
 - 0s - loss: 2165095.7891 - mean_squared_error: 2165095.7891 - val_loss: 2060063.3091 - val_mean_squared_error: 2060063.3091
Epoch 82/250
 - 0s - loss: 2159210.9014 - mean_squared_error: 2159210.9014 - val_loss: 2054764.6278 - val_mean_squared_error: 2054764.6278
Epoch 83/250
 - 0s - loss: 2154804.6965 - mean_squared_error: 2154804.6965 - val_loss: 2042656.1980 - val_mean_squared_error: 2042656.1980
Epoch 84/250
 - 0s - loss: 2151000.8027 - mean_squared_error: 2151000.8027 - val_loss: 2041542.0261 - val_mean_squared_error: 2041542.0261
Epoch 85/250
 - 0s - loss: 2144402.9209 - mean_squared_error: 2144402.9209 - val_loss: 2034710.8433 - val_mean_squared_error: 2034710.8433
Epoch 86/250
 - 0s - loss: 2139382.6877 - mean_squared_error: 2139382.6877 - val_loss: 2029521.7206 - val_mean_squared_error: 2029521.7206
Epoch 87/250
 - 0s - loss: 2134508.9045 - mean_squared_error: 2134508.9045 - val_loss: 2025544.5526 - val_mean_squared_error: 2025544.5526
Epoch 88/250
 - 0s - loss: 2130530.0560 - mean_squared_error: 2130530.0560 - val_loss: 2020708.7063 - val_mean_squared_error: 2020708.7063
Epoch 89/250
 - 0s - loss: 2124692.8997 - mean_squared_error: 2124692.8997 - val_loss: 2016661.2367 - val_mean_squared_error: 2016661.2367
Epoch 90/250
 - 0s - loss: 2119100.6322 - mean_squared_error: 2119100.6322 - val_loss: 2024581.7835 - val_mean_squared_error: 2024581.7835
Epoch 91/250
 - 0s - loss: 2115483.7229 - mean_squared_error: 2115483.7229 - val_loss: 2008754.8749 - val_mean_squared_error: 2008754.8749
Epoch 92/250
 - 0s - loss: 2110360.7427 - mean_squared_error: 2110360.7427 - val_loss: 2007724.7695 - val_mean_squared_error: 2007724.7695
Epoch 93/250
 - 0s - loss: 2104714.5825 - mean_squared_error: 2104714.5825 - val_loss: 2008926.0319 - val_mean_squared_error: 2008926.0319
Epoch 94/250
 - 0s - loss: 2100296.9009 - mean_squared_error: 2100296.9009 - val_loss: 1995537.1630 - val_mean_squared_error: 1995537.1630
Epoch 95/250
 - 0s - loss: 2095775.2807 - mean_squared_error: 2095775.2807 - val_loss: 1998770.4627 - val_mean_squared_error: 1998770.4627
Epoch 96/250
 - 0s - loss: 2090610.8210 - mean_squared_error: 2090610.8210 - val_loss: 1991205.4927 - val_mean_squared_error: 1991205.4927
Epoch 97/250
 - 0s - loss: 2085764.6586 - mean_squared_error: 2085764.6586 - val_loss: 1982456.1479 - val_mean_squared_error: 1982456.1479
Epoch 98/250
 - 0s - loss: 2081778.4795 - mean_squared_error: 2081778.4795 - val_loss: 1984038.7297 - val_mean_squared_error: 1984038.7297
Epoch 99/250
 - 0s - loss: 2076921.0596 - mean_squared_error: 2076921.0596 - val_loss: 1974410.2433 - val_mean_squared_error: 1974410.2433
Epoch 100/250
 - 0s - loss: 2071642.4116 - mean_squared_error: 2071642.4116 - val_loss: 1970844.3068 - val_mean_squared_error: 1970844.3068
Epoch 101/250
 - 0s - loss: 2067992.1074 - mean_squared_error: 2067992.1074 - val_loss: 1966067.1146 - val_mean_squared_error: 1966067.1146
Epoch 102/250
 - 0s - loss: 2062705.6920 - mean_squared_error: 2062705.6920 - val_loss: 1964978.2044 - val_mean_squared_error: 1964978.2044
Epoch 103/250
 - 1s - loss: 2058635.8663 - mean_squared_error: 2058635.8663 - val_loss: 1960653.1206 - val_mean_squared_error: 1960653.1206
Epoch 104/250
 - 1s - loss: 2053453.2750 - mean_squared_error: 2053453.2750 - val_loss: 1956940.2522 - val_mean_squared_error: 1956940.2522
Epoch 105/250
 - 0s - loss: 2049969.1384 - mean_squared_error: 2049969.1384 - val_loss: 1950447.4171 - val_mean_squared_error: 1950447.4171
Epoch 106/250
 - 0s - loss: 2044646.2613 - mean_squared_error: 2044646.2613 - val_loss: 1946335.1700 - val_mean_squared_error: 1946335.1700
Epoch 107/250
 - 0s - loss: 2040847.5135 - mean_squared_error: 2040847.5135 - val_loss: 1950765.8945 - val_mean_squared_error: 1950765.8945
Epoch 108/250
 - 0s - loss: 2035333.6843 - mean_squared_error: 2035333.6843 - val_loss: 1943112.7308 - val_mean_squared_error: 1943112.7308
Epoch 109/250
 - 0s - loss: 2031570.0148 - mean_squared_error: 2031570.0148 - val_loss: 1937313.6635 - val_mean_squared_error: 1937313.6635
Epoch 110/250
 - 0s - loss: 2026515.8787 - mean_squared_error: 2026515.8787 - val_loss: 1930995.4182 - val_mean_squared_error: 1930995.4182
Epoch 111/250
 - 0s - loss: 2023262.6958 - mean_squared_error: 2023262.6958 - val_loss: 1926765.3571 - val_mean_squared_error: 1926765.3571
Epoch 112/250
 - 0s - loss: 2018275.2594 - mean_squared_error: 2018275.2594 - val_loss: 1923056.6220 - val_mean_squared_error: 1923056.6220
Epoch 113/250
 - 0s - loss: 2013793.3882 - mean_squared_error: 2013793.3882 - val_loss: 1920843.6845 - val_mean_squared_error: 1920843.6845
Epoch 114/250
 - 0s - loss: 2009802.3657 - mean_squared_error: 2009802.3657 - val_loss: 1916405.0942 - val_mean_squared_error: 1916405.0942
Epoch 115/250
 - 0s - loss: 2005557.3843 - mean_squared_error: 2005557.3843 - val_loss: 1920216.2247 - val_mean_squared_error: 1920216.2247
Epoch 116/250
 - 0s - loss: 2000834.9872 - mean_squared_error: 2000834.9872 - val_loss: 1913231.6625 - val_mean_squared_error: 1913231.6625
Epoch 117/250
 - 0s - loss: 1996924.4391 - mean_squared_error: 1996924.4391 - val_loss: 1905361.8010 - val_mean_squared_error: 1905361.8010
Epoch 118/250
 - 0s - loss: 1991738.4791 - mean_squared_error: 1991738.4791 - val_loss: 1901166.0414 - val_mean_squared_error: 1901166.0414
Epoch 119/250
 - 0s - loss: 1987464.5905 - mean_squared_error: 1987464.5905 - val_loss: 1900646.9869 - val_mean_squared_error: 1900646.9869
Epoch 120/250
 - 0s - loss: 1982866.4694 - mean_squared_error: 1982866.4694 - val_loss: 1896726.6348 - val_mean_squared_error: 1896726.6348
Epoch 121/250
 - 0s - loss: 1979190.1764 - mean_squared_error: 1979190.1764 - val_loss: 1889626.5674 - val_mean_squared_error: 1889626.5674
Epoch 122/250
 - 0s - loss: 1974747.3787 - mean_squared_error: 1974747.3787 - val_loss: 1889790.7669 - val_mean_squared_error: 1889790.7669
Epoch 123/250
 - 0s - loss: 1970600.7402 - mean_squared_error: 1970600.7402 - val_loss: 1881613.5393 - val_mean_squared_error: 1881613.5393
Epoch 124/250
 - 0s - loss: 1966440.0516 - mean_squared_error: 1966440.0516 - val_loss: 1881952.2009 - val_mean_squared_error: 1881952.2009
Epoch 125/250
 - 0s - loss: 1963144.5010 - mean_squared_error: 1963144.5010 - val_loss: 1874578.3827 - val_mean_squared_error: 1874578.3827
Epoch 126/250
 - 0s - loss: 1958002.3114 - mean_squared_error: 1958002.3114 - val_loss: 1881907.8777 - val_mean_squared_error: 1881907.8777
Epoch 127/250
 - 0s - loss: 1953781.4717 - mean_squared_error: 1953781.4717 - val_loss: 1866496.5014 - val_mean_squared_error: 1866496.5014
Epoch 128/250
 - 0s - loss: 1949579.8435 - mean_squared_error: 1949579.8435 - val_loss: 1862999.9314 - val_mean_squared_error: 1862999.9314
Epoch 129/250
 - 0s - loss: 1944837.7633 - mean_squared_error: 1944837.7633 - val_loss: 1859692.0708 - val_mean_squared_error: 1859692.0708
Epoch 130/250
 - 0s - loss: 1941181.8801 - mean_squared_error: 1941181.8801 - val_loss: 1870401.3512 - val_mean_squared_error: 1870401.3512
Epoch 131/250
 - 0s - loss: 1937206.9816 - mean_squared_error: 1937206.9816 - val_loss: 1855640.7789 - val_mean_squared_error: 1855640.7789
Epoch 132/250
 - 1s - loss: 1932482.7280 - mean_squared_error: 1932482.7280 - val_loss: 1853098.5790 - val_mean_squared_error: 1853098.5790
Epoch 133/250
 - 1s - loss: 1928831.6393 - mean_squared_error: 1928831.6393 - val_loss: 1845656.8632 - val_mean_squared_error: 1845656.8632
Epoch 134/250
 - 0s - loss: 1923718.3025 - mean_squared_error: 1923718.3025 - val_loss: 1841296.4944 - val_mean_squared_error: 1841296.4944
Epoch 135/250
 - 0s - loss: 1919285.1301 - mean_squared_error: 1919285.1301 - val_loss: 1839304.6138 - val_mean_squared_error: 1839304.6138
Epoch 136/250
 - 0s - loss: 1915512.9725 - mean_squared_error: 1915512.9725 - val_loss: 1833941.8848 - val_mean_squared_error: 1833941.8848
Epoch 137/250
 - 0s - loss: 1910338.2096 - mean_squared_error: 1910338.2096 - val_loss: 1829266.4789 - val_mean_squared_error: 1829266.4789
Epoch 138/250
 - 0s - loss: 1906807.1133 - mean_squared_error: 1906807.1133 - val_loss: 1826667.6707 - val_mean_squared_error: 1826667.6707
Epoch 139/250
 - 0s - loss: 1901958.9682 - mean_squared_error: 1901958.9682 - val_loss: 1822769.3903 - val_mean_squared_error: 1822769.3903
Epoch 140/250
 - 0s - loss: 1898180.0895 - mean_squared_error: 1898180.0895 - val_loss: 1818848.6565 - val_mean_squared_error: 1818848.6565
Epoch 141/250
 - 0s - loss: 1893384.4182 - mean_squared_error: 1893384.4182 - val_loss: 1825904.4577 - val_mean_squared_error: 1825904.4577
Epoch 142/250
 - 0s - loss: 1888857.0500 - mean_squared_error: 1888857.0500 - val_loss: 1812047.4093 - val_mean_squared_error: 1812047.4093
Epoch 143/250
 - 0s - loss: 1885649.5577 - mean_squared_error: 1885649.5577 - val_loss: 1815988.5720 - val_mean_squared_error: 1815988.5720
Epoch 144/250
 - 0s - loss: 1882728.8948 - mean_squared_error: 1882728.8948 - val_loss: 1804221.2971 - val_mean_squared_error: 1804221.2971
Epoch 145/250
 - 0s - loss: 1878070.7728 - mean_squared_error: 1878070.7728 - val_loss: 1800390.6373 - val_mean_squared_error: 1800390.6373
Epoch 146/250
 - 0s - loss: 1873359.5868 - mean_squared_error: 1873359.5868 - val_loss: 1796718.0096 - val_mean_squared_error: 1796718.0096
Epoch 147/250
 - 0s - loss: 1870450.4402 - mean_squared_error: 1870450.4402 - val_loss: 1793502.3480 - val_mean_squared_error: 1793502.3480
Epoch 148/250
 - 0s - loss: 1864935.4132 - mean_squared_error: 1864935.4132 - val_loss: 1790719.0737 - val_mean_squared_error: 1790719.0737
Epoch 149/250
 - 0s - loss: 1860335.5980 - mean_squared_error: 1860335.5980 - val_loss: 1787109.7601 - val_mean_squared_error: 1787109.7601
Epoch 150/250
 - 0s - loss: 1857528.6761 - mean_squared_error: 1857528.6761 - val_loss: 1782251.8564 - val_mean_squared_error: 1782251.8564
Epoch 151/250
 - 0s - loss: 1853086.1135 - mean_squared_error: 1853086.1135 - val_loss: 1779972.3222 - val_mean_squared_error: 1779972.3222
Epoch 152/250
 - 0s - loss: 1849172.5880 - mean_squared_error: 1849172.5880 - val_loss: 1775980.5159 - val_mean_squared_error: 1775980.5159
Epoch 153/250
 - 0s - loss: 1844933.4104 - mean_squared_error: 1844933.4104 - val_loss: 1771529.7060 - val_mean_squared_error: 1771529.7060
Epoch 154/250
 - 0s - loss: 1839683.2720 - mean_squared_error: 1839683.2720 - val_loss: 1767939.3851 - val_mean_squared_error: 1767939.3851
Epoch 155/250
 - 0s - loss: 1836064.3526 - mean_squared_error: 1836064.3526 - val_loss: 1764414.1924 - val_mean_squared_error: 1764414.1924
Epoch 156/250
 - 0s - loss: 1832377.5910 - mean_squared_error: 1832377.5910 - val_loss: 1761295.6983 - val_mean_squared_error: 1761295.6983
Epoch 157/250
 - 0s - loss: 1828378.2116 - mean_squared_error: 1828378.2116 - val_loss: 1756511.1158 - val_mean_squared_error: 1756511.1158
Epoch 158/250
 - 0s - loss: 1824890.7548 - mean_squared_error: 1824890.7548 - val_loss: 1754338.1330 - val_mean_squared_error: 1754338.1330
Epoch 159/250
 - 0s - loss: 1820081.3972 - mean_squared_error: 1820081.3972 - val_loss: 1751247.4256 - val_mean_squared_error: 1751247.4256
Epoch 160/250
 - 0s - loss: 1816636.5487 - mean_squared_error: 1816636.5487 - val_loss: 1756630.1609 - val_mean_squared_error: 1756630.1609
Epoch 161/250
 - 0s - loss: 1811579.5376 - mean_squared_error: 1811579.5376 - val_loss: 1743509.7337 - val_mean_squared_error: 1743509.7337
Epoch 162/250
 - 0s - loss: 1807536.9920 - mean_squared_error: 1807536.9920 - val_loss: 1742150.8448 - val_mean_squared_error: 1742150.8448
Epoch 163/250
 - 0s - loss: 1803971.9994 - mean_squared_error: 1803971.9994 - val_loss: 1736175.7619 - val_mean_squared_error: 1736175.7619
Epoch 164/250
 - 0s - loss: 1800349.7850 - mean_squared_error: 1800349.7850 - val_loss: 1743091.8461 - val_mean_squared_error: 1743091.8461
Epoch 165/250
 - 0s - loss: 1794217.3592 - mean_squared_error: 1794217.3592 - val_loss: 1727758.6505 - val_mean_squared_error: 1727758.6505
Epoch 166/250
 - 0s - loss: 1791519.7027 - mean_squared_error: 1791519.7027 - val_loss: 1749954.5221 - val_mean_squared_error: 1749954.5221
Epoch 167/250
 - 0s - loss: 1789118.2473 - mean_squared_error: 1789118.2473 - val_loss: 1721773.4305 - val_mean_squared_error: 1721773.4305
Epoch 168/250
 - 0s - loss: 1784228.6996 - mean_squared_error: 1784228.6996 - val_loss: 1717498.3095 - val_mean_squared_error: 1717498.3095
Epoch 169/250
 - 0s - loss: 1779127.8237 - mean_squared_error: 1779127.8237 - val_loss: 1713949.0157 - val_mean_squared_error: 1713949.0157
Epoch 170/250
 - 0s - loss: 1775922.7718 - mean_squared_error: 1775922.7718 - val_loss: 1711449.6809 - val_mean_squared_error: 1711449.6809
Epoch 171/250
 - 0s - loss: 1771560.6863 - mean_squared_error: 1771560.6863 - val_loss: 1709218.2926 - val_mean_squared_error: 1709218.2926
Epoch 172/250
 - 0s - loss: 1768529.1427 - mean_squared_error: 1768529.1427 - val_loss: 1706059.7845 - val_mean_squared_error: 1706059.7845
Epoch 173/250
 - 0s - loss: 1764611.6279 - mean_squared_error: 1764611.6279 - val_loss: 1704987.3673 - val_mean_squared_error: 1704987.3673
Epoch 174/250
 - 0s - loss: 1759797.3488 - mean_squared_error: 1759797.3488 - val_loss: 1696676.8063 - val_mean_squared_error: 1696676.8063
Epoch 175/250
 - 0s - loss: 1756353.1683 - mean_squared_error: 1756353.1683 - val_loss: 1693596.5106 - val_mean_squared_error: 1693596.5106
Epoch 176/250
 - 0s - loss: 1752005.8658 - mean_squared_error: 1752005.8658 - val_loss: 1689543.5374 - val_mean_squared_error: 1689543.5374
Epoch 177/250
 - 0s - loss: 1747951.0195 - mean_squared_error: 1747951.0195 - val_loss: 1686419.0251 - val_mean_squared_error: 1686419.0251
Epoch 178/250
 - 0s - loss: 1744804.0021 - mean_squared_error: 1744804.0021 - val_loss: 1682881.5861 - val_mean_squared_error: 1682881.5861
Epoch 179/250
 - 0s - loss: 1741434.6224 - mean_squared_error: 1741434.6224 - val_loss: 1691390.7072 - val_mean_squared_error: 1691390.7072
Epoch 180/250
 - 0s - loss: 1737770.1080 - mean_squared_error: 1737770.1080 - val_loss: 1675420.7468 - val_mean_squared_error: 1675420.7468
Epoch 181/250
 - 0s - loss: 1732393.8311 - mean_squared_error: 1732393.8311 - val_loss: 1675718.3392 - val_mean_squared_error: 1675718.3392
Epoch 182/250
 - 0s - loss: 1728406.2045 - mean_squared_error: 1728406.2045 - val_loss: 1668087.5327 - val_mean_squared_error: 1668087.5327
Epoch 183/250
 - 0s - loss: 1724420.8879 - mean_squared_error: 1724420.8879 - val_loss: 1664868.2181 - val_mean_squared_error: 1664868.2181
Epoch 184/250
 - 0s - loss: 1720156.1015 - mean_squared_error: 1720156.1015 - val_loss: 1669234.7821 - val_mean_squared_error: 1669234.7821
Epoch 185/250
 - 0s - loss: 1716196.7695 - mean_squared_error: 1716196.7695 - val_loss: 1669442.6978 - val_mean_squared_error: 1669442.6978
Epoch 186/250
 - 0s - loss: 1712488.4788 - mean_squared_error: 1712488.4788 - val_loss: 1653888.1396 - val_mean_squared_error: 1653888.1396
Epoch 187/250
 - 0s - loss: 1708032.9664 - mean_squared_error: 1708032.9664 - val_loss: 1650641.3101 - val_mean_squared_error: 1650641.3101
Epoch 188/250
 - 0s - loss: 1704076.8909 - mean_squared_error: 1704076.8909 - val_loss: 1647999.1042 - val_mean_squared_error: 1647999.1042
Epoch 189/250
 - 0s - loss: 1701110.1930 - mean_squared_error: 1701110.1930 - val_loss: 1646898.5643 - val_mean_squared_error: 1646898.5643
Epoch 190/250
 - 0s - loss: 1697091.7084 - mean_squared_error: 1697091.7084 - val_loss: 1642208.3946 - val_mean_squared_error: 1642208.3946
Epoch 191/250
 - 0s - loss: 1693360.1219 - mean_squared_error: 1693360.1219 - val_loss: 1636948.2817 - val_mean_squared_error: 1636948.2817
Epoch 192/250
 - 0s - loss: 1689616.1710 - mean_squared_error: 1689616.1710 - val_loss: 1634705.0452 - val_mean_squared_error: 1634705.0452
Epoch 193/250
 - 0s - loss: 1685067.2261 - mean_squared_error: 1685067.2261 - val_loss: 1630226.0061 - val_mean_squared_error: 1630226.0061
Epoch 194/250
 - 0s - loss: 1681237.7659 - mean_squared_error: 1681237.7659 - val_loss: 1639004.3018 - val_mean_squared_error: 1639004.3018
Epoch 195/250
 - 0s - loss: 1678298.5193 - mean_squared_error: 1678298.5193 - val_loss: 1623961.1689 - val_mean_squared_error: 1623961.1689
Epoch 196/250
 - 0s - loss: 1673192.0288 - mean_squared_error: 1673192.0288 - val_loss: 1619567.3824 - val_mean_squared_error: 1619567.3824
Epoch 197/250
 - 0s - loss: 1669092.1021 - mean_squared_error: 1669092.1021 - val_loss: 1619046.1713 - val_mean_squared_error: 1619046.1713
Epoch 198/250
 - 0s - loss: 1664606.5096 - mean_squared_error: 1664606.5096 - val_loss: 1615387.4159 - val_mean_squared_error: 1615387.4159
Epoch 199/250
 - 0s - loss: 1662564.2851 - mean_squared_error: 1662564.2851 - val_loss: 1609142.0948 - val_mean_squared_error: 1609142.0948
Epoch 200/250
 - 0s - loss: 1657685.3866 - mean_squared_error: 1657685.3866 - val_loss: 1605894.4098 - val_mean_squared_error: 1605894.4098
Epoch 201/250
 - 0s - loss: 1654749.7125 - mean_squared_error: 1654749.7125 - val_loss: 1606032.8303 - val_mean_squared_error: 1606032.8303
Epoch 202/250
 - 0s - loss: 1649906.3164 - mean_squared_error: 1649906.3164 - val_loss: 1598597.9039 - val_mean_squared_error: 1598597.9039
Epoch 203/250
 - 0s - loss: 1646614.9474 - mean_squared_error: 1646614.9474 - val_loss: 1600799.1936 - val_mean_squared_error: 1600799.1936
Epoch 204/250
 - 0s - loss: 1643058.2099 - mean_squared_error: 1643058.2099 - val_loss: 1596763.3237 - val_mean_squared_error: 1596763.3237
Epoch 205/250
 - 0s - loss: 1638300.3648 - mean_squared_error: 1638300.3648 - val_loss: 1593418.9744 - val_mean_squared_error: 1593418.9744
Epoch 206/250
 - 0s - loss: 1634605.7919 - mean_squared_error: 1634605.7919 - val_loss: 1591944.4761 - val_mean_squared_error: 1591944.4761
Epoch 207/250
 - 0s - loss: 1632396.7522 - mean_squared_error: 1632396.7522 - val_loss: 1594654.7668 - val_mean_squared_error: 1594654.7668
Epoch 208/250
 - 0s - loss: 1626509.9061 - mean_squared_error: 1626509.9061 - val_loss: 1581620.4115 - val_mean_squared_error: 1581620.4115
Epoch 209/250
 - 0s - loss: 1623937.1295 - mean_squared_error: 1623937.1295 - val_loss: 1579898.4873 - val_mean_squared_error: 1579898.4873
Epoch 210/250
 - 0s - loss: 1621066.5514 - mean_squared_error: 1621066.5514 - val_loss: 1571717.8550 - val_mean_squared_error: 1571717.8550
Epoch 211/250
 - 0s - loss: 1616139.5395 - mean_squared_error: 1616139.5395 - val_loss: 1569710.2550 - val_mean_squared_error: 1569710.2550
Epoch 212/250
 - 0s - loss: 1612243.5690 - mean_squared_error: 1612243.5690 - val_loss: 1567843.0187 - val_mean_squared_error: 1567843.0187
Epoch 213/250
 - 0s - loss: 1609260.8194 - mean_squared_error: 1609260.8194 - val_loss: 1572308.2530 - val_mean_squared_error: 1572308.2530
Epoch 214/250
 - 0s - loss: 1605613.5753 - mean_squared_error: 1605613.5753 - val_loss: 1557996.3463 - val_mean_squared_error: 1557996.3463
Epoch 215/250
 - 0s - loss: 1600952.7402 - mean_squared_error: 1600952.7402 - val_loss: 1555764.3567 - val_mean_squared_error: 1555764.3567
Epoch 216/250
 - 0s - loss: 1597516.2842 - mean_squared_error: 1597516.2842 - val_loss: 1552335.2397 - val_mean_squared_error: 1552335.2397
Epoch 217/250
 - 0s - loss: 1595406.1265 - mean_squared_error: 1595406.1265 - val_loss: 1548897.1456 - val_mean_squared_error: 1548897.1456
Epoch 218/250
 - 0s - loss: 1591035.0155 - mean_squared_error: 1591035.0155 - val_loss: 1546232.4290 - val_mean_squared_error: 1546232.4290
Epoch 219/250
 - 0s - loss: 1587376.0179 - mean_squared_error: 1587376.0179 - val_loss: 1541615.9383 - val_mean_squared_error: 1541615.9383
Epoch 220/250
 - 0s - loss: 1583196.2946 - mean_squared_error: 1583196.2946 - val_loss: 1539573.1838 - val_mean_squared_error: 1539573.1838
Epoch 221/250
 - 0s - loss: 1580048.1778 - mean_squared_error: 1580048.1778 - val_loss: 1539254.7977 - val_mean_squared_error: 1539254.7977
Epoch 222/250
 - 0s - loss: 1576428.4779 - mean_squared_error: 1576428.4779 - val_loss: 1537425.3814 - val_mean_squared_error: 1537425.3814
Epoch 223/250
 - 0s - loss: 1571698.2321 - mean_squared_error: 1571698.2321 - val_loss: 1533045.4937 - val_mean_squared_error: 1533045.4937
Epoch 224/250
 - 1s - loss: 1569309.8116 - mean_squared_error: 1569309.8116 - val_loss: 1524448.8505 - val_mean_squared_error: 1524448.8505
Epoch 225/250
 - 0s - loss: 1565261.5082 - mean_squared_error: 1565261.5082 - val_loss: 1521244.5449 - val_mean_squared_error: 1521244.5449
Epoch 226/250
 - 0s - loss: 1561696.2988 - mean_squared_error: 1561696.2988 - val_loss: 1521136.5315 - val_mean_squared_error: 1521136.5315
Epoch 227/250
 - 0s - loss: 1557760.5976 - mean_squared_error: 1557760.5976 - val_loss: 1514913.8519 - val_mean_squared_error: 1514913.8519
Epoch 228/250
 - 0s - loss: 1554509.0372 - mean_squared_error: 1554509.0372 - val_loss: 1511611.5119 - val_mean_squared_error: 1511611.5119
Epoch 229/250
 - 0s - loss: 1550976.7215 - mean_squared_error: 1550976.7215 - val_loss: 1509094.8388 - val_mean_squared_error: 1509094.8388
Epoch 230/250
 - 0s - loss: 1546977.6634 - mean_squared_error: 1546977.6634 - val_loss: 1505029.9383 - val_mean_squared_error: 1505029.9383
Epoch 231/250
 - 0s - loss: 1543970.9746 - mean_squared_error: 1543970.9746 - val_loss: 1502910.6582 - val_mean_squared_error: 1502910.6582
Epoch 232/250
 - 0s - loss: 1540305.0970 - mean_squared_error: 1540305.0970 - val_loss: 1498495.6655 - val_mean_squared_error: 1498495.6655
Epoch 233/250
 - 0s - loss: 1537501.6374 - mean_squared_error: 1537501.6374 - val_loss: 1495304.1388 - val_mean_squared_error: 1495304.1388
Epoch 234/250
 - 0s - loss: 1533388.2508 - mean_squared_error: 1533388.2508 - val_loss: 1492806.1005 - val_mean_squared_error: 1492806.1005
Epoch 235/250
 - 0s - loss: 1530941.4702 - mean_squared_error: 1530941.4702 - val_loss: 1499393.1403 - val_mean_squared_error: 1499393.1403
Epoch 236/250
 - 0s - loss: 1526883.5424 - mean_squared_error: 1526883.5424 - val_loss: 1490247.0621 - val_mean_squared_error: 1490247.0621
Epoch 237/250
 - 0s - loss: 1524231.6881 - mean_squared_error: 1524231.6881 - val_loss: 1493653.0241 - val_mean_squared_error: 1493653.0241
Epoch 238/250
 - 0s - loss: 1519628.2648 - mean_squared_error: 1519628.2648 - val_loss: 1482725.0214 - val_mean_squared_error: 1482725.0214
Epoch 239/250
 - 0s - loss: 1517536.3750 - mean_squared_error: 1517536.3750 - val_loss: 1476558.6559 - val_mean_squared_error: 1476558.6559
Epoch 240/250
 - 0s - loss: 1514165.7144 - mean_squared_error: 1514165.7144 - val_loss: 1475985.6456 - val_mean_squared_error: 1475985.6456
Epoch 241/250
 - 0s - loss: 1510455.2794 - mean_squared_error: 1510455.2794 - val_loss: 1470677.5349 - val_mean_squared_error: 1470677.5349
Epoch 242/250
 - 0s - loss: 1508992.9663 - mean_squared_error: 1508992.9663 - val_loss: 1474781.7924 - val_mean_squared_error: 1474781.7924
Epoch 243/250
 - 0s - loss: 1503767.5517 - mean_squared_error: 1503767.5517 - val_loss: 1465611.5074 - val_mean_squared_error: 1465611.5074
Epoch 244/250
 - 0s - loss: 1501195.4531 - mean_squared_error: 1501195.4531 - val_loss: 1464832.4664 - val_mean_squared_error: 1464832.4664
Epoch 245/250
 - 0s - loss: 1497824.0992 - mean_squared_error: 1497824.0992 - val_loss: 1462425.5150 - val_mean_squared_error: 1462425.5150
Epoch 246/250
 - 0s - loss: 1495264.5258 - mean_squared_error: 1495264.5258 - val_loss: 1459882.0771 - val_mean_squared_error: 1459882.0771
Epoch 247/250
 - 0s - loss: 1493924.6338 - mean_squared_error: 1493924.6338 - val_loss: 1454529.6073 - val_mean_squared_error: 1454529.6073
Epoch 248/250
 - 0s - loss: 1489410.9016 - mean_squared_error: 1489410.9016 - val_loss: 1451906.8467 - val_mean_squared_error: 1451906.8467
Epoch 249/250
 - 0s - loss: 1486580.2059 - mean_squared_error: 1486580.2059 - val_loss: 1449524.4469 - val_mean_squared_error: 1449524.4469
Epoch 250/250
 - 0s - loss: 1482848.0573 - mean_squared_error: 1482848.0573 - val_loss: 1445007.3935 - val_mean_squared_error: 1445007.3935

   32/13485 [..............................] - ETA: 0s
 3968/13485 [=======>......................] - ETA: 0s
 6944/13485 [==============>...............] - ETA: 0s
10880/13485 [=======================>......] - ETA: 0s
13485/13485 [==============================] - 0s 14us/step

root mean_squared_error: 1207.313111

After all this hard work we are closer to the MSE we got from linear regression, but purely using a shallow feed-forward neural network.

Training: Gradient Descent

A family of numeric optimization techniques, where we solve a problem with the following pattern:

Describe the error in the model output: this is usually some difference between the the true values and the model's predicted values, as a function of the model parameters (weights)
Compute the gradient, or directional derivative, of the error -- the "slope toward lower error"
Adjust the parameters of the model variables in the indicated direction
Repeat

Some ideas to help build your intuition

What happens if the variables (imagine just 2, to keep the mental picture simple) are on wildly different scales ... like one ranges from -1 to 1 while another from -1e6 to +1e6?
What if some of the variables are correlated? I.e., a change in one corresponds to, say, a linear change in another?
Other things being equal, an approximate solution with fewer variables is easier to work with than one with more -- how could we get rid of some less valuable parameters? (e.g., L1 penalty)
How do we know how far to "adjust" our parameters with each step?

What if we have billions of data points? Does it makes sense to use all of them for each update? Is there a shortcut?

Yes: Stochastic Gradient Descent

Stochastic gradient descent is an iterative learning algorithm that uses a training dataset to update a model. - The batch size is a hyperparameter of gradient descent that controls the number of training samples to work through before the model's internal parameters are updated. - The number of epochs is a hyperparameter of gradient descent that controls the number of complete passes through the training dataset.

See https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9.

But SGD has some shortcomings, so we typically use a "smarter" version of SGD, which has rules for adjusting the learning rate and even direction in order to avoid common problems.

What about that "Adam" optimizer? Adam is short for "adaptive moment" and is a variant of SGD that includes momentum calculations that change over time. For more detail on optimizers, see the chapter "Training Deep Neural Nets" in Aurélien Géron's book: Hands-On Machine Learning with Scikit-Learn and TensorFlow (http://shop.oreilly.com/product/0636920052289.do)

See https://keras.io/optimizers/ and references therein.

Training: Backpropagation

With a simple, flat model, we could use SGD or a related algorithm to derive the weights, since the error depends directly on those weights.

With a deeper network, we have a couple of challenges:

The error is computed from the final layer, so the gradient of the error doesn't tell us immediately about problems in other-layer weights
Our tiny diamonds model has almost a thousand weights. Bigger models can easily have millions of weights. Each of those weights may need to move a little at a time, and we have to watch out for underflow or undersignificance situations.

The insight is to iteratively calculate errors, one layer at a time, starting at the output. This is called backpropagation. It is neither magical nor surprising. The challenge is just doing it fast and not losing information.

Ok so we've come up with a very slow way to perform a linear regression.

Welcome to Neural Networks in the 1960s!

Watch closely now because this is where the magic happens...

Non-Linearity + Perceptron = Universal Approximation

Where does the non-linearity fit in?

We start with the inputs to a perceptron -- these could be from source data, for example.
We multiply each input by its respective weight, which gets us the $x \cdot w$
Then add the "bias" -- an extra learnable parameter, to get ${x \cdot w} + b$
- This value (so far) is sometimes called the "pre-activation"
Now, apply a non-linear "activation function" to this value, such as the logistic sigmoid

Now the network can "learn" non-linear functions

To gain some intuition, consider that where the sigmoid is close to 1, we can think of that neuron as being "on" or activated, giving a specific output. When close to zero, it is "off."

So each neuron is a bit like a switch. If we have enough of them, we can theoretically express arbitrarily many different signals.

In some ways this is like the original artificial neuron, with the thresholding output -- the main difference is that the sigmoid gives us a smooth (arbitrarily differentiable) output that we can optimize over using gradient descent to learn the weights.

Where does the signal "go" from these neurons?

In a regression problem, like the diamonds dataset, the activations from the hidden layer can feed into a single output neuron, with a simple linear activation representing the final output of the calculation.
Frequently we want a classification output instead -- e.g., with MNIST digits, where we need to choose from 10 classes. In that case, we can feed the outputs from these hidden neurons forward into a final layer of 10 neurons, and compare those final neurons' activation levels.

Ok, before we talk any more theory, let's run it and see if we can do better on our diamonds dataset adding this "sigmoid activation."

While that's running, let's look at the code:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].values.flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).values
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='sigmoid')) # <- change to nonlinear activation
model.add(Dense(1, kernel_initializer='normal', activation='linear')) # <- activation is linear in output layer for this regression

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=2000, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples
Epoch 1/2000
 - 1s - loss: 31397235.6558 - mean_squared_error: 31397235.6558 - val_loss: 32383574.7958 - val_mean_squared_error: 32383574.7958
Epoch 2/2000
 - 0s - loss: 31327383.7912 - mean_squared_error: 31327383.7912 - val_loss: 32308639.4652 - val_mean_squared_error: 32308639.4652
Epoch 3/2000
 - 0s - loss: 31252282.8223 - mean_squared_error: 31252282.8223 - val_loss: 32231016.3836 - val_mean_squared_error: 32231016.3836
Epoch 4/2000
 - 0s - loss: 31170289.7380 - mean_squared_error: 31170289.7380 - val_loss: 32144145.4375 - val_mean_squared_error: 32144145.4375
Epoch 5/2000
 - 0s - loss: 31083260.5032 - mean_squared_error: 31083260.5032 - val_loss: 32057233.8833 - val_mean_squared_error: 32057233.8833
Epoch 6/2000
 - 0s - loss: 31001470.8423 - mean_squared_error: 31001470.8423 - val_loss: 31976492.0554 - val_mean_squared_error: 31976492.0554
Epoch 7/2000
 - 0s - loss: 30922894.7525 - mean_squared_error: 30922894.7525 - val_loss: 31897767.1992 - val_mean_squared_error: 31897767.1992
Epoch 8/2000
 - 0s - loss: 30845616.8024 - mean_squared_error: 30845616.8024 - val_loss: 31820051.4988 - val_mean_squared_error: 31820051.4988
Epoch 9/2000
 - 0s - loss: 30769199.3371 - mean_squared_error: 30769199.3371 - val_loss: 31742996.5240 - val_mean_squared_error: 31742996.5240
Epoch 10/2000
 - 0s - loss: 30693446.8908 - mean_squared_error: 30693446.8908 - val_loss: 31666641.7430 - val_mean_squared_error: 31666641.7430
Epoch 11/2000
 - 0s - loss: 30618136.4908 - mean_squared_error: 30618136.4908 - val_loss: 31590480.7336 - val_mean_squared_error: 31590480.7336
Epoch 12/2000
 - 0s - loss: 30543210.7944 - mean_squared_error: 30543210.7944 - val_loss: 31514667.7736 - val_mean_squared_error: 31514667.7736
Epoch 13/2000
 - 0s - loss: 30462755.6179 - mean_squared_error: 30462755.6179 - val_loss: 31430098.2818 - val_mean_squared_error: 31430098.2818
Epoch 14/2000
 - 0s - loss: 30382840.9173 - mean_squared_error: 30382840.9173 - val_loss: 31350592.4350 - val_mean_squared_error: 31350592.4350
Epoch 15/2000
 - 0s - loss: 30304952.8825 - mean_squared_error: 30304952.8825 - val_loss: 31272120.3500 - val_mean_squared_error: 31272120.3500
Epoch 16/2000
 - 0s - loss: 30227645.3064 - mean_squared_error: 30227645.3064 - val_loss: 31194103.1488 - val_mean_squared_error: 31194103.1488
Epoch 17/2000
 - 0s - loss: 30150993.0111 - mean_squared_error: 30150993.0111 - val_loss: 31116695.8389 - val_mean_squared_error: 31116695.8389
Epoch 18/2000
 - 0s - loss: 30069837.8761 - mean_squared_error: 30069837.8761 - val_loss: 31026249.2397 - val_mean_squared_error: 31026249.2397
Epoch 19/2000
 - 0s - loss: 29980447.6209 - mean_squared_error: 29980447.6209 - val_loss: 30939547.4424 - val_mean_squared_error: 30939547.4424
Epoch 20/2000
 - 0s - loss: 29896402.2454 - mean_squared_error: 29896402.2454 - val_loss: 30855597.2289 - val_mean_squared_error: 30855597.2289
Epoch 21/2000
 - 0s - loss: 29814127.5312 - mean_squared_error: 29814127.5312 - val_loss: 30772653.2219 - val_mean_squared_error: 30772653.2219
Epoch 22/2000
 - 0s - loss: 29732678.1205 - mean_squared_error: 29732678.1205 - val_loss: 30690551.4029 - val_mean_squared_error: 30690551.4029
Epoch 23/2000
 - 0s - loss: 29651890.3895 - mean_squared_error: 29651890.3895 - val_loss: 30608966.2224 - val_mean_squared_error: 30608966.2224
Epoch 24/2000
 - 0s - loss: 29571321.6483 - mean_squared_error: 29571321.6483 - val_loss: 30527517.6352 - val_mean_squared_error: 30527517.6352
Epoch 25/2000
 - 0s - loss: 29491328.6854 - mean_squared_error: 29491328.6854 - val_loss: 30446735.6540 - val_mean_squared_error: 30446735.6540
Epoch 26/2000
 - 0s - loss: 29411719.3353 - mean_squared_error: 29411719.3353 - val_loss: 30366152.0227 - val_mean_squared_error: 30366152.0227
Epoch 27/2000
 - 0s - loss: 29332314.8944 - mean_squared_error: 29332314.8944 - val_loss: 30285836.7108 - val_mean_squared_error: 30285836.7108
Epoch 28/2000
 - 0s - loss: 29253317.4112 - mean_squared_error: 29253317.4112 - val_loss: 30205941.9634 - val_mean_squared_error: 30205941.9634
Epoch 29/2000
 - 0s - loss: 29174452.6544 - mean_squared_error: 29174452.6544 - val_loss: 30126117.8626 - val_mean_squared_error: 30126117.8626
Epoch 30/2000
 - 0s - loss: 29095966.3842 - mean_squared_error: 29095966.3842 - val_loss: 30046719.3356 - val_mean_squared_error: 30046719.3356
Epoch 31/2000
 - 0s - loss: 29017649.8682 - mean_squared_error: 29017649.8682 - val_loss: 29967484.1167 - val_mean_squared_error: 29967484.1167
Epoch 32/2000
 - 0s - loss: 28939521.5281 - mean_squared_error: 28939521.5281 - val_loss: 29888416.4854 - val_mean_squared_error: 29888416.4854
Epoch 33/2000
 - 0s - loss: 28861697.5828 - mean_squared_error: 28861697.5828 - val_loss: 29809702.4281 - val_mean_squared_error: 29809702.4281
Epoch 34/2000
 - 0s - loss: 28784209.9575 - mean_squared_error: 28784209.9575 - val_loss: 29731281.6065 - val_mean_squared_error: 29731281.6065
Epoch 35/2000
 - 0s - loss: 28706831.1842 - mean_squared_error: 28706831.1842 - val_loss: 29652956.9580 - val_mean_squared_error: 29652956.9580
Epoch 36/2000
 - 0s - loss: 28629657.6606 - mean_squared_error: 28629657.6606 - val_loss: 29574874.1651 - val_mean_squared_error: 29574874.1651
Epoch 37/2000
 - 0s - loss: 28552852.6089 - mean_squared_error: 28552852.6089 - val_loss: 29497087.4385 - val_mean_squared_error: 29497087.4385
Epoch 38/2000
 - 0s - loss: 28476126.3093 - mean_squared_error: 28476126.3093 - val_loss: 29419495.4592 - val_mean_squared_error: 29419495.4592
Epoch 39/2000
 - 0s - loss: 28399812.4279 - mean_squared_error: 28399812.4279 - val_loss: 29342199.2081 - val_mean_squared_error: 29342199.2081
Epoch 40/2000
 - 0s - loss: 28323720.1345 - mean_squared_error: 28323720.1345 - val_loss: 29265191.3465 - val_mean_squared_error: 29265191.3465
Epoch 41/2000
 - 0s - loss: 28247835.7470 - mean_squared_error: 28247835.7470 - val_loss: 29188324.3025 - val_mean_squared_error: 29188324.3025
Epoch 42/2000
 - 0s - loss: 28172102.9738 - mean_squared_error: 28172102.9738 - val_loss: 29111682.9619 - val_mean_squared_error: 29111682.9619
Epoch 43/2000
 - 0s - loss: 28096726.5860 - mean_squared_error: 28096726.5860 - val_loss: 29035421.8458 - val_mean_squared_error: 29035421.8458
Epoch 44/2000
 - 0s - loss: 28021588.4910 - mean_squared_error: 28021588.4910 - val_loss: 28959328.9956 - val_mean_squared_error: 28959328.9956
Epoch 45/2000
 - 0s - loss: 27946707.1624 - mean_squared_error: 27946707.1624 - val_loss: 28883512.4706 - val_mean_squared_error: 28883512.4706
Epoch 46/2000
 - 0s - loss: 27872025.8892 - mean_squared_error: 27872025.8892 - val_loss: 28807863.5976 - val_mean_squared_error: 28807863.5976
Epoch 47/2000
 - 0s - loss: 27797425.4399 - mean_squared_error: 27797425.4399 - val_loss: 28732382.9471 - val_mean_squared_error: 28732382.9471
Epoch 48/2000
 - 0s - loss: 27723191.4074 - mean_squared_error: 27723191.4074 - val_loss: 28657206.3253 - val_mean_squared_error: 28657206.3253
Epoch 49/2000
 - 0s - loss: 27649192.8975 - mean_squared_error: 27649192.8975 - val_loss: 28582285.1824 - val_mean_squared_error: 28582285.1824
Epoch 50/2000
 - 0s - loss: 27575479.0100 - mean_squared_error: 27575479.0100 - val_loss: 28507610.2086 - val_mean_squared_error: 28507610.2086
Epoch 51/2000
 - 0s - loss: 27501288.6076 - mean_squared_error: 27501288.6076 - val_loss: 28429080.9985 - val_mean_squared_error: 28429080.9985
Epoch 52/2000
 - 0s - loss: 27421728.8021 - mean_squared_error: 27421728.8021 - val_loss: 28349501.5383 - val_mean_squared_error: 28349501.5383
Epoch 53/2000
 - 0s - loss: 27344500.0112 - mean_squared_error: 27344500.0112 - val_loss: 28271872.5428 - val_mean_squared_error: 28271872.5428
Epoch 54/2000
 - 0s - loss: 27268463.9309 - mean_squared_error: 27268463.9309 - val_loss: 28195175.1132 - val_mean_squared_error: 28195175.1132
Epoch 55/2000
 - 0s - loss: 27193075.2804 - mean_squared_error: 27193075.2804 - val_loss: 28118943.3356 - val_mean_squared_error: 28118943.3356
Epoch 56/2000
 - 0s - loss: 27118098.0572 - mean_squared_error: 27118098.0572 - val_loss: 28043017.4800 - val_mean_squared_error: 28043017.4800
Epoch 57/2000
 - 0s - loss: 27043562.7233 - mean_squared_error: 27043562.7233 - val_loss: 27967574.5348 - val_mean_squared_error: 27967574.5348
Epoch 58/2000
 - 0s - loss: 26969328.4123 - mean_squared_error: 26969328.4123 - val_loss: 27892381.2684 - val_mean_squared_error: 27892381.2684
Epoch 59/2000
 - 0s - loss: 26895302.7169 - mean_squared_error: 26895302.7169 - val_loss: 27817410.1503 - val_mean_squared_error: 27817410.1503
Epoch 60/2000
 - 0s - loss: 26821600.7915 - mean_squared_error: 26821600.7915 - val_loss: 27742831.9179 - val_mean_squared_error: 27742831.9179
Epoch 61/2000
 - 0s - loss: 26748231.5481 - mean_squared_error: 26748231.5481 - val_loss: 27668465.3149 - val_mean_squared_error: 27668465.3149
Epoch 62/2000
 - 0s - loss: 26674943.8663 - mean_squared_error: 26674943.8663 - val_loss: 27594222.4913 - val_mean_squared_error: 27594222.4913
Epoch 63/2000
 - 0s - loss: 26601994.2711 - mean_squared_error: 26601994.2711 - val_loss: 27520427.2951 - val_mean_squared_error: 27520427.2951
Epoch 64/2000
 - 0s - loss: 26529282.3881 - mean_squared_error: 26529282.3881 - val_loss: 27446688.2739 - val_mean_squared_error: 27446688.2739
Epoch 65/2000
 - 0s - loss: 26457019.9369 - mean_squared_error: 26457019.9369 - val_loss: 27373482.1631 - val_mean_squared_error: 27373482.1631
Epoch 66/2000
 - 0s - loss: 26384992.6683 - mean_squared_error: 26384992.6683 - val_loss: 27300497.3604 - val_mean_squared_error: 27300497.3604
Epoch 67/2000
 - 0s - loss: 26313104.0731 - mean_squared_error: 26313104.0731 - val_loss: 27227665.3999 - val_mean_squared_error: 27227665.3999
Epoch 68/2000
 - 0s - loss: 26241511.5586 - mean_squared_error: 26241511.5586 - val_loss: 27155084.1582 - val_mean_squared_error: 27155084.1582
Epoch 69/2000
 - 0s - loss: 26170093.5288 - mean_squared_error: 26170093.5288 - val_loss: 27082711.1349 - val_mean_squared_error: 27082711.1349
Epoch 70/2000
 - 0s - loss: 26099043.1046 - mean_squared_error: 26099043.1046 - val_loss: 27010760.2066 - val_mean_squared_error: 27010760.2066
Epoch 71/2000
 - 0s - loss: 26028235.2419 - mean_squared_error: 26028235.2419 - val_loss: 26938868.5180 - val_mean_squared_error: 26938868.5180
Epoch 72/2000
 - 0s - loss: 25957681.7494 - mean_squared_error: 25957681.7494 - val_loss: 26867434.4795 - val_mean_squared_error: 26867434.4795
Epoch 73/2000
 - 0s - loss: 25887375.9710 - mean_squared_error: 25887375.9710 - val_loss: 26796228.9352 - val_mean_squared_error: 26796228.9352
Epoch 74/2000
 - 0s - loss: 25817222.8240 - mean_squared_error: 25817222.8240 - val_loss: 26725061.6935 - val_mean_squared_error: 26725061.6935
Epoch 75/2000
 - 0s - loss: 25747292.7817 - mean_squared_error: 25747292.7817 - val_loss: 26654194.4212 - val_mean_squared_error: 26654194.4212
Epoch 76/2000
 - 0s - loss: 25677706.2037 - mean_squared_error: 25677706.2037 - val_loss: 26583594.1147 - val_mean_squared_error: 26583594.1147
Epoch 77/2000
 - 0s - loss: 25608337.1430 - mean_squared_error: 25608337.1430 - val_loss: 26513296.5477 - val_mean_squared_error: 26513296.5477
Epoch 78/2000
 - 0s - loss: 25539156.0119 - mean_squared_error: 25539156.0119 - val_loss: 26443137.3040 - val_mean_squared_error: 26443137.3040
Epoch 79/2000
 - 0s - loss: 25470427.8616 - mean_squared_error: 25470427.8616 - val_loss: 26373518.7711 - val_mean_squared_error: 26373518.7711
Epoch 80/2000
 - 0s - loss: 25401866.0610 - mean_squared_error: 25401866.0610 - val_loss: 26303972.2363 - val_mean_squared_error: 26303972.2363
Epoch 81/2000
 - 0s - loss: 25333487.7670 - mean_squared_error: 25333487.7670 - val_loss: 26234631.2130 - val_mean_squared_error: 26234631.2130
Epoch 82/2000
 - 0s - loss: 25265550.7323 - mean_squared_error: 25265550.7323 - val_loss: 26165749.3228 - val_mean_squared_error: 26165749.3228
Epoch 83/2000
 - 0s - loss: 25197691.5988 - mean_squared_error: 25197691.5988 - val_loss: 26097002.0771 - val_mean_squared_error: 26097002.0771
Epoch 84/2000
 - 0s - loss: 25130257.1538 - mean_squared_error: 25130257.1538 - val_loss: 26028667.4909 - val_mean_squared_error: 26028667.4909
Epoch 85/2000
 - 0s - loss: 25063019.1388 - mean_squared_error: 25063019.1388 - val_loss: 25960258.2946 - val_mean_squared_error: 25960258.2946
Epoch 86/2000
 - 0s - loss: 24995931.5351 - mean_squared_error: 24995931.5351 - val_loss: 25892334.4004 - val_mean_squared_error: 25892334.4004
Epoch 87/2000
 - 0s - loss: 24929045.9940 - mean_squared_error: 24929045.9940 - val_loss: 25824543.6985 - val_mean_squared_error: 25824543.6985
Epoch 88/2000
 - 0s - loss: 24862549.7085 - mean_squared_error: 24862549.7085 - val_loss: 25757038.2304 - val_mean_squared_error: 25757038.2304
Epoch 89/2000
 - 0s - loss: 24796340.2191 - mean_squared_error: 24796340.2191 - val_loss: 25689859.1270 - val_mean_squared_error: 25689859.1270
Epoch 90/2000
 - 0s - loss: 24730308.6975 - mean_squared_error: 24730308.6975 - val_loss: 25622882.4706 - val_mean_squared_error: 25622882.4706
Epoch 91/2000
 - 0s - loss: 24664616.4917 - mean_squared_error: 24664616.4917 - val_loss: 25556223.6144 - val_mean_squared_error: 25556223.6144
Epoch 92/2000
 - 0s - loss: 24599087.1876 - mean_squared_error: 24599087.1876 - val_loss: 25489791.0321 - val_mean_squared_error: 25489791.0321
Epoch 93/2000
 - 0s - loss: 24533830.7211 - mean_squared_error: 24533830.7211 - val_loss: 25423553.8339 - val_mean_squared_error: 25423553.8339
Epoch 94/2000
 - 0s - loss: 24468854.4123 - mean_squared_error: 24468854.4123 - val_loss: 25357663.3732 - val_mean_squared_error: 25357663.3732
Epoch 95/2000
 - 0s - loss: 24404026.7401 - mean_squared_error: 24404026.7401 - val_loss: 25291782.8176 - val_mean_squared_error: 25291782.8176
Epoch 96/2000
 - 0s - loss: 24339535.6913 - mean_squared_error: 24339535.6913 - val_loss: 25226405.7410 - val_mean_squared_error: 25226405.7410
Epoch 97/2000
 - 0s - loss: 24275298.2234 - mean_squared_error: 24275298.2234 - val_loss: 25161186.7612 - val_mean_squared_error: 25161186.7612
Epoch 98/2000
 - 0s - loss: 24211340.8869 - mean_squared_error: 24211340.8869 - val_loss: 25096314.0040 - val_mean_squared_error: 25096314.0040
Epoch 99/2000
 - 0s - loss: 24147694.9561 - mean_squared_error: 24147694.9561 - val_loss: 25031670.1819 - val_mean_squared_error: 25031670.1819
Epoch 100/2000
 - 0s - loss: 24084192.7054 - mean_squared_error: 24084192.7054 - val_loss: 24967208.1681 - val_mean_squared_error: 24967208.1681
Epoch 101/2000
 - 0s - loss: 24020997.9685 - mean_squared_error: 24020997.9685 - val_loss: 24903012.1028 - val_mean_squared_error: 24903012.1028
Epoch 102/2000
 - 0s - loss: 23958014.3527 - mean_squared_error: 23958014.3527 - val_loss: 24839094.8265 - val_mean_squared_error: 24839094.8265
Epoch 103/2000
 - 0s - loss: 23895302.7699 - mean_squared_error: 23895302.7699 - val_loss: 24775430.5388 - val_mean_squared_error: 24775430.5388
Epoch 104/2000
 - 0s - loss: 23832711.7829 - mean_squared_error: 23832711.7829 - val_loss: 24711884.1275 - val_mean_squared_error: 24711884.1275
Epoch 105/2000
 - 0s - loss: 23770447.2922 - mean_squared_error: 23770447.2922 - val_loss: 24648619.4217 - val_mean_squared_error: 24648619.4217
Epoch 106/2000
 - 0s - loss: 23708384.5371 - mean_squared_error: 23708384.5371 - val_loss: 24585604.6703 - val_mean_squared_error: 24585604.6703
Epoch 107/2000
 - 0s - loss: 23646681.5160 - mean_squared_error: 23646681.5160 - val_loss: 24523010.6960 - val_mean_squared_error: 24523010.6960
Epoch 108/2000
 - 0s - loss: 23585118.3183 - mean_squared_error: 23585118.3183 - val_loss: 24460488.8799 - val_mean_squared_error: 24460488.8799
Epoch 109/2000
 - 0s - loss: 23523883.4182 - mean_squared_error: 23523883.4182 - val_loss: 24398303.1092 - val_mean_squared_error: 24398303.1092
Epoch 110/2000
 - 0s - loss: 23462910.2811 - mean_squared_error: 23462910.2811 - val_loss: 24336406.8443 - val_mean_squared_error: 24336406.8443
Epoch 111/2000
 - 0s - loss: 23402204.0326 - mean_squared_error: 23402204.0326 - val_loss: 24274770.0376 - val_mean_squared_error: 24274770.0376
Epoch 112/2000
 - 0s - loss: 23341832.9642 - mean_squared_error: 23341832.9642 - val_loss: 24213379.6490 - val_mean_squared_error: 24213379.6490
Epoch 113/2000
 - 0s - loss: 23279795.8057 - mean_squared_error: 23279795.8057 - val_loss: 24148311.3791 - val_mean_squared_error: 24148311.3791
Epoch 114/2000
 - 0s - loss: 23217236.7834 - mean_squared_error: 23217236.7834 - val_loss: 24086002.9946 - val_mean_squared_error: 24086002.9946
Epoch 115/2000
 - 0s - loss: 23156625.9503 - mean_squared_error: 23156625.9503 - val_loss: 24024608.4825 - val_mean_squared_error: 24024608.4825
Epoch 116/2000
 - 0s - loss: 23096589.0903 - mean_squared_error: 23096589.0903 - val_loss: 23963636.2195 - val_mean_squared_error: 23963636.2195
Epoch 117/2000
 - 0s - loss: 23036876.2545 - mean_squared_error: 23036876.2545 - val_loss: 23902991.3623 - val_mean_squared_error: 23902991.3623
Epoch 118/2000
 - 0s - loss: 22977437.5939 - mean_squared_error: 22977437.5939 - val_loss: 23842603.3732 - val_mean_squared_error: 23842603.3732
Epoch 119/2000
 - 0s - loss: 22918273.3646 - mean_squared_error: 22918273.3646 - val_loss: 23782522.0158 - val_mean_squared_error: 23782522.0158
Epoch 120/2000
 - 0s - loss: 22859272.2713 - mean_squared_error: 22859272.2713 - val_loss: 23722539.8853 - val_mean_squared_error: 23722539.8853
Epoch 121/2000
 - 0s - loss: 22800737.8453 - mean_squared_error: 22800737.8453 - val_loss: 23663112.8057 - val_mean_squared_error: 23663112.8057
Epoch 122/2000
 - 0s - loss: 22742394.6360 - mean_squared_error: 22742394.6360 - val_loss: 23603825.1330 - val_mean_squared_error: 23603825.1330
Epoch 123/2000
 - 0s - loss: 22684245.9053 - mean_squared_error: 22684245.9053 - val_loss: 23544635.8824 - val_mean_squared_error: 23544635.8824
Epoch 124/2000
 - 0s - loss: 22626413.5138 - mean_squared_error: 22626413.5138 - val_loss: 23485876.2778 - val_mean_squared_error: 23485876.2778
Epoch 125/2000
 - 0s - loss: 22568720.6338 - mean_squared_error: 22568720.6338 - val_loss: 23427270.5783 - val_mean_squared_error: 23427270.5783
Epoch 126/2000
 - 0s - loss: 22511420.9494 - mean_squared_error: 22511420.9494 - val_loss: 23368986.0880 - val_mean_squared_error: 23368986.0880
Epoch 127/2000
 - 0s - loss: 22454394.1190 - mean_squared_error: 22454394.1190 - val_loss: 23311068.8235 - val_mean_squared_error: 23311068.8235
Epoch 128/2000
 - 0s - loss: 22397541.9148 - mean_squared_error: 22397541.9148 - val_loss: 23253145.5877 - val_mean_squared_error: 23253145.5877
Epoch 129/2000
 - 0s - loss: 22341051.0658 - mean_squared_error: 22341051.0658 - val_loss: 23195790.0475 - val_mean_squared_error: 23195790.0475
Epoch 130/2000
 - 0s - loss: 22284655.2844 - mean_squared_error: 22284655.2844 - val_loss: 23138376.5378 - val_mean_squared_error: 23138376.5378
Epoch 131/2000
 - 0s - loss: 22228695.7513 - mean_squared_error: 22228695.7513 - val_loss: 23081453.2160 - val_mean_squared_error: 23081453.2160
Epoch 132/2000
 - 0s - loss: 22172854.7761 - mean_squared_error: 22172854.7761 - val_loss: 23024590.7385 - val_mean_squared_error: 23024590.7385
Epoch 133/2000
 - 0s - loss: 22117285.0639 - mean_squared_error: 22117285.0639 - val_loss: 22968118.0573 - val_mean_squared_error: 22968118.0573
Epoch 134/2000
 - 0s - loss: 22061987.1151 - mean_squared_error: 22061987.1151 - val_loss: 22911898.1137 - val_mean_squared_error: 22911898.1137
Epoch 135/2000
 - 0s - loss: 22007029.5824 - mean_squared_error: 22007029.5824 - val_loss: 22855873.1587 - val_mean_squared_error: 22855873.1587
Epoch 136/2000
 - 0s - loss: 21952235.7935 - mean_squared_error: 21952235.7935 - val_loss: 22800335.4088 - val_mean_squared_error: 22800335.4088
Epoch 137/2000
 - 0s - loss: 21897579.2996 - mean_squared_error: 21897579.2996 - val_loss: 22744598.8492 - val_mean_squared_error: 22744598.8492
Epoch 138/2000
 - 0s - loss: 21843216.3979 - mean_squared_error: 21843216.3979 - val_loss: 22689297.0005 - val_mean_squared_error: 22689297.0005
Epoch 139/2000
 - 0s - loss: 21789164.4874 - mean_squared_error: 21789164.4874 - val_loss: 22634314.9501 - val_mean_squared_error: 22634314.9501
Epoch 140/2000
 - 1s - loss: 21735301.7342 - mean_squared_error: 21735301.7342 - val_loss: 22579402.9283 - val_mean_squared_error: 22579402.9283
Epoch 141/2000
 - 1s - loss: 21681605.8874 - mean_squared_error: 21681605.8874 - val_loss: 22524827.5185 - val_mean_squared_error: 22524827.5185
Epoch 142/2000
 - 0s - loss: 21628268.6495 - mean_squared_error: 21628268.6495 - val_loss: 22470437.9377 - val_mean_squared_error: 22470437.9377
Epoch 143/2000
 - 0s - loss: 21575064.8829 - mean_squared_error: 21575064.8829 - val_loss: 22416369.4118 - val_mean_squared_error: 22416369.4118
Epoch 144/2000
 - 0s - loss: 21522082.2842 - mean_squared_error: 21522082.2842 - val_loss: 22362375.5818 - val_mean_squared_error: 22362375.5818
Epoch 145/2000
 - 0s - loss: 21469363.4439 - mean_squared_error: 21469363.4439 - val_loss: 22308673.4740 - val_mean_squared_error: 22308673.4740
Epoch 146/2000
 - 0s - loss: 21416936.7842 - mean_squared_error: 21416936.7842 - val_loss: 22255289.7617 - val_mean_squared_error: 22255289.7617
Epoch 147/2000
 - 0s - loss: 21364871.8014 - mean_squared_error: 21364871.8014 - val_loss: 22202402.4785 - val_mean_squared_error: 22202402.4785
Epoch 148/2000
 - 0s - loss: 21313050.3111 - mean_squared_error: 21313050.3111 - val_loss: 22149539.6560 - val_mean_squared_error: 22149539.6560
Epoch 149/2000
 - 0s - loss: 21261446.3621 - mean_squared_error: 21261446.3621 - val_loss: 22096959.3178 - val_mean_squared_error: 22096959.3178
Epoch 150/2000
 - 0s - loss: 21210021.5085 - mean_squared_error: 21210021.5085 - val_loss: 22044621.9871 - val_mean_squared_error: 22044621.9871
Epoch 151/2000
 - 0s - loss: 21158849.3865 - mean_squared_error: 21158849.3865 - val_loss: 21992473.2417 - val_mean_squared_error: 21992473.2417
Epoch 152/2000
 - 0s - loss: 21107857.0819 - mean_squared_error: 21107857.0819 - val_loss: 21940490.6703 - val_mean_squared_error: 21940490.6703
Epoch 153/2000
 - 0s - loss: 21057117.0225 - mean_squared_error: 21057117.0225 - val_loss: 21888891.6738 - val_mean_squared_error: 21888891.6738
Epoch 154/2000
 - 0s - loss: 21006774.0317 - mean_squared_error: 21006774.0317 - val_loss: 21837635.9951 - val_mean_squared_error: 21837635.9951
Epoch 155/2000
 - 0s - loss: 20956700.9287 - mean_squared_error: 20956700.9287 - val_loss: 21786561.2378 - val_mean_squared_error: 21786561.2378
Epoch 156/2000
 - 0s - loss: 20906693.4021 - mean_squared_error: 20906693.4021 - val_loss: 21735692.4320 - val_mean_squared_error: 21735692.4320
Epoch 157/2000
 - 0s - loss: 20857150.1686 - mean_squared_error: 20857150.1686 - val_loss: 21685168.9184 - val_mean_squared_error: 21685168.9184
Epoch 158/2000
 - 0s - loss: 20807666.7034 - mean_squared_error: 20807666.7034 - val_loss: 21634740.0801 - val_mean_squared_error: 21634740.0801
Epoch 159/2000
 - 0s - loss: 20758500.1218 - mean_squared_error: 20758500.1218 - val_loss: 21584612.7355 - val_mean_squared_error: 21584612.7355
Epoch 160/2000
 - 0s - loss: 20709524.1605 - mean_squared_error: 20709524.1605 - val_loss: 21534721.3228 - val_mean_squared_error: 21534721.3228
Epoch 161/2000
 - 0s - loss: 20660910.6885 - mean_squared_error: 20660910.6885 - val_loss: 21485147.7420 - val_mean_squared_error: 21485147.7420
Epoch 162/2000
 - 1s - loss: 20612586.9465 - mean_squared_error: 20612586.9465 - val_loss: 21435754.7781 - val_mean_squared_error: 21435754.7781
Epoch 163/2000
 - 0s - loss: 20564474.1311 - mean_squared_error: 20564474.1311 - val_loss: 21386777.7944 - val_mean_squared_error: 21386777.7944
Epoch 164/2000
 - 0s - loss: 20516645.5917 - mean_squared_error: 20516645.5917 - val_loss: 21338018.3618 - val_mean_squared_error: 21338018.3618
Epoch 165/2000
 - 0s - loss: 20468905.3257 - mean_squared_error: 20468905.3257 - val_loss: 21289250.6525 - val_mean_squared_error: 21289250.6525
Epoch 166/2000
 - 0s - loss: 20421415.9548 - mean_squared_error: 20421415.9548 - val_loss: 21240928.0890 - val_mean_squared_error: 21240928.0890
Epoch 167/2000
 - 0s - loss: 20374153.5797 - mean_squared_error: 20374153.5797 - val_loss: 21192654.9896 - val_mean_squared_error: 21192654.9896
Epoch 168/2000
 - 0s - loss: 20327158.7635 - mean_squared_error: 20327158.7635 - val_loss: 21144689.6471 - val_mean_squared_error: 21144689.6471
Epoch 169/2000
 - 1s - loss: 20280455.0811 - mean_squared_error: 20280455.0811 - val_loss: 21097043.4177 - val_mean_squared_error: 21097043.4177
Epoch 170/2000
 - 0s - loss: 20233902.1935 - mean_squared_error: 20233902.1935 - val_loss: 21049538.9530 - val_mean_squared_error: 21049538.9530
Epoch 171/2000
 - 0s - loss: 20187576.5592 - mean_squared_error: 20187576.5592 - val_loss: 21002178.7395 - val_mean_squared_error: 21002178.7395
Epoch 172/2000
 - 0s - loss: 20141522.7279 - mean_squared_error: 20141522.7279 - val_loss: 20955227.2457 - val_mean_squared_error: 20955227.2457
Epoch 173/2000
 - 0s - loss: 20095770.7149 - mean_squared_error: 20095770.7149 - val_loss: 20908545.9822 - val_mean_squared_error: 20908545.9822

*** WARNING: skipped 241388 bytes of output ***

Epoch 1833/2000
 - 0s - loss: 14086458.4326 - mean_squared_error: 14086458.4326 - val_loss: 14688438.1349 - val_mean_squared_error: 14688438.1349
Epoch 1834/2000
 - 0s - loss: 14085434.0234 - mean_squared_error: 14085434.0234 - val_loss: 14687099.8651 - val_mean_squared_error: 14687099.8651
Epoch 1835/2000
 - 0s - loss: 14084509.4712 - mean_squared_error: 14084509.4712 - val_loss: 14685995.8003 - val_mean_squared_error: 14685995.8003
Epoch 1836/2000
 - 0s - loss: 14083522.1710 - mean_squared_error: 14083522.1710 - val_loss: 14685020.8038 - val_mean_squared_error: 14685020.8038
Epoch 1837/2000
 - 0s - loss: 14082602.0144 - mean_squared_error: 14082602.0144 - val_loss: 14684060.2743 - val_mean_squared_error: 14684060.2743
Epoch 1838/2000
 - 0s - loss: 14081612.7300 - mean_squared_error: 14081612.7300 - val_loss: 14683103.0138 - val_mean_squared_error: 14683103.0138
Epoch 1839/2000
 - 0s - loss: 14080601.5729 - mean_squared_error: 14080601.5729 - val_loss: 14682194.6535 - val_mean_squared_error: 14682194.6535
Epoch 1840/2000
 - 0s - loss: 14079617.4989 - mean_squared_error: 14079617.4989 - val_loss: 14681039.8512 - val_mean_squared_error: 14681039.8512
Epoch 1841/2000
 - 0s - loss: 14078604.7551 - mean_squared_error: 14078604.7551 - val_loss: 14680020.2279 - val_mean_squared_error: 14680020.2279
Epoch 1842/2000
 - 0s - loss: 14077606.6323 - mean_squared_error: 14077606.6323 - val_loss: 14679032.0391 - val_mean_squared_error: 14679032.0391
Epoch 1843/2000
 - 0s - loss: 14076695.6481 - mean_squared_error: 14076695.6481 - val_loss: 14678025.0559 - val_mean_squared_error: 14678025.0559
Epoch 1844/2000
 - 0s - loss: 14075708.6411 - mean_squared_error: 14075708.6411 - val_loss: 14677170.4572 - val_mean_squared_error: 14677170.4572
Epoch 1845/2000
 - 0s - loss: 14074733.2668 - mean_squared_error: 14074733.2668 - val_loss: 14676151.7696 - val_mean_squared_error: 14676151.7696
Epoch 1846/2000
 - 0s - loss: 14073722.6215 - mean_squared_error: 14073722.6215 - val_loss: 14675369.2526 - val_mean_squared_error: 14675369.2526
Epoch 1847/2000
 - 1s - loss: 14072775.8814 - mean_squared_error: 14072775.8814 - val_loss: 14674029.9896 - val_mean_squared_error: 14674029.9896
Epoch 1848/2000
 - 0s - loss: 14071836.2345 - mean_squared_error: 14071836.2345 - val_loss: 14673136.9338 - val_mean_squared_error: 14673136.9338
Epoch 1849/2000
 - 0s - loss: 14070810.8924 - mean_squared_error: 14070810.8924 - val_loss: 14672194.3228 - val_mean_squared_error: 14672194.3228
Epoch 1850/2000
 - 0s - loss: 14069863.6761 - mean_squared_error: 14069863.6761 - val_loss: 14671299.9367 - val_mean_squared_error: 14671299.9367
Epoch 1851/2000
 - 0s - loss: 14068889.2048 - mean_squared_error: 14068889.2048 - val_loss: 14670078.4859 - val_mean_squared_error: 14670078.4859
Epoch 1852/2000
 - 0s - loss: 14067938.4834 - mean_squared_error: 14067938.4834 - val_loss: 14669063.5971 - val_mean_squared_error: 14669063.5971
Epoch 1853/2000
 - 0s - loss: 14066922.4883 - mean_squared_error: 14066922.4883 - val_loss: 14668091.0554 - val_mean_squared_error: 14668091.0554
Epoch 1854/2000
 - 0s - loss: 14065955.6033 - mean_squared_error: 14065955.6033 - val_loss: 14667112.7553 - val_mean_squared_error: 14667112.7553
Epoch 1855/2000
 - 0s - loss: 14064991.1939 - mean_squared_error: 14064991.1939 - val_loss: 14666075.4078 - val_mean_squared_error: 14666075.4078
Epoch 1856/2000
 - 0s - loss: 14063978.3373 - mean_squared_error: 14063978.3373 - val_loss: 14665110.3134 - val_mean_squared_error: 14665110.3134
Epoch 1857/2000
 - 0s - loss: 14063012.4079 - mean_squared_error: 14063012.4079 - val_loss: 14664084.2921 - val_mean_squared_error: 14664084.2921
Epoch 1858/2000
 - 0s - loss: 14062046.5409 - mean_squared_error: 14062046.5409 - val_loss: 14663314.2165 - val_mean_squared_error: 14663314.2165
Epoch 1859/2000
 - 0s - loss: 14061095.0863 - mean_squared_error: 14061095.0863 - val_loss: 14662105.3094 - val_mean_squared_error: 14662105.3094
Epoch 1860/2000
 - 0s - loss: 14060109.7960 - mean_squared_error: 14060109.7960 - val_loss: 14661178.3495 - val_mean_squared_error: 14661178.3495
Epoch 1861/2000
 - 0s - loss: 14059102.8300 - mean_squared_error: 14059102.8300 - val_loss: 14660171.9862 - val_mean_squared_error: 14660171.9862
Epoch 1862/2000
 - 0s - loss: 14058160.4495 - mean_squared_error: 14058160.4495 - val_loss: 14659101.1468 - val_mean_squared_error: 14659101.1468
Epoch 1863/2000
 - 0s - loss: 14057130.4759 - mean_squared_error: 14057130.4759 - val_loss: 14658154.8270 - val_mean_squared_error: 14658154.8270
Epoch 1864/2000
 - 0s - loss: 14056215.9189 - mean_squared_error: 14056215.9189 - val_loss: 14657098.5195 - val_mean_squared_error: 14657098.5195
Epoch 1865/2000
 - 0s - loss: 14055249.4822 - mean_squared_error: 14055249.4822 - val_loss: 14656309.6268 - val_mean_squared_error: 14656309.6268
Epoch 1866/2000
 - 0s - loss: 14054291.7944 - mean_squared_error: 14054291.7944 - val_loss: 14655135.5764 - val_mean_squared_error: 14655135.5764
Epoch 1867/2000
 - 0s - loss: 14053316.9188 - mean_squared_error: 14053316.9188 - val_loss: 14654136.7983 - val_mean_squared_error: 14654136.7983
Epoch 1868/2000
 - 0s - loss: 14052269.8117 - mean_squared_error: 14052269.8117 - val_loss: 14653555.8601 - val_mean_squared_error: 14653555.8601
Epoch 1869/2000
 - 0s - loss: 14051339.1897 - mean_squared_error: 14051339.1897 - val_loss: 14652159.2521 - val_mean_squared_error: 14652159.2521
Epoch 1870/2000
 - 0s - loss: 14050346.3569 - mean_squared_error: 14050346.3569 - val_loss: 14651238.4543 - val_mean_squared_error: 14651238.4543
Epoch 1871/2000
 - 0s - loss: 14049374.9380 - mean_squared_error: 14049374.9380 - val_loss: 14650198.4182 - val_mean_squared_error: 14650198.4182
Epoch 1872/2000
 - 0s - loss: 14048339.0222 - mean_squared_error: 14048339.0222 - val_loss: 14649223.7830 - val_mean_squared_error: 14649223.7830
Epoch 1873/2000
 - 0s - loss: 14047460.8214 - mean_squared_error: 14047460.8214 - val_loss: 14648165.3683 - val_mean_squared_error: 14648165.3683
Epoch 1874/2000
 - 0s - loss: 14046460.4335 - mean_squared_error: 14046460.4335 - val_loss: 14647232.9313 - val_mean_squared_error: 14647232.9313
Epoch 1875/2000
 - 0s - loss: 14045494.6317 - mean_squared_error: 14045494.6317 - val_loss: 14646206.6401 - val_mean_squared_error: 14646206.6401
Epoch 1876/2000
 - 0s - loss: 14044518.6021 - mean_squared_error: 14044518.6021 - val_loss: 14645205.3974 - val_mean_squared_error: 14645205.3974
Epoch 1877/2000
 - 0s - loss: 14043542.4435 - mean_squared_error: 14043542.4435 - val_loss: 14644302.8868 - val_mean_squared_error: 14644302.8868
Epoch 1878/2000
 - 0s - loss: 14042623.7559 - mean_squared_error: 14042623.7559 - val_loss: 14643214.0633 - val_mean_squared_error: 14643214.0633
Epoch 1879/2000
 - 0s - loss: 14041576.4189 - mean_squared_error: 14041576.4189 - val_loss: 14642297.2813 - val_mean_squared_error: 14642297.2813
Epoch 1880/2000
 - 0s - loss: 14040577.5329 - mean_squared_error: 14040577.5329 - val_loss: 14641284.8616 - val_mean_squared_error: 14641284.8616
Epoch 1881/2000
 - 0s - loss: 14039654.2488 - mean_squared_error: 14039654.2488 - val_loss: 14640227.1527 - val_mean_squared_error: 14640227.1527
Epoch 1882/2000
 - 0s - loss: 14038650.1586 - mean_squared_error: 14038650.1586 - val_loss: 14639322.6555 - val_mean_squared_error: 14639322.6555
Epoch 1883/2000
 - 0s - loss: 14037704.3331 - mean_squared_error: 14037704.3331 - val_loss: 14638298.8942 - val_mean_squared_error: 14638298.8942
Epoch 1884/2000
 - 0s - loss: 14036720.1617 - mean_squared_error: 14036720.1617 - val_loss: 14637364.9995 - val_mean_squared_error: 14637364.9995
Epoch 1885/2000
 - 0s - loss: 14035740.1378 - mean_squared_error: 14035740.1378 - val_loss: 14636306.6515 - val_mean_squared_error: 14636306.6515
Epoch 1886/2000
 - 0s - loss: 14034767.4048 - mean_squared_error: 14034767.4048 - val_loss: 14635277.0662 - val_mean_squared_error: 14635277.0662
Epoch 1887/2000
 - 0s - loss: 14033839.9824 - mean_squared_error: 14033839.9824 - val_loss: 14634301.1147 - val_mean_squared_error: 14634301.1147
Epoch 1888/2000
 - 0s - loss: 14032845.4022 - mean_squared_error: 14032845.4022 - val_loss: 14633356.0568 - val_mean_squared_error: 14633356.0568
Epoch 1889/2000
 - 0s - loss: 14031883.9451 - mean_squared_error: 14031883.9451 - val_loss: 14632473.1379 - val_mean_squared_error: 14632473.1379
Epoch 1890/2000
 - 0s - loss: 14030868.8508 - mean_squared_error: 14030868.8508 - val_loss: 14631417.2615 - val_mean_squared_error: 14631417.2615
Epoch 1891/2000
 - 0s - loss: 14029889.9388 - mean_squared_error: 14029889.9388 - val_loss: 14630325.6421 - val_mean_squared_error: 14630325.6421
Epoch 1892/2000
 - 0s - loss: 14028935.6314 - mean_squared_error: 14028935.6314 - val_loss: 14629574.6817 - val_mean_squared_error: 14629574.6817
Epoch 1893/2000
 - 0s - loss: 14027996.0493 - mean_squared_error: 14027996.0493 - val_loss: 14628529.3243 - val_mean_squared_error: 14628529.3243
Epoch 1894/2000
 - 0s - loss: 14027005.3169 - mean_squared_error: 14027005.3169 - val_loss: 14627391.1967 - val_mean_squared_error: 14627391.1967
Epoch 1895/2000
 - 0s - loss: 14026048.6426 - mean_squared_error: 14026048.6426 - val_loss: 14626388.6139 - val_mean_squared_error: 14626388.6139
Epoch 1896/2000
 - 0s - loss: 14025031.5184 - mean_squared_error: 14025031.5184 - val_loss: 14625377.2590 - val_mean_squared_error: 14625377.2590
Epoch 1897/2000
 - 0s - loss: 14024054.8359 - mean_squared_error: 14024054.8359 - val_loss: 14624599.2032 - val_mean_squared_error: 14624599.2032
Epoch 1898/2000
 - 0s - loss: 14023097.5869 - mean_squared_error: 14023097.5869 - val_loss: 14623391.8893 - val_mean_squared_error: 14623391.8893
Epoch 1899/2000
 - 0s - loss: 14022133.5670 - mean_squared_error: 14022133.5670 - val_loss: 14622396.1473 - val_mean_squared_error: 14622396.1473
Epoch 1900/2000
 - 0s - loss: 14021144.6183 - mean_squared_error: 14021144.6183 - val_loss: 14621375.9590 - val_mean_squared_error: 14621375.9590
Epoch 1901/2000
 - 0s - loss: 14020168.4813 - mean_squared_error: 14020168.4813 - val_loss: 14620391.5175 - val_mean_squared_error: 14620391.5175
Epoch 1902/2000
 - 0s - loss: 14019185.7647 - mean_squared_error: 14019185.7647 - val_loss: 14619394.6965 - val_mean_squared_error: 14619394.6965
Epoch 1903/2000
 - 0s - loss: 14018240.0450 - mean_squared_error: 14018240.0450 - val_loss: 14618478.2061 - val_mean_squared_error: 14618478.2061
Epoch 1904/2000
 - 0s - loss: 14017278.0611 - mean_squared_error: 14017278.0611 - val_loss: 14617430.2546 - val_mean_squared_error: 14617430.2546
Epoch 1905/2000
 - 0s - loss: 14016316.0847 - mean_squared_error: 14016316.0847 - val_loss: 14616452.4731 - val_mean_squared_error: 14616452.4731
Epoch 1906/2000
 - 0s - loss: 14015343.6848 - mean_squared_error: 14015343.6848 - val_loss: 14615430.9802 - val_mean_squared_error: 14615430.9802
Epoch 1907/2000
 - 0s - loss: 14014326.0258 - mean_squared_error: 14014326.0258 - val_loss: 14614445.1933 - val_mean_squared_error: 14614445.1933
Epoch 1908/2000
 - 0s - loss: 14013395.7700 - mean_squared_error: 14013395.7700 - val_loss: 14613461.9624 - val_mean_squared_error: 14613461.9624
Epoch 1909/2000
 - 0s - loss: 14012363.3592 - mean_squared_error: 14012363.3592 - val_loss: 14612518.7103 - val_mean_squared_error: 14612518.7103
Epoch 1910/2000
 - 0s - loss: 14011435.9528 - mean_squared_error: 14011435.9528 - val_loss: 14611503.5304 - val_mean_squared_error: 14611503.5304
Epoch 1911/2000
 - 0s - loss: 14010426.8371 - mean_squared_error: 14010426.8371 - val_loss: 14610503.6649 - val_mean_squared_error: 14610503.6649
Epoch 1912/2000
 - 0s - loss: 14009488.0206 - mean_squared_error: 14009488.0206 - val_loss: 14609634.6718 - val_mean_squared_error: 14609634.6718
Epoch 1913/2000
 - 0s - loss: 14008505.2308 - mean_squared_error: 14008505.2308 - val_loss: 14608588.0242 - val_mean_squared_error: 14608588.0242
Epoch 1914/2000
 - 1s - loss: 14007534.5843 - mean_squared_error: 14007534.5843 - val_loss: 14607539.7721 - val_mean_squared_error: 14607539.7721
Epoch 1915/2000
 - 1s - loss: 14006574.5376 - mean_squared_error: 14006574.5376 - val_loss: 14606539.5245 - val_mean_squared_error: 14606539.5245
Epoch 1916/2000
 - 1s - loss: 14005556.4157 - mean_squared_error: 14005556.4157 - val_loss: 14605521.3895 - val_mean_squared_error: 14605521.3895
Epoch 1917/2000
 - 0s - loss: 14004657.1730 - mean_squared_error: 14004657.1730 - val_loss: 14604527.3683 - val_mean_squared_error: 14604527.3683
Epoch 1918/2000
 - 0s - loss: 14003640.9749 - mean_squared_error: 14003640.9749 - val_loss: 14603704.9026 - val_mean_squared_error: 14603704.9026
Epoch 1919/2000
 - 0s - loss: 14002662.2360 - mean_squared_error: 14002662.2360 - val_loss: 14602829.4483 - val_mean_squared_error: 14602829.4483
Epoch 1920/2000
 - 1s - loss: 14001717.1359 - mean_squared_error: 14001717.1359 - val_loss: 14601569.4266 - val_mean_squared_error: 14601569.4266
Epoch 1921/2000
 - 1s - loss: 14000717.3413 - mean_squared_error: 14000717.3413 - val_loss: 14600629.5428 - val_mean_squared_error: 14600629.5428
Epoch 1922/2000
 - 1s - loss: 13999781.5390 - mean_squared_error: 13999781.5390 - val_loss: 14599680.2531 - val_mean_squared_error: 14599680.2531
Epoch 1923/2000
 - 0s - loss: 13998813.2340 - mean_squared_error: 13998813.2340 - val_loss: 14598647.6990 - val_mean_squared_error: 14598647.6990
Epoch 1924/2000
 - 0s - loss: 13997831.6735 - mean_squared_error: 13997831.6735 - val_loss: 14597635.7968 - val_mean_squared_error: 14597635.7968
Epoch 1925/2000
 - 1s - loss: 13996861.3573 - mean_squared_error: 13996861.3573 - val_loss: 14596606.2121 - val_mean_squared_error: 14596606.2121
Epoch 1926/2000
 - 1s - loss: 13995880.4640 - mean_squared_error: 13995880.4640 - val_loss: 14595651.4765 - val_mean_squared_error: 14595651.4765
Epoch 1927/2000
 - 0s - loss: 13994892.9043 - mean_squared_error: 13994892.9043 - val_loss: 14594648.9130 - val_mean_squared_error: 14594648.9130
Epoch 1928/2000
 - 1s - loss: 13993922.0608 - mean_squared_error: 13993922.0608 - val_loss: 14593825.7469 - val_mean_squared_error: 14593825.7469
Epoch 1929/2000
 - 0s - loss: 13992974.8030 - mean_squared_error: 13992974.8030 - val_loss: 14592629.0475 - val_mean_squared_error: 14592629.0475
Epoch 1930/2000
 - 1s - loss: 13991971.5907 - mean_squared_error: 13991971.5907 - val_loss: 14591685.5942 - val_mean_squared_error: 14591685.5942
Epoch 1931/2000
 - 1s - loss: 13991008.8681 - mean_squared_error: 13991008.8681 - val_loss: 14590691.3915 - val_mean_squared_error: 14590691.3915
Epoch 1932/2000
 - 1s - loss: 13990055.1244 - mean_squared_error: 13990055.1244 - val_loss: 14589704.5507 - val_mean_squared_error: 14589704.5507
Epoch 1933/2000
 - 1s - loss: 13989088.0260 - mean_squared_error: 13989088.0260 - val_loss: 14588763.9466 - val_mean_squared_error: 14588763.9466
Epoch 1934/2000
 - 1s - loss: 13988146.8797 - mean_squared_error: 13988146.8797 - val_loss: 14587764.2353 - val_mean_squared_error: 14587764.2353
Epoch 1935/2000
 - 1s - loss: 13987125.8917 - mean_squared_error: 13987125.8917 - val_loss: 14586716.4291 - val_mean_squared_error: 14586716.4291
Epoch 1936/2000
 - 1s - loss: 13986166.2039 - mean_squared_error: 13986166.2039 - val_loss: 14585790.5264 - val_mean_squared_error: 14585790.5264
Epoch 1937/2000
 - 1s - loss: 13985232.6452 - mean_squared_error: 13985232.6452 - val_loss: 14584743.5902 - val_mean_squared_error: 14584743.5902
Epoch 1938/2000
 - 1s - loss: 13984278.0603 - mean_squared_error: 13984278.0603 - val_loss: 14583766.9377 - val_mean_squared_error: 14583766.9377
Epoch 1939/2000
 - 1s - loss: 13983262.8058 - mean_squared_error: 13983262.8058 - val_loss: 14583150.8240 - val_mean_squared_error: 14583150.8240
Epoch 1940/2000
 - 1s - loss: 13982284.5357 - mean_squared_error: 13982284.5357 - val_loss: 14581842.4533 - val_mean_squared_error: 14581842.4533
Epoch 1941/2000
 - 1s - loss: 13981343.8120 - mean_squared_error: 13981343.8120 - val_loss: 14580730.2536 - val_mean_squared_error: 14580730.2536
Epoch 1942/2000
 - 1s - loss: 13980322.3916 - mean_squared_error: 13980322.3916 - val_loss: 14579760.0781 - val_mean_squared_error: 14579760.0781
Epoch 1943/2000
 - 1s - loss: 13979362.7327 - mean_squared_error: 13979362.7327 - val_loss: 14578797.9862 - val_mean_squared_error: 14578797.9862
Epoch 1944/2000
 - 1s - loss: 13978394.7608 - mean_squared_error: 13978394.7608 - val_loss: 14577774.0717 - val_mean_squared_error: 14577774.0717
Epoch 1945/2000
 - 1s - loss: 13977455.0814 - mean_squared_error: 13977455.0814 - val_loss: 14576858.7963 - val_mean_squared_error: 14576858.7963
Epoch 1946/2000
 - 1s - loss: 13976429.1993 - mean_squared_error: 13976429.1993 - val_loss: 14575857.0514 - val_mean_squared_error: 14575857.0514
Epoch 1947/2000
 - 1s - loss: 13975504.5684 - mean_squared_error: 13975504.5684 - val_loss: 14575230.7785 - val_mean_squared_error: 14575230.7785
Epoch 1948/2000
 - 1s - loss: 13974535.1979 - mean_squared_error: 13974535.1979 - val_loss: 14573892.8655 - val_mean_squared_error: 14573892.8655
Epoch 1949/2000
 - 1s - loss: 13973541.2834 - mean_squared_error: 13973541.2834 - val_loss: 14572876.0929 - val_mean_squared_error: 14572876.0929
Epoch 1950/2000
 - 1s - loss: 13972627.5943 - mean_squared_error: 13972627.5943 - val_loss: 14571915.0870 - val_mean_squared_error: 14571915.0870
Epoch 1951/2000
 - 1s - loss: 13971644.4700 - mean_squared_error: 13971644.4700 - val_loss: 14570882.7696 - val_mean_squared_error: 14570882.7696
Epoch 1952/2000
 - 1s - loss: 13970696.7312 - mean_squared_error: 13970696.7312 - val_loss: 14569891.8141 - val_mean_squared_error: 14569891.8141
Epoch 1953/2000
 - 1s - loss: 13969663.0166 - mean_squared_error: 13969663.0166 - val_loss: 14568972.2818 - val_mean_squared_error: 14568972.2818
Epoch 1954/2000
 - 1s - loss: 13968782.9991 - mean_squared_error: 13968782.9991 - val_loss: 14567947.0801 - val_mean_squared_error: 14567947.0801
Epoch 1955/2000
 - 1s - loss: 13967730.0860 - mean_squared_error: 13967730.0860 - val_loss: 14567092.4567 - val_mean_squared_error: 14567092.4567
Epoch 1956/2000
 - 1s - loss: 13966790.8138 - mean_squared_error: 13966790.8138 - val_loss: 14565992.3465 - val_mean_squared_error: 14565992.3465
Epoch 1957/2000
 - 1s - loss: 13965827.6534 - mean_squared_error: 13965827.6534 - val_loss: 14564962.4770 - val_mean_squared_error: 14564962.4770
Epoch 1958/2000
 - 1s - loss: 13964874.5129 - mean_squared_error: 13964874.5129 - val_loss: 14564002.0074 - val_mean_squared_error: 14564002.0074
Epoch 1959/2000
 - 1s - loss: 13963875.4002 - mean_squared_error: 13963875.4002 - val_loss: 14563131.7825 - val_mean_squared_error: 14563131.7825
Epoch 1960/2000
 - 1s - loss: 13962938.1506 - mean_squared_error: 13962938.1506 - val_loss: 14562012.5324 - val_mean_squared_error: 14562012.5324
Epoch 1961/2000
 - 1s - loss: 13961932.7147 - mean_squared_error: 13961932.7147 - val_loss: 14561060.4523 - val_mean_squared_error: 14561060.4523
Epoch 1962/2000
 - 1s - loss: 13960977.4902 - mean_squared_error: 13960977.4902 - val_loss: 14560080.2239 - val_mean_squared_error: 14560080.2239
Epoch 1963/2000
 - 1s - loss: 13959991.4240 - mean_squared_error: 13959991.4240 - val_loss: 14559581.7108 - val_mean_squared_error: 14559581.7108
Epoch 1964/2000
 - 1s - loss: 13959042.5384 - mean_squared_error: 13959042.5384 - val_loss: 14558281.3435 - val_mean_squared_error: 14558281.3435
Epoch 1965/2000
 - 1s - loss: 13958002.1029 - mean_squared_error: 13958002.1029 - val_loss: 14557103.2595 - val_mean_squared_error: 14557103.2595
Epoch 1966/2000
 - 1s - loss: 13957110.6674 - mean_squared_error: 13957110.6674 - val_loss: 14556089.4385 - val_mean_squared_error: 14556089.4385
Epoch 1967/2000
 - 1s - loss: 13956126.5047 - mean_squared_error: 13956126.5047 - val_loss: 14555112.7884 - val_mean_squared_error: 14555112.7884
Epoch 1968/2000
 - 1s - loss: 13955138.1453 - mean_squared_error: 13955138.1453 - val_loss: 14554135.6223 - val_mean_squared_error: 14554135.6223
Epoch 1969/2000
 - 1s - loss: 13954155.2029 - mean_squared_error: 13954155.2029 - val_loss: 14553200.1097 - val_mean_squared_error: 14553200.1097
Epoch 1970/2000
 - 1s - loss: 13953245.7021 - mean_squared_error: 13953245.7021 - val_loss: 14552137.4904 - val_mean_squared_error: 14552137.4904
Epoch 1971/2000
 - 1s - loss: 13952238.0690 - mean_squared_error: 13952238.0690 - val_loss: 14551263.7726 - val_mean_squared_error: 14551263.7726
Epoch 1972/2000
 - 1s - loss: 13951232.4036 - mean_squared_error: 13951232.4036 - val_loss: 14550146.3342 - val_mean_squared_error: 14550146.3342
Epoch 1973/2000
 - 1s - loss: 13950299.8464 - mean_squared_error: 13950299.8464 - val_loss: 14549223.6060 - val_mean_squared_error: 14549223.6060
Epoch 1974/2000
 - 1s - loss: 13949369.3179 - mean_squared_error: 13949369.3179 - val_loss: 14548158.0573 - val_mean_squared_error: 14548158.0573
Epoch 1975/2000
 - 1s - loss: 13948345.8805 - mean_squared_error: 13948345.8805 - val_loss: 14547203.9649 - val_mean_squared_error: 14547203.9649
Epoch 1976/2000
 - 1s - loss: 13947369.9629 - mean_squared_error: 13947369.9629 - val_loss: 14546559.4800 - val_mean_squared_error: 14546559.4800
Epoch 1977/2000
 - 1s - loss: 13946421.5726 - mean_squared_error: 13946421.5726 - val_loss: 14545221.1962 - val_mean_squared_error: 14545221.1962
Epoch 1978/2000
 - 1s - loss: 13945421.9250 - mean_squared_error: 13945421.9250 - val_loss: 14544338.6955 - val_mean_squared_error: 14544338.6955
Epoch 1979/2000
 - 1s - loss: 13944505.0388 - mean_squared_error: 13944505.0388 - val_loss: 14543509.0623 - val_mean_squared_error: 14543509.0623
Epoch 1980/2000
 - 1s - loss: 13943541.5878 - mean_squared_error: 13943541.5878 - val_loss: 14542235.4656 - val_mean_squared_error: 14542235.4656
Epoch 1981/2000
 - 1s - loss: 13942526.7038 - mean_squared_error: 13942526.7038 - val_loss: 14541299.3579 - val_mean_squared_error: 14541299.3579
Epoch 1982/2000
 - 1s - loss: 13941562.8653 - mean_squared_error: 13941562.8653 - val_loss: 14540301.8443 - val_mean_squared_error: 14540301.8443
Epoch 1983/2000
 - 1s - loss: 13940599.2324 - mean_squared_error: 13940599.2324 - val_loss: 14539327.7350 - val_mean_squared_error: 14539327.7350
Epoch 1984/2000
 - 1s - loss: 13939643.7045 - mean_squared_error: 13939643.7045 - val_loss: 14538470.0386 - val_mean_squared_error: 14538470.0386
Epoch 1985/2000
 - 1s - loss: 13938686.7978 - mean_squared_error: 13938686.7978 - val_loss: 14537338.1977 - val_mean_squared_error: 14537338.1977
Epoch 1986/2000
 - 1s - loss: 13937676.0775 - mean_squared_error: 13937676.0775 - val_loss: 14536332.8077 - val_mean_squared_error: 14536332.8077
Epoch 1987/2000
 - 1s - loss: 13936761.4252 - mean_squared_error: 13936761.4252 - val_loss: 14535352.2719 - val_mean_squared_error: 14535352.2719
Epoch 1988/2000
 - 1s - loss: 13935780.5833 - mean_squared_error: 13935780.5833 - val_loss: 14534363.9086 - val_mean_squared_error: 14534363.9086
Epoch 1989/2000
 - 1s - loss: 13934811.6088 - mean_squared_error: 13934811.6088 - val_loss: 14533393.1532 - val_mean_squared_error: 14533393.1532
Epoch 1990/2000
 - 1s - loss: 13933816.7718 - mean_squared_error: 13933816.7718 - val_loss: 14532369.0000 - val_mean_squared_error: 14532369.0000
Epoch 1991/2000
 - 1s - loss: 13932870.7646 - mean_squared_error: 13932870.7646 - val_loss: 14531386.6288 - val_mean_squared_error: 14531386.6288
Epoch 1992/2000
 - 1s - loss: 13931881.5788 - mean_squared_error: 13931881.5788 - val_loss: 14530503.4108 - val_mean_squared_error: 14530503.4108
Epoch 1993/2000
 - 1s - loss: 13930933.1328 - mean_squared_error: 13930933.1328 - val_loss: 14529401.6886 - val_mean_squared_error: 14529401.6886
Epoch 1994/2000
 - 1s - loss: 13929969.5578 - mean_squared_error: 13929969.5578 - val_loss: 14528616.6149 - val_mean_squared_error: 14528616.6149
Epoch 1995/2000
 - 1s - loss: 13929016.3818 - mean_squared_error: 13929016.3818 - val_loss: 14527630.0900 - val_mean_squared_error: 14527630.0900
Epoch 1996/2000
 - 1s - loss: 13928019.3752 - mean_squared_error: 13928019.3752 - val_loss: 14526495.1280 - val_mean_squared_error: 14526495.1280
Epoch 1997/2000
 - 1s - loss: 13927022.6285 - mean_squared_error: 13927022.6285 - val_loss: 14525727.0074 - val_mean_squared_error: 14525727.0074
Epoch 1998/2000
 - 1s - loss: 13926088.6883 - mean_squared_error: 13926088.6883 - val_loss: 14524562.8072 - val_mean_squared_error: 14524562.8072
Epoch 1999/2000
 - 1s - loss: 13925121.2566 - mean_squared_error: 13925121.2566 - val_loss: 14523548.8596 - val_mean_squared_error: 14523548.8596
Epoch 2000/2000
 - 1s - loss: 13924164.6088 - mean_squared_error: 13924164.6088 - val_loss: 14523216.8319 - val_mean_squared_error: 14523216.8319

   32/13485 [..............................] - ETA: 0s
 2592/13485 [====>.........................] - ETA: 0s
 5088/13485 [==========>...................] - ETA: 0s
 8032/13485 [================>.............] - ETA: 0s
10720/13485 [======================>.......] - ETA: 0s
13485/13485 [==============================] - 0s 19us/step

root mean_squared_error: 3707.069044

What is different here?

We've changed the activation in the hidden layer to "sigmoid" per our discussion.
Next, notice that we're running 2000 training epochs!

Even so, it takes a looooong time to converge. If you experiment a lot, you'll find that ... it still takes a long time to converge. Around the early part of the most recent deep learning renaissance, researchers started experimenting with other non-linearities.

(Remember, we're talking about non-linear activations in the hidden layer. The output here is still using "linear" rather than "softmax" because we're performing regression, not classification.)

In theory, any non-linearity should allow learning, and maybe we can use one that "works better"

By "works better" we mean

Simpler gradient - faster to compute
Less prone to "saturation" -- where the neuron ends up way off in the 0 or 1 territory of the sigmoid and can't easily learn anything
Keeps gradients "big" -- avoiding the large, flat, near-zero gradient areas of the sigmoid

Turns out that a big breakthrough and popular solution is a very simple hack:

Rectified Linear Unit (ReLU)

Go change your hidden-layer activation from 'sigmoid' to 'relu'

Start your script and watch the error for a bit!

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd

input_file = "/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv"

df = pd.read_csv(input_file, header = 0)
df.drop(df.columns[0], axis=1, inplace=True)
df = pd.get_dummies(df, prefix=['cut_', 'color_', 'clarity_'])

y = df.iloc[:,3:4].values.flatten()
y.flatten()

X = df.drop(df.columns[3], axis=1).values
np.shape(X)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = Sequential()
model.add(Dense(30, input_dim=26, kernel_initializer='normal', activation='relu')) # <--- CHANGE IS HERE
model.add(Dense(1, kernel_initializer='normal', activation='linear'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
history = model.fit(X_train, y_train, epochs=2000, batch_size=100, validation_split=0.1, verbose=2)

scores = model.evaluate(X_test, y_test)
print("\nroot %s: %f" % (model.metrics_names[1], np.sqrt(scores[1])))

Train on 36409 samples, validate on 4046 samples
Epoch 1/2000
 - 1s - loss: 30189420.8985 - mean_squared_error: 30189420.8985 - val_loss: 28636809.7133 - val_mean_squared_error: 28636809.7133
Epoch 2/2000
 - 1s - loss: 23198066.2223 - mean_squared_error: 23198066.2223 - val_loss: 19609172.2017 - val_mean_squared_error: 19609172.2017
Epoch 3/2000
 - 1s - loss: 16603911.3175 - mean_squared_error: 16603911.3175 - val_loss: 16048687.6584 - val_mean_squared_error: 16048687.6584
Epoch 4/2000
 - 1s - loss: 15165760.0960 - mean_squared_error: 15165760.0960 - val_loss: 15658225.8557 - val_mean_squared_error: 15658225.8557
Epoch 5/2000
 - 1s - loss: 14989417.4870 - mean_squared_error: 14989417.4870 - val_loss: 15506647.2674 - val_mean_squared_error: 15506647.2674
Epoch 6/2000
 - 1s - loss: 14827583.5424 - mean_squared_error: 14827583.5424 - val_loss: 15322988.9224 - val_mean_squared_error: 15322988.9224
Epoch 7/2000
 - 1s - loss: 14640567.4536 - mean_squared_error: 14640567.4536 - val_loss: 15118571.4770 - val_mean_squared_error: 15118571.4770
Epoch 8/2000
 - 1s - loss: 14430938.9404 - mean_squared_error: 14430938.9404 - val_loss: 14892813.4118 - val_mean_squared_error: 14892813.4118
Epoch 9/2000
 - 1s - loss: 14196182.8864 - mean_squared_error: 14196182.8864 - val_loss: 14636089.3959 - val_mean_squared_error: 14636089.3959
Epoch 10/2000
 - 1s - loss: 13933527.4948 - mean_squared_error: 13933527.4948 - val_loss: 14348018.4563 - val_mean_squared_error: 14348018.4563
Epoch 11/2000
 - 1s - loss: 13639193.3295 - mean_squared_error: 13639193.3295 - val_loss: 14035286.0519 - val_mean_squared_error: 14035286.0519
Epoch 12/2000
 - 1s - loss: 13308997.3142 - mean_squared_error: 13308997.3142 - val_loss: 13662616.8888 - val_mean_squared_error: 13662616.8888
Epoch 13/2000
 - 1s - loss: 12935229.2288 - mean_squared_error: 12935229.2288 - val_loss: 13251498.6856 - val_mean_squared_error: 13251498.6856
Epoch 14/2000
 - 1s - loss: 12511291.7037 - mean_squared_error: 12511291.7037 - val_loss: 12788027.2852 - val_mean_squared_error: 12788027.2852
Epoch 15/2000
 - 1s - loss: 12037879.5633 - mean_squared_error: 12037879.5633 - val_loss: 12273845.5472 - val_mean_squared_error: 12273845.5472
Epoch 16/2000
 - 1s - loss: 11514312.3183 - mean_squared_error: 11514312.3183 - val_loss: 11705433.3421 - val_mean_squared_error: 11705433.3421
Epoch 17/2000
 - 0s - loss: 10942791.8511 - mean_squared_error: 10942791.8511 - val_loss: 11090956.9812 - val_mean_squared_error: 11090956.9812
Epoch 18/2000
 - 0s - loss: 10324517.6534 - mean_squared_error: 10324517.6534 - val_loss: 10432108.8576 - val_mean_squared_error: 10432108.8576
Epoch 19/2000
 - 0s - loss: 9672784.8852 - mean_squared_error: 9672784.8852 - val_loss: 9742762.5522 - val_mean_squared_error: 9742762.5522
Epoch 20/2000
 - 0s - loss: 8999411.3819 - mean_squared_error: 8999411.3819 - val_loss: 9035512.7736 - val_mean_squared_error: 9035512.7736
Epoch 21/2000
 - 0s - loss: 8324484.4254 - mean_squared_error: 8324484.4254 - val_loss: 8336192.3166 - val_mean_squared_error: 8336192.3166
Epoch 22/2000
 - 0s - loss: 7667359.0580 - mean_squared_error: 7667359.0580 - val_loss: 7670299.3245 - val_mean_squared_error: 7670299.3245
Epoch 23/2000
 - 0s - loss: 7044304.7153 - mean_squared_error: 7044304.7153 - val_loss: 7043340.9703 - val_mean_squared_error: 7043340.9703
Epoch 24/2000
 - 0s - loss: 6463977.2148 - mean_squared_error: 6463977.2148 - val_loss: 6448855.0040 - val_mean_squared_error: 6448855.0040
Epoch 25/2000
 - 0s - loss: 5929861.5535 - mean_squared_error: 5929861.5535 - val_loss: 5912099.5652 - val_mean_squared_error: 5912099.5652
Epoch 26/2000
 - 0s - loss: 5448215.9329 - mean_squared_error: 5448215.9329 - val_loss: 5432970.9488 - val_mean_squared_error: 5432970.9488
Epoch 27/2000
 - 0s - loss: 5022827.4511 - mean_squared_error: 5022827.4511 - val_loss: 5010718.1720 - val_mean_squared_error: 5010718.1720
Epoch 28/2000
 - 0s - loss: 4649669.4373 - mean_squared_error: 4649669.4373 - val_loss: 4636147.5659 - val_mean_squared_error: 4636147.5659
Epoch 29/2000
 - 0s - loss: 4327426.4939 - mean_squared_error: 4327426.4939 - val_loss: 4310399.2614 - val_mean_squared_error: 4310399.2614
Epoch 30/2000
 - 0s - loss: 4046344.3449 - mean_squared_error: 4046344.3449 - val_loss: 4023325.6366 - val_mean_squared_error: 4023325.6366
Epoch 31/2000
 - 0s - loss: 3797185.8155 - mean_squared_error: 3797185.8155 - val_loss: 3777603.3353 - val_mean_squared_error: 3777603.3353
Epoch 32/2000
 - 0s - loss: 3580273.5487 - mean_squared_error: 3580273.5487 - val_loss: 3549745.9894 - val_mean_squared_error: 3549745.9894
Epoch 33/2000
 - 0s - loss: 3387090.7587 - mean_squared_error: 3387090.7587 - val_loss: 3343150.4650 - val_mean_squared_error: 3343150.4650
Epoch 34/2000
 - 0s - loss: 3214145.2624 - mean_squared_error: 3214145.2624 - val_loss: 3162656.5896 - val_mean_squared_error: 3162656.5896
Epoch 35/2000
 - 0s - loss: 3058334.4194 - mean_squared_error: 3058334.4194 - val_loss: 3004862.6919 - val_mean_squared_error: 3004862.6919
Epoch 36/2000
 - 0s - loss: 2917225.2674 - mean_squared_error: 2917225.2674 - val_loss: 2846089.5386 - val_mean_squared_error: 2846089.5386
Epoch 37/2000
 - 0s - loss: 2785693.9761 - mean_squared_error: 2785693.9761 - val_loss: 2707869.5716 - val_mean_squared_error: 2707869.5716
Epoch 38/2000
 - 0s - loss: 2663386.8056 - mean_squared_error: 2663386.8056 - val_loss: 2570811.6094 - val_mean_squared_error: 2570811.6094
Epoch 39/2000
 - 0s - loss: 2550473.5284 - mean_squared_error: 2550473.5284 - val_loss: 2448269.9601 - val_mean_squared_error: 2448269.9601
Epoch 40/2000
 - 0s - loss: 2447714.5338 - mean_squared_error: 2447714.5338 - val_loss: 2339941.2572 - val_mean_squared_error: 2339941.2572
Epoch 41/2000
 - 0s - loss: 2354000.8045 - mean_squared_error: 2354000.8045 - val_loss: 2230314.2221 - val_mean_squared_error: 2230314.2221
Epoch 42/2000
 - 0s - loss: 2265501.9560 - mean_squared_error: 2265501.9560 - val_loss: 2135751.3114 - val_mean_squared_error: 2135751.3114
Epoch 43/2000
 - 0s - loss: 2184753.3058 - mean_squared_error: 2184753.3058 - val_loss: 2047332.1391 - val_mean_squared_error: 2047332.1391
Epoch 44/2000
 - 0s - loss: 2112292.5049 - mean_squared_error: 2112292.5049 - val_loss: 1965993.4016 - val_mean_squared_error: 1965993.4016
Epoch 45/2000
 - 0s - loss: 2044595.5458 - mean_squared_error: 2044595.5458 - val_loss: 1891222.9977 - val_mean_squared_error: 1891222.9977
Epoch 46/2000
 - 0s - loss: 1981209.5128 - mean_squared_error: 1981209.5128 - val_loss: 1822348.1388 - val_mean_squared_error: 1822348.1388
Epoch 47/2000
 - 0s - loss: 1924395.7817 - mean_squared_error: 1924395.7817 - val_loss: 1764530.3743 - val_mean_squared_error: 1764530.3743
Epoch 48/2000
 - 0s - loss: 1869240.8064 - mean_squared_error: 1869240.8064 - val_loss: 1699779.9201 - val_mean_squared_error: 1699779.9201
Epoch 49/2000
 - 0s - loss: 1818964.0647 - mean_squared_error: 1818964.0647 - val_loss: 1642916.4305 - val_mean_squared_error: 1642916.4305
Epoch 50/2000
 - 0s - loss: 1772949.7733 - mean_squared_error: 1772949.7733 - val_loss: 1589150.5234 - val_mean_squared_error: 1589150.5234
Epoch 51/2000
 - 0s - loss: 1728289.2533 - mean_squared_error: 1728289.2533 - val_loss: 1540550.6556 - val_mean_squared_error: 1540550.6556
Epoch 52/2000
 - 0s - loss: 1686578.8149 - mean_squared_error: 1686578.8149 - val_loss: 1493753.4956 - val_mean_squared_error: 1493753.4956
Epoch 53/2000
 - 0s - loss: 1648652.4508 - mean_squared_error: 1648652.4508 - val_loss: 1458841.1140 - val_mean_squared_error: 1458841.1140
Epoch 54/2000
 - 0s - loss: 1611433.7462 - mean_squared_error: 1611433.7462 - val_loss: 1409693.2003 - val_mean_squared_error: 1409693.2003
Epoch 55/2000
 - 0s - loss: 1576917.2249 - mean_squared_error: 1576917.2249 - val_loss: 1372174.2692 - val_mean_squared_error: 1372174.2692
Epoch 56/2000
 - 0s - loss: 1544672.2450 - mean_squared_error: 1544672.2450 - val_loss: 1336006.0685 - val_mean_squared_error: 1336006.0685
Epoch 57/2000
 - 0s - loss: 1515766.5707 - mean_squared_error: 1515766.5707 - val_loss: 1304437.5853 - val_mean_squared_error: 1304437.5853
Epoch 58/2000
 - 0s - loss: 1487106.1103 - mean_squared_error: 1487106.1103 - val_loss: 1271618.0805 - val_mean_squared_error: 1271618.0805
Epoch 59/2000
 - 0s - loss: 1457997.5480 - mean_squared_error: 1457997.5480 - val_loss: 1242240.4967 - val_mean_squared_error: 1242240.4967
Epoch 60/2000
 - 0s - loss: 1433937.3561 - mean_squared_error: 1433937.3561 - val_loss: 1213240.8568 - val_mean_squared_error: 1213240.8568
Epoch 61/2000
 - 0s - loss: 1409274.3087 - mean_squared_error: 1409274.3087 - val_loss: 1191901.4261 - val_mean_squared_error: 1191901.4261
Epoch 62/2000
 - 0s - loss: 1387124.1232 - mean_squared_error: 1387124.1232 - val_loss: 1166198.4036 - val_mean_squared_error: 1166198.4036
Epoch 63/2000
 - 0s - loss: 1367341.6098 - mean_squared_error: 1367341.6098 - val_loss: 1144052.0850 - val_mean_squared_error: 1144052.0850
Epoch 64/2000
 - 0s - loss: 1348362.6727 - mean_squared_error: 1348362.6727 - val_loss: 1122819.6877 - val_mean_squared_error: 1122819.6877
Epoch 65/2000
 - 0s - loss: 1329934.3085 - mean_squared_error: 1329934.3085 - val_loss: 1101036.6601 - val_mean_squared_error: 1101036.6601
Epoch 66/2000
 - 0s - loss: 1313865.1430 - mean_squared_error: 1313865.1430 - val_loss: 1087398.2983 - val_mean_squared_error: 1087398.2983
Epoch 67/2000
 - 0s - loss: 1297733.6369 - mean_squared_error: 1297733.6369 - val_loss: 1068921.9223 - val_mean_squared_error: 1068921.9223
Epoch 68/2000
 - 0s - loss: 1283105.7054 - mean_squared_error: 1283105.7054 - val_loss: 1052849.1446 - val_mean_squared_error: 1052849.1446
Epoch 69/2000
 - 0s - loss: 1269226.2389 - mean_squared_error: 1269226.2389 - val_loss: 1038531.1632 - val_mean_squared_error: 1038531.1632
Epoch 70/2000
 - 0s - loss: 1254929.9186 - mean_squared_error: 1254929.9186 - val_loss: 1023195.2031 - val_mean_squared_error: 1023195.2031
Epoch 71/2000
 - 0s - loss: 1242275.6744 - mean_squared_error: 1242275.6744 - val_loss: 1004922.5253 - val_mean_squared_error: 1004922.5253
Epoch 72/2000
 - 0s - loss: 1231293.5488 - mean_squared_error: 1231293.5488 - val_loss: 991712.2193 - val_mean_squared_error: 991712.2193
Epoch 73/2000
 - 0s - loss: 1219624.4148 - mean_squared_error: 1219624.4148 - val_loss: 979516.0057 - val_mean_squared_error: 979516.0057
Epoch 74/2000
 - 0s - loss: 1208870.5152 - mean_squared_error: 1208870.5152 - val_loss: 968570.3547 - val_mean_squared_error: 968570.3547
Epoch 75/2000
 - 0s - loss: 1198596.5433 - mean_squared_error: 1198596.5433 - val_loss: 957311.5155 - val_mean_squared_error: 957311.5155
Epoch 76/2000
 - 0s - loss: 1188799.8814 - mean_squared_error: 1188799.8814 - val_loss: 945335.0963 - val_mean_squared_error: 945335.0963
Epoch 77/2000
 - 0s - loss: 1179789.8610 - mean_squared_error: 1179789.8610 - val_loss: 940609.4587 - val_mean_squared_error: 940609.4587
Epoch 78/2000
 - 0s - loss: 1170511.9571 - mean_squared_error: 1170511.9571 - val_loss: 925838.4326 - val_mean_squared_error: 925838.4326
Epoch 79/2000
 - 0s - loss: 1162717.7094 - mean_squared_error: 1162717.7094 - val_loss: 918393.0366 - val_mean_squared_error: 918393.0366
Epoch 80/2000
 - 0s - loss: 1155802.0771 - mean_squared_error: 1155802.0771 - val_loss: 909008.6373 - val_mean_squared_error: 909008.6373
Epoch 81/2000
 - 0s - loss: 1147283.3683 - mean_squared_error: 1147283.3683 - val_loss: 899639.5847 - val_mean_squared_error: 899639.5847
Epoch 82/2000
 - 0s - loss: 1139978.5240 - mean_squared_error: 1139978.5240 - val_loss: 893100.5091 - val_mean_squared_error: 893100.5091
Epoch 83/2000
 - 0s - loss: 1133857.4649 - mean_squared_error: 1133857.4649 - val_loss: 886380.4719 - val_mean_squared_error: 886380.4719
Epoch 84/2000
 - 0s - loss: 1126496.1330 - mean_squared_error: 1126496.1330 - val_loss: 883940.9976 - val_mean_squared_error: 883940.9976
Epoch 85/2000
 - 0s - loss: 1119351.9238 - mean_squared_error: 1119351.9238 - val_loss: 873083.9504 - val_mean_squared_error: 873083.9504
Epoch 86/2000
 - 0s - loss: 1113990.1621 - mean_squared_error: 1113990.1621 - val_loss: 865655.3887 - val_mean_squared_error: 865655.3887
Epoch 87/2000
 - 0s - loss: 1107928.8959 - mean_squared_error: 1107928.8959 - val_loss: 873964.5185 - val_mean_squared_error: 873964.5185
Epoch 88/2000
 - 0s - loss: 1101927.3729 - mean_squared_error: 1101927.3729 - val_loss: 861648.0959 - val_mean_squared_error: 861648.0959
Epoch 89/2000
 - 0s - loss: 1096059.7178 - mean_squared_error: 1096059.7178 - val_loss: 845345.2097 - val_mean_squared_error: 845345.2097
Epoch 90/2000
 - 0s - loss: 1090183.0421 - mean_squared_error: 1090183.0421 - val_loss: 838976.3949 - val_mean_squared_error: 838976.3949
Epoch 91/2000
 - 0s - loss: 1085125.6741 - mean_squared_error: 1085125.6741 - val_loss: 834308.7424 - val_mean_squared_error: 834308.7424
Epoch 92/2000
 - 0s - loss: 1079960.3373 - mean_squared_error: 1079960.3373 - val_loss: 827018.7170 - val_mean_squared_error: 827018.7170
Epoch 93/2000
 - 0s - loss: 1075016.3585 - mean_squared_error: 1075016.3585 - val_loss: 824307.1723 - val_mean_squared_error: 824307.1723
Epoch 94/2000
 - 0s - loss: 1069779.9771 - mean_squared_error: 1069779.9771 - val_loss: 815420.9564 - val_mean_squared_error: 815420.9564
Epoch 95/2000
 - 0s - loss: 1065536.1922 - mean_squared_error: 1065536.1922 - val_loss: 815212.7480 - val_mean_squared_error: 815212.7480
Epoch 96/2000
 - 0s - loss: 1060357.2113 - mean_squared_error: 1060357.2113 - val_loss: 806200.7524 - val_mean_squared_error: 806200.7524
Epoch 97/2000
 - 0s - loss: 1055606.6725 - mean_squared_error: 1055606.6725 - val_loss: 815764.0978 - val_mean_squared_error: 815764.0978
Epoch 98/2000
 - 0s - loss: 1052198.3024 - mean_squared_error: 1052198.3024 - val_loss: 796517.7568 - val_mean_squared_error: 796517.7568
Epoch 99/2000
 - 0s - loss: 1047813.9664 - mean_squared_error: 1047813.9664 - val_loss: 792303.8808 - val_mean_squared_error: 792303.8808
Epoch 100/2000
 - 0s - loss: 1043633.2012 - mean_squared_error: 1043633.2012 - val_loss: 792735.2539 - val_mean_squared_error: 792735.2539
Epoch 101/2000
 - 0s - loss: 1039242.1259 - mean_squared_error: 1039242.1259 - val_loss: 785669.7569 - val_mean_squared_error: 785669.7569
Epoch 102/2000
 - 0s - loss: 1035018.1389 - mean_squared_error: 1035018.1389 - val_loss: 779840.4313 - val_mean_squared_error: 779840.4313
Epoch 103/2000
 - 0s - loss: 1030598.0931 - mean_squared_error: 1030598.0931 - val_loss: 781102.4054 - val_mean_squared_error: 781102.4054
Epoch 104/2000
 - 0s - loss: 1026942.8171 - mean_squared_error: 1026942.8171 - val_loss: 774531.1024 - val_mean_squared_error: 774531.1024
Epoch 105/2000
 - 0s - loss: 1023330.2583 - mean_squared_error: 1023330.2583 - val_loss: 767568.0108 - val_mean_squared_error: 767568.0108
Epoch 106/2000
 - 0s - loss: 1019163.5211 - mean_squared_error: 1019163.5211 - val_loss: 773755.5281 - val_mean_squared_error: 773755.5281
Epoch 107/2000
 - 0s - loss: 1015504.6994 - mean_squared_error: 1015504.6994 - val_loss: 761746.3565 - val_mean_squared_error: 761746.3565
Epoch 108/2000
 - 0s - loss: 1012240.3183 - mean_squared_error: 1012240.3183 - val_loss: 758921.7005 - val_mean_squared_error: 758921.7005
Epoch 109/2000
 - 0s - loss: 1008086.6043 - mean_squared_error: 1008086.6043 - val_loss: 752427.9384 - val_mean_squared_error: 752427.9384
Epoch 110/2000
 - 0s - loss: 1004775.1231 - mean_squared_error: 1004775.1231 - val_loss: 748668.6899 - val_mean_squared_error: 748668.6899
Epoch 111/2000
 - 0s - loss: 1001015.0934 - mean_squared_error: 1001015.0934 - val_loss: 745433.2375 - val_mean_squared_error: 745433.2375
Epoch 112/2000
 - 0s - loss: 997149.6093 - mean_squared_error: 997149.6093 - val_loss: 746147.8102 - val_mean_squared_error: 746147.8102
Epoch 113/2000
 - 0s - loss: 994264.0145 - mean_squared_error: 994264.0145 - val_loss: 740447.2325 - val_mean_squared_error: 740447.2325
Epoch 114/2000
 - 0s - loss: 990874.0751 - mean_squared_error: 990874.0751 - val_loss: 752904.6843 - val_mean_squared_error: 752904.6843
Epoch 115/2000
 - 0s - loss: 988689.3808 - mean_squared_error: 988689.3808 - val_loss: 738193.1483 - val_mean_squared_error: 738193.1483
Epoch 116/2000
 - 0s - loss: 984721.8802 - mean_squared_error: 984721.8802 - val_loss: 728698.0477 - val_mean_squared_error: 728698.0477
Epoch 117/2000
 - 0s - loss: 981724.4628 - mean_squared_error: 981724.4628 - val_loss: 726934.4692 - val_mean_squared_error: 726934.4692
Epoch 118/2000
 - 0s - loss: 978402.5133 - mean_squared_error: 978402.5133 - val_loss: 724588.1015 - val_mean_squared_error: 724588.1015
Epoch 119/2000
 - 0s - loss: 974883.3142 - mean_squared_error: 974883.3142 - val_loss: 719651.9469 - val_mean_squared_error: 719651.9469
Epoch 120/2000
 - 0s - loss: 972348.5150 - mean_squared_error: 972348.5150 - val_loss: 723171.7312 - val_mean_squared_error: 723171.7312
Epoch 121/2000
 - 0s - loss: 969273.4995 - mean_squared_error: 969273.4995 - val_loss: 715898.5582 - val_mean_squared_error: 715898.5582
Epoch 122/2000
 - 0s - loss: 966046.4831 - mean_squared_error: 966046.4831 - val_loss: 721566.7988 - val_mean_squared_error: 721566.7988
Epoch 123/2000
 - 0s - loss: 963870.3004 - mean_squared_error: 963870.3004 - val_loss: 713585.8379 - val_mean_squared_error: 713585.8379
Epoch 124/2000
 - 0s - loss: 960399.8889 - mean_squared_error: 960399.8889 - val_loss: 706410.6038 - val_mean_squared_error: 706410.6038
Epoch 125/2000
 - 0s - loss: 958589.1116 - mean_squared_error: 958589.1116 - val_loss: 705770.9541 - val_mean_squared_error: 705770.9541
Epoch 126/2000
 - 0s - loss: 955909.8628 - mean_squared_error: 955909.8628 - val_loss: 707573.5642 - val_mean_squared_error: 707573.5642
Epoch 127/2000
 - 0s - loss: 953370.5705 - mean_squared_error: 953370.5705 - val_loss: 703662.7318 - val_mean_squared_error: 703662.7318
Epoch 128/2000
 - 0s - loss: 950355.3206 - mean_squared_error: 950355.3206 - val_loss: 705298.0371 - val_mean_squared_error: 705298.0371
Epoch 129/2000
 - 0s - loss: 947470.5564 - mean_squared_error: 947470.5564 - val_loss: 694944.4235 - val_mean_squared_error: 694944.4235
Epoch 130/2000
 - 0s - loss: 945568.4066 - mean_squared_error: 945568.4066 - val_loss: 692030.7021 - val_mean_squared_error: 692030.7021
Epoch 131/2000
 - 0s - loss: 943050.2850 - mean_squared_error: 943050.2850 - val_loss: 699362.5001 - val_mean_squared_error: 699362.5001
Epoch 132/2000
 - 0s - loss: 939632.1378 - mean_squared_error: 939632.1378 - val_loss: 691716.7484 - val_mean_squared_error: 691716.7484
Epoch 133/2000
 - 0s - loss: 937285.4758 - mean_squared_error: 937285.4758 - val_loss: 690304.0487 - val_mean_squared_error: 690304.0487
Epoch 134/2000
 - 0s - loss: 935539.2472 - mean_squared_error: 935539.2472 - val_loss: 684200.6772 - val_mean_squared_error: 684200.6772
Epoch 135/2000
 - 0s - loss: 932167.0026 - mean_squared_error: 932167.0026 - val_loss: 681388.0945 - val_mean_squared_error: 681388.0945
Epoch 136/2000
 - 0s - loss: 929939.8893 - mean_squared_error: 929939.8893 - val_loss: 682788.7851 - val_mean_squared_error: 682788.7851
Epoch 137/2000
 - 0s - loss: 927668.1191 - mean_squared_error: 927668.1191 - val_loss: 679113.7417 - val_mean_squared_error: 679113.7417
Epoch 138/2000
 - 0s - loss: 926008.5818 - mean_squared_error: 926008.5818 - val_loss: 676147.3046 - val_mean_squared_error: 676147.3046
Epoch 139/2000
 - 0s - loss: 923048.2713 - mean_squared_error: 923048.2713 - val_loss: 677156.4186 - val_mean_squared_error: 677156.4186
Epoch 140/2000
 - 0s - loss: 920688.1663 - mean_squared_error: 920688.1663 - val_loss: 672482.7448 - val_mean_squared_error: 672482.7448
Epoch 141/2000
 - 0s - loss: 919837.8943 - mean_squared_error: 919837.8943 - val_loss: 670414.9576 - val_mean_squared_error: 670414.9576
Epoch 142/2000
 - 0s - loss: 916313.0555 - mean_squared_error: 916313.0555 - val_loss: 685471.9791 - val_mean_squared_error: 685471.9791
Epoch 143/2000
 - 0s - loss: 915272.9670 - mean_squared_error: 915272.9670 - val_loss: 668524.5762 - val_mean_squared_error: 668524.5762
Epoch 144/2000
 - 0s - loss: 912336.0104 - mean_squared_error: 912336.0104 - val_loss: 674249.9691 - val_mean_squared_error: 674249.9691
Epoch 145/2000
 - 0s - loss: 910291.7740 - mean_squared_error: 910291.7740 - val_loss: 663560.6815 - val_mean_squared_error: 663560.6815
Epoch 146/2000
 - 0s - loss: 908166.9091 - mean_squared_error: 908166.9091 - val_loss: 661399.2248 - val_mean_squared_error: 661399.2248
Epoch 147/2000
 - 0s - loss: 905912.1078 - mean_squared_error: 905912.1078 - val_loss: 667400.1303 - val_mean_squared_error: 667400.1303
Epoch 148/2000
 - 0s - loss: 903832.7466 - mean_squared_error: 903832.7466 - val_loss: 664925.2466 - val_mean_squared_error: 664925.2466
Epoch 149/2000
 - 0s - loss: 901111.1322 - mean_squared_error: 901111.1322 - val_loss: 656424.3131 - val_mean_squared_error: 656424.3131
Epoch 150/2000
 - 0s - loss: 899936.9588 - mean_squared_error: 899936.9588 - val_loss: 655574.3051 - val_mean_squared_error: 655574.3051
Epoch 151/2000
 - 0s - loss: 897709.8877 - mean_squared_error: 897709.8877 - val_loss: 653072.2465 - val_mean_squared_error: 653072.2465
Epoch 152/2000
 - 0s - loss: 895961.3970 - mean_squared_error: 895961.3970 - val_loss: 653385.8794 - val_mean_squared_error: 653385.8794
Epoch 153/2000
 - 0s - loss: 894145.2104 - mean_squared_error: 894145.2104 - val_loss: 650049.8083 - val_mean_squared_error: 650049.8083
Epoch 154/2000
 - 0s - loss: 892313.8828 - mean_squared_error: 892313.8828 - val_loss: 651325.9268 - val_mean_squared_error: 651325.9268
Epoch 155/2000
 - 0s - loss: 890755.4445 - mean_squared_error: 890755.4445 - val_loss: 651107.3247 - val_mean_squared_error: 651107.3247
Epoch 156/2000
 - 0s - loss: 888268.6103 - mean_squared_error: 888268.6103 - val_loss: 649560.8445 - val_mean_squared_error: 649560.8445
Epoch 157/2000
 - 0s - loss: 887101.2549 - mean_squared_error: 887101.2549 - val_loss: 649862.3474 - val_mean_squared_error: 649862.3474
Epoch 158/2000
 - 0s - loss: 884886.7809 - mean_squared_error: 884886.7809 - val_loss: 642837.5409 - val_mean_squared_error: 642837.5409
Epoch 159/2000
 - 0s - loss: 883092.6271 - mean_squared_error: 883092.6271 - val_loss: 645735.0666 - val_mean_squared_error: 645735.0666
Epoch 160/2000
 - 0s - loss: 881120.0721 - mean_squared_error: 881120.0721 - val_loss: 642229.4987 - val_mean_squared_error: 642229.4987
Epoch 161/2000
 - 0s - loss: 879317.6468 - mean_squared_error: 879317.6468 - val_loss: 649237.6002 - val_mean_squared_error: 649237.6002
Epoch 162/2000
 - 0s - loss: 877933.4986 - mean_squared_error: 877933.4986 - val_loss: 645492.1847 - val_mean_squared_error: 645492.1847
Epoch 163/2000
 - 0s - loss: 876311.9927 - mean_squared_error: 876311.9927 - val_loss: 638299.4104 - val_mean_squared_error: 638299.4104
Epoch 164/2000
 - 0s - loss: 874368.5659 - mean_squared_error: 874368.5659 - val_loss: 639309.5914 - val_mean_squared_error: 639309.5914
Epoch 165/2000
 - 0s - loss: 871988.8340 - mean_squared_error: 871988.8340 - val_loss: 636048.8807 - val_mean_squared_error: 636048.8807
Epoch 166/2000
 - 0s - loss: 871099.2410 - mean_squared_error: 871099.2410 - val_loss: 633811.7354 - val_mean_squared_error: 633811.7354
Epoch 167/2000
 - 0s - loss: 869259.6353 - mean_squared_error: 869259.6353 - val_loss: 631700.6498 - val_mean_squared_error: 631700.6498
Epoch 168/2000
 - 0s - loss: 867846.2387 - mean_squared_error: 867846.2387 - val_loss: 631306.9374 - val_mean_squared_error: 631306.9374
Epoch 169/2000
 - 0s - loss: 867124.9870 - mean_squared_error: 867124.9870 - val_loss: 630908.5872 - val_mean_squared_error: 630908.5872
Epoch 170/2000
 - 0s - loss: 865043.7359 - mean_squared_error: 865043.7359 - val_loss: 628584.2976 - val_mean_squared_error: 628584.2976
Epoch 171/2000
 - 0s - loss: 862503.4103 - mean_squared_error: 862503.4103 - val_loss: 636332.0420 - val_mean_squared_error: 636332.0420
Epoch 172/2000
 - 0s - loss: 861153.7802 - mean_squared_error: 861153.7802 - val_loss: 627266.2563 - val_mean_squared_error: 627266.2563
Epoch 173/2000
 - 0s - loss: 859494.3078 - mean_squared_error: 859494.3078 - val_loss: 624305.4543 - val_mean_squared_error: 624305.4543
Epoch 174/2000
 - 0s - loss: 858547.6955 - mean_squared_error: 858547.6955 - val_loss: 630155.4827 - val_mean_squared_error: 630155.4827
Epoch 175/2000
 - 0s - loss: 856202.1055 - mean_squared_error: 856202.1055 - val_loss: 623894.7843 - val_mean_squared_error: 623894.7843
Epoch 176/2000
 - 0s - loss: 854444.2765 - mean_squared_error: 854444.2765 - val_loss: 623929.4103 - val_mean_squared_error: 623929.4103
Epoch 177/2000
 - 0s - loss: 853613.5416 - mean_squared_error: 853613.5416 - val_loss: 622482.8054 - val_mean_squared_error: 622482.8054
Epoch 178/2000
 - 0s - loss: 852153.3314 - mean_squared_error: 852153.3314 - val_loss: 621689.6013 - val_mean_squared_error: 621689.6013
Epoch 179/2000
 - 0s - loss: 849991.4039 - mean_squared_error: 849991.4039 - val_loss: 618582.9728 - val_mean_squared_error: 618582.9728
Epoch 180/2000
 - 0s - loss: 848670.8722 - mean_squared_error: 848670.8722 - val_loss: 617514.6323 - val_mean_squared_error: 617514.6323

*** WARNING: skipped 225777 bytes of output ***

Epoch 1823/2000
 - 0s - loss: 352445.9751 - mean_squared_error: 352445.9751 - val_loss: 322069.3224 - val_mean_squared_error: 322069.3224
Epoch 1824/2000
 - 0s - loss: 352472.1113 - mean_squared_error: 352472.1113 - val_loss: 322258.6551 - val_mean_squared_error: 322258.6551
Epoch 1825/2000
 - 0s - loss: 352828.0406 - mean_squared_error: 352828.0406 - val_loss: 322122.4397 - val_mean_squared_error: 322122.4397
Epoch 1826/2000
 - 0s - loss: 352703.0477 - mean_squared_error: 352703.0477 - val_loss: 323740.2058 - val_mean_squared_error: 323740.2058
Epoch 1827/2000
 - 0s - loss: 352640.1232 - mean_squared_error: 352640.1232 - val_loss: 321345.6100 - val_mean_squared_error: 321345.6100
Epoch 1828/2000
 - 0s - loss: 352718.0216 - mean_squared_error: 352718.0216 - val_loss: 322241.4297 - val_mean_squared_error: 322241.4297
Epoch 1829/2000
 - 0s - loss: 353036.8003 - mean_squared_error: 353036.8003 - val_loss: 321266.4523 - val_mean_squared_error: 321266.4523
Epoch 1830/2000
 - 0s - loss: 352526.1196 - mean_squared_error: 352526.1196 - val_loss: 322149.5655 - val_mean_squared_error: 322149.5655
Epoch 1831/2000
 - 0s - loss: 352638.5939 - mean_squared_error: 352638.5939 - val_loss: 322820.1009 - val_mean_squared_error: 322820.1009
Epoch 1832/2000
 - 0s - loss: 352792.9434 - mean_squared_error: 352792.9434 - val_loss: 321974.9352 - val_mean_squared_error: 321974.9352
Epoch 1833/2000
 - 0s - loss: 352735.2175 - mean_squared_error: 352735.2175 - val_loss: 321391.9115 - val_mean_squared_error: 321391.9115
Epoch 1834/2000
 - 0s - loss: 351935.7235 - mean_squared_error: 351935.7235 - val_loss: 323753.1287 - val_mean_squared_error: 323753.1287
Epoch 1835/2000
 - 0s - loss: 352531.7978 - mean_squared_error: 352531.7978 - val_loss: 322526.4762 - val_mean_squared_error: 322526.4762
Epoch 1836/2000
 - 0s - loss: 352604.3952 - mean_squared_error: 352604.3952 - val_loss: 321563.6542 - val_mean_squared_error: 321563.6542
Epoch 1837/2000
 - 0s - loss: 352392.8832 - mean_squared_error: 352392.8832 - val_loss: 321702.4486 - val_mean_squared_error: 321702.4486
Epoch 1838/2000
 - 0s - loss: 352626.8102 - mean_squared_error: 352626.8102 - val_loss: 321687.4584 - val_mean_squared_error: 321687.4584
Epoch 1839/2000
 - 0s - loss: 352328.3696 - mean_squared_error: 352328.3696 - val_loss: 322284.7248 - val_mean_squared_error: 322284.7248
Epoch 1840/2000
 - 0s - loss: 352630.2977 - mean_squared_error: 352630.2977 - val_loss: 321221.5514 - val_mean_squared_error: 321221.5514
Epoch 1841/2000
 - 0s - loss: 352444.8096 - mean_squared_error: 352444.8096 - val_loss: 321824.6473 - val_mean_squared_error: 321824.6473
Epoch 1842/2000
 - 0s - loss: 352482.0319 - mean_squared_error: 352482.0319 - val_loss: 321641.6989 - val_mean_squared_error: 321641.6989
Epoch 1843/2000
 - 0s - loss: 352372.8971 - mean_squared_error: 352372.8971 - val_loss: 321319.8558 - val_mean_squared_error: 321319.8558
Epoch 1844/2000
 - 0s - loss: 352389.4766 - mean_squared_error: 352389.4766 - val_loss: 323707.5949 - val_mean_squared_error: 323707.5949
Epoch 1845/2000
 - 0s - loss: 352781.0235 - mean_squared_error: 352781.0235 - val_loss: 321291.5840 - val_mean_squared_error: 321291.5840
Epoch 1846/2000
 - 0s - loss: 352106.0627 - mean_squared_error: 352106.0627 - val_loss: 321071.3208 - val_mean_squared_error: 321071.3208
Epoch 1847/2000
 - 0s - loss: 351785.5042 - mean_squared_error: 351785.5042 - val_loss: 321158.4504 - val_mean_squared_error: 321158.4504
Epoch 1848/2000
 - 0s - loss: 352376.8591 - mean_squared_error: 352376.8591 - val_loss: 321186.0486 - val_mean_squared_error: 321186.0486
Epoch 1849/2000
 - 0s - loss: 352178.2893 - mean_squared_error: 352178.2893 - val_loss: 323155.9485 - val_mean_squared_error: 323155.9485
Epoch 1850/2000
 - 0s - loss: 352153.0114 - mean_squared_error: 352153.0114 - val_loss: 321153.2629 - val_mean_squared_error: 321153.2629
Epoch 1851/2000
 - 0s - loss: 352782.0023 - mean_squared_error: 352782.0023 - val_loss: 321800.5722 - val_mean_squared_error: 321800.5722
Epoch 1852/2000
 - 0s - loss: 352016.6540 - mean_squared_error: 352016.6540 - val_loss: 322145.6720 - val_mean_squared_error: 322145.6720
Epoch 1853/2000
 - 0s - loss: 352021.9134 - mean_squared_error: 352021.9134 - val_loss: 321694.4788 - val_mean_squared_error: 321694.4788
Epoch 1854/2000
 - 0s - loss: 352119.9222 - mean_squared_error: 352119.9222 - val_loss: 321353.8271 - val_mean_squared_error: 321353.8271
Epoch 1855/2000
 - 0s - loss: 352235.1926 - mean_squared_error: 352235.1926 - val_loss: 321369.2222 - val_mean_squared_error: 321369.2222
Epoch 1856/2000
 - 0s - loss: 352193.0640 - mean_squared_error: 352193.0640 - val_loss: 322583.4591 - val_mean_squared_error: 322583.4591
Epoch 1857/2000
 - 0s - loss: 352017.5066 - mean_squared_error: 352017.5066 - val_loss: 322900.6989 - val_mean_squared_error: 322900.6989
Epoch 1858/2000
 - 0s - loss: 352078.2558 - mean_squared_error: 352078.2558 - val_loss: 321475.4010 - val_mean_squared_error: 321475.4010
Epoch 1859/2000
 - 0s - loss: 352198.4636 - mean_squared_error: 352198.4636 - val_loss: 321521.3116 - val_mean_squared_error: 321521.3116
Epoch 1860/2000
 - 0s - loss: 352062.0915 - mean_squared_error: 352062.0915 - val_loss: 321479.3511 - val_mean_squared_error: 321479.3511
Epoch 1861/2000
 - 0s - loss: 352102.0045 - mean_squared_error: 352102.0045 - val_loss: 322015.3600 - val_mean_squared_error: 322015.3600
Epoch 1862/2000
 - 0s - loss: 352234.2320 - mean_squared_error: 352234.2320 - val_loss: 322595.9791 - val_mean_squared_error: 322595.9791
Epoch 1863/2000
 - 0s - loss: 352006.5176 - mean_squared_error: 352006.5176 - val_loss: 321737.5715 - val_mean_squared_error: 321737.5715
Epoch 1864/2000
 - 0s - loss: 352435.9867 - mean_squared_error: 352435.9867 - val_loss: 320787.8923 - val_mean_squared_error: 320787.8923
Epoch 1865/2000
 - 0s - loss: 352326.2756 - mean_squared_error: 352326.2756 - val_loss: 322110.2077 - val_mean_squared_error: 322110.2077
Epoch 1866/2000
 - 0s - loss: 352004.5124 - mean_squared_error: 352004.5124 - val_loss: 321217.3982 - val_mean_squared_error: 321217.3982
Epoch 1867/2000
 - 0s - loss: 352266.8707 - mean_squared_error: 352266.8707 - val_loss: 321929.5202 - val_mean_squared_error: 321929.5202
Epoch 1868/2000
 - 0s - loss: 352129.3507 - mean_squared_error: 352129.3507 - val_loss: 320808.0614 - val_mean_squared_error: 320808.0614
Epoch 1869/2000
 - 0s - loss: 352086.6336 - mean_squared_error: 352086.6336 - val_loss: 323234.1755 - val_mean_squared_error: 323234.1755
Epoch 1870/2000
 - 0s - loss: 352069.6200 - mean_squared_error: 352069.6200 - val_loss: 321823.5260 - val_mean_squared_error: 321823.5260
Epoch 1871/2000
 - 0s - loss: 351834.2334 - mean_squared_error: 351834.2334 - val_loss: 321119.7924 - val_mean_squared_error: 321119.7924
Epoch 1872/2000
 - 0s - loss: 351934.5263 - mean_squared_error: 351934.5263 - val_loss: 320743.5473 - val_mean_squared_error: 320743.5473
Epoch 1873/2000
 - 0s - loss: 351754.7197 - mean_squared_error: 351754.7197 - val_loss: 320728.0788 - val_mean_squared_error: 320728.0788
Epoch 1874/2000
 - 0s - loss: 351686.1255 - mean_squared_error: 351686.1255 - val_loss: 325863.9035 - val_mean_squared_error: 325863.9035
Epoch 1875/2000
 - 0s - loss: 352027.5565 - mean_squared_error: 352027.5565 - val_loss: 322377.7509 - val_mean_squared_error: 322377.7509
Epoch 1876/2000
 - 0s - loss: 351773.6168 - mean_squared_error: 351773.6168 - val_loss: 321306.0384 - val_mean_squared_error: 321306.0384
Epoch 1877/2000
 - 0s - loss: 351731.8323 - mean_squared_error: 351731.8323 - val_loss: 322280.8299 - val_mean_squared_error: 322280.8299
Epoch 1878/2000
 - 0s - loss: 351939.9910 - mean_squared_error: 351939.9910 - val_loss: 320719.5448 - val_mean_squared_error: 320719.5448
Epoch 1879/2000
 - 0s - loss: 351811.5054 - mean_squared_error: 351811.5054 - val_loss: 322086.1814 - val_mean_squared_error: 322086.1814
Epoch 1880/2000
 - 0s - loss: 351969.0682 - mean_squared_error: 351969.0682 - val_loss: 323696.7555 - val_mean_squared_error: 323696.7555
Epoch 1881/2000
 - 0s - loss: 351740.8710 - mean_squared_error: 351740.8710 - val_loss: 321175.8996 - val_mean_squared_error: 321175.8996
Epoch 1882/2000
 - 0s - loss: 351783.0421 - mean_squared_error: 351783.0421 - val_loss: 321210.5479 - val_mean_squared_error: 321210.5479
Epoch 1883/2000
 - 0s - loss: 351670.6111 - mean_squared_error: 351670.6111 - val_loss: 324955.2986 - val_mean_squared_error: 324955.2986
Epoch 1884/2000
 - 0s - loss: 351904.2298 - mean_squared_error: 351904.2298 - val_loss: 320764.6226 - val_mean_squared_error: 320764.6226
Epoch 1885/2000
 - 0s - loss: 351816.9915 - mean_squared_error: 351816.9915 - val_loss: 320791.8610 - val_mean_squared_error: 320791.8610
Epoch 1886/2000
 - 0s - loss: 351615.2335 - mean_squared_error: 351615.2335 - val_loss: 320851.1270 - val_mean_squared_error: 320851.1270
Epoch 1887/2000
 - 0s - loss: 351285.4173 - mean_squared_error: 351285.4173 - val_loss: 322818.1600 - val_mean_squared_error: 322818.1600
Epoch 1888/2000
 - 0s - loss: 352008.8270 - mean_squared_error: 352008.8270 - val_loss: 323102.7775 - val_mean_squared_error: 323102.7775
Epoch 1889/2000
 - 0s - loss: 351963.0233 - mean_squared_error: 351963.0233 - val_loss: 321013.9407 - val_mean_squared_error: 321013.9407
Epoch 1890/2000
 - 0s - loss: 351481.8547 - mean_squared_error: 351481.8547 - val_loss: 321091.4695 - val_mean_squared_error: 321091.4695
Epoch 1891/2000
 - 0s - loss: 351505.0111 - mean_squared_error: 351505.0111 - val_loss: 321116.0141 - val_mean_squared_error: 321116.0141
Epoch 1892/2000
 - 0s - loss: 351433.4253 - mean_squared_error: 351433.4253 - val_loss: 321061.0411 - val_mean_squared_error: 321061.0411
Epoch 1893/2000
 - 0s - loss: 351122.9481 - mean_squared_error: 351122.9481 - val_loss: 320691.0843 - val_mean_squared_error: 320691.0843
Epoch 1894/2000
 - 0s - loss: 351521.8679 - mean_squared_error: 351521.8679 - val_loss: 324941.1356 - val_mean_squared_error: 324941.1356
Epoch 1895/2000
 - 0s - loss: 351561.7527 - mean_squared_error: 351561.7527 - val_loss: 321182.1784 - val_mean_squared_error: 321182.1784
Epoch 1896/2000
 - 0s - loss: 351470.0308 - mean_squared_error: 351470.0308 - val_loss: 323297.2951 - val_mean_squared_error: 323297.2951
Epoch 1897/2000
 - 0s - loss: 351745.6374 - mean_squared_error: 351745.6374 - val_loss: 320517.4713 - val_mean_squared_error: 320517.4713
Epoch 1898/2000
 - 0s - loss: 351691.2853 - mean_squared_error: 351691.2853 - val_loss: 320498.0968 - val_mean_squared_error: 320498.0968
Epoch 1899/2000
 - 0s - loss: 351567.3757 - mean_squared_error: 351567.3757 - val_loss: 320597.0670 - val_mean_squared_error: 320597.0670
Epoch 1900/2000
 - 0s - loss: 351300.7174 - mean_squared_error: 351300.7174 - val_loss: 324114.6680 - val_mean_squared_error: 324114.6680
Epoch 1901/2000
 - 0s - loss: 351173.0977 - mean_squared_error: 351173.0977 - val_loss: 321130.5498 - val_mean_squared_error: 321130.5498
Epoch 1902/2000
 - 0s - loss: 351405.3581 - mean_squared_error: 351405.3581 - val_loss: 321523.2287 - val_mean_squared_error: 321523.2287
Epoch 1903/2000
 - 0s - loss: 351206.5453 - mean_squared_error: 351206.5453 - val_loss: 320541.2431 - val_mean_squared_error: 320541.2431
Epoch 1904/2000
 - 0s - loss: 351668.0912 - mean_squared_error: 351668.0912 - val_loss: 320546.0671 - val_mean_squared_error: 320546.0671
Epoch 1905/2000
 - 0s - loss: 351069.9819 - mean_squared_error: 351069.9819 - val_loss: 320459.3314 - val_mean_squared_error: 320459.3314
Epoch 1906/2000
 - 0s - loss: 351435.6701 - mean_squared_error: 351435.6701 - val_loss: 321989.0115 - val_mean_squared_error: 321989.0115
Epoch 1907/2000
 - 0s - loss: 351298.0716 - mean_squared_error: 351298.0716 - val_loss: 322274.4590 - val_mean_squared_error: 322274.4590
Epoch 1908/2000
 - 0s - loss: 350774.8583 - mean_squared_error: 350774.8583 - val_loss: 320461.4206 - val_mean_squared_error: 320461.4206
Epoch 1909/2000
 - 0s - loss: 351546.8358 - mean_squared_error: 351546.8358 - val_loss: 322772.2442 - val_mean_squared_error: 322772.2442
Epoch 1910/2000
 - 0s - loss: 351531.8571 - mean_squared_error: 351531.8571 - val_loss: 320255.2415 - val_mean_squared_error: 320255.2415
Epoch 1911/2000
 - 0s - loss: 351123.9902 - mean_squared_error: 351123.9902 - val_loss: 323927.2046 - val_mean_squared_error: 323927.2046
Epoch 1912/2000
 - 0s - loss: 351175.4219 - mean_squared_error: 351175.4219 - val_loss: 322185.1594 - val_mean_squared_error: 322185.1594
Epoch 1913/2000
 - 0s - loss: 351124.5860 - mean_squared_error: 351124.5860 - val_loss: 320269.5490 - val_mean_squared_error: 320269.5490
Epoch 1914/2000
 - 0s - loss: 351164.6861 - mean_squared_error: 351164.6861 - val_loss: 320922.3566 - val_mean_squared_error: 320922.3566
Epoch 1915/2000
 - 0s - loss: 351430.4632 - mean_squared_error: 351430.4632 - val_loss: 321206.3645 - val_mean_squared_error: 321206.3645
Epoch 1916/2000
 - 0s - loss: 351039.9810 - mean_squared_error: 351039.9810 - val_loss: 322146.4838 - val_mean_squared_error: 322146.4838
Epoch 1917/2000
 - 0s - loss: 351092.1443 - mean_squared_error: 351092.1443 - val_loss: 320477.6637 - val_mean_squared_error: 320477.6637
Epoch 1918/2000
 - 0s - loss: 351279.6311 - mean_squared_error: 351279.6311 - val_loss: 322849.2993 - val_mean_squared_error: 322849.2993
Epoch 1919/2000
 - 0s - loss: 351378.9222 - mean_squared_error: 351378.9222 - val_loss: 320415.2793 - val_mean_squared_error: 320415.2793
Epoch 1920/2000
 - 0s - loss: 351077.6502 - mean_squared_error: 351077.6502 - val_loss: 320414.5563 - val_mean_squared_error: 320414.5563
Epoch 1921/2000
 - 0s - loss: 351523.3945 - mean_squared_error: 351523.3945 - val_loss: 321847.9682 - val_mean_squared_error: 321847.9682
Epoch 1922/2000
 - 0s - loss: 351072.6680 - mean_squared_error: 351072.6680 - val_loss: 321248.8032 - val_mean_squared_error: 321248.8032
Epoch 1923/2000
 - 0s - loss: 351459.1636 - mean_squared_error: 351459.1636 - val_loss: 321194.2757 - val_mean_squared_error: 321194.2757
Epoch 1924/2000
 - 0s - loss: 350877.2161 - mean_squared_error: 350877.2161 - val_loss: 320426.4567 - val_mean_squared_error: 320426.4567
Epoch 1925/2000
 - 0s - loss: 350936.2435 - mean_squared_error: 350936.2435 - val_loss: 322513.9072 - val_mean_squared_error: 322513.9072
Epoch 1926/2000
 - 0s - loss: 351073.9743 - mean_squared_error: 351073.9743 - val_loss: 320405.1168 - val_mean_squared_error: 320405.1168
Epoch 1927/2000
 - 0s - loss: 351355.3434 - mean_squared_error: 351355.3434 - val_loss: 320018.8288 - val_mean_squared_error: 320018.8288
Epoch 1928/2000
 - 0s - loss: 351079.9940 - mean_squared_error: 351079.9940 - val_loss: 320791.3080 - val_mean_squared_error: 320791.3080
Epoch 1929/2000
 - 0s - loss: 351064.3151 - mean_squared_error: 351064.3151 - val_loss: 320712.5592 - val_mean_squared_error: 320712.5592
Epoch 1930/2000
 - 0s - loss: 351300.7137 - mean_squared_error: 351300.7137 - val_loss: 321289.1198 - val_mean_squared_error: 321289.1198
Epoch 1931/2000
 - 0s - loss: 351233.0456 - mean_squared_error: 351233.0456 - val_loss: 322679.0501 - val_mean_squared_error: 322679.0501
Epoch 1932/2000
 - 0s - loss: 350893.3236 - mean_squared_error: 350893.3236 - val_loss: 320317.3456 - val_mean_squared_error: 320317.3456
Epoch 1933/2000
 - 0s - loss: 351329.4404 - mean_squared_error: 351329.4404 - val_loss: 319852.4713 - val_mean_squared_error: 319852.4713
Epoch 1934/2000
 - 0s - loss: 351124.6956 - mean_squared_error: 351124.6956 - val_loss: 320436.1364 - val_mean_squared_error: 320436.1364
Epoch 1935/2000
 - 0s - loss: 351053.7903 - mean_squared_error: 351053.7903 - val_loss: 320834.8588 - val_mean_squared_error: 320834.8588
Epoch 1936/2000
 - 0s - loss: 350984.9909 - mean_squared_error: 350984.9909 - val_loss: 321249.0250 - val_mean_squared_error: 321249.0250
Epoch 1937/2000
 - 0s - loss: 350841.7996 - mean_squared_error: 350841.7996 - val_loss: 320661.3310 - val_mean_squared_error: 320661.3310
Epoch 1938/2000
 - 0s - loss: 350720.7031 - mean_squared_error: 350720.7031 - val_loss: 320589.8654 - val_mean_squared_error: 320589.8654
Epoch 1939/2000
 - 0s - loss: 350983.4347 - mean_squared_error: 350983.4347 - val_loss: 320438.1442 - val_mean_squared_error: 320438.1442
Epoch 1940/2000
 - 0s - loss: 350860.4392 - mean_squared_error: 350860.4392 - val_loss: 321121.9078 - val_mean_squared_error: 321121.9078
Epoch 1941/2000
 - 0s - loss: 351001.5372 - mean_squared_error: 351001.5372 - val_loss: 320003.6929 - val_mean_squared_error: 320003.6929
Epoch 1942/2000
 - 0s - loss: 350936.1459 - mean_squared_error: 350936.1459 - val_loss: 320647.4402 - val_mean_squared_error: 320647.4402
Epoch 1943/2000
 - 0s - loss: 350834.0426 - mean_squared_error: 350834.0426 - val_loss: 320384.9495 - val_mean_squared_error: 320384.9495
Epoch 1944/2000
 - 0s - loss: 351162.9750 - mean_squared_error: 351162.9750 - val_loss: 322697.6413 - val_mean_squared_error: 322697.6413
Epoch 1945/2000
 - 0s - loss: 351042.0928 - mean_squared_error: 351042.0928 - val_loss: 320229.0246 - val_mean_squared_error: 320229.0246
Epoch 1946/2000
 - 1s - loss: 350967.1519 - mean_squared_error: 350967.1519 - val_loss: 321109.6796 - val_mean_squared_error: 321109.6796
Epoch 1947/2000
 - 1s - loss: 351546.0674 - mean_squared_error: 351546.0674 - val_loss: 319761.1340 - val_mean_squared_error: 319761.1340
Epoch 1948/2000
 - 0s - loss: 351032.9363 - mean_squared_error: 351032.9363 - val_loss: 321014.5154 - val_mean_squared_error: 321014.5154
Epoch 1949/2000
 - 1s - loss: 350831.0772 - mean_squared_error: 350831.0772 - val_loss: 320220.1645 - val_mean_squared_error: 320220.1645
Epoch 1950/2000
 - 0s - loss: 350525.6807 - mean_squared_error: 350525.6807 - val_loss: 321441.4327 - val_mean_squared_error: 321441.4327
Epoch 1951/2000
 - 0s - loss: 350881.6454 - mean_squared_error: 350881.6454 - val_loss: 320592.7997 - val_mean_squared_error: 320592.7997
Epoch 1952/2000
 - 0s - loss: 350383.6098 - mean_squared_error: 350383.6098 - val_loss: 322400.0443 - val_mean_squared_error: 322400.0443
Epoch 1953/2000
 - 0s - loss: 350707.1438 - mean_squared_error: 350707.1438 - val_loss: 319757.0388 - val_mean_squared_error: 319757.0388
Epoch 1954/2000
 - 0s - loss: 350510.8410 - mean_squared_error: 350510.8410 - val_loss: 321124.5681 - val_mean_squared_error: 321124.5681
Epoch 1955/2000
 - 0s - loss: 350382.6248 - mean_squared_error: 350382.6248 - val_loss: 321917.0947 - val_mean_squared_error: 321917.0947
Epoch 1956/2000
 - 0s - loss: 350494.4437 - mean_squared_error: 350494.4437 - val_loss: 323771.4122 - val_mean_squared_error: 323771.4122
Epoch 1957/2000
 - 0s - loss: 350762.2462 - mean_squared_error: 350762.2462 - val_loss: 319887.6764 - val_mean_squared_error: 319887.6764
Epoch 1958/2000
 - 0s - loss: 350812.8616 - mean_squared_error: 350812.8616 - val_loss: 319785.3546 - val_mean_squared_error: 319785.3546
Epoch 1959/2000
 - 0s - loss: 350633.5645 - mean_squared_error: 350633.5645 - val_loss: 320639.5500 - val_mean_squared_error: 320639.5500
Epoch 1960/2000
 - 0s - loss: 350431.9455 - mean_squared_error: 350431.9455 - val_loss: 320044.3539 - val_mean_squared_error: 320044.3539
Epoch 1961/2000
 - 0s - loss: 350607.4654 - mean_squared_error: 350607.4654 - val_loss: 320627.9426 - val_mean_squared_error: 320627.9426
Epoch 1962/2000
 - 0s - loss: 350769.9184 - mean_squared_error: 350769.9184 - val_loss: 320445.9118 - val_mean_squared_error: 320445.9118
Epoch 1963/2000
 - 0s - loss: 350721.9007 - mean_squared_error: 350721.9007 - val_loss: 320749.6331 - val_mean_squared_error: 320749.6331
Epoch 1964/2000
 - 0s - loss: 350791.8435 - mean_squared_error: 350791.8435 - val_loss: 320953.2111 - val_mean_squared_error: 320953.2111
Epoch 1965/2000
 - 0s - loss: 350626.2471 - mean_squared_error: 350626.2471 - val_loss: 320379.1204 - val_mean_squared_error: 320379.1204
Epoch 1966/2000
 - 0s - loss: 350645.8384 - mean_squared_error: 350645.8384 - val_loss: 321258.9025 - val_mean_squared_error: 321258.9025
Epoch 1967/2000
 - 0s - loss: 351144.3000 - mean_squared_error: 351144.3000 - val_loss: 319555.7377 - val_mean_squared_error: 319555.7377
Epoch 1968/2000
 - 0s - loss: 350257.5325 - mean_squared_error: 350257.5325 - val_loss: 320498.8982 - val_mean_squared_error: 320498.8982
Epoch 1969/2000
 - 0s - loss: 350440.9468 - mean_squared_error: 350440.9468 - val_loss: 323353.8991 - val_mean_squared_error: 323353.8991
Epoch 1970/2000
 - 0s - loss: 350514.8672 - mean_squared_error: 350514.8672 - val_loss: 319926.7191 - val_mean_squared_error: 319926.7191
Epoch 1971/2000
 - 0s - loss: 350476.5234 - mean_squared_error: 350476.5234 - val_loss: 320403.7104 - val_mean_squared_error: 320403.7104
Epoch 1972/2000
 - 0s - loss: 350614.9112 - mean_squared_error: 350614.9112 - val_loss: 320656.1458 - val_mean_squared_error: 320656.1458
Epoch 1973/2000
 - 0s - loss: 350340.0376 - mean_squared_error: 350340.0376 - val_loss: 323562.8329 - val_mean_squared_error: 323562.8329
Epoch 1974/2000
 - 0s - loss: 350855.2276 - mean_squared_error: 350855.2276 - val_loss: 319600.7327 - val_mean_squared_error: 319600.7327
Epoch 1975/2000
 - 0s - loss: 350561.9088 - mean_squared_error: 350561.9088 - val_loss: 321137.1200 - val_mean_squared_error: 321137.1200
Epoch 1976/2000
 - 0s - loss: 350554.1135 - mean_squared_error: 350554.1135 - val_loss: 319576.2050 - val_mean_squared_error: 319576.2050
Epoch 1977/2000
 - 0s - loss: 350576.2284 - mean_squared_error: 350576.2284 - val_loss: 319929.7035 - val_mean_squared_error: 319929.7035
Epoch 1978/2000
 - 0s - loss: 350541.0335 - mean_squared_error: 350541.0335 - val_loss: 319699.2241 - val_mean_squared_error: 319699.2241
Epoch 1979/2000
 - 0s - loss: 350542.8354 - mean_squared_error: 350542.8354 - val_loss: 320522.0850 - val_mean_squared_error: 320522.0850
Epoch 1980/2000
 - 0s - loss: 350310.6930 - mean_squared_error: 350310.6930 - val_loss: 321860.9097 - val_mean_squared_error: 321860.9097
Epoch 1981/2000
 - 0s - loss: 350367.3628 - mean_squared_error: 350367.3628 - val_loss: 322932.4098 - val_mean_squared_error: 322932.4098
Epoch 1982/2000
 - 0s - loss: 350613.5599 - mean_squared_error: 350613.5599 - val_loss: 320085.8123 - val_mean_squared_error: 320085.8123
Epoch 1983/2000
 - 0s - loss: 350550.4906 - mean_squared_error: 350550.4906 - val_loss: 321715.1233 - val_mean_squared_error: 321715.1233
Epoch 1984/2000
 - 0s - loss: 350098.3904 - mean_squared_error: 350098.3904 - val_loss: 319653.5809 - val_mean_squared_error: 319653.5809
Epoch 1985/2000
 - 0s - loss: 350215.2669 - mean_squared_error: 350215.2669 - val_loss: 319889.6822 - val_mean_squared_error: 319889.6822
Epoch 1986/2000
 - 0s - loss: 350394.0994 - mean_squared_error: 350394.0994 - val_loss: 320383.7723 - val_mean_squared_error: 320383.7723
Epoch 1987/2000
 - 0s - loss: 350067.7060 - mean_squared_error: 350067.7060 - val_loss: 320083.8967 - val_mean_squared_error: 320083.8967
Epoch 1988/2000
 - 0s - loss: 349942.0462 - mean_squared_error: 349942.0462 - val_loss: 320788.4456 - val_mean_squared_error: 320788.4456
Epoch 1989/2000
 - 0s - loss: 350594.1286 - mean_squared_error: 350594.1286 - val_loss: 320672.2185 - val_mean_squared_error: 320672.2185
Epoch 1990/2000
 - 0s - loss: 350586.9137 - mean_squared_error: 350586.9137 - val_loss: 320443.1261 - val_mean_squared_error: 320443.1261
Epoch 1991/2000
 - 0s - loss: 350447.7632 - mean_squared_error: 350447.7632 - val_loss: 319521.4387 - val_mean_squared_error: 319521.4387
Epoch 1992/2000
 - 0s - loss: 350198.1232 - mean_squared_error: 350198.1232 - val_loss: 319705.7904 - val_mean_squared_error: 319705.7904
Epoch 1993/2000
 - 0s - loss: 350177.6373 - mean_squared_error: 350177.6373 - val_loss: 319399.6656 - val_mean_squared_error: 319399.6656
Epoch 1994/2000
 - 0s - loss: 349822.2746 - mean_squared_error: 349822.2746 - val_loss: 319906.7587 - val_mean_squared_error: 319906.7587
Epoch 1995/2000
 - 0s - loss: 349979.3041 - mean_squared_error: 349979.3041 - val_loss: 320456.3252 - val_mean_squared_error: 320456.3252
Epoch 1996/2000
 - 0s - loss: 349674.5740 - mean_squared_error: 349674.5740 - val_loss: 320019.5364 - val_mean_squared_error: 320019.5364
Epoch 1997/2000
 - 0s - loss: 350329.6704 - mean_squared_error: 350329.6704 - val_loss: 321159.2947 - val_mean_squared_error: 321159.2947
Epoch 1998/2000
 - 0s - loss: 350249.9685 - mean_squared_error: 350249.9685 - val_loss: 320483.8665 - val_mean_squared_error: 320483.8665
Epoch 1999/2000
 - 0s - loss: 349857.2837 - mean_squared_error: 349857.2837 - val_loss: 319652.7423 - val_mean_squared_error: 319652.7423
Epoch 2000/2000
 - 0s - loss: 350730.7542 - mean_squared_error: 350730.7542 - val_loss: 319270.1511 - val_mean_squared_error: 319270.1511

   32/13485 [..............................] - ETA: 0s
 3840/13485 [=======>......................] - ETA: 0s
 7584/13485 [===============>..............] - ETA: 0s
11392/13485 [========================>.....] - ETA: 0s
13485/13485 [==============================] - 0s 13us/step

root mean_squared_error: 595.088727

Would you look at that?!

We break $1000 RMSE around epoch 112
$900 around epoch 220
$800 around epoch 450
By around epoch 2000, my RMSE is < $600

...

Same theory; different activation function. Huge difference

Multilayer Networks

If a single-layer perceptron network learns the importance of different combinations of features in the data...

What would another network learn if it had a second (hidden) layer of neurons?

It depends on how we train the network. We'll talk in the next section about how this training works, but the general idea is that we still work backward from the error gradient.

That is, the last layer learns from error in the output; the second-to-last layer learns from error transmitted through that last layer, etc. It's a touch hand-wavy for now, but we'll make it more concrete later.

Given this approach, we can say that:

The second (hidden) layer is learning features composed of activations in the first (hidden) layer
The first (hidden) layer is learning feature weights that enable the second layer to perform best
- Why? Earlier, the first hidden layer just learned feature weights because that's how it was judged
- Now, the first hidden layer is judged on the error in the second layer, so it learns to contribute to that second layer
The second layer is learning new features that aren't explicit in the data, and is teaching the first layer to supply it with the necessary information to compose these new features

So instead of just feature weighting and combining, we have new feature learning!

This concept is the foundation of the "Deep Feed-Forward Network"

Let's try it!

Add a layer to your Keras network, perhaps another 20 neurons, and see how the training goes.

if you get stuck, there is a solution in the Keras-DFFN notebook

I'm getting RMSE < $1000 by epoch 35 or so

< $800 by epoch 90

In this configuration, mine makes progress to around 700 epochs or so and then stalls with RMSE around $560

Our network has "gone meta"

It's now able to exceed where a simple decision tree can go, because it can create new features and then split on those

Congrats! You have built your first deep-learning model!

So does that mean we can just keep adding more layers and solve anything?

Well, theoretically maybe ... try reconfiguring your network, watch the training, and see what happens.

sds-3.x/ScaDaMaLe

Keras is a High-Level API for Neural Networks and Deep Learning