03_streamable_trend_calculus_estimators

ScaDaMaLe Course site and book

Markov Model for Trend Calculus

Johannes Graner (LinkedIn), Albert Nilsson (LinkedIn) and Raazesh Sainudiin (LinkedIn)

2020, Uppsala, Sweden

This project was supported by Combient Mix AB through summer internships at:

Combient Competence Centre for Data Engineering Sciences, Department of Mathematics, Uppsala University, Uppsala, Sweden

We use the dataset generated in the last notebook to build a simple, proof of concept Markov model for predicting trends.

This is merely to verify using a simple model if there is indeed any predictive value in the trends and their reversals that many real-worl traders bet on every instant in the market.

Resources

This builds on the following library and its antecedents therein:

https://github.com/lamastex/spark-trend-calculus

This work was inspired by:

Antoine Aamennd's texata-2017
Andrew Morgan's Trend Calculus Library


"./000a_finance_utils"


import java.sql.Timestamp
import io.delta.tables._
import org.apache.spark.sql._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.streaming.{GroupState, GroupStateTimeout, OutputMode, Trigger}
import org.apache.spark.sql.types._
import org.apache.spark.sql.expressions.{Window, WindowSpec}
import org.lamastex.spark.trendcalculus._
import scala.util.Random


import java.sql.Timestamp
import io.delta.tables._
import org.apache.spark.sql._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.streaming.{GroupState, GroupStateTimeout, OutputMode, Trigger}
import org.apache.spark.sql.types._
import org.apache.spark.sql.expressions.{Window, WindowSpec}
import org.lamastex.spark.trendcalculus._
import scala.util.Random


defined object TrendUtils

Reading the joined dataset from the last notebook.

We train the model using both oil and gold data and predict trends in oil data. We show that this yields better results than just training on the oil data.


val rootPath = TrendUtils.getStreamableTrendCalculusPath
val maxRevPath = rootPath + "maxRev"
val maxRevDS = spark.read.format("delta").load(maxRevPath).as[FlatReversal]

We want to predict what the trend of the next data point will be given the trend reversals we have observed.

For this, we use an m-th order Markov model. We look at the reversal state of the last m points and use this to predict the trends in the next n points. k is the maximum order of reversal that is considered when training the model.

trainingRatio is the ratio of the data used for training the model, the rest is used for testing.


val modelPath = rootPath + "estimators/"
val maxRevDSWithLagCountPath = modelPath + "maxRevDSWithLag"

val numPartitions = 10
val partialModelPaths = (1 to numPartitions).map( i => modelPath + s"partialModel${i}" )
val fullModelPath = modelPath + "fullModel"

val m = 10
val n = 1
val k = 18
val trainingRatio = 0.7
type FinalModel = Map[Seq[Int], Map[Seq[Int], Double]]


def truncRev(k: Int)(rev: Int): Int = {
  if (math.abs(rev) > k) k*rev.signum else rev
}
val truncRevUDF = udf{ rev: Int => rev.signum }
def truncRevsUDF(k: Int) = udf{ revs: Seq[Int] => revs.map(truncRev(k)) }

def lagColumn(df: DataFrame, orderColumnName: String, lagKeyName: String, lagValueName: String, m: Int, n: Int): DataFrame = {
  val windowSpec = Window.partitionBy("ticker").orderBy(orderColumnName)
  val laggedKeyColNames = (1 to m).map( i => s"lagKey$i" ).toSeq
  val laggedValueColNames = (1 to n).map( i => s"lagValue$i" ).toSeq
  val dfWithLaggedKeyColumns = (n+1 to m+n)
    .foldLeft(df)( (df: DataFrame, i: Int) => df.withColumn(laggedKeyColNames(i-n-1), lag(lagKeyName, i-1, Int.MaxValue).over(windowSpec)) )
  val dfWithLaggedKeyValueColumns = (1 to n)
    .foldLeft(dfWithLaggedKeyColumns)( (df: DataFrame, i: Int) => df.withColumn(laggedValueColNames(i-1), lag(lagValueName, i-1, Int.MaxValue).over(windowSpec)) )
  
  dfWithLaggedKeyValueColumns
    .withColumn("lagKey", array(laggedKeyColNames.reverse.take(m).map(col(_)):_*))
    .withColumn("lagValue", array(laggedValueColNames.reverse.takeRight(n).map(col(_)):_*))
    .withColumn("lagKeyFirst", col(laggedKeyColNames.last))
    .filter($"lagKeyFirst" =!= Int.MaxValue)
    .drop("lagKeyFirst")
    .drop(laggedKeyColNames:_*)
    .drop(laggedValueColNames:_*)
}


truncRev: (k: Int)(rev: Int)Int
truncRevUDF: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$9209/1126103335@6b64702c,IntegerType,List(Some(class[value[0]: int])),None,false,true)
truncRevsUDF: (k: Int)org.apache.spark.sql.expressions.UserDefinedFunction
lagColumn: (df: org.apache.spark.sql.DataFrame, orderColumnName: String, lagKeyName: String, lagValueName: String, m: Int, n: Int)org.apache.spark.sql.DataFrame

The trend at each point can be extracted from the trend reversals by taking the sum of all previous 1-st order trend reversals. This sum will always be either 0 (up trend) or -1 (down trend) and 0 is therefore mapped to 1 to get (1, -1) as (up, down).


val maxRevDSWithLag = lagColumn(
  maxRevDS
    .orderBy("x")
    .toDF
    .withColumn("truncRev", truncRevUDF($"reversal"))
    .withColumn(
      "tmpTrend",
      sum("truncRev").over(
        Window
          .partitionBy("ticker")
          .orderBy("x")
          .rowsBetween(Window.unboundedPreceding, Window.currentRow)
      )
    )
    .withColumn("trend", when($"tmpTrend" === 0, 1).otherwise(-1))
    .drop("truncRev", "tmpTrend"),
  "x", 
  "reversal",
  "trend",
  m, 
  n
)


maxRevDSWithLag: org.apache.spark.sql.DataFrame = [ticker: string, x: timestamp ... 5 more fields]

We now want to predict lagValue from lagKey.


maxRevDSWithLag.show(20, false)


+------+-------------------+------+--------+-----+--------------------------------+--------+
|ticker|x                  |y     |reversal|trend|lagKey                          |lagValue|
+------+-------------------+------+--------+-----+--------------------------------+--------+
|XAUUSD|2009-03-15 18:09:00|925.4 |4       |1    |[-6, 0, 0, 1, -1, 0, 0, 0, 0, 0]|[1]     |
|XAUUSD|2009-03-15 18:10:00|925.75|-2      |-1   |[0, 0, 1, -1, 0, 0, 0, 0, 0, 4] |[-1]    |
|XAUUSD|2009-03-15 18:11:00|925.7 |0       |-1   |[0, 1, -1, 0, 0, 0, 0, 0, 4, -2]|[-1]    |
|XAUUSD|2009-03-15 18:12:00|925.65|1       |1    |[1, -1, 0, 0, 0, 0, 0, 4, -2, 0]|[1]     |
|XAUUSD|2009-03-15 18:13:00|925.65|0       |1    |[-1, 0, 0, 0, 0, 0, 4, -2, 0, 1]|[1]     |
|XAUUSD|2009-03-15 18:14:00|925.75|0       |1    |[0, 0, 0, 0, 0, 4, -2, 0, 1, 0] |[1]     |
|XAUUSD|2009-03-15 18:15:00|925.75|-1      |-1   |[0, 0, 0, 0, 4, -2, 0, 1, 0, 0] |[-1]    |
|XAUUSD|2009-03-15 18:16:00|925.65|0       |-1   |[0, 0, 0, 4, -2, 0, 1, 0, 0, -1]|[-1]    |
|XAUUSD|2009-03-15 18:17:00|925.6 |2       |1    |[0, 0, 4, -2, 0, 1, 0, 0, -1, 0]|[1]     |
|XAUUSD|2009-03-15 18:18:00|925.85|0       |1    |[0, 4, -2, 0, 1, 0, 0, -1, 0, 2]|[1]     |
|XAUUSD|2009-03-15 18:19:00|926.05|0       |1    |[4, -2, 0, 1, 0, 0, -1, 0, 2, 0]|[1]     |
|XAUUSD|2009-03-15 18:20:00|925.95|0       |1    |[-2, 0, 1, 0, 0, -1, 0, 2, 0, 0]|[1]     |
|XAUUSD|2009-03-15 18:21:00|926.55|0       |1    |[0, 1, 0, 0, -1, 0, 2, 0, 0, 0] |[1]     |
|XAUUSD|2009-03-15 18:22:00|926.95|0       |1    |[1, 0, 0, -1, 0, 2, 0, 0, 0, 0] |[1]     |
|XAUUSD|2009-03-15 18:23:00|927.25|0       |1    |[0, 0, -1, 0, 2, 0, 0, 0, 0, 0] |[1]     |
|XAUUSD|2009-03-15 18:24:00|927.45|-4      |-1   |[0, -1, 0, 2, 0, 0, 0, 0, 0, 0] |[-1]    |
|XAUUSD|2009-03-15 18:25:00|927.2 |0       |-1   |[-1, 0, 2, 0, 0, 0, 0, 0, 0, -4]|[-1]    |
|XAUUSD|2009-03-15 18:26:00|926.85|0       |-1   |[0, 2, 0, 0, 0, 0, 0, 0, -4, 0] |[-1]    |
|XAUUSD|2009-03-15 18:27:00|926.7 |0       |-1   |[2, 0, 0, 0, 0, 0, 0, -4, 0, 0] |[-1]    |
|XAUUSD|2009-03-15 18:28:00|926.5 |0       |-1   |[0, 0, 0, 0, 0, 0, -4, 0, 0, 0] |[-1]    |
+------+-------------------+------+--------+-----+--------------------------------+--------+
only showing top 20 rows

Cleaning up last run and writing model training input to delta tables.


dbutils.fs.rm(maxRevDSWithLagCountPath, recurse=true)

maxRevDSWithLag
  .withColumn("count", lit(1L))
  .write
  .format("delta")
  .mode("overwrite")
  .save(maxRevDSWithLagCountPath)


val divUDF = udf{ (a: Long, b: Long) => a.toDouble/b }
val maxRevDSWithLagCount = spark.read.format("delta").load(maxRevDSWithLagCountPath)
val numberOfRows = maxRevDSWithLagCount.count


divUDF: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$9266/71459450@6a0e8651,DoubleType,List(Some(class[value[0]: bigint]), Some(class[value[0]: bigint])),None,false,true)
maxRevDSWithLagCount: org.apache.spark.sql.DataFrame = [ticker: string, x: timestamp ... 6 more fields]
numberOfRows: Long = 6845798

The data is split into training and testing data. This is not done randomly as there is a dependence on previous data points. We don't want to train on data that is dependent on the testing data and therefore the training data consists of the first (for example) 70% of the data and the last 30% is saved for testing. This also reflects how the model would be used since we can only train on data points that have already been observed.


val tickers = maxRevDSWithLagCount.select("ticker").distinct.as[String].collect.toSeq
val tickerDFs = tickers.map( ticker => maxRevDSWithLagCount.filter($"ticker" === ticker))
val trainingDF = tickerDFs.map( df => df.limit((df.count*trainingRatio).toInt) ).reduce( _.union(_) ).orderBy("x")
val trainingRows = trainingDF.count

val testingDF = maxRevDSWithLagCount.except(trainingDF)


tickers: Seq[String] = WrappedArray(XAUUSD, BCOUSD)
tickerDFs: Seq[org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]] = ArrayBuffer([ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields])
trainingDF: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [ticker: string, x: timestamp ... 6 more fields]
trainingRows: Long = 4792058
testingDF: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [ticker: string, x: timestamp ... 6 more fields]

Create numTrainingSets training set of increasing size to get snapshots of how a partially trained model looks like. The sizes are spaced logarithmically since the improvement in the model is fastest in the beginning.


val rowsInPartitions = (1 to numPartitions).map{ i: Int => (math.exp(math.log(trainingRows)*i/numPartitions)).toInt }//.scanLeft(0.0)(_-_)


rowsInPartitions: scala.collection.immutable.IndexedSeq[Int] = Vector(4, 21, 100, 470, 2189, 10193, 47464, 221012, 1029129, 4792058)

Model is trained by counting how many times each (lagKey, lagValue) pair is observed and dividing by how many times lagKey is observed to get an estimation of the transition probabilities.


val keyValueCountPartialDFs = rowsInPartitions.map(
  trainingDF
    .limit(_)
    .withColumn("keyValueObs", sum("count").over(Window.partitionBy($"lagKey", $"lagValue")))
    .withColumn("totalKeyObs", sum("count").over(Window.partitionBy($"lagKey")))
    .drop("count")
)


keyValueCountPartialDFs: scala.collection.immutable.IndexedSeq[org.apache.spark.sql.DataFrame] = Vector([ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields], [ticker: string, x: timestamp ... 7 more fields])


keyValueCountPartialDFs.last.orderBy($"keyValueObs".desc).show(20,false)


+------+-------------------+------+--------+-----+------------------------------+--------+-----------+-----------+
|ticker|x                  |y     |reversal|trend|lagKey                        |lagValue|keyValueObs|totalKeyObs|
+------+-------------------+------+--------+-----+------------------------------+--------+-----------+-----------+
|XAUUSD|2009-03-15 19:41:00|926.55|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-17 05:27:00|921.73|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 04:51:00|928.75|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 04:52:00|928.7 |0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 04:53:00|928.68|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 06:24:00|925.1 |-5      |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 09:19:00|917.73|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 09:20:00|917.83|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 09:21:00|917.0 |0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 14:47:00|922.9 |0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 14:48:00|922.88|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 14:49:00|922.93|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 14:50:00|922.83|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 22:49:00|922.28|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 22:50:00|922.25|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 22:51:00|922.13|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-16 22:52:00|922.08|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-17 05:11:00|923.8 |-1      |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-17 05:24:00|922.23|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
|XAUUSD|2009-03-17 05:25:00|922.03|0       |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]|[-1]    |53544      |106773     |
+------+-------------------+------+--------+-----+------------------------------+--------+-----------+-----------+
only showing top 20 rows


keyValueCountPartialDFs
  .map( df =>
    df
      .withColumn("probability", divUDF($"keyValueObs", $"totalKeyObs"))
      .drop("keyValueObs", "totalKeyObs")
  )
  .zip(partialModelPaths).map{ case (df: DataFrame, path: String) =>
    df.write.mode("overwrite").format("delta").save(path)  
  }


val probDFs = partialModelPaths.map(spark.read.format("delta").load(_))


probDFs: scala.collection.immutable.IndexedSeq[org.apache.spark.sql.DataFrame] = Vector([ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields], [ticker: string, x: timestamp ... 6 more fields])


probDFs.last.orderBy("probability").show(20,false)


+------+-------------------+-------+--------+-----+---------------------------------+--------+--------------------+
|ticker|x                  |y      |reversal|trend|lagKey                           |lagValue|probability         |
+------+-------------------+-------+--------+-----+---------------------------------+--------+--------------------+
|BCOUSD|2011-04-29 11:59:00|125.61 |2       |1    |[0, -4, 0, 0, 0, 1, 0, 0, 0, -1] |[1]     |0.007142857142857143|
|XAUUSD|2015-07-31 00:05:00|1084.69|-1      |-1   |[-2, 0, 0, 0, 1, -1, 0, 0, 0, 3] |[-1]    |0.013513513513513514|
|XAUUSD|2016-09-07 11:09:00|1344.18|-2      |-1   |[0, 0, 0, 0, 6, 0, 0, 0, -1, 1]  |[-1]    |0.014492753623188406|
|XAUUSD|2015-11-06 00:05:00|1108.14|-1      |-1   |[0, 4, 0, 0, -1, 0, 0, 0, 0, 1]  |[-1]    |0.014705882352941176|
|XAUUSD|2010-05-13 01:54:00|1237.83|2       |1    |[-3, 0, 0, 1, 0, 0, 0, 0, 0, -1] |[1]     |0.01639344262295082 |
|BCOUSD|2011-04-05 16:30:00|121.63 |-3      |-1   |[-2, 0, 0, 0, 0, 1, 0, -1, 0, 3] |[-1]    |0.016666666666666666|
|XAUUSD|2009-09-30 22:16:00|1006.65|1       |1    |[0, 0, 0, 0, 1, 0, 0, -1, 1, -1] |[1]     |0.016666666666666666|
|XAUUSD|2010-11-05 04:43:00|1386.98|1       |1    |[0, 0, 0, 0, 1, 0, 0, -1, 1, -1] |[1]     |0.016666666666666666|
|XAUUSD|2010-11-19 15:13:00|1351.4 |-1      |-1   |[0, 0, 0, 0, 0, 1, -1, 0, 0, 5]  |[-1]    |0.017094017094017096|
|BCOUSD|2012-10-10 14:23:00|114.27 |-1      |-1   |[0, 0, 0, 0, 0, 1, -1, 0, 0, 5]  |[-1]    |0.017094017094017096|
|XAUUSD|2015-06-28 19:18:00|1185.37|2       |1    |[-4, 0, 0, 0, 1, 0, 0, 0, 0, -1] |[1]     |0.017241379310344827|
|BCOUSD|2015-10-15 01:18:00|49.75  |2       |1    |[0, -3, 0, 1, 0, -1, 0, 0, 1, -1]|[1]     |0.017543859649122806|
|XAUUSD|2012-02-27 20:23:00|1767.83|1       |1    |[0, -5, 0, 0, 0, 1, 0, 0, 0, -1] |[1]     |0.01818181818181818 |
|BCOUSD|2015-04-27 23:58:00|63.96  |-1      |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 8]   |[-1]    |0.0196078431372549  |
|XAUUSD|2009-04-08 15:54:00|880.88 |-1      |-1   |[0, 0, 0, 0, 0, 0, 0, 0, 0, 8]   |[-1]    |0.0196078431372549  |
|BCOUSD|2015-09-11 11:06:00|47.27  |-1      |-1   |[0, 0, 0, 0, 1, -1, 0, 0, 0, 6]  |[-1]    |0.0196078431372549  |
|XAUUSD|2012-01-24 23:36:00|1666.11|1       |1    |[0, 1, 0, 0, 0, -1, 0, 1, 0, -4] |[1]     |0.02040816326530612 |
|XAUUSD|2009-12-28 23:56:00|1104.68|-1      |-1   |[0, -2, 2, 0, 0, 0, 0, -1, 0, 1] |[-1]    |0.020833333333333332|
|BCOUSD|2014-03-12 09:33:00|107.74 |-3      |-1   |[0, 0, 7, 0, 0, 0, 0, -1, 0, 1]  |[-1]    |0.020833333333333332|
|XAUUSD|2016-04-06 09:50:00|1221.57|2       |1    |[0, -1, 0, 1, -3, 0, 0, 0, 1, -1]|[1]     |0.02127659574468085 |
+------+-------------------+-------+--------+-----+---------------------------------+--------+--------------------+
only showing top 20 rows

The prediction is given by taking

$V \in argmax(P_K(V))$

where *P_K(V)* is the probability that V is the next trend when the last m points have had reversals K. If there are more than one elements in argmax, an element is chosen uniformly at random.


val aggWindow = Window.partitionBy("lagKey").orderBy('probability desc)
val testedDFs = probDFs
  .map { df =>
     val predictionDF = df
      .select("lagKey", "lagValue", "probability")
      .distinct
      .withColumn("rank", rank().over(aggWindow))
      .filter("rank == 1")
      .groupBy("lagKey")
      .agg(collect_list("lagValue"))
    
    testingDF
      .filter($"ticker" === "BCOUSD")
      .join(predictionDF, Seq("lagKey"), "left")
      .withColumnRenamed("collect_list(lagValue)", "test")
  }


warning: there was one feature warning; for details, enable `:setting -feature' or `:replay -feature'
aggWindow: org.apache.spark.sql.expressions.WindowSpec = org.apache.spark.sql.expressions.WindowSpec@2dbd5c17
testedDFs: scala.collection.immutable.IndexedSeq[org.apache.spark.sql.DataFrame] = Vector([lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields], [lagKey: array<int>, ticker: string ... 7 more fields])

We use a binary loss function that indicates if the prediction was correct or not.

getRandomUDF is used to choose an element at random in argmax(P_K(V)). safeArr is used since we might see patterns in the test data that were not present in the training data.


val getRandomUDF = udf( (arr: Seq[Seq[Int]]) => {
  val safeArr = Option(arr).getOrElse(Seq[Seq[Int]]())
  if (safeArr.isEmpty) Seq[Int]() else safeArr(Random.nextInt(safeArr.size))
} )

val lossUDF = udf{ (value: Seq[Int], pred: Seq[Int]) =>
  if (value == pred) 0 else 1
}


getRandomUDF: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$9416/1051216045@79bf6d73,ArrayType(IntegerType,false),List(Some(class[value[0]: array<array<int>>])),None,true,true)
lossUDF: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$9417/777777096@6e1c4ce7,IntegerType,List(Some(class[value[0]: array<int>]), Some(class[value[0]: array<int>])),None,false,true)


val lossDFs = testedDFs.map(_.withColumn("prediction", getRandomUDF($"test")).withColumn("loss", lossUDF($"lagValue", $"prediction")))


lossDFs: scala.collection.immutable.IndexedSeq[org.apache.spark.sql.DataFrame] = Vector([lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields], [lagKey: array<int>, ticker: string ... 9 more fields])

Already on line 2 we see a case where lagKey was previously not observed (indicated by null in test column).


lossDFs.last.show(20,false)


+---------------------------------+------+-------------------+-----+--------+-----+--------+-----+------+----------+----+
|lagKey                           |ticker|x                  |y    |reversal|trend|lagValue|count|test  |prediction|loss|
+---------------------------------+------+-------------------+-----+--------+-----+--------+-----+------+----------+----+
|[0, 0, 0, 0, 0, 0, -2, 0, 0, 0]  |BCOUSD|2017-10-02 02:03:00|56.56|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[-4, 0, 1, 0, 0, 0, -1, 0, 0, 0] |BCOUSD|2017-10-03 07:07:00|56.0 |1       |1    |[1]     |1    |[[-1]]|[-1]      |1   |
|[0, 1, 0, -1, 3, -1, 0, 1, 0, 0] |BCOUSD|2017-10-03 13:23:00|55.92|-2      |-1   |[-1]    |1    |[[1]] |[1]       |1   |
|[0, 2, 0, 0, -2, 0, 0, 0, 0, 1]  |BCOUSD|2017-10-13 06:59:00|57.43|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, 0, 0, 0, 1, -1, 2, 0, 0, -2] |BCOUSD|2017-10-18 07:21:00|58.21|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[0, 0, 0, -1, 0, 0, 0, 2, 0, 0]  |BCOUSD|2017-10-25 06:08:00|58.21|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, -4, 1, 0, -1, 1, 0, -1, 0, 0]|BCOUSD|2017-10-31 08:44:00|60.52|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[0, 0, -1, 0, 1, 0, -1, 0, 1, 0] |BCOUSD|2017-11-15 12:35:00|61.96|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, 0, 0, 0, 0, 0, 0, 0, -4, 0]  |BCOUSD|2017-11-15 13:28:00|62.02|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[1, 0, -1, 0, 2, -2, 0, 0, 0, 0] |BCOUSD|2017-11-16 11:37:00|61.7 |0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[0, 0, 0, -2, 0, 0, 0, 2, 0, 0]  |BCOUSD|2017-12-08 09:10:00|63.37|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, 0, 0, -1, 0, 0, 2, 0, 0, 0]  |BCOUSD|2017-12-15 06:19:00|63.45|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]   |BCOUSD|2017-12-19 01:46:00|63.56|0       |1    |[1]     |1    |[[-1]]|[-1]      |1   |
|[1, 0, -1, 0, 0, 0, 0, 0, 3, 0]  |BCOUSD|2017-12-21 07:09:00|64.39|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[-1, 0, 0, 0, 0, 0, 0, 1, 0, -1] |BCOUSD|2017-12-22 13:50:00|65.18|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[0, 1, 0, -1, 0, 0, 0, 0, 0, 1]  |BCOUSD|2017-12-27 14:51:00|65.92|-1      |-1   |[-1]    |1    |[[1]] |[1]       |1   |
|[0, 0, 0, 0, 1, 0, 0, -2, 1, -1] |BCOUSD|2018-01-03 22:15:00|67.91|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[0, 0, 0, -3, 0, 0, 1, 0, -1, 0] |BCOUSD|2018-01-08 10:17:00|67.72|0       |-1   |[-1]    |1    |[[-1]]|[-1]      |0   |
|[-1, 0, 1, 0, 0, 0, -1, 0, 1, 0] |BCOUSD|2018-01-23 16:24:00|70.15|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
|[0, 0, 1, 0, -1, 0, 0, 2, 0, 0]  |BCOUSD|2018-01-30 07:53:00|69.02|0       |1    |[1]     |1    |[[1]] |[1]       |0   |
+---------------------------------+------+-------------------+-----+--------+-----+--------+-----+------+----------+----+
only showing top 20 rows


val testLength = testingDF.count
val oilTestDF = testingDF.filter($"ticker" === "BCOUSD")
val oilTestLength = oilTestDF.count


testLength: Long = 2053732
oilTestDF: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [ticker: string, x: timestamp ... 6 more fields]
oilTestLength: Long = 857750

Out of roughly 2 million observations, about 860k were oil price.

We find the mean loss for each training dataset of increasing size. As one can see, the loss decreases as more data is supplied.

Further, training on both oil and gold data yields a better result than just oil, suggesting that trends behave similarly in the two commodities.


val losses = lossDFs.map( _.agg(sum("loss")).select($"sum(loss)".as("totalLoss")).collect.head.getLong(0).toDouble/oilTestLength )
// Data up to 2019
// k=max, m=2 ,n=1: (0.5489615950080244, 0.4014721726541194, 0.3660973816818738, 0.36497087640146797, 0.36485163238050344)
// k=max, m=10,n=1: (0.9812730246821787, 0.8523390131056561, 0.5819573469954631, 0.3552177121357085, 0.2807503135425113)
// k=max,m=100,n=1: (1.0, 1.0, 1.0, 1.0, 1.0)
// k=5  , m=10,n=1: (0.9812730246821787, 0.8522267831239777, 0.5820597568537447, 0.3550507700379618, 0.2806352778112909)
// k=1  , m=10,n=1: (0.9812730246821787, 0.852200128503329, 0.5821242890932098, 0.3553383593660128, 0.2806549180580846)

// k=max, m=10,n=2: (0.9879380657977248, 0.9049354606556204, 0.7243136776273427, 0.5472986906951395, 0.4783823708897465)

// k=max(17), m=10,n=1, 10 training sets: (0.9977203284971564, 0.9812730246821787, 0.9455375956409875, 0.8523810993487855, 0.7183644724769999, 0.5819320952495854, 0.44607349380350214, 0.35513915114853356, 0.3029480010437388, 0.280629666312207)

// Data up to last month
// k=max(18), m=10,n=1, 10 training sets: (0.9973104051296998, 0.9843882250072865, 0.9510789857184494, 0.8574351501020111, 0.7515744680851064, 0.6360547945205479, 0.5099656076945497, 0.4249921305741766, 0.3735668901194987, 0.34609501603031184)
// Could the difference be due to Corona?

// Trained on both oil and gold. testing on oil as previously.
// k=max(18), m=10,n=1, 10 training sets: (0.9999778490236083, 0.9980728650539201, 0.9527158262897114, 0.8921317400174876, 0.7559988341591373, 0.5820915185077237, 0.46675488195861264, 0.3923101136694841, 0.3574876129408336, 0.3355837948120082)


losses: scala.collection.immutable.IndexedSeq[Double] = Vector(0.9999778490236083, 0.9980728650539201, 0.9528044301952784, 0.8921084232002332, 0.7558216263480035, 0.5819551151267852, 0.4666686097347712, 0.3920524628388225, 0.35745380355581463, 0.3355488195861265)

Visualizing the improvement of the model as the size of training data increases.


val trainingSizes = probDFs.map(_.count)
val lossesDS = sc
  .parallelize(losses.zip(trainingSizes))
  .toDF("loss", "size")
  .withColumn("training", lit("Oil and Gold"))
  .union(
    sc
      .parallelize(Seq(0.9973104051296998, 0.9843882250072865, 0.9510789857184494, 0.8574351501020111, 0.7515744680851064, 0.6360547945205479, 0.5099656076945497, 0.4249921305741766, 0.3735668901194987, 0.34609501603031184).zip(Seq(4, 18, 77, 331, 1414, 6036, 25759, 109918, 469036, 2001430)))
      .toDF("loss", "size")
      .withColumn("training", lit("Oil"))
  )
  .as[(Double,Long,String)]


trainingSizes: scala.collection.immutable.IndexedSeq[Long] = Vector(4, 21, 100, 470, 2189, 10193, 47464, 221012, 1029129, 4792058)
lossesDS: org.apache.spark.sql.Dataset[(Double, Long, String)] = [loss: double, size: bigint ... 1 more field]


display(lossesDS)

loss	size	training
0.9999778490236083	4.0	Oil and Gold
0.9980728650539201	21.0	Oil and Gold
0.9528044301952784	100.0	Oil and Gold
0.8921084232002332	470.0	Oil and Gold
0.7558216263480035	2189.0	Oil and Gold
0.5819551151267852	10193.0	Oil and Gold
0.4666686097347712	47464.0	Oil and Gold
0.3920524628388225	221012.0	Oil and Gold
0.35745380355581463	1029129.0	Oil and Gold
0.3355488195861265	4792058.0	Oil and Gold
0.9973104051296998	4.0	Oil
0.9843882250072865	18.0	Oil
0.9510789857184494	77.0	Oil
0.8574351501020111	331.0	Oil
0.7515744680851064	1414.0	Oil
0.6360547945205479	6036.0	Oil
0.5099656076945497	25759.0	Oil
0.4249921305741766	109918.0	Oil
0.3735668901194987	469036.0	Oil
0.34609501603031184	2001430.0	Oil

trendcalculusmcmodelperformance

The series using only oil data is very similar to the one with both oil and gold data. The final data point has a loss of about 0.35, meaning that the model accuracy is around 65% for predicting the the trend of the next point in the time series.

To better understand the difference between the different (partially trained) models, we want to calculate the total variation distance between them.

The first step is to collect the models and work with them locally.


val partialModels = probDFs.map{df => 
  val tmpMap = df
    .select("lagKey", "lagValue", "probability")
    .distinct
    .collect
    .map{ r => 
      (r.getAs[Seq[Int]](0), r.getAs[Seq[Int]](1), r.getDouble(2))
    }
    .groupBy(_._1) // Grouping by lagKey
    .mapValues(_.map(tup => Map(tup._2 -> tup._3)).flatten.toMap)
  
  tmpMap: FinalModel
}

If a lagKey sequence only belongs to one of the models we are comparing, the variation distance is maximal, i.e. 1. Otherwise, we compute the total variation distance between the probabilities of observing a sequence of n trends, following the sequence of trends lagKey.


def totalVarDist(m1: FinalModel, m2: FinalModel): Map[Seq[Int], Double] = {
  val allKeys = m1.keys.toSet.union(m2.keys.toSet)
  val sharedKeys = m1.keys.toSet.intersect(m2.keys.toSet)
  val totalVarDists = allKeys.toSeq.map{ key =>
    if (!sharedKeys.contains(key)) {
      1.0
    } else {
      val val1 = m1.getOrElse(key, Map())
      val val2 = m2.getOrElse(key, Map())
      val allValKeys = val1.keys.toSet.union(val2.keys.toSet)
      allValKeys.map( valKey => 0.5*math.abs(val1.getOrElse(valKey, 0.0) - val2.getOrElse(valKey, 0.0)) ).sum
    }
  }
  allKeys.zip(totalVarDists).toMap
}


totalVarDist: (m1: FinalModel, m2: FinalModel)Map[Seq[Int],Double]

Computing the variation distance for all possible pairs of partially trained models.


val totalVariationDistances = partialModels.map( m1 => partialModels.map( m2 => totalVarDist(m1,m2) ) )


totalVariationDistances: scala.collection.immutable.IndexedSeq[scala.collection.immutable.IndexedSeq[Map[Seq[Int],Double]]] = Vector(Vector(Map(WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 4, -2) -> 0.0, WrappedArray(-6, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 0.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 0, 4) -> 0.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 4, -2, 0) -> 0.0), Map(WrappedArray(-6, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 0.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 4, -2) -> 0.0, WrappedArray(-1, 0, 2, 0, 0, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(0, 4, -2, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 4, -2, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(-2, 0, 1, 0, 0, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(4, -2, 0, 1, 0, 0, -1, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 0, 4) -> 0.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 4, -2, 0) -> 0.0, WrappedArray(0, 1, 0, 0, -1, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, -4, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -4, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 4, -2, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 4, -2, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 4, -2, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 4, -2, 0, 1, 0, 0) -> 1.0), Map(WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-6, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 0.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 3, 0) -> 1.0, WrappedArray(0, 1, -1, 3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 4, -2) -> 0.0, WrappedArray(1, 0, -3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -4, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(0, 4, -2, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 4, -2, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(-1, 3, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 1, 0, -3, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -2, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(-2, 0, 1, 0, 0, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -4, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 3, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, 0, 0, -1, 0, 1, 0, -3) -> 1.0, WrappedArray(0, 1, 0, 0, -1, 0, 1, 0, -3, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 1, -1, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, -1, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(4, -2, 0, 1, 0, 0, -1, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 0, 4) -> 0.0, WrappedArray(0, 0, -1, 0, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 4, -2, 0) -> 0.0, WrappedArray(2, 0, 0, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 3, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 1, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 1, 0, 0, -1, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 2, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(-3, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, -4, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 1, -1, 0, 2, 0) -> 1.0, WrappedArray(1, -1, 3, 0, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, -1, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -4, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, -3, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 0, 1, -1, 3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -4, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 4, -2, 0, 1, 0) -> 1.0, WrappedArray(0, -3, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, -2, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 4, -2, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(-4, 0, 0, 0, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(0, 2, 0, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 3, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 2, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 2, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -3, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 4, -2, 0, 1) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, -4, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(3, 0, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 4, -2, 0, 1, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0), Map(WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 2, 0, -1, 0, 1, 0, -3) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 6, -2) -> 1.0, WrappedArray(0, 0, -3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 1, -1, 1, -1, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 2, 0, -1, 0) -> 1.0, WrappedArray(1, -1, 0, 1, -1, 0, 0, 3, -1, 0) -> 1.0, WrappedArray(0, 0, 4, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -2, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(-6, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 0.0, WrappedArray(0, 0, -4, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 5, 0, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 5, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -4, 0, 1, 0, -1, 0, 1) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -3, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 0, 3, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 1, -1, 0, 0) -> 1.0, WrappedArray(6, -2, 0, 0, 0, 2, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 3, 0) -> 1.0, WrappedArray(0, 0, 2, 0, -1, 0, 1, -1, 0, 1) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 0, 1, -1, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(-5, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, -5, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -1, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 2, -2, 3, -1, 1) -> 1.0, WrappedArray(1, 0, -1, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, -1, 1) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 1, 0, -2, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 5, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(-3, 0, 1, 0, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 3, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 4, -2) -> 0.0, WrappedArray(-1, 1, -2, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -2, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 4, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -1, 0, 2, -2, 3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, 0, -2, 0, 2, 0, -1) -> 1.0, WrappedArray(-2, 3, -1, 1, 0, 0, -3, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 2, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(1, 0, -3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(2, 0, -1, 0, 1, 0, -2, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, -1, 1, -2) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 0, 2, -2) -> 1.0, WrappedArray(3, -1, 1, 0, 0, -3, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -4, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 1, -1, 0, 0, 3) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, -1, 1, -2, 0, 0) -> 1.0, WrappedArray(0, 4, -2, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 1, -1, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 4, -2, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(1, 0, -2, 0, 2, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(3, 0, -1, 1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 1, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 3, 0, -1, 1) -> 1.0, WrappedArray(0, 0, -4, 0, 1, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 0, 4) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 1, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 3, -1, 0, 1, 0, 0, -3, 1) -> 1.0, WrappedArray(0, -3, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 3, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 4, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 1, -1, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 6, -2, 0, 0, 0, 2) -> 1.0, WrappedArray(-1, 0, 1, 0, -2, 0, 2, 0, -1, 0) -> 1.0, WrappedArray(0, 6, -2, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 0, 0, 3, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -2, 0, 0, 0) -> 1.0, WrappedArray(0, 2, -2, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -5, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 1, 0, -1) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -4, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 3, 0, -1, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 1, 0, -2) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 1, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 1, 0, -3, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 6, -2, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, -4, 0, 1, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(1, 0, 0, -3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 6, -2, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 2, 0, -1, 0, 1, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 1, 0, -1, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, -2, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -3, 1, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 3, 0, -1) -> 1.0, WrappedArray(-2, 0, 2, 0, -1, 0, 1, 0, -3, 1) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 1, 0, -3, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, -3, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -2, 0, 0, 1) -> 1.0, WrappedArray(0, -1, 1, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(-2, 0, 1, 0, 0, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, -2, 1, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 2, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, -1, 1, -2, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(-1, 0, 0, 0, 1, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -4, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, -2, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, -5, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 3, 0, 0, 0, 0) -> 1.0, WrappedArray(-4, 0, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, -1, 0, 2, -2) -> 1.0, WrappedArray(-4, 0, 1, 0, -1, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 2, 0, -1, 0, 1, 0, -3, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 1, 0, -1) -> 1.0, WrappedArray(-2, 0, 0, 1, -1, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -2, 1, 0, -1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, -3, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 2, -2, 0, 1, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, -2, 1, 0, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -5, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 2, 0, -1, 0, 1, 0, -2) -> 1.0, WrappedArray(0, 1, 0, -2, 0, 2, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(2, -2, 0, 1, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, -1, 0, 1, 0, -3) -> 1.0, WrappedArray(0, 0, 2, -2, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 1, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 1, 0, 0, -1, 0, 1, 0, -3, 0) -> 1.0, WrappedArray(3, -1, 0, 1, 0, 0, -3, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 1, -1, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 0, 3) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 5, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 2, 0, -1) -> 1.0, WrappedArray(0, 4, 0, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 3, -1, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -3, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 4, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, -4, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, -4, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(-2, 0, 2, 0, -1, 0, 1, 0, -2, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(-1, 0, 0, 3, -1, 0, 1, 0, 0, -3) -> 1.0, WrappedArray(0, 0, 0, 4, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(4, 0, 0, 0, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 2, -2, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, -1, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(-1, 0, 0, 2, 0, -1, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(4, -2, 0, 1, 0, 0, -1, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 0, 4) -> 0.0, WrappedArray(0, 3, 0, -1, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, -1, 1, -2, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(2, -2, 3, -1, 1, 0, 0, -3, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 5, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 2, -2, 3, -1, 1, 0, 0, -3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 3, 0, -1, 1, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 4, -2, 0) -> 0.0, WrappedArray(-1, 0, 0, 0, 1, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -1, 0, 0, 2) -> 1.0, WrappedArray(0, -1, 0, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 3, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(-2, 1, 0, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 1, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 1, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 5, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 6, -2, 0, 0) -> 1.0, WrappedArray(1, 0, -2, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -5, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 0, 0, 0, 6) -> 1.0, WrappedArray(0, 1, 0, 0, -1, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -3, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(-1, 1, -1, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 1, 0, -3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 2, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, -1, 1, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-3, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 0, 2, -2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -5, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 1, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, -4, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -5, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 1, -1, 0, 1, -1) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 1, 0, -2, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 5, 0) -> 1.0, WrappedArray(0, 0, 6, -2, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(-3, 1, 0, 0, -1, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 2, 0, -1, 0, 1, 0, -2, 0, 2) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 1, -1, 0, 2, 0) -> 1.0, WrappedArray(1, -1, 3, 0, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(0, -3, 1, 0, 0, -1, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 3, -1, 0, 1, 0, 0, -3, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 1, 0, 0, -3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(-1, 1, 0, 0, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -3, 0, 1, 0, -1) -> 1.0, WrappedArray(2, 0, 0, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 4, 0) -> 1.0, WrappedArray(0, -3, 0, 0, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -5, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 4, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, -4, 0, 1) -> 1.0, WrappedArray(0, 0, -1, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 2, 0, -1, 0, 1, -1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 2, -2, 3, -1, 1, 0, 0, -3, 0) -> 1.0, WrappedArray(-2, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, 0, 0, 0, -5) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 1, 0, -2, 1) -> 1.0, WrappedArray(0, 0, -3, 1, 0, 0, -1, 0, 0, 2) -> 1.0, WrappedArray(1, 0, 0, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 2, -2, 3, -1, 1, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, -1, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, -1, 0, 0, 1) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(-3, 0, 0, 0, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -4, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 0, 1, -1, 1) -> 1.0, WrappedArray(0, 0, 0, 0, -4, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, -3, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 0, 1, -1, 3) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -4, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 4, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, -1, 0, 2, -2, 3, -1) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 5) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, -4, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 5, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 2, -2, 0, 1) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 4, -2, 0, 1, 0) -> 1.0, WrappedArray(0, -3, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, -2, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 1, 0, -2, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 4, -2, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 1, -1, 0, 0, 3, -1) -> 1.0, WrappedArray(0, -1, 0, 1, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 1, -1) -> 1.0, WrappedArray(0, 0, 1, 0, -1, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(-4, 0, 0, 0, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(0, 2, 0, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, -1, 0, 1) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 1, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 3, 0, -1, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 3, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 2, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(1, 0, -3, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 1, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, -1, 0, 2, -2, 3, -1, 1, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 2, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, 0, -2, 0, 2, 0, -1, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -3, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 4, -2, 0, 1) -> 1.0, WrappedArray(2, 0, -1, 0, 1, 0, -3, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 1, -2, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 0, 6, -2, 0) -> 1.0, WrappedArray(0, 0, -4, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 4, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, -2, 1, 0, -1) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -2, 0, 2, 0, -1) -> 1.0, WrappedArray(0, 2, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 3, 0, -1, 1, 0, 0, 0) -> 1.0, WrappedArray(3, 0, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(2, 0, -1, 0, 1, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -3, 0, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 2, -2, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(5, 0, 0, 0, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(4, 0, 0, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 4, -2, 0, 1, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 0, 0, 1, -1) -> 1.0), Map(WrappedArray(-1, 0, 2, -1, 0, 0, 0, 1, -3, 3) -> 1.0, WrappedArray(-3, 0, 0, 0, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, 0, 0, -3) -> 1.0, WrappedArray(2, 0, 0, 0, -1, 0, 1, 0, 0, -4) -> 1.0, WrappedArray(-1, 0, 0, 0, 1, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 2, -2, 0, 1, -1, 0, 0, 1) -> 1.0, WrappedArray(-1, 1, 0, -2, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 1, -1) -> 1.0, WrappedArray(1, 0, 0, 0, -1, 1, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 5, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, -2, 0, 1, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 1, 0, -2) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 3, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 2, 0, -1, 0, 1, 0, -3) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 6, -2) -> 1.0, WrappedArray(0, 0, -3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 1, 0, -1, 0) -> 1.0, WrappedArray(2, -2, 0, 6, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(1, 0, -2, 0, 1, 0, -1, 0, 0, 2) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, 0, 0, -6) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 0, 0, 0, 0, 3) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 1, -1, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 2, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 3, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(-4, 0, 0, 0, 0, 2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 1, 0, -1, 0, 0, 2, 0) -> 1.0, WrappedArray(0, -1, 2, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, -3, 0, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 1, -1, 0, 0, 3, -1, 0) -> 1.0, WrappedArray(0, 0, 4, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -3, 0, 1, 0, -1, 0, 2, -2) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 3, -1, 0, 1, -1, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -2, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(-2, 0, 1, -1, 0, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, -1, 0, 1, 0, 0, -2) -> 1.0, WrappedArray(1, 0, -2, 0, 0, 0, 0, 0, 3, -1) -> 1.0, WrappedArray(0, 0, 0, 2, 0, -1, 1, 0, 0, -2) -> 1.0, WrappedArray(0, 0, 0, -2, 2, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, -2, 1, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 2, 0, -1, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 2, -2) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, 0, 0, -1, 1) -> 1.0, WrappedArray(0, -3, 0, 0, 2, -2, 0, 1, 0, -1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -2, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 1, 0, -1, 1) -> 1.0, WrappedArray(1, -1, 0, 0, 2, -2, 1, 0, -1, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, 0, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -4, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 3) -> 1.0, WrappedArray(0, 0, 3, -1, 1, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(-1, 0, 0, 2, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, 0, -1, 1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 1, -2, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 1, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -3, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(-5, 0, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 6, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, -1, 0, 5, -1, 0) -> 1.0, WrappedArray(0, -1, 1, 0, -1, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(4, 0, 0, 0, 0, 0, 0, 0, -1, 1) -> 1.0, WrappedArray(0, 0, -1, 2, 0, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(-6, 0, 0, 1, -1, 0, 0, 0, 0, 0) -> 0.0, WrappedArray(0, 0, 0, 0, 0, 0, -2, 2, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, -4, 0, 1, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, -4, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 2, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(-1, 1, 0, 0, 0, 0, -2, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 5, 0, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(-1, 1, 0, 0, -1, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, -1, 5, -1, 1) -> 1.0, WrappedArray(1, 0, -3, 0, 4, -2, 2, -1, 0, 0) -> 1.0, WrappedArray(0, 6, 0, 0, 0, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 1, -2, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 1, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, -3, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 5, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 5, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 5, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 1, -3, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 1, 0, -1, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -4, 0, 1, 0, -1, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, 0, 0, 0, -5) -> 1.0, WrappedArray(0, -1, 0, 1, 0, -3, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -1, 1) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 1, 0, -1, 1, 0) -> 1.0, WrappedArray(0, 6, 0, 0, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 3, 0, -2, 2, 0, -1, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 0, 0, 0, 2) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(-3, 1, 0, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 0, 3, -1, 0, 1) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 2, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, -1, 1, -1, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 3, -1, 0, 1) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(6, -2, 0, 0, 0, 2, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 2, 0, 0, -3, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 1, -1, 3, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 3, 0, 0, -3, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 1, 0, 0) -> 1.0, WrappedArray(-1, 1, 0, 0, -2, 0, 2, 0, -1, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, -1, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, -1, 0, 1, -1, 0, 1) -> 1.0, WrappedArray(-1, 0, 1, 0, -2, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 0, 1, -1, 0) -> 1.0, WrappedArray(-4, 0, 1, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 2, 0, -2, 1, 0) -> 1.0, WrappedArray(2, 0, -1, 0, 0, 1, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, -4, 0, 0, 4, 0, -2) -> 1.0, WrappedArray(0, 1, -1, 0, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 2, 0, 0, 0, -2) -> 1.0, WrappedArray(-5, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 1, -1, 0, 1, -1, 0) -> 1.0, WrappedArray(-5, 0, 0, 0, 0, 1, -1, 0, 0, 4) -> 1.0, WrappedArray(-1, 2, 0, 0, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(-3, 0, 0, 2, -2, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(-3, 0, 0, 2, -2, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 1, -3, 0, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, 0, -5, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -4, 0, 2, -1, 1, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -2, 0, 1, -1, 2, 0, -1) -> 1.0, WrappedArray(0, 1, -1, 3, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, -1, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -5, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -5, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 3, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 4, 0, 0, -3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -4, 0, 0, 4, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 5, 0) -> 1.0, WrappedArray(1, 0, 0, -1, 0, 2, -2, 3, -1, 1) -> 1.0, WrappedArray(0, -1, 0, 1, -1, 0, 1, -3, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, -4, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 1, 0, -3, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 6, 0, 0, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, -2, 0, 0, 1, -1, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, 0, 0, -1, 1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, -1, 0, 2, 0, 0) -> 1.0, WrappedArray(0, 2, -1, 0, 0, 0, 1, -3, 3, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 2, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(-2, 2, 0, -1, 0, 1, -1, 0, 1, -3) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 1, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 1, 0, 0, -1, 0, 4, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 5, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 1, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 2, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 2, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 1, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, -2, 0, 1, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -1, 0, 1, -2) -> 1.0, WrappedArray(0, 0, 0, 4, 0, 0, -3, 1, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(5, -1, 1, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -6, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, -2, 0, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 7, 0, 0, -1, 1, 0) -> 1.0, WrappedArray(3, 0, 0, -2, 0, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 1, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(-3, 0, 1, 0, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(-1, 1, 0, -1, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 2, 0, -1, 0, 1) -> 1.0, WrappedArray(1, -1, 0, 0, 3, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 0, 4, -2) -> 0.0, WrappedArray(0, 0, 0, 0, 1, 0, -1, 0, 2, 0) -> 1.0, WrappedArray(-1, 1, -2, 0, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 4, 0, 0, -2, 0, 0, 2, -1, 0) -> 1.0, WrappedArray(1, 0, -2, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 1, 0, -1, 0, 0, 2, 0) -> 1.0, WrappedArray(0, 1, 0, -2, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 7, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 0, 4, 0, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -1, 0, 2, -2, 3) -> 1.0, WrappedArray(0, -1, 1, -2, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 0, 0, 3, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, -3, 0, 3, 0, -2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, -2, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, 0, -2, 0, 2, 0, -1) -> 1.0, WrappedArray(0, 0, -1, 1, -2, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, 0, -4, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, -1, 0, 2, 0) -> 1.0, WrappedArray(0, 2, 0, -1, 1, 0, 0, -2, 0, 2) -> 1.0, WrappedArray(-2, 3, -1, 1, 0, 0, -3, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 1, -1, 0, 0, 2, 0) -> 1.0, WrappedArray(-2, 1, 0, 0, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -5, 0, 0, 0, 0, 0, 2) -> 1.0, WrappedArray(1, 0, -2, 0, 0, 1, -1, 2, 0, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 2, -1, 1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 5, -1, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(-1, 0, 1, 0, -2, 0, 0, 3, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 0, 0, 1) -> 1.0, WrappedArray(-2, 0, 0, 0, 2, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(1, 0, -3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 0, 0, 3, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -2, 2, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(2, 0, -1, 0, 1, 0, -2, 0, 2, 0) -> 1.0, WrappedArray(-5, 0, 0, 0, 0, 0, 2, 0, -2, 1) -> 1.0, WrappedArray(1, 0, 0, -1, 2, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(2, -1, 0, 0, 0, 1, 0, -2, 0, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, -1, 1, -2) -> 1.0, WrappedArray(0, 0, 2, -2, 0, 1, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 3, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -3, 1, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 1, -1, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, -3, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 1, 0, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(1, 0, -2, 1, 0, -1, 0, 0, 0, 3) -> 1.0, WrappedArray(2, -1, 0, 1, 0, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 0, 0, 0, 1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 0, 2, -2) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -4, 0, 1, -1, 0) -> 1.0, WrappedArray(-1, 0, 0, 1, 0, -1, 0, 0, 0, 2) -> 1.0, WrappedArray(3, -1, 1, 0, 0, -3, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 4, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -1, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 1, 0, 0, -4, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 4, 0, 0, -2, 1, -1, 0, 2, -1) -> 1.0, WrappedArray(0, 0, -3, 0, 0, 2, -2, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 3, 0, 0, 0, 0, -1) -> 1.0, WrappedArray(-1, 0, 3, 0, 0, 0, -2, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, -4, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 2, 0, 0, 0, 0, 0, 0, -4) -> 1.0, WrappedArray(2, 0, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 1, -6, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 1, -1, 0, 0, 3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -2, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 3, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 1, -1, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 2, -2, 1, 0, -1, 0, 0, 3, -1) -> 1.0, WrappedArray(0, 1, -3, 0, 0, 0, 0, 0, 3, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 1, 0, 0, -4) -> 1.0, WrappedArray(0, 2, -2, 0, 1, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(1, 0, -2, 0, 1, 0, -1, 1, 0, -1) -> 1.0, WrappedArray(-1, 0, 3, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, -1, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 1, 0, -2, 0, 0, 1, -1, 2) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, -4, 0) -> 1.0, WrappedArray(0, 0, 0, 2, 0, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 1, 0, 0, -2) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 2, 0, 0, -3, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, -1, 1, -2, 0, 0) -> 1.0, WrappedArray(2, 0, -1, 0, 1, -1, 0, 1, 0, -2) -> 1.0, WrappedArray(0, 0, 1, 0, -1, 0, 3, 0, 0, 0) -> 1.0, WrappedArray(0, 4, -2, 0, 1, 0, 0, -1, 0, 2) -> 1.0, WrappedArray(0, 1, 0, -2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(2, 0, 0, 0, 0, -2, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 2, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, -2, 1, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 1, -2, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 0, 4, 0, 0) -> 1.0, WrappedArray(6, 0, 0, 0, -1, 0, 0, 1, 0, -2) -> 1.0, WrappedArray(0, 0, 0, -5, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 4, 0, 0, -2, 0) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, -2, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(-3, 0, 0, 0, 0, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -4, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 0, 2, 0) -> 1.0, WrappedArray(0, 0, -1, 0, 0, 2, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 1, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 3, 0, 0, -2, 2, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, -4, 0, 0, 4) -> 1.0, WrappedArray(-1, 1, 0, 0, 0, -1, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 0, -2, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, -3, 0, 0, 2) -> 1.0, WrappedArray(0, -1, 0, 1, 0, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, -2, 0, 0, 2) -> 1.0, WrappedArray(-2, 0, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 0, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 0, 4, -2, 0, 1, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 2, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 1, -1, 2, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(1, 0, 0, 0, 0, -2, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, -1, 0, 0, 2, -1) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 0, 1, 0, -1) -> 1.0, WrappedArray(1, 0, -2, 0, 2, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, 0, -2, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 3, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 2, -2, 0, 6, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 0, 2, 0, 0) -> 1.0, WrappedArray(-2, 1, 0, -1, 0, 0, 0, 3, 0, -1) -> 1.0, WrappedArray(0, 1, -1, 0, 0, 0, 3, 0, 0, -2) -> 1.0, WrappedArray(-1, 1, 0, 0, 0, 0, -1, 1, -2, 1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 5, 0, -1, 1, -2, 0, 0, 2) -> 1.0, WrappedArray(0, 0, 0, -2, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 7, 0, 0) -> 1.0, WrappedArray(3, 0, -1, 1, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, 0, 0, 0, -5, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, 0, -1, 1, 0, 0) -> 1.0, WrappedArray(0, 0, -2, 2, 0, 0, -1, 1, 0, 0) -> 1.0, WrappedArray(-2, 0, 0, 0, 0, 0, 0, 0, 0, 3) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -6, 0, 0, 0, 0, 0, 4) -> 1.0, WrappedArray(1, 0, -1, 0, 1, 0, -1, 0, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 1, -1, 0, 2, 0) -> 1.0, WrappedArray(0, -2, 0, 1, -1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 1, 0, 0, -4, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 0, 1, -1, 0, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, -1, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 3, -1, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 1, 0, -2) -> 1.0, WrappedArray(0, 1, -1, 0, 1, 0, -2, 1, 0, -1) -> 1.0, WrappedArray(0, 0, 1, 0, 0, 0, 0, -1, 0, 1) -> 1.0, WrappedArray(0, -1, 1, 0, -2, 0, 1, 0, -1, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 1, 0, -1, 0, 0, 2) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 1, 0, -2, 0, 1) -> 1.0, WrappedArray(0, -4, 0, 0, 0, 0, 2, 0, 0, 0) -> 1.0, WrappedArray(0, 2, 0, 0, 0, 0, -2, 1, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 3, 0, 0, -2, 0, 0, 1) -> 1.0, WrappedArray(0, 0, 2, 0, -1, 1, 0, 0, -2, 0) -> 1.0, WrappedArray(-1, 0, 1, 0, 0, -2, 0, 0, 0, 1) -> 1.0, WrappedArray(0, -1, 1, -6, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, -3, 0, 0, 0, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 3, 0, -1, 1) -> 1.0, WrappedArray(0, 0, 0, 1, -2, 0, 1, -1, 2, 0) -> 1.0, WrappedArray(0, 0, -4, 0, 1, 0, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 1, 0, 0, 0, -1, 0, 1, 0, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 1, -1, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(0, 0, 0, 1, -1, 0, 0, 0, 0, 4) -> 1.0, WrappedArray(1, 0, -1, 0, 1, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(-1, 0, 0, 0, 1, -1, 0, 2, -2, 0) -> 1.0, WrappedArray(-1, 0, 0, 2, -1, 0, 0, 0, 1, 0) -> 1.0, WrappedArray(1, 0, -1, 0, 0, 0, 2, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 1, 0, -2, 0, 1, 0) -> 1.0, WrappedArray(0, -1, 0, 0, 0, 1, 0, 0, 0, -1) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 1, -2, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 0, 2, -1) -> 1.0, WrappedArray(0, 0, 0, 0, 2, 0, 0, 0, 0, -2) -> 1.0, WrappedArray(-1, 4, 0, -1, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 2, 0, -1, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 3, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 1, 0, -1, 0, 0, 1, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 1, 0, -1, 0, 3, 0) -> 1.0, WrappedArray(2, 0, -1, 1, 0, 0, 0, -1, 0, 0) -> 1.0, WrappedArray(0, -2, 0, 0, 0, 0, 0, 3, -1, 0) -> 1.0, WrappedArray(0, 0, 1, 0, -1, 0, 0, 0, 1, -1) -> 1.0, WrappedArray(-1, 0, 1, -1, 0, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -2, 2, -1, 0, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, -1, 0, 0, 1, 0, 0) -> 1.0, WrappedArray(-4, 0, 0, 1, 0, 0, 0, 0, -1, 0) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 3, -1, 0, 1, 0, 0, -3, 1) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 1, -1, 4) -> 1.0, WrappedArray(0, -6, 0, 0, 0, 0, 0, 4, 0, 0) -> 1.0, WrappedArray(0, -3, 0, 1, 0, -1, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, -1, 0, 0, 2, -1, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 0, 0, -5) -> 1.0, WrappedArray(-1, 3, 0, 0, 0, 0, 0, 0, -2, 0) -> 1.0, WrappedArray(0, 0, 1, -1, 0, 2, 0, 0, 0, -3) -> 1.0, WrappedArray(0, 0, 0, 1, 0, -1, 0, 0, 1, 0) -> 1.0, WrappedArray(-1, 0, 0, 4, 0, 0, -2, 1, -1, 0) -> 1.0, WrappedArray(0, 0, 2, 0, 0, 0, 0, -2, 1, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, -5, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 3, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 0, 0, 0, 0, 4, 0, 0, 0) -> 1.0, WrappedArray(1, -1, 1, -1, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(0, 5, 0, -1, 1, -2, 0, 0, 2, 0) -> 1.0, WrappedArray(0, -1, 0, 1, -2, 0, 0, 0, 0, 0) -> 1.0, WrappedArray(0, 0, 0, 0, 0, 0, 0, 1, -1, 0) -> 1.0, WrappedArray(-8, 0, 0, 0, 0, 0, 0, 0, 0, 0) -> ...


def aggToMatrix(totalVarDists: Seq[Seq[Map[Seq[Int],Double]]], aggFunc: Seq[Double] => Double): Seq[Seq[Double]] = {
  totalVariationDistances.map(_.map( t => aggFunc(t.values.toSeq)))
}

def printMatrix(mat: Seq[Seq[Double]]): Unit = {
  mat.map( s => { s.map( a => print(f"$a%2.3f ") ); println() } )
}


aggToMatrix: (totalVarDists: Seq[Seq[Map[Seq[Int],Double]]], aggFunc: Seq[Double] => Double)Seq[Seq[Double]]
printMatrix: (mat: Seq[Seq[Double]])Unit

We compute three different metrics of the total variation distances.

maxDists is the maximal total variation distance over the different lagKeys.
minDists is the minimal total variation distance over the different lagKeys.
meanDists is an estimate of the average total variation distance over the different lagKeys.


val maxDists = aggToMatrix(totalVariationDistances, s => s.max)
val minDists = aggToMatrix(totalVariationDistances, s => s.min)
val meanDists = aggToMatrix(totalVariationDistances, s => s.sum/s.size)


maxDists: Seq[Seq[Double]] = Vector(Vector(0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0), Vector(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0))
minDists: Seq[Seq[Double]] = Vector(Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1111111111111111), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), Vector(0.1111111111111111, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))
meanDists: Seq[Seq[Double]] = Vector(Vector(0.0, 0.8095238095238095, 0.9595959595959596, 0.9902676399026764, 0.9975932611311673, 0.9992509363295881, 0.999785112600997, 0.9999212988949179, 0.9999637822897781, 0.9999828303297199), Vector(0.8095238095238095, 0.0, 0.7878787878787878, 0.948905109489051, 0.9879412354592861, 0.9962818797931157, 0.9988235824354509, 0.9995277161089322, 0.9997797200520187, 0.9999006682605777), Vector(0.9595959595959596, 0.7878787878787878, 0.0, 0.7675182481751823, 0.9456465498353374, 0.9839165434703036, 0.9949339063117071, 0.9979342365055828, 0.9990263999070653, 0.9995453963182023), Vector(0.9902676399026764, 0.948905109489051, 0.7675182481751823, 0.0, 0.76569626045364, 0.9313390461281573, 0.9778513135067101, 0.9912980359273889, 0.9959874223204074, 0.998122961240012), Vector(0.9975932611311673, 0.9879412354592861, 0.9456465498353374, 0.76569626045364, 0.0, 0.709855350171288, 0.9069532060773262, 0.9635540738278638, 0.9834585383266629, 0.992240400344003), Vector(0.9992509363295881, 0.9962818797931157, 0.9839165434703036, 0.9313390461281573, 0.709855350171288, 0.0, 0.6846310864524279, 0.8780374739844375, 0.9450969813814848, 0.9743411917827652), Vector(0.999785112600997, 0.9988235824354509, 0.9949339063117071, 0.9778513135067101, 0.9069532060773262, 0.6846310864524279, 0.0, 0.6217124078957903, 0.8326361431674983, 0.9226862715145945), Vector(0.9999212988949179, 0.9995277161089322, 0.9979342365055828, 0.9912980359273889, 0.9635540738278638, 0.8780374739844375, 0.6217124078957903, 0.0, 0.57277338181383, 0.8064805024139434), Vector(0.9999637822897781, 0.9997797200520187, 0.9990263999070653, 0.9959874223204074, 0.9834585383266629, 0.9450969813814848, 0.8326361431674983, 0.57277338181383, 0.0, 0.5653338124646969), Vector(0.9999828303297199, 0.9999006682605777, 0.9995453963182023, 0.998122961240012, 0.992240400344003, 0.9743411917827652, 0.9226862715145945, 0.8064805024139434, 0.5653338124646969, 0.0))

Each model is a mapping {key: {value: probability}} where key (lagKey) is a sequence of reversals and non-reversals of length m, value is a sequence of trends of length n and probability is the estimated probability that value is observed immediately following key.

Hence, for any two models A and B, we can calculate the total variation distance between the mappings {value: probability} for a given key in the union of the keys for A and B. If a key is not present in one of the models, the total variation distance is 1 for that key.

In the matrix below, the (i,j)-th position is the arithmetic mean of the total variation distances for all keys in the union of the keysets. The matrix is symmetric with the smallest model in the top row and leftmost column and the largest model in the bottom row and rightmost column. All diagonal elements are 0 since the total variation distance from one model to itself is always 0.

If there are three models labeled M1, M2, M3 and Vi_j is the arithmetic mean described above, the matrix is

$\begin{matrix} V_{1,1} & V_{1,2} & V_{1,3} \end{matrix}$ $\begin{matrix} V_{2,1} & V_{2,2} & V_{2,3} \end{matrix}$ $\begin{matrix} V_{3,1} & V_{3,2} & V_{3,3} \end{matrix}$

As one can see, the models differ a lot from each other, suggesting that the estimate can still be improved given more data.


printMatrix(meanDists)


0.000 0.810 0.960 0.990 0.998 0.999 1.000 1.000 1.000 1.000 
0.810 0.000 0.788 0.949 0.988 0.996 0.999 1.000 1.000 1.000 
0.960 0.788 0.000 0.768 0.946 0.984 0.995 0.998 0.999 1.000 
0.990 0.949 0.768 0.000 0.766 0.931 0.978 0.991 0.996 0.998 
0.998 0.988 0.946 0.766 0.000 0.710 0.907 0.964 0.983 0.992 
0.999 0.996 0.984 0.931 0.710 0.000 0.685 0.878 0.945 0.974 
1.000 0.999 0.995 0.978 0.907 0.685 0.000 0.622 0.833 0.923 
1.000 1.000 0.998 0.991 0.964 0.878 0.622 0.000 0.573 0.806 
1.000 1.000 0.999 0.996 0.983 0.945 0.833 0.573 0.000 0.565 
1.000 1.000 1.000 0.998 0.992 0.974 0.923 0.806 0.565 0.000

sds-3.x/ScaDaMaLe

Markov Model for Trend Calculus