1. Test Modules
  2. Training Characteristics
    1. Input Learning
      1. Gradient Descent
      2. Conjugate Gradient Descent
      3. Limited-Memory BFGS
    2. Results
  3. Results

Target Description: The type Assert dimensions layer.

Report Description: The type Basic.

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 2855730146805100544

Training Characteristics

Input Learning

In this apply, we use a network to learn this target input, given it's pre-evaluated output:

TrainingTester.java:445 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
      try {
        return x.prettyPrint();
      } finally {
        x.freeRef();
      }
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ [ 0.496, 0.08 ], [ 0.7, -0.128 ] ]
    [ [ 0.08, 0.496 ], [ 0.7, -0.128 ] ]
    [ [ -0.128, 0.08 ], [ 0.7, 0.496 ] ]
    [ [ 0.7, -0.128 ], [ 0.496, 0.08 ] ]
    [ [ 0.7, -0.128 ], [ 0.496, 0.08 ] ]

Gradient Descent

First, we train using basic gradient descent method apply weak line search conditions.

TrainingTester.java:638 executed in 0.33 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 3894017868285
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 3894048439471
Constructing line search parameters: GD
th(0)=16.169926427548848;dx=-6.8896640000000004E22
New Minimum: 16.169926427548848 > 0.04113794444684464
Armijo: th(2.154434690031884)=0.04113794444684464; dx=-6.889664000078004E10 evalInputDelta=16.128788483102003
Armijo: th(1.077217345015942)=0.09451886944630736; dx=-6.889664000079366E10 evalInputDelta=16.07540755810254
Armijo: th(0.3590724483386473)=0.36622933791252255; dx=-6.889664000123738E10 evalInputDelta=15.803697089636326
Armijo: th(0.08976811208466183)=0.8958355405184305; dx=-6.88966400034462E10 evalInputDelta=15.274090887030418
Armijo: th(0.017953622416932366)=1.2385132659475149; dx=-6.889664000690414E10 evalInputDelta=14.931413161601334
Armijo: th(0.002992270402822061)=1.3547495983536328; dx=-6.889664000878992E10 evalInputDelta=14.815176829195215
Armijo: th(4.2746720040315154E-4)=1.3778437597877013; dx=-6.889664000922585E10 evalInputDelta=14.792082667761147
Armijo: th(5.343340005039394E-5)=1.3813071007473823; dx=-6.889664000929318E10 evalInputDelta=14.788619326801467
Armijo: th(5.9370444500437714E-6)=1.3817486978562186; dx=-6.88966400093018E10 evalInputDelta=14.78817772969263
Armijo: th(5.937044450043771E-7)=1.3817984031708268; dx=-6.889664000930278E10 evalInputDelta=14.788128024378022
Armijo: th(5.397313136403428E-8)=1.3818034241982362; dx=-6.889664000930287E10 evalInputDelta=14.788123003350613
Armijo: th(4.4977609470028565E-9)=1.3818038844617313; dx=-6.889664000930289E10 evalInputDelta=14.788122543087116
Armijo: th(3.4598161130791205E-10)=1.3818039230852617; dx=-6.889664000930289E10 evalInputDelta=14.788122504463587
Armijo: th(2.4712972236279432E-11)=1.4029076278058599; dx=-6.95979801983369E10 evalInputDelta=14.767018799742988
Armijo: th(1.6475314824186289E-12)=3.1113160236615975; dx=-2.5600000085050432E20 evalInputDelta=13.05861040388725
Armijo: th(1.029707176511643E-13)=16.169926427547885; dx=-6.8896640000000004E22 evalInputDelta=9.627854069549358E-13
Armijo: th(6.057101038303783E-15)=16.16992642754879; dx=-6.8896640000000004E22 evalInputDelta=5.6843418860808015E-14
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.04113794444684464
Fitness changed from 16.169926427548848 to 0.04113794444684464
Iteration 1 complete. Error: 0.04113794444684464 Total: 0.1337; Orientation: 0.0031; Line Search: 0.0932
th(0)=0.04113794444684464;dx=-0.1521108068937866
New Minimum: 0.04113794444684464 > 0.04113794444684461
WOLFE (weak): th(2.154434690031884E-15)=0.04113794444684461; dx=-0.1521108068937866 evalInputDelta=2.7755575615628914E-17
New Minimum: 0.04113794444684461 > 0.041137944446844585
WOLFE (weak): th(4.308869380063768E-15)=0.041137944446844585; dx=-0.1521108068937866 evalInputDelta=5.551115123125783E-17
New Minimum: 0.041137944446844585 > 0.04113794444684445
WOLFE (weak): th(1.2926608140191303E-14)=0.04113794444684445; dx=-0.1521108068937866 evalInputDelta=1.8735013540549517E-16
New Minimum: 0.04113794444684445 > 0.041137944446843884
WOLFE (weak): th(5.1706432560765214E-14)=0.041137944446843884; dx=-0.15211080689378653 evalInputDelta=7.563394355258879E-16
New Minimum: 0.041137944446843884 > 0.04113794444684089
WOLFE (weak): th(2.5853216280382605E-13)=0.04113794444684089; dx=-0.15211080689378623 evalInputDelta=3.747002708109903E-15
New Minimum: 0.04113794444684089 > 0.04113794444682216
WOLFE (weak): th(1.5511929768229563E-12)=0.04113794444682216; dx=-0.1521108068937843 evalInputDelta=2.248201624865942E-14
New Minimum: 0.04113794444682216 > 0.04113794444668728
WOLFE (weak): th(1.0858350837760695E-11)=0.04113794444668728; dx=-0.15211080689377057 evalInputDelta=1.5736023595280813E-13
New Minimum: 0.04113794444668728 > 0.041137944445585745
WOLFE (weak): th(8.686680670208556E-11)=0.041137944445585745; dx=-0.1521108068936584 evalInputDelta=1.2588957654102728E-12
New Minimum: 0.041137944445585745 > 0.041137944435514585
WOLFE (weak): th(7.8180126031877E-10)=0.041137944435514585; dx=-0.1521108068926328 evalInputDelta=1.1330054949798551E-11
New Minimum: 0.041137944435514585 > 0.04113794433354408
WOLFE (weak): th(7.818012603187701E-9)=0.04113794433354408; dx=-0.15211080688224857 evalInputDelta=1.1330056337577332E-10
New Minimum: 0.04113794433354408 > 0.04113794320053839
WOLFE (weak): th(8.599813863506471E-8)=0.04113794320053839; dx=-0.15211080676686814 evalInputDelta=1.2463062526446578E-9
New Minimum: 0.04113794320053839 > 0.04113792949117033
WOLFE (weak): th(1.0319776636207765E-6)=0.04113792949117033; dx=-0.15211080537076527 evalInputDelta=1.4955674310090927E-8
New Minimum: 0.04113792949117033 > 0.041137750023201275
WOLFE (weak): th(1.3415709627070094E-5)=0.041137750023201275; dx=-0.15211078709453596 evalInputDelta=1.9442364336541562E-7
New Minimum: 0.041137750023201275 > 0.04113522254000879
WOLFE (weak): th(1.878199347789813E-4)=0.04113522254000879; dx=-0.15211052970954397 evalInputDelta=2.7219068358536047E-6
New Minimum: 0.04113522254000879 > 0.04109712130960218
WOLFE (weak): th(0.0028172990216847197)=0.04109712130960218; dx=-0.15210665032081377 evalInputDelta=4.082313724246278E-5
New Minimum: 0.04109712130960218 > 0.040486175048326724
WOLFE (weak): th(0.045076784346955515)=0.040486175048326724; dx=-0.15204460636078893 evalInputDelta=6.517693985179165E-4
New Minimum: 0.040486175048326724 > 0.03132216233983654
WOLFE (weak): th(0.7663053338982437)=0.03132216233983654; dx=-0.151160471641101 evalInputDelta=0.009815782107008099
New Minimum: 0.03132216233983654 > 0.0023889230230832725
WOLFE (weak): th(13.793496010168386)=0.0023889230230832725; dx=-0.1500558586644081 evalInputDelta=0.03874902142376137
New Minimum: 0.0023889230230832725 > 0.0
WOLFE (weak): th(262.07642419319933)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(5241.528483863986)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(110072.09816114372)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(2421586.1595451618)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(1265829.1288531527)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(687950.6135071482)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(399011.355834146)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(254541.72699764484)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(326776.5414158954)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(290659.1342067701)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
Armijo: th(272600.4306022075)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(263571.07879992615)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(268085.7547010668)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
WOLFE (weak): th(270343.09265163715)=0.0; dx=-0.14998359583880083 evalInputDelta=0.04113794444684464
mu ~= nu (270343.09265163715): th(262.07642419319933)=0.0
Fitness changed from 0.04113794444684464 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.1270; Orientation: 0.0013; Line Search: 0.1223
th(0)=0.0;dx=-0.14848319999999998
Armijo: th(584868.1806131077)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(292434.0903065539)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(97478.03010218462)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(24369.507525546156)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(4873.901505109231)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(812.3169175182052)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(116.04527393117216)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(14.50565924139652)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(1.6117399157107244)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(0.16117399157107243)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(0.014652181051915675)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(0.0012210150876596395)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(9.392423751227996E-5)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(6.7088741080199975E-6)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(4.4725827386799985E-7)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(2.795364211674999E-8)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(1.6443318892205876E-9)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(9.135177162336598E-11)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(4.807987980177157E-12)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(2.4039939900885786E-13)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Armijo: th(1.1447590428993231E-14)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
MIN ALPHA (5.203450194996924E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0555; Orientation: 0.0007; Line Search: 0.0528
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.316s (< 30.000s)

Returns

    0.0

Training Converged

Conjugate Gradient Descent

First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.

TrainingTester.java:603 executed in 0.11 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 3894340619215
Reset training subject: 3894341895104
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=16.169926427548848}, derivative=-6.8896640000000004E22}
New Minimum: 16.169926427548848 > 1.3818039253736005
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=1.3818039253736005}, derivative=-6.889664000930289E10}, evalInputDelta = -14.788122502175248
New Minimum: 1.3818039253736005 > 1.3818039197918688
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=1.3818039197918688}, derivative=-6.889664000930289E10}, evalInputDelta = -14.78812250775698
New Minimum: 1.3818039197918688 > 1.3818038807197472
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=1.3818038807197472}, derivative=-6.889664000930289E10}, evalInputDelta = -14.7881225468291
New Minimum: 1.3818038807197472 > 1.3818036072149875
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=1.3818036072149875}, derivative=-6.889664000930287E10}, evalInputDelta = -14.78812282033386
New Minimum: 1.3818036072149875 > 1.3818016926860672
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=1.3818016926860672}, derivative=-6.889664000930284E10}, evalInputDelta = -14.78812473486278
New Minimum: 1.3818016926860672 > 1.3817882911991815
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=1.3817882911991815}, derivative=-6.889664000930258E10}, evalInputDelta = -14.788138136349666
New Minimum: 1.3817882911991815 > 1.3816944913515639
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=1.3816944913515639}, derivative=-6.889664000930074E10}, evalInputDelta = -14.788231936197285
New Minimum: 1.3816944913515639 > 1.3810384093155363
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=1.3810384093155363}, derivative=-6.889664000928793E10}, evalInputDelta = -14.788888018233312
New Minimum: 1.3810384093155363 > 1.3764709688358279
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=1.3764709688358279}, derivative=-6.88966400091993E10}, evalInputDelta = -14.79345545871302
New Minimum: 1.3764709688358279 > 1.3456677158758286
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=1.3456677158758286}, derivative=-6.889664000862454E10}, evalInputDelta = -14.82425871167302
New Minimum: 1.3456677158758286 > 1.1721839029806258
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=1.1721839029806258}, derivative=-6.889664000602399E10}, evalInputDelta = -14.997742524568222
New Minimum: 1.1721839029806258 > 0.6168331602056226
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.6168331602056226}, derivative=-6.889664000199237E10}, evalInputDelta = -15.553093267343225
New Minimum: 0.6168331602056226 > 0.07760809906208668
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.07760809906208668}, derivative=-6.889664000078874E10}, evalInputDelta = -16.092318328486762
New Minimum: 0.07760809906208668 > 0.008109322689957793
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.008109322689957793}, derivative=-6.889664000077478E10}, evalInputDelta = -16.16181710485889
New Minimum: 0.008109322689957793 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
0.0 <= 16.169926427548848
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.88966400007745E10}, evalInputDelta = -16.169926427548848
Right bracket at 1.0E10
Converged to right
Fitness changed from 16.169926427548848 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0638; Orientation: 0.0004; Line Search: 0.0596
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
0.0 <= 0.0
F(5.0E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 5.0E9
F(2.5E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 2.5E9
F(1.25E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 1.25E9
F(6.25E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 6.25E8
F(3.125E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 3.125E8
F(1.5625E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 1.5625E8
F(7.8125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 7.8125E7
F(3.90625E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 3.90625E7
F(1.953125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 1.953125E7
F(9765625.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 9765625.0
F(4882812.5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Right bracket at 4882812.5
F(2441406.25) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.14848319999999998}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0433; Orientation: 0.0004; Line Search: 0.0415
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.108s (< 30.000s)

Returns

    0.0

Training Converged

Limited-Memory BFGS

Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.

TrainingTester.java:674 executed in 1.10 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new LBFGS());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setIterationsPerSample(100);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 3894452017904
Reset training subject: 3894452901653
Adding measurement 49e8704b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 16.169926427548848 < 16.169926427548848. Total: 1
th(0)=16.169926427548848;dx=-6.8896640000000004E22
Adding measurement 2295c060 to history. Total: 1
New Minimum: 16.169926427548848 > 0.04113794444684464
Armijo: th(2.154434690031884)=0.04113794444684464; dx=-6.889664000078004E10 evalInputDelta=16.128788483102003
Non-optimal measurement 0.09451886944630736 < 0.04113794444684464. Total: 2
Armijo: th(1.077217345015942)=0.09451886944630736; dx=-6.889664000079366E10 evalInputDelta=16.07540755810254
Non-optimal measurement 0.36622933791252255 < 0.04113794444684464. Total: 2
Armijo: th(0.3590724483386473)=0.36622933791252255; dx=-6.889664000123738E10 evalInputDelta=15.803697089636326
Non-optimal measurement 0.8958355405184305 < 0.04113794444684464. Total: 2
Armijo: th(0.08976811208466183)=0.8958355405184305; dx=-6.88966400034462E10 evalInputDelta=15.274090887030418
Non-optimal measurement 1.2385132659475149 < 0.04113794444684464. Total: 2
Armijo: th(0.017953622416932366)=1.2385132659475149; dx=-6.889664000690414E10 evalInputDelta=14.931413161601334
Non-optimal measurement 1.3547495983536328 < 0.04113794444684464. Total: 2
Armijo: th(0.002992270402822061)=1.3547495983536328; dx=-6.889664000878992E10 evalInputDelta=14.815176829195215
Non-optimal measurement 1.3778437597877013 < 0.04113794444684464. Total: 2
Armijo: th(4.2746720040315154E-4)=1.3778437597877013; dx=-6.889664000922585E10 evalInputDelta=14.792082667761147
Non-optimal measurement 1.3813071007473823 < 0.04113794444684464. Total: 2
Armijo: th(5.343340005039394E-5)=1.3813071007473823; dx=-6.889664000929318E10 evalInputDelta=14.788619326801467
Non-optimal measurement 1.3817486978562186 < 0.04113794444684464. Total: 2
Armijo: th(5.9370444500437714E-6)=1.3817486978562186; dx=-6.88966400093018E10 evalInputDelta=14.78817772969263
Non-optimal measurement 1.3817984031708268 < 0.04113794444684464. Total: 2
Armijo: th(5.937044450043771E-7)=1.3817984031708268; dx=-6.889664000930278E10 evalInputDelta=14.788128024378022
Non-optimal measurement 1.3818034241982362 < 0.04113794444684464. Total: 2
Armijo: th(5.397313136403428E-8)=1.3818034241982362; dx=-6.889664000930287E10 evalInputDelta=14.788123003350613
Non-optimal measurement 1.3818038844617313 < 0.04113794444684464. Total: 2
Armijo: th(4.4977609470028565E-9)=1.3818038844617313; dx=-6.889664000930289E10 evalInputDelta=14.788122543087116
Non-optimal measurement 1.3818039230852617 < 0.04113794444684464. Total: 2
Armijo: th(3.4598161130791205E-10)=1.3818039230852617; dx=-6.889664000930289E10 evalInputDelta=14.788122504463587
Non-optimal measurement 1.4029076278058599 < 0.04113794444684464. Total: 2
Armijo: th(2.4712972236279432E-11)=1.4029076278058599; dx=-6.95979801983369E10 evalInputDelta=14.767018799742988
Non-optimal measurement 3.1113160236615975 < 0.04113794444684464. Total: 2
Armijo: th(1.6475314824186289E-12)=3.1113160236615975; dx=-2.5600000085050432E20 evalInputDelta=13.05861040388725
Non-optimal measurement 16.169926427547885 < 0.04113794444684464. Total: 2
Armijo: th(1.029707176511643E-13)=16.169926427547885; dx=-6.8896640000000004E22 evalInputDelta=9.627854069549358E-13
Non-optimal measurement 16.16992642754879 < 0.04113794444684464. Total: 2
Armijo: th(6.057101038303783E-15)=16.16992642754879; dx=-6.8896640000000004E22 evalInputDelta=5.6843418860808015E-14
Non-optimal measurement 0.04113794444684464 < 0.04113794444684464. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.04113794444684464
Fitness changed from 16.169926427548848 to 0.04113794444684464
Iteration 1 complete. Error: 0.04113794444684464 Total: 0.0442; Orientation: 0.0027; Line Search: 0.0386
Non-optimal measurement 0.04113794444684464 < 0.04113794444684464. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.04113794444684464 < 0.04113794444684464. Total: 2
th(0)=0.04113794444684464;dx=-0.1521108068937866
Adding measurement a50902c to history. Total: 2
New Minimum: 0.04113794444684464 > 0.04113794444684461
WOLFE (weak): th(2.154434690031884E-15)=0.04113794444684461; dx=-0.1521108068937866 evalInputDelta=2.7755575615628914E-17
Adding measurement 2f4f16db to history. Total: 3
New Minimum: 0.04113794444684461 > 0.041137944446844585
WOLFE (weak): th(4.308869380063768E-15)=0.041137944446844585; dx=-0.1521108068937866 evalInputDelta=5.551115123125783E-17
Adding measurement 61c6b4b9 to history. Total: 4
New Minimum: 0.041137944446844585 > 0.04113794444684445
WOLFE (weak): th(1.2926608140191303E-14)=0.04113794444684445; dx=-0.1521108068937866 evalInputDelta=1.8735013540549517E-16
Adding measurement 46cc0109 to history. Total: 5
New Minimum: 0.04113794444684445 > 0.041137944446843884
WOLFE (weak): th(5.1706432560765214E-14)=0.041137944446843884; dx=-0.15211080689378653 evalInputDelta=7.563394355258879E-16
Adding measurement 3c4f0781 to history. Total: 6
New Minimum: 0.041137944446843884 > 0.04113794444684089
WOLFE (weak): th(2.5853216280382605E-13)=0.04113794444684089; dx=-0.15211080689378623 evalInputDelta=3.747002708109903E-15
Adding measurement 4db6238f to history. Total: 7
New Minimum: 0.04113794444684089 > 0.04113794444682216
WOLFE (weak): th(1.5511929768229563E-12)=0.04113794444682216; dx=-0.1521108068937843 evalInputDelta=2.248201624865942E-14
Adding measurement 2d3108b3 to history. Total: 8
New Minimum: 0.04113794444682216 > 0.04113794444668728
WOLFE (weak): th(1.0858350837760695E-11)=0.04113794444668728; dx=-0.1521108068937706 evalInputDelta=1.5736023595280813E-13
Adding measurement f220b0a to history. Total: 9
New Minimum: 0.04113794444668728 > 0.041137944445585745
WOLFE (weak): th(8.686680670208556E-11)=0.041137944445585745; dx=-0.1521108068936584 evalInputDelta=1.2588957654102728E-12
Adding measurement 3b73295e to history. Total: 10
New Minimum: 0.041137944445585745 > 0.041137944435514585
WOLFE (weak): th(7.8180126031877E-10)=0.041137944435514585; dx=-0.1521108068926328 evalInputDelta=1.1330054949798551E-11
Adding measurement c8c85e0 to history. Total: 11
New Minimum: 0.041137944435514585 > 0.04113794433354408
WOLFE (weak): th(7.818012603187701E-9)=0.04113794433354408; dx=-0.15211080688224857 evalInputDelta=1.1330056337577332E-10
Adding measurement 558b17a0 to history. Total: 12
New Minimum: 0.04113794433354408 > 0.04113794320053839
WOLFE (weak): th(8.599813863506471E-8)=0.04113794320053839; dx=-0.15211080676686814 evalInputDelta=1.2463062526446578E-9
Adding measurement 693d9ea8 to history. Total: 13
New Minimum: 0.04113794320053839 > 0.04113792949117033
WOLFE (weak): th(1.0319776636207765E-6)=0.04113792949117033; dx=-0.15211080537076527 evalInputDelta=1.4955674310090927E-8
Adding measurement 2afc5fec to history. Total: 14
New Minimum: 0.04113792949117033 > 0.041137750023201275
WOLFE (weak): th(1.3415709627070094E-5)=0.041137750023201275; dx=-0.15211078709453596 evalInputDelta=1.9442364336541562E-7
Adding measurement 597bf550 to history. Total: 15
New Minimum: 0.041137750023201275 > 0.04113522254000879
WOLFE (weak): th(1.878199347789813E-4)=0.04113522254000879; dx=-0.15211052970954397 evalInputDelta=2.7219068358536047E-6
Adding measurement 6bc81db to history. Total: 16
New Minimum: 0.04113522254000879 > 0.04109712130960218
WOLFE (weak): th(0.0028172990216847197)=0.04109712130960218; dx=-0.15210665032081377 evalInputDelta=4.082313724246278E-5
Adding measurement 41d4ecc3 to history. Total: 17
New Minimum: 0.04109712130960218 > 0.040486175048326724
WOLFE (weak): th(0.045076784346955515)=0.040486175048326724; dx=-0.15204460636078893 evalInputDelta=6.517693985179165E-4
Adding measurement 2cf2c231 to history. Total: 18
New Minimum: 0.040486175048326724 > 0.03132216233983654
WOLFE (weak): th(0.7663053338982437)=0.03132216233983654; dx=-0.151160471641101 evalInputDelta=0.009815782107008099
Adding measurement 60047b7 to history. Total: 19
New Minimum: 0.03132216233983654 > 0.0023889230230832725
WOLFE (weak): th(13.793496010168386)=0.0023889230230832725; dx=-0.1500558586644081 evalInputDelta=0.03874902142376137
Adding mea

...skipping 10079 bytes...

7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879, 0.041137750023201275, 0.04113792949117033, 0.04113794320053839, 0.04113794433354408, 0.041137944435514585
Rejected: LBFGS Orientation magnitude: 5.444e+01, gradient 3.853e-01, dot -0.980; [07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, 609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879, 0.041137750023201275, 0.04113792949117033, 0.04113794320053839, 0.04113794433354408
Rejected: LBFGS Orientation magnitude: 5.444e+01, gradient 3.853e-01, dot -0.980; [1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00, 609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879, 0.041137750023201275, 0.04113792949117033, 0.04113794320053839
Rejected: LBFGS Orientation magnitude: 5.444e+01, gradient 3.853e-01, dot -0.980; [1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879, 0.041137750023201275, 0.04113792949117033
Rejected: LBFGS Orientation magnitude: 5.937e+01, gradient 3.853e-01, dot -0.982; [609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00, 1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879, 0.041137750023201275
Rejected: LBFGS Orientation magnitude: 7.649e+01, gradient 3.853e-01, dot -0.992; [ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00, 609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218, 0.04113522254000879
Rejected: LBFGS Orientation magnitude: 7.739e+01, gradient 3.853e-01, dot -0.992; [609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, 07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724, 0.04109712130960218
Rejected: LBFGS Orientation magnitude: 9.262e+01, gradient 3.853e-01, dot -0.992; [07ac6050-fc91-4e40-a798-10334d5fa3c1 = 1.000/1.000e+00, 1dc200ae-7260-4a12-9986-d63193bac940 = 1.000/1.000e+00, ae8ef5c4-6614-40f0-a340-c0c3e1dff3cb = 1.000/1.000e+00, 609ce7b3-9ac6-423f-9ede-95bcfd903b35 = 1.000/1.000e+00, 024540be-91e8-4435-9854-d25a7e57876d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.0023889230230832725, 0.03132216233983654, 0.040486175048326724
LBFGS Accumulation History: 3 points
Removed measurement 407d8561 to history. Total: 20
Removed measurement 60047b7 to history. Total: 19
Removed measurement 2cf2c231 to history. Total: 18
Removed measurement 41d4ecc3 to history. Total: 17
Removed measurement 6bc81db to history. Total: 16
Removed measurement 597bf550 to history. Total: 15
Removed measurement 2afc5fec to history. Total: 14
Removed measurement 693d9ea8 to history. Total: 13
Removed measurement 558b17a0 to history. Total: 12
Removed measurement c8c85e0 to history. Total: 11
Removed measurement 3b73295e to history. Total: 10
Removed measurement f220b0a to history. Total: 9
Removed measurement 2d3108b3 to history. Total: 8
Removed measurement 4db6238f to history. Total: 7
Removed measurement 3c4f0781 to history. Total: 6
Removed measurement 46cc0109 to history. Total: 5
Removed measurement 61c6b4b9 to history. Total: 4
Removed measurement 2f4f16db to history. Total: 3
Adding measurement 666b68de to history. Total: 3
th(0)=0.0;dx=-0.14848319999999998
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(584868.1806131077)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(292434.0903065539)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(97478.03010218462)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(24369.507525546156)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4873.901505109231)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(812.3169175182052)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(116.04527393117216)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(14.50565924139652)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.6117399157107244)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.16117399157107243)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.014652181051915675)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.0012210150876596395)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(9.392423751227996E-5)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(6.7088741080199975E-6)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.4725827386799985E-7)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.795364211674999E-8)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.6443318892205876E-9)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(9.135177162336598E-11)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.807987980177157E-12)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.4039939900885786E-13)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.1447590428993231E-14)=0.0; dx=-0.14848319999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (5.203450194996924E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.9636; Orientation: 0.9297; Line Search: 0.0330
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 1.101s (< 30.000s)

Returns

    0.0

Training Converged

TrainingTester.java:576 executed in 0.10 seconds (0.000 gc):

    return TestUtil.compare(title + " vs Iteration", runs);
Logging
Plotting range=[1.0, -2.385757412633895], [2.0, -0.38575741263389474]; valueStats=DoubleSummaryStatistics{count=2, sum=0.082276, min=0.041138, average=0.041138, max=0.041138}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

TrainingTester.java:579 executed in 0.01 seconds (0.000 gc):

    return TestUtil.compareTime(title + " vs Time", runs);
Logging
Plotting range=[0.0, -2.385757412633895], [0.127, -0.38575741263389474]; valueStats=DoubleSummaryStatistics{count=2, sum=0.082276, min=0.041138, average=0.041138, max=0.041138}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

Results

TrainingTester.java:350 executed in 0.00 seconds (0.000 gc):

    return grid(inputLearning, modelLearning, completeLearning);

Returns

Result

TrainingTester.java:353 executed in 0.00 seconds (0.000 gc):

    return new ComponentResult(null == inputLearning ? null : inputLearning.value,
        null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);

Returns

    {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}

LayerTests.java:605 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

detailsresult
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "2.322",
      "gc_time": "0.385"
    },
    "created_on": 1587005913346,
    "file_name": "trainingTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.AssertDimensionsLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/c9a1867488dc7e77a975f095285b5882c0486db6/src/test/java/com/simiacryptus/mindseye/layers/java/AssertDimensionsLayerTest.java",
      "javaDoc": "The type Basic."
    },
    "training_analysis": {
      "input": {
        "LBFGS": {
          "type": "Converged",
          "value": 0.0
        },
        "CjGD": {
          "type": "Converged",
          "value": 0.0
        },
        "GD": {
          "type": "Converged",
          "value": 0.0
        }
      }
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/AssertDimensionsLayer/Basic/trainingTest/202004165833",
    "id": "0549e0e4-eb17-42b2-8ca7-a36a54015396",
    "report_type": "Components",
    "display_name": "Comparative Training",
    "target": {
      "simpleName": "AssertDimensionsLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.AssertDimensionsLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/c9a1867488dc7e77a975f095285b5882c0486db6/src/main/java/com/simiacryptus/mindseye/layers/java/AssertDimensionsLayer.java",
      "javaDoc": "The type Assert dimensions layer."
    }
  }