1. Test Modules
  2. Training Characteristics
    1. Input Learning
      1. Gradient Descent
      2. Conjugate Gradient Descent
      3. Limited-Memory BFGS
    2. Results
  3. Results

Target Description: The type Img pixel softmax layer.

Report Description: The type Basic.

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 5996652123750114304

Training Characteristics

Input Learning

In this apply, we use a network to learn this target input, given it's pre-evaluated output:

TrainingTester.java:445 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
      try {
        return x.prettyPrint();
      } finally {
        x.freeRef();
      }
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [
    	[ [ -0.608, -0.384, 0.08 ], [ 1.208, -1.72, 1.764 ] ],
    	[ [ -1.028, 1.524, -0.128 ], [ 0.7, 0.048, 0.496 ] ]
    ]
    [
    	[ [ 0.7, 0.048, -0.384 ], [ -0.608, -1.028, -1.72 ] ],
    	[ [ 0.496, 1.208, 0.08 ], [ 1.764, -0.128, 1.524 ] ]
    ]
    [
    	[ [ 1.208, -0.608, -0.128 ], [ 0.08, 1.764, -1.72 ] ],
    	[ [ -0.384, 1.524, -1.028 ], [ 0.496, 0.7, 0.048 ] ]
    ]
    [
    	[ [ 0.7, 1.208, 0.048 ], [ -0.384, 1.524, -1.028 ] ],
    	[ [ -1.72, 0.496, -0.128 ], [ 0.08, 1.764, -0.608 ] ]
    ]
    [
    	[ [ 0.496, -1.72, -1.028 ], [ 1.764, 1.208, 0.048 ] ],
    	[ [ 0.08, -0.128, -0.384 ], [ 0.7, 1.524, -0.608 ] ]
    ]

Gradient Descent

First, we train using basic gradient descent method apply weak line search conditions.

TrainingTester.java:638 executed in 1.73 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 2981522337015
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 2981558951839
Constructing line search parameters: GD
th(0)=6.045431069096695;dx=-0.2967382679520778
New Minimum: 6.045431069096695 > 5.443494326030401
END: th(2.154434690031884)=5.443494326030401; dx=-0.2611224607540802 evalInputDelta=0.6019367430662941
Fitness changed from 6.045431069096695 to 5.443494326030401
Iteration 1 complete. Error: 5.443494326030401 Total: 0.0864; Orientation: 0.0035; Line Search: 0.0261
th(0)=5.443494326030401;dx=-0.23058263544533114
New Minimum: 5.443494326030401 > 4.52495953774945
END: th(4.641588833612779)=4.52495953774945; dx=-0.16350790924874908 evalInputDelta=0.9185347882809509
Fitness changed from 5.443494326030401 to 4.52495953774945
Iteration 2 complete. Error: 4.52495953774945 Total: 0.0243; Orientation: 0.0011; Line Search: 0.0171
th(0)=4.52495953774945;dx=-0.11925811689650978
New Minimum: 4.52495953774945 > 3.6825080195421096
END: th(10.000000000000002)=3.6825080195421096; dx=-0.04934001070866413 evalInputDelta=0.8424515182073407
Fitness changed from 4.52495953774945 to 3.6825080195421096
Iteration 3 complete. Error: 3.6825080195421096 Total: 0.0197; Orientation: 0.0010; Line Search: 0.0136
th(0)=3.6825080195421096;dx=-0.025769919873265178
New Minimum: 3.6825080195421096 > 3.461322637357123
WOLF (strong): th(21.544346900318843)=3.461322637357123; dx=0.006351749065164181 evalInputDelta=0.2211853821849865
END: th(10.772173450159421)=3.4832561728246105; dx=-0.010574542925683253 evalInputDelta=0.19925184671749907
Fitness changed from 3.6825080195421096 to 3.461322637357123
Iteration 4 complete. Error: 3.461322637357123 Total: 0.0280; Orientation: 0.0011; Line Search: 0.0209
th(0)=3.461322637357123;dx=-0.0045029705975177985
New Minimum: 3.461322637357123 > 3.437576236767408
WOLF (strong): th(23.2079441680639)=3.437576236767408; dx=0.002585945221962065 evalInputDelta=0.023746400589715222
New Minimum: 3.437576236767408 > 3.4287267616506845
END: th(11.60397208403195)=3.4287267616506845; dx=-0.0010564411263141898 evalInputDelta=0.032595875706438626
Fitness changed from 3.461322637357123 to 3.4287267616506845
Iteration 5 complete. Error: 3.4287267616506845 Total: 0.0323; Orientation: 0.0010; Line Search: 0.0264
th(0)=3.4287267616506845;dx=-4.800392107450423E-4
New Minimum: 3.4287267616506845 > 3.4212681135192957
END: th(25.000000000000007)=3.4212681135192957; dx=-1.192463256958196E-4 evalInputDelta=0.007458648131388745
Fitness changed from 3.4287267616506845 to 3.4212681135192957
Iteration 6 complete. Error: 3.4212681135192957 Total: 0.0215; Orientation: 0.0010; Line Search: 0.0124
th(0)=3.4212681135192957;dx=-1.8648426960825662E-4
Armijo: th(53.860867250797114)=3.4214054000322145; dx=1.8638234809732952E-4 evalInputDelta=-1.3728651291877725E-4
New Minimum: 3.4212681135192957 > 3.418824901142794
WOLF (strong): th(26.930433625398557)=3.418824901142794; dx=3.8508168325752104E-6 evalInputDelta=0.002443212376501691
END: th(8.976811208466186)=3.419882911206482; dx=-1.2225657544607044E-4 evalInputDelta=0.0013852023128135826
Fitness changed from 3.4212681135192957 to 3.418824901142794
Iteration 7 complete. Error: 3.418824901142794 Total: 0.0348; Orientation: 0.0009; Line Search: 0.0260
th(0)=3.418824901142794;dx=-1.3023732138082815E-4
New Minimum: 3.418824901142794 > 3.417630595200957
WOLF (strong): th(19.33995347338659)=3.417630595200957; dx=7.121265834586686E-6 evalInputDelta=0.001194305941837115
END: th(9.669976736693295)=3.4178956262801563; dx=-6.185099521118663E-5 evalInputDelta=9.29274862637719E-4
Fitness changed from 3.418824901142794 to 3.417630595200957
Iteration 8 complete. Error: 3.417630595200957 Total: 0.0255; Orientation: 0.0009; Line Search: 0.0199
th(0)=3.417630595200957;dx=-4.945389783558409E-5
New Minimum: 3.417630595200957 > 3.4170453573830186
END: th(20.833333333333346)=3.4170453573830186; dx=-6.8270501113385635E-6 evalInputDelta=5.852378179382889E-4
Fitness changed from 3.417630595200957 to 3.4170453573830186
Iteration 9 complete. Error: 3.4170453573830186 Total: 0.0178; Orientation: 0.0009; Line Search: 0.0122
th(0)=3.4170453573830186;dx=-2.5366867306649873E-5
New Minimum: 3.4170453573830186 > 3.416805923369594
WOLF (strong): th(44.88405604233094)=3.416805923369594; dx=1.469154978128534E-5 evalInputDelta=2.3943401342441106E-4
New Minimum: 3.416805923369594 > 3.4167008798535052
END: th(22.44202802116547)=3.4167008798535052; dx=-5.3329316096465496E-6 evalInputDelta=3.444775295133873E-4
Fitness changed from 3.4170453573830186 to 3.4167008798535052
Iteration 10 complete. Error: 3.4167008798535052 Total: 0.0323; Orientation: 0.0010; Line Search: 0.0217
th(0)=3.4167008798535052;dx=-1.5108743462744073E-5
New Minimum: 3.4167008798535052 > 3.4165863373680914
WOLF (strong): th(48.34988368346648)=3.4165863373680914; dx=1.0270457790549041E-5 evalInputDelta=1.1454248541387102E-4
New Minimum: 3.4165863373680914 > 3.416490218000169
END: th(24.17494184173324)=3.416490218000169; dx=-2.3439778418170055E-6 evalInputDelta=2.1066185333618037E-4
Fitness changed from 3.4167008798535052 to 3.416490218000169
Iteration 11 complete. Error: 3.416490218000169 Total: 0.0221; Orientation: 0.0008; Line Search: 0.0169
th(0)=3.416490218000169;dx=-1.0386798123664074E-5
Armijo: th(52.08333333333337)=3.4165188339505512; dx=1.149104804237487E-5 evalInputDelta=-2.8615950382171462E-5
New Minimum: 3.416490218000169 > 3.4163620839751188
WOLF (strong): th(26.041666666666686)=3.4163620839751188; dx=5.480739071636717E-7 evalInputDelta=1.2813402505029714E-4
END: th(8.680555555555562)=3.416415867912977; dx=-6.743198450066421E-6 evalInputDelta=7.435008719225422E-5
Fitness changed from 3.416490218000169 to 3.4163620839751188
Iteration 12 complete. Error: 3.4163620839751188 Total: 0.0254; Orientation: 0.0007; Line Search: 0.0202
th(0)=3.4163620839751188;dx=-8.818185408212769E-6
New Minimum: 3.4163620839751188 > 3.4162770589003033
END: th(18.701690017637894)=3.4162770589003033; dx=-2.827246538245922E-7 evalInputDelta=8.502507481544797E-5
Fitness changed from 3.4163620839751188 to 3.4162770589003033
Iteration 13 complete. Error: 3.4162770589003033 Total: 0.0154; Orientation: 0.0008; Line Search: 0.0097
th(0)=3.4162770589003033;dx=-3.82376326852941E-6
New Minimum: 3.4162770589003033 > 3.4162251178450234
WOLF (strong): th(40.29156973622207)=3.4162251178450234; dx=1.2429201914921114E-6 evalInputDelta=5.194105527994708E-5
END: th(20.145784868111036)=3.4162255701836513; dx=-1.2884837140940802E-6 evalInputDelta=5.148871665205945E-5
Fitness changed from 3.4162770589003033 to 3.4162251178450234
Iteration 14 complete. Error: 3.4162251178450234 Total: 0.0193; Orientation: 0.0007; Line Search: 0.0154
th(0)=3.4162251178450234;dx=-7.917881955049003E-6
Armijo: th(43.402777777777814)=3.4163997050149675; dx=1.5917080167091156E-5 evalInputDelta=-1.7458716994411816E-4
New Minimum: 3.4162251178450234 > 3.416183088751602
WOLF (strong): th(21.701388888888907)=3.416183088751602; dx=4.033935936853968E-6 evalInputDelta=4.202909342154726E-5
New Minimum: 3.416183088751602 > 3.416182280177778
END: th(7.233796296296302)=3.416182280177778; dx=-3.926980238214066E-6 evalInputDelta=4.2837667245443356E-5
Fitness changed from 3.4162251178450234 to 3.416182280177778
Iteration 15 complete. Error: 3.416182280177778 Total: 0.0228; Orientation: 0.0005; Line Search: 0.0195
th(0)=3.416182280177778;dx=-2.2466523999994534E-6
New Minimum: 3.416182280177778 > 3.4161594634661343
END: th(15.584741681364912)=3.4161594634661343; dx=-6.821018359247388E-7 evalInputDelta=2.2816711643613985E-5
Fitness changed from 3.416182280177778 to 3.4161594634661343
Iteration 16 complete. Error: 3.4161594634661343 Total: 0.0506; Orientation: 0.0268; Line Search: 0.0198
th(0)=3.4161594634661343;dx=-9.462959212627136E-7
New Minimum: 3.4161594634661343 > 3.416134622883104
END: th(33.576308113518394)=3.416134622883104; dx=-5.341908408586028E-7 evalInputDelta=2.484058303009462E-5
Fitness changed from 3.4161594634661343 to 3.416134622883104
Iteration 17 complete. Error: 3.41613462288310

...skipping 20959 bytes...

692217E-13
Fitness changed from 3.416092718565036 to 3.4160927185648786
Iteration 59 complete. Error: 3.4160927185648786 Total: 0.0108; Orientation: 0.0005; Line Search: 0.0077
Low gradient: 7.413253464435921E-8
th(0)=3.4160927185648786;dx=-5.4956326927971186E-15
New Minimum: 3.4160927185648786 > 3.416092718564652
END: th(50.70739962412083)=3.416092718564652; dx=-3.4483278563473154E-15 evalInputDelta=2.2648549702353193E-13
Fitness changed from 3.4160927185648786 to 3.416092718564652
Iteration 60 complete. Error: 3.416092718564652 Total: 0.0107; Orientation: 0.0005; Line Search: 0.0076
Low gradient: 6.367263946436267E-8
th(0)=3.416092718564652;dx=-4.054205016358715E-15
Armijo: th(109.24578079151563)=3.4160927185649625; dx=9.738773387031305E-15 evalInputDelta=-3.1041835768519377E-13
New Minimum: 3.416092718564652 > 3.416092718564619
WOLF (strong): th(54.622890395757814)=3.416092718564619; dx=2.842286859280314E-15 evalInputDelta=3.3306690738754696E-14
New Minimum: 3.416092718564619 > 3.416092718564599
END: th(18.20763013191927)=3.416092718564599; dx=-1.7553737973634124E-15 evalInputDelta=5.3290705182007514E-14
Fitness changed from 3.416092718564652 to 3.416092718564599
Iteration 61 complete. Error: 3.416092718564599 Total: 0.0393; Orientation: 0.0005; Line Search: 0.0360
Low gradient: 5.1025673012806425E-8
th(0)=3.416092718564599;dx=-2.603619306409841E-15
New Minimum: 3.416092718564599 > 3.4160927185645322
END: th(39.22714997947668)=3.4160927185645322; dx=-8.186858146098316E-16 evalInputDelta=6.661338147750939E-14
Fitness changed from 3.416092718564599 to 3.4160927185645322
Iteration 62 complete. Error: 3.4160927185645322 Total: 0.0295; Orientation: 0.0005; Line Search: 0.0244
Low gradient: 5.887791246274488E-8
th(0)=3.4160927185645322;dx=-3.466608575970648E-15
Armijo: th(84.51233270686805)=3.41609271856491; dx=1.2423443767190102E-14 evalInputDelta=-3.779199175824033E-13
Armijo: th(42.256166353434025)=3.4160927185645535; dx=4.4784194049667606E-15 evalInputDelta=-2.1316282072803006E-14
New Minimum: 3.4160927185645322 > 3.4160927185645016
END: th(14.085388784478008)=3.4160927185645016; dx=-8.18265513334136E-16 evalInputDelta=3.064215547965432E-14
Fitness changed from 3.4160927185645322 to 3.4160927185645016
Iteration 63 complete. Error: 3.4160927185645016 Total: 0.0293; Orientation: 0.0006; Line Search: 0.0256
Low gradient: 3.5015842880880897E-8
th(0)=3.4160927185645016;dx=-1.2261092526585374E-15
New Minimum: 3.4160927185645016 > 3.416092718564471
END: th(30.34605021986545)=3.416092718564471; dx=-8.110986401004003E-16 evalInputDelta=3.064215547965432E-14
Fitness changed from 3.4160927185645016 to 3.416092718564471
Iteration 64 complete. Error: 3.416092718564471 Total: 0.0092; Orientation: 0.0005; Line Search: 0.0063
Low gradient: 3.16541643398587E-8
th(0)=3.416092718564471;dx=-1.0019861200547823E-15
New Minimum: 3.416092718564471 > 3.4160927185644545
WOLF (strong): th(65.3785832991278)=3.4160927185644545; dx=5.022754441651344E-16 evalInputDelta=1.6431300764452317E-14
New Minimum: 3.4160927185644545 > 3.4160927185644505
END: th(32.6892916495639)=3.4160927185644505; dx=-2.4985523789430764E-16 evalInputDelta=2.042810365310288E-14
Fitness changed from 3.416092718564471 to 3.4160927185644505
Iteration 65 complete. Error: 3.4160927185644505 Total: 0.0119; Orientation: 0.0004; Line Search: 0.0091
Low gradient: 3.50713841808653E-8
th(0)=3.4160927185644505;dx=-1.2300019883618486E-15
Armijo: th(70.42694392239005)=3.416092718564522; dx=3.2574643730035113E-15 evalInputDelta=-7.149836278586008E-14
New Minimum: 3.4160927185644505 > 3.416092718564447
WOLF (strong): th(35.213471961195026)=3.416092718564447; dx=1.013731070763291E-15 evalInputDelta=3.552713678800501E-15
New Minimum: 3.416092718564447 > 3.4160927185644403
END: th(11.737823987065008)=3.4160927185644403; dx=-4.820909955827057E-16 evalInputDelta=1.021405182655144E-14
Fitness changed from 3.4160927185644505 to 3.4160927185644403
Iteration 66 complete. Error: 3.4160927185644403 Total: 0.0166; Orientation: 0.0005; Line Search: 0.0137
Low gradient: 2.138098993230171E-8
th(0)=3.4160927185644403;dx=-4.571467304851871E-16
New Minimum: 3.4160927185644403 > 3.4160927185644296
END: th(25.288375183221213)=3.4160927185644296; dx=-3.906816090161139E-16 evalInputDelta=1.0658141036401503E-14
Fitness changed from 3.4160927185644403 to 3.4160927185644296
Iteration 67 complete. Error: 3.4160927185644296 Total: 0.0107; Orientation: 0.0006; Line Search: 0.0071
Low gradient: 1.8360106887332317E-8
th(0)=3.4160927185644296;dx=-3.370935249142675E-16
New Minimum: 3.4160927185644296 > 3.416092718564414
END: th(54.482152749273176)=3.416092718564414; dx=-2.2850559743164263E-16 evalInputDelta=1.554312234475219E-14
Fitness changed from 3.4160927185644296 to 3.416092718564414
Iteration 68 complete. Error: 3.416092718564414 Total: 0.0267; Orientation: 0.0006; Line Search: 0.0232
Low gradient: 1.3406580774159894E-8
th(0)=3.416092718564414;dx=-1.797364080540737E-16
New Minimum: 3.416092718564414 > 3.4160927185644105
WOLF (strong): th(117.3782398706501)=3.4160927185644105; dx=1.1879633528872284E-16 evalInputDelta=3.552713678800501E-15
New Minimum: 3.4160927185644105 > 3.4160927185644083
END: th(58.68911993532505)=3.4160927185644083; dx=-3.047001384843266E-17 evalInputDelta=5.773159728050814E-15
Fitness changed from 3.416092718564414 to 3.4160927185644083
Iteration 69 complete. Error: 3.4160927185644083 Total: 0.0165; Orientation: 0.0007; Line Search: 0.0124
Low gradient: 2.0239137922070786E-8
th(0)=3.4160927185644083;dx=-4.096227038286038E-16
Armijo: th(126.44187591610608)=3.4160927185646037; dx=3.50937877984941E-15 evalInputDelta=-1.9539925233402755E-13
Armijo: th(63.22093795805304)=3.416092718564444; dx=1.549877949559782E-15 evalInputDelta=-3.552713678800501E-14
New Minimum: 3.4160927185644083 > 3.416092718564406
WOLF (strong): th(21.07364598601768)=3.416092718564406; dx=2.435441612748851E-16 evalInputDelta=2.220446049250313E-15
END: th(5.26841149650442)=3.416092718564406; dx=-2.4633098961186517E-16 evalInputDelta=2.220446049250313E-15
Fitness changed from 3.4160927185644083 to 3.416092718564406
Iteration 70 complete. Error: 3.416092718564406 Total: 0.0735; Orientation: 0.0314; Line Search: 0.0293
Low gradient: 1.836984535573676E-8
th(0)=3.416092718564406;dx=-3.374512183936834E-16
New Minimum: 3.416092718564406 > 3.4160927185644043
END: th(11.350448489431914)=3.4160927185644043; dx=-4.411768977887988E-17 evalInputDelta=1.7763568394002505E-15
Fitness changed from 3.416092718564406 to 3.4160927185644043
Iteration 71 complete. Error: 3.4160927185644043 Total: 0.0106; Orientation: 0.0005; Line Search: 0.0070
Low gradient: 7.05668035734322E-9
th(0)=3.4160927185644043;dx=-4.9796737665713636E-17
New Minimum: 3.4160927185644043 > 3.4160927185644026
END: th(24.45379997305211)=3.4160927185644026; dx=-4.229194490919111E-17 evalInputDelta=1.7763568394002505E-15
Fitness changed from 3.4160927185644043 to 3.4160927185644026
Iteration 72 complete. Error: 3.4160927185644026 Total: 0.0101; Orientation: 0.0005; Line Search: 0.0069
Low gradient: 6.10626412906136E-9
th(0)=3.4160927185644026;dx=-3.728646161386148E-17
New Minimum: 3.4160927185644026 > 3.416092718564401
END: th(52.68411496504421)=3.416092718564401; dx=-2.4128923076447118E-17 evalInputDelta=1.3322676295501878E-15
Fitness changed from 3.4160927185644026 to 3.416092718564401
Iteration 73 complete. Error: 3.416092718564401 Total: 0.0094; Orientation: 0.0005; Line Search: 0.0062
Low gradient: 5.000077195079652E-9
th(0)=3.416092718564401;dx=-2.5000771956755602E-17
Armijo: th(113.50448489431915)=3.4160927185644034; dx=5.63067396050421E-17 evalInputDelta=-2.220446049250313E-15
WOLF (strong): th(56.752242447159574)=3.416092718564401; dx=1.5652984484165836E-17 evalInputDelta=0.0
END: th(18.91741414905319)=3.416092718564401; dx=-1.1449519582987418E-17 evalInputDelta=0.0
Fitness changed from 3.416092718564401 to 3.416092718564401
Static Iteration Total: 0.0160; Orientation: 0.0004; Line Search: 0.0130
Iteration 74 failed. Error: 3.416092718564401
Previous Error: 0.0 -> 3.416092718564401
Optimization terminated 74
Final threshold in iteration 74: 3.416092718564401 (> 0.0) after 1.718s (< 30.000s)

Returns

    3.416092718564401

This training apply resulted in the following configuration:

TrainingTester.java:785 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

    

And regressed input:

TrainingTester.java:797 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [
    	[ [ -0.15466666666626083, 0.06933333333419436, 0.5333333333320769 ], [ 0.9679996917772243, -1.9599993793724917, 1.5239996875952662 ] ],
    	[ [ 0.07333333333333447, 2.6253333333333346, 0.9733333333333345 ], [ -0.6146666666619873, -1.266666666667268, -0.8186666666707331 ] ]
    ]

To produce the following output:

TrainingTester.java:808 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [
    	[ [ 0.23580439245020213, 0.29500804164849426, 0.4691875659013035 ], [ 0.3575019194928961, 0.019127778690794774, 0.6233703018163091 ] ],
    	[ [ 0.06137851203438363, 0.7876547088076621, 0.15096677915795423 ], [ 0.4279969771831134, 0.22298759294520362, 0.3490154298716829 ] ]
    ]

Conjugate Gradient Descent

First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.

TrainingTester.java:603 executed in 4.13 seconds (0.152 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 2983258428607
Reset training subject: 2983261378312
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=6.045431069096695}, derivative=-0.2967382679520778}
New Minimum: 6.045431069096695 > 6.04543106906702
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=6.04543106906702}, derivative=-0.29673826795055697}, evalInputDelta = -2.9674929180600884E-11
New Minimum: 6.04543106906702 > 6.045431068888978
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=6.045431068888978}, derivative=-0.2967382679414316}, evalInputDelta = -2.077173988368486E-10
New Minimum: 6.045431068888978 > 6.045431067642677
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=6.045431067642677}, derivative=-0.29673826787755414}, evalInputDelta = -1.4540182391442613E-9
New Minimum: 6.045431067642677 > 6.045431058918573
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=6.045431058918573}, derivative=-0.296738267430412}, evalInputDelta = -1.0178122344939311E-8
New Minimum: 6.045431058918573 > 6.045430997849837
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=6.045430997849837}, derivative=-0.2967382643004171}, evalInputDelta = -7.124685819093202E-8
New Minimum: 6.045430997849837 > 6.04543057036871
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=6.04543057036871}, derivative=-0.2967382423904513}, evalInputDelta = -4.987279851320636E-7
New Minimum: 6.04543057036871 > 6.045427578001698
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=6.045427578001698}, derivative=-0.29673808902061655}, evalInputDelta = -3.4910949970878846E-6
New Minimum: 6.045427578001698 > 6.045406631475929
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=6.045406631475929}, derivative=-0.2967370154281538}, evalInputDelta = -2.443762076609346E-5
New Minimum: 6.045406631475929 > 6.045260007917537
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=6.045260007917537}, derivative=-0.29672950010354227}, evalInputDelta = -1.7106117915854924E-4
New Minimum: 6.045260007917537 > 6.044233746997964
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=6.044233746997964}, derivative=-0.2966768841403352}, evalInputDelta = -0.001197322098731135
New Minimum: 6.044233746997964 > 6.037055020028108
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=6.037055020028108}, derivative=-0.2963081466436327}, evalInputDelta = -0.008376049068587577
New Minimum: 6.037055020028108 > 5.987055174428777
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=5.987055174428777}, derivative=-0.2937061596134986}, evalInputDelta = -0.05837589466791826
New Minimum: 5.987055174428777 > 5.649830610772154
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=5.649830610772154}, derivative=-0.27449147912756827}, evalInputDelta = -0.39560045832454094
New Minimum: 5.649830610772154 > 4.027021976019279
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=4.027021976019279}, derivative=-0.11350439759299578}, evalInputDelta = -2.018409093077416
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=9.803132644625412}, derivative=0.1635685271249639}, evalInputDelta = 3.757701575528717
F(5.217100560376924) = LineSearchPoint{point=PointSample{avg=4.731612553100253}, derivative=-0.2025686941352995}, evalInputDelta = -1.3138185159964424
F(36.51970392263847) = LineSearchPoint{point=PointSample{avg=5.083047642332035}, derivative=0.1246765975254771}, evalInputDelta = -0.96238342676466
5.083047642332035 <= 6.045431069096695
New Minimum: 4.027021976019279 > 3.97932042933625
F(25.715262027707524) = LineSearchPoint{point=PointSample{avg=3.97932042933625}, derivative=0.07358725040334468}, evalInputDelta = -2.066110639760445
Right bracket at 25.715262027707524
New Minimum: 3.97932042933625 > 3.6980666171534464
F(20.60539156988946) = LineSearchPoint{point=PointSample{avg=3.6980666171534464}, derivative=0.034446998639072336}, evalInputDelta = -2.347364451943249
Right bracket at 20.60539156988946
New Minimum: 3.6980666171534464 > 3.645852743901172
F(18.462198719943697) = LineSearchPoint{point=PointSample{avg=3.645852743901172}, derivative=0.013765815735508818}, evalInputDelta = -2.3995783251955234
Right bracket at 18.462198719943697
New Minimum: 3.645852743901172 > 3.6381374716627106
F(17.643699901400613) = LineSearchPoint{point=PointSample{avg=3.6381374716627106}, derivative=0.004999132181963302}, evalInputDelta = -2.4072935974339846
Right bracket at 17.643699901400613
New Minimum: 3.6381374716627106 > 3.6371510268652765
F(17.351382184250497) = LineSearchPoint{point=PointSample{avg=3.6371510268652765}, derivative=0.0017382257314116157}, evalInputDelta = -2.408280042231419
Right bracket at 17.351382184250497
Converged to right
Fitness changed from 6.045431069096695 to 3.6371510268652765
Iteration 1 complete. Error: 3.6371510268652765 Total: 0.1258; Orientation: 0.0004; Line Search: 0.1180
F(0.0) = LineSearchPoint{point=PointSample{avg=3.6371510268652765}, derivative=-0.02209867843198117}
New Minimum: 3.6371510268652765 > 3.4459306966143153
F(17.351382184250497) = LineSearchPoint{point=PointSample{avg=3.4459306966143153}, derivative=0.0010008008072533293}, evalInputDelta = -0.1912203302509612
3.4459306966143153 <= 3.6371510268652765
New Minimum: 3.4459306966143153 > 3.445580008558916
F(16.59962163081499) = LineSearchPoint{point=PointSample{avg=3.445580008558916}, derivative=-6.846689629228919E-5}, evalInputDelta = -0.19157101830636059
Left bracket at 16.59962163081499
Converged to left
Fitness changed from 3.6371510268652765 to 3.445580008558916
Iteration 2 complete. Error: 3.445580008558916 Total: 0.0173; Orientation: 0.0004; Line Search: 0.0142
F(0.0) = LineSearchPoint{point=PointSample{avg=3.445580008558916}, derivative=-0.0018591276813321393}
New Minimum: 3.445580008558916 > 3.426382834107261
F(16.59962163081499) = LineSearchPoint{point=PointSample{avg=3.426382834107261}, derivative=-4.4622355701698406E-4}, evalInputDelta = -0.01919717445165503
F(116.19735141570493) = LineSearchPoint{point=PointSample{avg=3.8258215783293252}, derivative=0.00842476770566546}, evalInputDelta = 0.38024156977040935
F(8.93825780120807) = LineSearchPoint{point=PointSample{avg=3.4323274472300667}, derivative=-0.0011040106561755023}, evalInputDelta = -0.013252561328849222
F(62.5678046084565) = LineSearchPoint{point=PointSample{avg=3.499534308988468}, derivative=0.003667721853377436}, evalInputDelta = 0.05395430042955196
F(4.812908046804346) = LineSearchPoint{point=PointSample{avg=3.4376051376832613}, derivative=-0.001454180318135172}, evalInputDelta = -0.007974870875654627
F(33.690356327630425) = LineSearchPoint{point=PointSample{avg=3.431517597339808}, derivative=0.0010544183293690911}, evalInputDelta = -0.014062411219107673
3.431517597339808 <= 3.445580008558916
New Minimum: 3.426382834107261 > 3.4252377677025962
F(21.49774666766503) = LineSearchPoint{point=PointSample{avg=3.4252377677025962}, derivative=-2.069264767361271E-5}, evalInputDelta = -0.020342240856319638
Left bracket at 21.49774666766503
New Minimum: 3.4252377677025962 > 3.4252353149101964
F(21.73241767494052) = LineSearchPoint{point=PointSample{avg=3.4252353149101964}, derivative=-2.1001271106310454E-7}, evalInputDelta = -0.020344693648719492
Left bracket at 21.73241767494052
Converged to left
Fitness changed from 3.445580008558916 to 3.4252353149101964
Iteration 3 complete. Error: 3.4252353149101964 Total: 0.0615; Orientation: 0.0004; Line Search: 0.0584
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4252353149101964}, derivative=-5.065816498878637E-4}
New Minimum: 3.4252353149101964 > 3.420721258639128
F(21.73241767494052) = LineSearchPoint{point=PointSample{avg=3.420721258639128}, derivative=9.241754396674634E-5}, evalInputDelta = -0.0045140562710686005
3.420721258639128 <= 3.4252353149101964
New Minimum: 3.420721258639128 > 3.4205670300933306
F(18.37939702552541) = LineSearchPoint{point=PointSample{avg=3.4205670300933306}, derivative=-4.119859441817185E-7}, evalInputDelta = -0.004668284816865764
Left bracket at 18.37939702552541
Converged to left
Fitness changed from 3.4252353149101964 to 3.4205670300933306
It

...skipping 109802 bytes...

erivative=-2.643813047611413E-16}, evalInputDelta = -2.6645352591003757E-15
F(62.41162758817775) = LineSearchPoint{point=PointSample{avg=3.416092718564426}, derivative=4.933379041524148E-16}, evalInputDelta = 3.552713678800501E-15
F(4.800894429859827) = LineSearchPoint{point=PointSample{avg=3.416092718564421}, derivative=-3.2266740727188746E-16}, evalInputDelta = -1.3322676295501878E-15
New Minimum: 3.4160927185644185 > 3.4160927185644177
F(33.606261009018795) = LineSearchPoint{point=PointSample{avg=3.4160927185644177}, derivative=8.533528043481299E-17}, evalInputDelta = -4.884981308350689E-15
3.4160927185644177 <= 3.4160927185644225
F(27.581511471934544) = LineSearchPoint{point=PointSample{avg=3.4160927185644177}, derivative=6.030826015952447E-24}, evalInputDelta = -4.884981308350689E-15
Right bracket at 27.581511471934544
Converged to right
Fitness changed from 3.4160927185644225 to 3.4160927185644177
Iteration 95 complete. Error: 3.4160927185644177 Total: 0.0663; Orientation: 0.0004; Line Search: 0.0380
Low gradient: 2.7549169508029585E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4160927185644177}, derivative=-7.58956740582147E-16}
F(27.581511471934544) = LineSearchPoint{point=PointSample{avg=3.4160927185644177}, derivative=7.582897726506118E-16}, evalInputDelta = 0.0
3.4160927185644177 <= 3.4160927185644177
New Minimum: 3.4160927185644177 > 3.4160927185644128
F(13.796818028249577) = LineSearchPoint{point=PointSample{avg=3.4160927185644128}, derivative=-5.0666212944915405E-23}, evalInputDelta = -4.884981308350689E-15
Left bracket at 13.796818028249577
Converged to left
Fitness changed from 3.4160927185644177 to 3.4160927185644128
Iteration 96 complete. Error: 3.4160927185644128 Total: 0.0301; Orientation: 0.0004; Line Search: 0.0274
Low gradient: 1.3384694854978817E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4160927185644128}, derivative=-1.7915005636089634E-16}
New Minimum: 3.4160927185644128 > 3.4160927185644105
F(13.796818028249577) = LineSearchPoint{point=PointSample{avg=3.4160927185644105}, derivative=-1.1941292925413043E-16}, evalInputDelta = -2.220446049250313E-15
F(96.57772619774704) = LineSearchPoint{point=PointSample{avg=3.4160927185644154}, derivative=2.3900975986926283E-16}, evalInputDelta = 2.6645352591003757E-15
F(7.429055861365157) = LineSearchPoint{point=PointSample{avg=3.4160927185644114}, derivative=-1.469839101316449E-16}, evalInputDelta = -1.3322676295501878E-15
New Minimum: 3.4160927185644105 > 3.4160927185644083
F(52.0033910295561) = LineSearchPoint{point=PointSample{avg=3.4160927185644083}, derivative=4.601294356827731E-17}, evalInputDelta = -4.440892098500626E-15
3.4160927185644083 <= 3.4160927185644128
F(41.376293781985666) = LineSearchPoint{point=PointSample{avg=3.4160927185644083}, derivative=4.170058743349635E-24}, evalInputDelta = -4.440892098500626E-15
Right bracket at 41.376293781985666
Converged to right
Fitness changed from 3.4160927185644128 to 3.4160927185644083
Iteration 97 complete. Error: 3.4160927185644083 Total: 0.0490; Orientation: 0.0004; Line Search: 0.0464
Low gradient: 2.4750937489137712E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4160927185644083}, derivative=-6.126089065912027E-16}
F(41.376293781985666) = LineSearchPoint{point=PointSample{avg=3.416092718564426}, derivative=1.4465092602924785E-15}, evalInputDelta = 1.7763568394002505E-14
New Minimum: 3.4160927185644083 > 3.416092718564407
F(3.182791829383513) = LineSearchPoint{point=PointSample{avg=3.416092718564407}, derivative=-4.542152288874888E-16}, evalInputDelta = -1.3322676295501878E-15
F(22.279542805684592) = LineSearchPoint{point=PointSample{avg=3.416092718564408}, derivative=4.96146932446211E-16}, evalInputDelta = -4.440892098500626E-16
3.416092718564408 <= 3.4160927185644083
New Minimum: 3.416092718564407 > 3.4160927185644057
F(12.309875517220862) = LineSearchPoint{point=PointSample{avg=3.4160927185644057}, derivative=-2.7057694197620316E-23}, evalInputDelta = -2.6645352591003757E-15
Left bracket at 12.309875517220862
Converged to left
Fitness changed from 3.4160927185644083 to 3.4160927185644057
Iteration 98 complete. Error: 3.4160927185644057 Total: 0.0207; Orientation: 0.0004; Line Search: 0.0179
Low gradient: 8.290646585851633E-9
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4160927185644057}, derivative=-6.873482081149334E-17}
New Minimum: 3.4160927185644057 > 3.4160927185644043
F(12.309875517220862) = LineSearchPoint{point=PointSample{avg=3.4160927185644043}, derivative=-5.569370133400396E-17}, evalInputDelta = -1.3322676295501878E-15
New Minimum: 3.4160927185644043 > 3.4160927185644034
F(86.16912862054603) = LineSearchPoint{point=PointSample{avg=3.4160927185644034}, derivative=2.255300357861724E-17}, evalInputDelta = -2.220446049250313E-15
3.4160927185644034 <= 3.4160927185644057
F(64.88071826430104) = LineSearchPoint{point=PointSample{avg=3.4160927185644034}, derivative=2.7721656990470496E-24}, evalInputDelta = -2.220446049250313E-15
Right bracket at 64.88071826430104
Converged to right
Fitness changed from 3.4160927185644057 to 3.4160927185644034
Iteration 99 complete. Error: 3.4160927185644034 Total: 0.0255; Orientation: 0.0072; Line Search: 0.0159
Low gradient: 1.9628850526039887E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=3.4160927185644034}, derivative=-3.852917729736165E-16}
F(64.88071826430104) = LineSearchPoint{point=PointSample{avg=3.4160927185644487}, derivative=1.800547789671716E-15}, evalInputDelta = 4.529709940470639E-14
New Minimum: 3.4160927185644034 > 3.416092718564401
F(4.990824481869311) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-2.1715030505626918E-16}, evalInputDelta = -2.220446049250313E-15
F(34.935771373085174) = LineSearchPoint{point=PointSample{avg=3.4160927185644105}, derivative=7.916986312022324E-16}, evalInputDelta = 7.105427357601002E-15
F(2.6873670286988594) = LineSearchPoint{point=PointSample{avg=3.4160927185644026}, derivative=-2.9475405999513486E-16}, evalInputDelta = -8.881784197001252E-16
F(18.811569200892016) = LineSearchPoint{point=PointSample{avg=3.416092718564402}, derivative=2.4847225314466548E-16}, evalInputDelta = -1.3322676295501878E-15
3.416092718564402 <= 3.4160927185644034
F(11.436343104262672) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-1.0157326169924185E-23}, evalInputDelta = -2.220446049250313E-15
Left bracket at 11.436343104262672
Converged to left
Fitness changed from 3.4160927185644034 to 3.416092718564401
Iteration 100 complete. Error: 3.416092718564401 Total: 0.0492; Orientation: 0.0004; Line Search: 0.0467
Reset training subject: 2987337486092
Low gradient: 1.1189397959192257E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-1.2520262668917584E-16}
F(11.436343104262672) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-9.7958171792441E-18}, evalInputDelta = 0.0
F(80.0544017298387) = LineSearchPoint{point=PointSample{avg=3.416092718564424}, derivative=6.826451534643843E-16}, evalInputDelta = 2.2648549702353193E-14
F(6.158030902295285) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-6.306049907544568E-17}, evalInputDelta = 0.0
F(43.10621631606699) = LineSearchPoint{point=PointSample{avg=3.4160927185644057}, derivative=3.097922986594016E-16}, evalInputDelta = 4.440892098500626E-15
F(3.315862793543615) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-9.174148131389848E-17}, evalInputDelta = 0.0
F(23.211039554805303) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=1.0902540057742579E-16}, evalInputDelta = 0.0
3.416092718564401 <= 3.416092718564401
F(12.40706825037739) = LineSearchPoint{point=PointSample{avg=3.416092718564401}, derivative=-2.768142387243597E-24}, evalInputDelta = 0.0
Left bracket at 12.40706825037739
Converged to left
Fitness changed from 3.416092718564401 to 3.416092718564401
Static Iteration Total: 0.0525; Orientation: 0.0004; Line Search: 0.0476
Iteration 101 failed. Error: 3.416092718564401
Previous Error: 0.0 -> 3.416092718564401
Optimization terminated 101
Final threshold in iteration 101: 3.416092718564401 (> 0.0) after 4.132s (< 30.000s)

Returns

    3.416092718564401

This training apply resulted in the following configuration:

TrainingTester.java:785 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

    

And regressed input:

TrainingTester.java:797 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [
    	[ [ -0.15466666666666512, 0.06933333333333486, 0.5333333333333348 ], [ 0.9679996302718412, -1.9599994330123738, 1.5239998027405286 ] ],
    	[ [ 0.07333333333333515, 2.625333333333335, 0.973333333333335 ], [ -0.6146666666666648, -1.2666666666666648, -0.8186666666666649 ] ]
    ]

To produce the following output:

TrainingTester.java:808 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [
    	[ [ 0.23580439245004992, 0.29500804164816957, 0.46918756590178057 ], [ 0.3575018800714406, 0.01912777673203921, 0.6233703431965202 ] ],
    	[ [ 0.06137851203438363, 0.7876547088076621, 0.15096677915795423 ], [ 0.4279969771813031, 0.22298759294543796, 0.349015429873259 ] ]
    ]

Limited-Memory BFGS

Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.

TrainingTester.java:674 executed in 9.25 seconds (0.411 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new LBFGS());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setIterationsPerSample(100);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 2987405533755
Reset training subject: 2987407932990
Adding measurement 572e2a5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 6.045431069096695 < 6.045431069096695. Total: 1
th(0)=6.045431069096695;dx=-0.2967382679520778
Adding measurement 562dedd2 to history. Total: 1
New Minimum: 6.045431069096695 > 5.443494326030401
END: th(2.154434690031884)=5.443494326030401; dx=-0.26112246075408024 evalInputDelta=0.6019367430662941
Fitness changed from 6.045431069096695 to 5.443494326030401
Iteration 1 complete. Error: 5.443494326030401 Total: 0.0178; Orientation: 0.0028; Line Search: 0.0082
Non-optimal measurement 5.443494326030401 < 5.443494326030401. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 5.443494326030401 < 5.443494326030401. Total: 2
th(0)=5.443494326030401;dx=-0.23058263544533114
Adding measurement 4653dc18 to history. Total: 2
New Minimum: 5.443494326030401 > 4.52495953774945
END: th(4.641588833612779)=4.52495953774945; dx=-0.16350790924874908 evalInputDelta=0.9185347882809509
Fitness changed from 5.443494326030401 to 4.52495953774945
Iteration 2 complete. Error: 4.52495953774945 Total: 0.0112; Orientation: 0.0015; Line Search: 0.0072
Non-optimal measurement 4.52495953774945 < 4.52495953774945. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 4.52495953774945 < 4.52495953774945. Total: 3
th(0)=4.52495953774945;dx=-0.11925811689650978
Adding measurement 723d3bf8 to history. Total: 3
New Minimum: 4.52495953774945 > 3.6825080195421096
END: th(10.000000000000002)=3.6825080195421096; dx=-0.04934001070866413 evalInputDelta=0.8424515182073407
Fitness changed from 4.52495953774945 to 3.6825080195421096
Iteration 3 complete. Error: 3.6825080195421096 Total: 0.0092; Orientation: 0.0009; Line Search: 0.0064
Non-optimal measurement 3.6825080195421096 < 3.6825080195421096. Total: 4
Rejected: LBFGS Orientation magnitude: 4.231e+00, gradient 1.605e-01, dot -0.993; [afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.6825080195421096, 4.52495953774945, 5.443494326030401, 6.045431069096695
LBFGS Accumulation History: 3 points
Removed measurement 723d3bf8 to history. Total: 3
Adding measurement 6029442 to history. Total: 3
th(0)=3.6825080195421096;dx=-0.025769919873265178
Adding measurement c0c8ed2 to history. Total: 4
New Minimum: 3.6825080195421096 > 3.461322637357123
WOLF (strong): th(21.544346900318843)=3.461322637357123; dx=0.006351749065164181 evalInputDelta=0.2211853821849865
Non-optimal measurement 3.4832561728246105 < 3.461322637357123. Total: 5
END: th(10.772173450159421)=3.4832561728246105; dx=-0.010574542925683253 evalInputDelta=0.19925184671749907
Fitness changed from 3.6825080195421096 to 3.461322637357123
Iteration 4 complete. Error: 3.461322637357123 Total: 0.1162; Orientation: 0.0810; Line Search: 0.0332
Non-optimal measurement 3.461322637357123 < 3.461322637357123. Total: 5
Rejected: LBFGS Orientation magnitude: 1.272e+00, gradient 6.710e-02, dot -0.953; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.461322637357123, 3.6825080195421096, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 1.140e+00, gradient 6.710e-02, dot -0.959; [022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.461322637357123, 3.6825080195421096, 4.52495953774945, 5.443494326030401
LBFGS Accumulation History: 3 points
Removed measurement c0c8ed2 to history. Total: 4
Removed measurement 6029442 to history. Total: 3
Adding measurement 3947f126 to history. Total: 3
th(0)=3.461322637357123;dx=-0.0045029705975177985
Adding measurement 186cf15d to history. Total: 4
New Minimum: 3.461322637357123 > 3.437576236767408
WOLF (strong): th(23.2079441680639)=3.437576236767408; dx=0.002585945221962065 evalInputDelta=0.023746400589715222
Adding measurement 6b05b7d to history. Total: 5
New Minimum: 3.437576236767408 > 3.4287267616506845
END: th(11.60397208403195)=3.4287267616506845; dx=-0.0010564411263141898 evalInputDelta=0.032595875706438626
Fitness changed from 3.461322637357123 to 3.4287267616506845
Iteration 5 complete. Error: 3.4287267616506845 Total: 0.0618; Orientation: 0.0489; Line Search: 0.0096
Non-optimal measurement 3.4287267616506845 < 3.4287267616506845. Total: 6
Rejected: LBFGS Orientation magnitude: 4.432e-01, gradient 2.191e-02, dot -0.998; [7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4287267616506845, 3.437576236767408, 3.461322637357123, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 4.162e-01, gradient 2.191e-02, dot -0.999; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4287267616506845, 3.437576236767408, 3.461322637357123, 4.52495953774945, 5.443494326030401
Rejected: LBFGS Orientation magnitude: 4.830e-01, gradient 2.191e-02, dot -0.989; [afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4287267616506845, 3.437576236767408, 3.461322637357123, 4.52495953774945
LBFGS Accumulation History: 3 points
Removed measurement 6b05b7d to history. Total: 5
Removed measurement 186cf15d to history. Total: 4
Removed measurement 3947f126 to history. Total: 3
Adding measurement 58b0bb87 to history. Total: 3
th(0)=3.4287267616506845;dx=-4.800392107450423E-4
Adding measurement 100da538 to history. Total: 4
New Minimum: 3.4287267616506845 > 3.4212681135192957
END: th(25.000000000000007)=3.4212681135192957; dx=-1.192463256958196E-4 evalInputDelta=0.007458648131388745
Fitness changed from 3.4287267616506845 to 3.4212681135192957
Iteration 6 complete. Error: 3.4212681135192957 Total: 0.2775; Orientation: 0.2668; Line Search: 0.0084
Non-optimal measurement 3.4212681135192957 < 3.4212681135192957. Total: 5
Rejected: LBFGS Orientation magnitude: 4.511e-01, gradient 1.366e-02, dot -0.935; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4212681135192957, 3.4287267616506845, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 4.133e-01, gradient 1.366e-02, dot -0.922; [022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-2

...skipping 125152 bytes...

4083;dx=-4.096227038286038E-16
Non-optimal measurement 3.4160927185646037 < 3.4160927185644083. Total: 4
Armijo: th(126.44187591610608)=3.4160927185646037; dx=3.5093787798494105E-15 evalInputDelta=-1.9539925233402755E-13
Non-optimal measurement 3.416092718564444 < 3.4160927185644083. Total: 4
Armijo: th(63.22093795805304)=3.416092718564444; dx=1.5498779495597817E-15 evalInputDelta=-3.552713678800501E-14
Adding measurement 19884633 to history. Total: 4
New Minimum: 3.4160927185644083 > 3.416092718564406
WOLF (strong): th(21.07364598601768)=3.416092718564406; dx=2.435441612748851E-16 evalInputDelta=2.220446049250313E-15
Non-optimal measurement 3.416092718564406 < 3.416092718564406. Total: 5
END: th(5.26841149650442)=3.416092718564406; dx=-2.4633098961186517E-16 evalInputDelta=2.220446049250313E-15
Fitness changed from 3.4160927185644083 to 3.416092718564406
Iteration 70 complete. Error: 3.416092718564406 Total: 0.4637; Orientation: 0.2063; Line Search: 0.2553
Non-optimal measurement 3.416092718564406 < 3.416092718564406. Total: 5
Rejected: LBFGS Orientation magnitude: 2.205e-07, gradient 1.837e-08, dot -0.914; [76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.416092718564406, 3.4160927185644083, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 2.048e-07, gradient 1.837e-08, dot -0.930; [7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.416092718564406, 3.4160927185644083, 4.52495953774945, 5.443494326030401
LBFGS Accumulation History: 3 points
Removed measurement 19884633 to history. Total: 4
Removed measurement 57cc1ab2 to history. Total: 3
Adding measurement 735bc4d5 to history. Total: 3
th(0)=3.416092718564406;dx=-3.3745121839368343E-16
Adding measurement 7d6ba8c8 to history. Total: 4
New Minimum: 3.416092718564406 > 3.4160927185644043
END: th(11.350448489431914)=3.4160927185644043; dx=-4.411768977887988E-17 evalInputDelta=1.7763568394002505E-15
Fitness changed from 3.416092718564406 to 3.4160927185644043
Iteration 71 complete. Error: 3.4160927185644043 Total: 0.0858; Orientation: 0.0791; Line Search: 0.0050
Non-optimal measurement 3.4160927185644043 < 3.4160927185644043. Total: 5
Rejected: LBFGS Orientation magnitude: 1.413e-07, gradient 7.057e-09, dot -0.956; [022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4160927185644043, 3.416092718564406, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 1.246e-07, gradient 7.057e-09, dot -0.951; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4160927185644043, 3.416092718564406, 4.52495953774945, 5.443494326030401
LBFGS Accumulation History: 3 points
Removed measurement 7d6ba8c8 to history. Total: 4
Removed measurement 735bc4d5 to history. Total: 3
Adding measurement 53f1624c to history. Total: 3
th(0)=3.4160927185644043;dx=-4.9796737665713636E-17
Adding measurement 170a994b to history. Total: 4
New Minimum: 3.4160927185644043 > 3.4160927185644026
END: th(24.45379997305211)=3.4160927185644026; dx=-4.229194490919111E-17 evalInputDelta=1.7763568394002505E-15
Fitness changed from 3.4160927185644043 to 3.4160927185644026
Iteration 72 complete. Error: 3.4160927185644026 Total: 0.1550; Orientation: 0.1368; Line Search: 0.0163
Non-optimal measurement 3.4160927185644026 < 3.4160927185644026. Total: 5
Rejected: LBFGS Orientation magnitude: 8.290e-07, gradient 6.106e-09, dot -0.924; [022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4160927185644026, 3.4160927185644043, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 8.082e-07, gradient 6.106e-09, dot -0.909; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.4160927185644026, 3.4160927185644043, 4.52495953774945, 5.443494326030401
LBFGS Accumulation History: 3 points
Removed measurement 170a994b to history. Total: 4
Removed measurement 53f1624c to history. Total: 3
Adding measurement 382e6a09 to history. Total: 3
th(0)=3.4160927185644026;dx=-3.728646161386148E-17
Adding measurement 442966cc to history. Total: 4
New Minimum: 3.4160927185644026 > 3.416092718564401
END: th(52.68411496504421)=3.416092718564401; dx=-2.4128923076447115E-17 evalInputDelta=1.3322676295501878E-15
Fitness changed from 3.4160927185644026 to 3.416092718564401
Iteration 73 complete. Error: 3.416092718564401 Total: 0.1010; Orientation: 0.0920; Line Search: 0.0066
Non-optimal measurement 3.416092718564401 < 3.416092718564401. Total: 5
Rejected: LBFGS Orientation magnitude: 6.043e-07, gradient 5.000e-09, dot -0.835; [afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00, 519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.416092718564401, 3.4160927185644026, 4.52495953774945, 5.443494326030401, 6.045431069096695
Rejected: LBFGS Orientation magnitude: 5.815e-07, gradient 5.000e-09, dot -0.818; [519fd5dd-2b1d-455a-988f-59148f60363e = 1.000/1.000e+00, 7602d3f9-612e-4e43-b741-b21e859c58d4 = 1.000/1.000e+00, 022038aa-c7e6-4db4-99a0-2d5f834109b0 = 1.000/1.000e+00, afcc040d-bcdd-4531-a7ed-25df1732375f = 1.000/1.000e+00, 76e46209-cc59-49d5-8f8c-8a7a492a4390 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.416092718564401, 3.4160927185644026, 4.52495953774945, 5.443494326030401
LBFGS Accumulation History: 3 points
Removed measurement 442966cc to history. Total: 4
Removed measurement 382e6a09 to history. Total: 3
Adding measurement 19321c53 to history. Total: 3
th(0)=3.416092718564401;dx=-2.50007719567556E-17
Non-optimal measurement 3.4160927185644034 < 3.416092718564401. Total: 4
Armijo: th(113.50448489431915)=3.4160927185644034; dx=5.63067396050421E-17 evalInputDelta=-2.220446049250313E-15
Non-optimal measurement 3.416092718564401 < 3.416092718564401. Total: 4
WOLF (strong): th(56.752242447159574)=3.416092718564401; dx=1.5652984484165836E-17 evalInputDelta=0.0
Non-optimal measurement 3.416092718564401 < 3.416092718564401. Total: 4
END: th(18.91741414905319)=3.416092718564401; dx=-1.1449519582987418E-17 evalInputDelta=0.0
Fitness changed from 3.416092718564401 to 3.416092718564401
Static Iteration Total: 0.1047; Orientation: 0.0888; Line Search: 0.0131
Iteration 74 failed. Error: 3.416092718564401
Previous Error: 0.0 -> 3.416092718564401
Optimization terminated 74
Final threshold in iteration 74: 3.416092718564401 (> 0.0) after 9.255s (< 30.000s)

Returns

    3.416092718564401

This training apply resulted in the following configuration:

TrainingTester.java:785 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

    

And regressed input:

TrainingTester.java:797 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [
    	[ [ -0.15466666666626083, 0.06933333333419436, 0.5333333333320769 ], [ 0.9679996917772243, -1.9599993793724917, 1.5239996875952662 ] ],
    	[ [ 0.07333333333333447, 2.6253333333333346, 0.9733333333333345 ], [ -0.6146666666619873, -1.266666666667268, -0.8186666666707331 ] ]
    ]

To produce the following output:

TrainingTester.java:808 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [
    	[ [ 0.23580439245020213, 0.29500804164849426, 0.4691875659013035 ], [ 0.3575019194928961, 0.019127778690794774, 0.6233703018163091 ] ],
    	[ [ 0.06137851203438363, 0.7876547088076621, 0.15096677915795423 ], [ 0.4279969771831134, 0.22298759294520362, 0.3490154298716829 ] ]
    ]

TrainingTester.java:576 executed in 0.10 seconds (0.000 gc):

    return TestUtil.compare(title + " vs Iteration", runs);
Logging
Plotting range=[1.0, 0.5335296496675602], [100.0, 0.735877774574145]; valueStats=DoubleSummaryStatistics{count=246, sum=847.575539, min=3.416093, average=3.445429, max=5.443494}
Plotting 73 points for GD
Plotting 100 points for CjGD
Plotting 73 points for LBFGS

Returns

Result

TrainingTester.java:579 executed in 0.01 seconds (0.000 gc):

    return TestUtil.compareTime(title + " vs Time", runs);
Logging
Plotting range=[0.0, 0.5335296496675602], [9.132, 0.735877774574145]; valueStats=DoubleSummaryStatistics{count=246, sum=847.575539, min=3.416093, average=3.445429, max=5.443494}
Plotting 73 points for GD
Plotting 100 points for CjGD
Plotting 73 points for LBFGS

Returns

Result

Results

TrainingTester.java:350 executed in 0.00 seconds (0.000 gc):

    return grid(inputLearning, modelLearning, completeLearning);

Returns

Result

TrainingTester.java:353 executed in 0.00 seconds (0.000 gc):

    return new ComponentResult(null == inputLearning ? null : inputLearning.value,
        null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);

Returns

    {"input":{ "LBFGS": { "type": "NonConverged", "value": 3.416092718564401 }, "CjGD": { "type": "NonConverged", "value": 3.416092718564401 }, "GD": { "type": "NonConverged", "value": 3.416092718564401 } }, "model":null, "complete":null}

LayerTests.java:605 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

detailsresult
{"input":{ "LBFGS": { "type": "NonConverged", "value": 3.416092718564401 }, "CjGD": { "type": "NonConverged", "value": 3.416092718564401 }, "GD": { "type": "NonConverged", "value": 3.416092718564401 } }, "model":null, "complete":null}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "15.860",
      "gc_time": "0.778"
    },
    "created_on": 1587005000848,
    "file_name": "trainingTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgPixelSoftmaxLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/c9a1867488dc7e77a975f095285b5882c0486db6/src/test/java/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayerTest.java",
      "javaDoc": "The type Basic."
    },
    "training_analysis": {
      "input": {
        "LBFGS": {
          "type": "NonConverged",
          "value": 3.416092718564401
        },
        "CjGD": {
          "type": "NonConverged",
          "value": 3.416092718564401
        },
        "GD": {
          "type": "NonConverged",
          "value": 3.416092718564401
        }
      }
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayer/Basic/trainingTest/202004164320",
    "id": "9e528ea8-4d50-4515-a709-26ed55090903",
    "report_type": "Components",
    "display_name": "Comparative Training",
    "target": {
      "simpleName": "ImgPixelSoftmaxLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgPixelSoftmaxLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/c9a1867488dc7e77a975f095285b5882c0486db6/src/main/java/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayer.java",
      "javaDoc": "The type Img pixel softmax layer."
    }
  }