Gregory Gay SBSE - March 31st Deliverable Variants tested: - keys5p (top 5% of values set each round) - keysCo (mitigations with equal frequency in "best" to the best are set each round) - keysFI (Fewer "instances" each round) - keysNBR (shifting step function for best/rest division) - keysNN (nearest neighborhood used in alternating rounds) - keysNN2 (combination of keysNN and keys2) Note 1: All results sorted best -> worst on the median values Note 2: MW charts are missing original Keys right now because I forgot to upload the data. I will redo them later. Note 3: keysNBR's results are shown in the quartiles, but I left them out of the mann-whitney charts because I think there may be an issue with it currently. It only returns non-zero cost results for a few models. I want to play with it some more. ---------- Model 21 Cost: keysCo: 0.0, 0.0, 0.0, 0.0, 0.0 keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keys: 0.0, 0.0, 0.0, 0.0, 75.0 keysFI: 0.0, 0.0, 0.0, 0.0, 200.0 keysNN: 0.0, 0.0, 0.0, 0.0, 900.0 keys2: 0.0, 0.0, 0.0, 25.0, 200.0 keysNN2: 0.0, 0.0, 0.0, 25.0, 750.0 keys5p: 325.0, 2500.0, 4200.0, 6675.0, 16275.0 #key, ties, win, loss, win-loss keysNN, 4, 1, 0, 1 keysNN2, 4, 1, 0, 1 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keys2, 4, 1, 0, 1 keys5p, 0, 0, 5, -5 Attainment: All: 25.0, 25.0, 25.0, 25.0, 25.0 #key, ties, win, loss, win-loss keysNN, 5, 0, 0, 0 keysNN2, 5, 0, 0, 0 keysFI, 5, 0, 0, 0 keysCo, 5, 0, 0, 0 keys5p, 5, 0, 0, 0 keys2, 5, 0, 0, 0 ---------- Model 22 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keys: 0.0, 0.0, 0.0, 0.0, 200.0 keysCo: 0.0, 0.0, 0.0, 0.0, 200.0 keysFI: 0.0, 0.0, 0.0, 40.0, 190.0 keysNN: 0.0, 0.0, 0.0, 25.0, 2025.0 keys2: 0.0, 75.0, 140.0, 190.0, 1250.0 keysNN2: 0.0, 100.0, 200.0, 315.0, 2750.0 keys5p: 2325.0, 10650.0, 15375.0, 20750.0, 35615.0 #key, ties, win, loss, win-loss keysNN, 2, 3, 0, 3 keysFI, 2, 3, 0, 3 keysCo, 2, 3, 0, 3 keys2, 0, 2, 3, -1 keysNN2, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 54.0, 54.0, 54.0, 54.0, 54.0 #key, ties, win, loss, win-loss keysNN, 5, 0, 0, 0 keysNN2, 5, 0, 0, 0 keysFI, 5, 0, 0, 0 keysCo, 5, 0, 0, 0 keys5p, 5, 0, 0, 0 keys2, 5, 0, 0, 0 ---------- Model 24 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keysCo: 0.0, 0.0, 25.0, 115.0, 1690.0 keys: 0.0, 65.0, 125.0, 200.0, 2205.0 keysNN: 25.0, 140.0, 275.0, 1250.0, 4200.0 keysFI: 270.0, 530.0, 670.0, 875.0, 2880.0 keys2: 255.0, 550.0, 710.0, 1290.0, 2700.0 keysNN2: 240.0, 615.0, 1430.0, 2155.0, 6345.0 keys5p: 7440.0, 19430.0, 26335.0, 33230.0, 47145.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keysFI, 1, 2, 2, 0 keys2, 1, 2, 2, 0 keysNN2, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 112.0, 112.0, 112.0, 112.0, 112.0 #key, ties, win, loss, win-loss keysNN, 5, 0, 0, 0 keysNN2, 5, 0, 0, 0 keysFI, 5, 0, 0, 0 keysCo, 5, 0, 0, 0 keys5p, 5, 0, 0, 0 keys2, 5, 0, 0, 0 ---------- Model 28 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keysCo: 25.0, 330.0, 775.0, 1515.0, 6365.0 keys: 290.0, 715.0, 1080.0, 1575.0, 4190.0 keysNN: 340.0, 1755.0, 3140.0, 4225.0, 8950.0 keys2: 1495.0, 2625.0, 3435.0, 4325.0, 8375.0 keysNN2: 1440.0, 5070.0, 6640.0, 8290.0, 15445.0 keysFI: 9780.0, 13970.0, 16325.0, 20475.0, 29100.0 keys5p: 23410.0, 44240.0, 49285.0, 56985.0, 80850.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 228.0, 228.0, 228.0, 228.0, 228.0 #key, ties, win, loss, win-loss keysNN, 5, 0, 0, 0 keysNN2, 5, 0, 0, 0 keysFI, 5, 0, 0, 0 keysCo, 5, 0, 0, 0 keys5p, 5, 0, 0, 0 keys2, 5, 0, 0, 0 ---------- Model 216 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keysCo: 1585.0, 6315.0, 7775.0, 12730.0, 24290.0 keys: 4740.0, 8495.0, 10815.0, 12650.0, 19935.0 keys5p: 115225.0, 153050.0, 166010.0, 180050.0, 216760.0 keysNN: 9725.0, 16585.0, 20875.0, 24335.0, 36000.0 keys2: 14500.0, 20025.0, 22030.0, 26040.0, 45560.0 keysNN2: 18460.0, 27120.0, 33335.0, 39045.0, 54990.0 keysFI: 69060.0, 85315.0, 93100.0, 103260.0, 140065.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 460.0, 460.0, 460.0, 460.0, 460.0 #key, ties, win, loss, win-loss keysNN, 5, 0, 0, 0 keysNN2, 5, 0, 0, 0 keysFI, 5, 0, 0, 0 keysCo, 5, 0, 0, 0 keys5p, 5, 0, 0, 0 keys2, 5, 0, 0, 0 ---------- Model 41 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 0.0 keysCo: 0.0, 0.0, 0.0, 0.0, 5000.0 keys: 0.0, 0.0, 0.0, 0.0, 30000.0 keysFI: 0.0, 0.0, 0.0, 0.0, 30000.0 keys2: 0.0, 0.0, 0.0, 5000.0, 75000.0 keysNN: 0.0, 0.0, 0.0, 0.0, 150000.0 keysNN2: 0.0, 0.0, 2000.0, 20000.0, 100000.0 keys5p: 55000.0, 215000.0, 395000.0, 707000.0, 1125000.0 #key, ties, win, loss, win-loss keysNN, 3, 2, 0, 2 keysFI, 3, 2, 0, 2 keysCo, 3, 2, 0, 2 keys2, 3, 2, 0, 2 keysNN2, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p, keysCo, keysFI, keysNN: 2591.0, 2591.0, 2591.0, 2591.0, 2591.0 keysNN2: 2234.5, 2587.4, 2590.5, 2591.0, 2591.0 keysNBR: 2581.0, 2581.0, 2581.0, 2581.0, 2581.0 #key, ties, win, loss, win-loss keysNN, 4, 1, 0, 1 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keys5p, 4, 1, 0, 1 keys2, 4, 1, 0, 1 keysNN2, 0, 0, 5, -5 ---------- Model 42 Cost: keysNBR: 0.0, 0.0, 0.0, 15000.0, 250000.0 keysCo: 0.0, 0.0, 1000.0, 3000.0, 175000.0 keys: 0.0, 0.0, 1000.0, 5000.0, 107000.0 keysNN: 0.0, 1000.0, 5000.0, 21000.0, 320000.0 keysFI: 0.0, 4500.0, 9000.0, 20500.0, 115500.0 keys2: 0.0, 19000.0, 36500.0, 53000.0, 269000.0 keysNN2: 0.0, 27000.0, 46000.0, 90000.0, 454000.0 keys5p: 694500.0, 1299000.0, 1778000.0, 2288500.0, 3909000.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keysFI, 0, 3, 2, 1 keys2, 0, 2, 3, -1 keysNN2, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p, keysCo, keysFI, keysNN: 3835.0, 3835.0, 3835.0, 3835.0, 3835.0 keysNN2: 3634.0, 3826.9, 3834.3, 3834.9, 3835.0 keysNBR: 3433.1, 3775.3, 3785.8, 3794.5, 3830.4 #key, ties, win, loss, win-loss keys2, 2, 3, 0, 3 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keysNN, 3, 1, 1, 0 keys5p, 3, 1, 1, 0 keysNN2, 0, 0, 5, -5 ---------- Model 44 Cost: keysNBR: 0.0, 0.0, 0.0, 5000.0, 305000.0 keysCo: 0.0, 15500.0, 35500.0, 61000.0, 175000.0 keys: 7000.0, 20500.0, 35500.0, 65000.0, 170500.0 keysNN: 0.0, 40500.0, 71000.0, 118000.0, 710500.0 keys2: 56500.0, 107000.0, 146000.0, 198000.0, 389500.0 keysNN2: 43000.0, 191500.0, 259000.0, 347000.0, 926500.0 keysFI: 211500.0, 373000.0, 458000.0, 562000.0, 984500.0 keys5p: 1241500.0, 2292500.0, 2795500.0, 3321500.0, 4831000.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p,keysCo,keysFI, keysNN: 8726.9, 8727.0, 8727.0, 8727.0, 8727.0 keysNN2: 8474.7, 8695.6, 8722.3, 8726.3, 8727.0 keysNBR: 8399.1, 8609.4, 8631.5, 8635.4, 8727.0 #key, ties, win, loss, win-loss keysNN, 2, 3, 0, 3 keysFI, 2, 3, 0, 3 keysCo, 2, 3, 0, 3 keys5p, 1, 1, 3, -2 keys2, 1, 1, 3, -2 keysNN2, 0, 0, 5, -5 ---------- Model 48 Cost: keysNBR: 0.0, 0.0, 20000.0, 90000.0, 321000.0 keysCo: 11500.0, 132000.0, 213500.0, 325000.0, 897500.0 keys: 67000.0, 165000.0, 220000.0, 268500.0, 556000.0 keysNN: 117000.0, 314000.0, 459000.0, 655500.0, 1223500.0 keys2: 337500.0, 469000.0, 577000.0, 688000.0, 1088000.0 keysNN2: 346000.0, 722500.0, 855500.0, 970500.0, 1545000.0 keysFI: 1373000.0, 2134500.0, 2558500.0, 2918500.0, 3561500.0 keys5p: 4089000.0, 5135500.0, 5689500.0, 6172000.0, 8716500.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p, keysCo, keysFI,keysNN: 18491.7, 18492.0, 18492.0, 18492.0, 18492.0 keysNN2: 18380.4, 18485.9, 18489.8, 18491.5, 18492.0 keysNBR: 17926.9, 18286.1, 18339.1, 18369.8, 18492.0 #key, ties, win, loss, win-loss keysNN, 3, 2, 0, 2 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keys5p, 4, 1, 0, 1 keys2, 3, 1, 1, 0 keysNN2, 0, 0, 5, -5 ---------- Model 416 Cost: keysNBR: 0.0, 0.0, 10000.0, 82000.0, 720000.0 keys: 704500.0, 1216000.0, 1422500.0, 1591500.0, 2437000.0 keysCo: 314000.0, 1209000.0, 1449500.0, 1805500.0, 3318500.0 keysNN: 1528500.0, 2106500.0, 2382000.0, 2902500.0, 4224500.0 keys2: 1619000.0, 2403000.0, 2733000.0, 3171000.0, 3897500.0 keysNN2: 1974000.0, 3542000.0, 3908000.0, 4388500.0, 5529000.0 keysFI: 7183500.0, 10129000.0, 11062500.0, 12118500.0, 15054500.0 keys5p: 12745500.0, 15602000.0, 16392000.0, 17472500.0, 20131500.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p, keysCo, keysFI, keysNN: 34974.0, 34975.0, 34975.0, 34975.0, 34975.0 keysNN2: 34837.9, 34952.4, 34971.0, 34973.8, 34975.0 keysNBR: 34455.5, 34680.1, 34707.5, 34715.7, 34760.3 #key, ties, win, loss, win-loss keysNN, 1, 4, 0, 4 keysFI, 3, 2, 0, 2 keysCo, 3, 1, 1, 0 keys2, 3, 1, 1, 0 keys5p, 2, 1, 2, -1 keysNN2, 0, 0, 5, -5 ---------- Model 51 Cost: keysCo: 0.0, 0.0, 1000.0, 1000.0, 16000.0 keys: 0.0, 0.0, 1000.0, 2000.0, 13025.0 keysFI: 0.0, 2000.0, 3000.0, 6000.0, 15000.0 keysNN: 0.0, 1000.0, 3025.0, 8025.0, 27000.0 keys2: 0.0, 3000.0, 6000.0, 9025.0, 25000.0 keysNN2: 1000.0, 6000.0, 9025.0, 16000.0, 51000.0 keysNBR: 0.0, 3000.0, 10000.0, 17000.0, 58000.0 keys5p: 53000.0, 150000.0, 174000.0, 235000.0, 377000.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 1, 3, 1, 2 keysFI, 1, 3, 1, 2 keys2, 0, 2, 3, -1 keysNN2, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 255.1, 255.1, 255.1, 255.1, 255.1 #key, ties, win, loss, win-loss keys5p, 2, 3, 0, 3 keysFI, 3, 2, 0, 2 keys2, 3, 2, 0, 2 keysCo, 3, 1, 1, 0 keysNN, 1, 1, 3, -2 keysNN2, 0, 0, 5, -5 ---------- Model 52 Cost: keysCo: 0.0, 2000.0, 7000.0, 15525.0, 63000.0 keys: 25.0, 8000.0, 13575.0, 21000.0, 42000.0 keysNBR: 0.0, 15000.0, 25000.0, 51000.0, 126000.0 keysNN: 6000.0, 21000.0, 30000.0, 40025.0, 124000.0 keys2: 7550.0, 33025.0, 43050.0, 58025.0, 94025.0 keysNN2: 26025.0, 50525.0, 67050.0, 82025.0, 194025.0 keysFI: 70075.0, 112025.0, 126550.0, 153050.0, 213575.0 keys5p: 292025.0, 492025.0, 583550.0, 675525.0, 1064525.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: All: 525.3, 525.3, 525.3, 525.3, 525.3 #key, ties, win, loss, win-loss keysNN, 4, 1, 0, 1 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keys5p, 4, 1, 0, 1 keys2, 4, 1, 0, 1 keysNN2, 0, 0, 5, -5 ---------- Model 54 Cost: keysNBR: 0.0, 12500.0, 29000.0, 72000.0, 178000.0 keysCo: 14000.0, 49000.0, 72000.0, 108000.0, 222000.0 keys: 33500.0, 78000.0, 93000.0, 111000.0, 153500.0 keysNN: 73500.0, 140500.0, 173000.0, 204500.0, 326500.0 keys2: 143000.0, 198500.0, 228000.0, 250500.0, 325000.0 keysNN2: 189500.0, 259000.0, 299000.0, 357500.0, 598500.0 keysFI: 552500.0, 704500.0, 801500.0, 913500.0, 1126000.0 keys5p: 1000000.0, 1340500.0, 1439500.0, 1590000.0, 1935000.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keys5p, keysCo, keysFI, keysNN, keysNN2: 845.2, 845.2, 845.2, 845.2, 845.2 keysNBR: 843.7, 844.0, 844.6, 845.2, 845.2 #key, ties, win, loss, win-loss keysNN, 4, 1, 0, 1 keysFI, 4, 1, 0, 1 keysCo, 4, 1, 0, 1 keys5p, 4, 1, 0, 1 keys2, 4, 1, 0, 1 keysNN2, 0, 0, 5, -5 ---------- Model 58 Cost: keysNBR: 0.0, 0.0, 0.0, 0.0, 2000.0 keysCo: 203000.0, 312050.0, 402575.0, 504525.0, 806550.0 keys: 288050.0, 402075.0, 430050.0, 485550.0, 693050.0 keysNN: 430525.0, 605050.0, 688550.0, 794575.0, 1127025.0 keys2: 628125.0, 745125.0, 796650.0, 875625.0, 1099575.0 keysNN2: 671550.0, 940025.0, 1008025.0, 1157600.0, 1456050.0 keysFI: 2032525.0, 2390525.0, 2536125.0, 2703625.0, 3397550.0 keys5p: 2798650.0, 3351550.0, 3543525.0, 3781050.0, 4197525.0 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 0, 4, 1, 3 keys2, 0, 3, 2, 1 keysNN2, 0, 2, 3, -1 keysFI, 0, 1, 4, -3 keys5p, 0, 0, 5, -5 Attainment: keys2, keys, keysCo, keysFI,keysNN: 1787.9, 1787.9, 1787.9, 1787.9, 1787.9 keys5p: 1787.7, 1787.9, 1787.9, 1787.9, 1787.9 keysNBR: 1778.5, 1778.5, 1778.5, 1778.5, 1778.7 keysNN2: 1787.3, 1787.9, 1787.9, 1787.9, 1787.9 #key, ties, win, loss, win-loss keysCo, 0, 5, 0, 5 keysNN, 3, 1, 1, 0 keysFI, 3, 1, 1, 0 keys2, 3, 1, 1, 0 keys5p, 4, 0, 1, -1 keysNN2, 1, 0, 4, -4 ----------