Coala Benchmarks

Paper Benchmarks

Some memouts are not shown in the table!
  default_one_wrap default_negation_one_wrap incremental_one encoding_one_wrap encoding_incremental_one encoding_reverse_one
  CPU time - avg CPU time - timeouts/count Solver time total - avg CPU time - avg CPU time - timeouts/count Solver time total - avg CPU time - avg CPU time - timeouts/count Solver time total - avg CPU time - avg CPU time - timeouts/count Solver time total - avg CPU time - avg CPU time - timeouts/count Solver time total - avg CPU time - avg CPU time - timeouts/count Solver time total - avg
blocksworld/b08_16 44.19 0/3 20.90 44.31 0/3 21.84 4.30 0/3 4.23 323.43 0/3 69.98 76.83 0/3 74.83 112.95 0/3 109.78
blocksworld/b09_16 53.56 0/3 9.27 55.78 0/3 9.58 11.68 0/3 11.58 283.90 0/3 280.33 338.64 0/3 332.95
blocksworld/b10_17 88.69 0/3 20.53 77.38 0/3 11.74 14.69 0/3 14.56 366.34 0/3 361.42 220.96 0/3 220.96
blocksworld/b11_18 403.31 0/3 276.31 403.78 0/3 247.84 117.53 0/3 117.37 559.73 0/3 559.73
blocksworld/b12_19 228.47 -1/3 160.37 290.91 0/3 163.54 69.99 0/3 69.81 537.90 0/3 159.51 0/3 159.51 88.63 0/3 88.63
ferryman/f03_15 29.38 0/3 6.80 29.79 0/3 7.26 2.67 0/3 2.63 49.84 0/3 7.43 7.40 0/3 6.99 13.09 0/3 12.31
ferryman/f04_17 65.40 0/3 14.48 64.00 0/3 13.26 3.48 0/3 3.44 96.89 0/3 14.20 11.84 0/3 11.17 26.96 0/3 25.71
ferryman/f05_19 132.22 0/3 26.67 100.30 -1/3 26.83 4.36 0/3 4.32 170.14 0/3 23.46 19.35 0/3 18.31 46.27 0/3 44.35
ferryman/f06_17 122.92 0/3 28.63 120.93 0/3 26.29 6.03 0/3 5.97 202.99 0/3 28.12 26.57 0/3 25.26 67.57 0/3 65.13
ferryman/f07_19 243.05 0/3 54.12 257.18 0/3 50.53 18.04 0/3 17.96 356.48 0/3 48.50 40.12 0/3 38.19 138.62 0/3 135.09
Hanoi/h03_31 85.89 0/3 9.77 88.48 0/3 10.68 1.83 0/3 1.80 50.80 0/3 5.01 4.66 0/3 4.50 281.18 0/3 280.86
Hanoi/h04_63 69.51 0/3 69.41 54.74 0/3 54.19
yale/y04_18 6.22 0/3 0.16 6.14 0/3 0.20 10.18 0/3 10.14 11.35 0/3 0.07 5.13 0/3 5.10 16.11 0/3 16.08
yale/y05_20 40.91 0/3 0.14 35.52 0/3 0.79 16.83 0/3 16.80 64.19 0/3 0.86 40.59 0/3 40.56 69.95 0/3 69.91
yale/y06_22 132.51 0/3 5.88 165.62 0/3 4.50 79.05 0/3 79.02 87.20 0/3 0.34 177.14 0/3 177.10
yale/y07_24 547.40 0/3 0.45 205.35 0/3 205.31 499.40 0/3 22.02

Preliminary Benchmarks

Observations

  • the incremental direct encoding performs best in all benchmark classes
  • the incremental non-direct encoding performs worse
    • often significantly worse than the direct incremental encoding
    • in the harder blocksworld and yale instances it is even worse then the non-incremental direct encoding. in the other instances it performs better
  • the reverse incremental non-direct encoding performs similar to the non-reverse (previous) one
    • it is never (one exception) faster, but similar in blocksworld and yale classes
    • in the hanoi and ferryman classes it needs twice the time
  • the non-direct non-incremental encoding is the worst of all
    • it can't solve hanoi and yale at all and times out on the harder blocksworld and ferryman instances
    • compared to its twin the direct non-incremental encoding is loses badly
  • the direct non-incremental encoding is surprisingly good
    • sometimes even better then the non-direct incremental encodings
  • the direct non-incremental encoding with simulated classical negation (more integrity constrains) is slightly worse than without simulated classical negation
    • has the same number of equivalences and constraint type distribution, but more choices and conflicts during search
    • grounding time is almost the same, but solving time is higher for harder instances

Conclusions

  • classical negation has still be simulated for fasted encoding, so push Roland to implement this in gringo
  • develop a direct reverse incremental encoding and compare its performance to the others
  • develop a direct LTL encoding to speed up LTL queries

One Solution

Times

Times, One solution, iClasp wrapper

  default_one_wrap default_negation_one_wrap encoding_one_wrap incremental_one encoding_incremental_one encoding_reverse_one
  CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg
blocksworld/p01_05 0.62 0.09 0.00 0.59 0.08 0.01 3.40 0.51 0.08 0.19 0.09 0.00 1.19 0.65 0.02 1.42 0.80 0.02
blocksworld/p02_06 0.60 0.14 0.00 0.60 0.15 0.00 2.47 0.32 0.01 0.15 0.07 0.00 0.74 0.38 0.01 0.87 0.48 0.01
blocksworld/p03_08 3.05 0.38 0.02 3.03 0.37 0.02 22.01 2.27 0.42 0.61 0.38 0.01 5.78 2.94 0.45 7.37 3.82 0.64
blocksworld/p04_09 11.47 1.38 0.20 11.58 1.41 0.17 94.37 8.78 1.61 2.15 1.36 0.09 24.56 11.52 3.12 30.01 15.14 2.58
blocksworld/p05_11 17.37 1.64 0.64 15.26 1.62 0.49 140.03 10.43 6.12 2.62 1.68 0.13 31.24 13.96 4.47 36.68 18.68 2.18
blocksworld/p06_13 40.64 10.70 0.37 41.57 10.82 0.65 154.48 8.71 14.84 3.04 1.44 0.93 42.80 11.29 20.83 52.73 15.45 24.34
blocksworld/p07_14 85.16 28.08 1.91 85.89 28.15 2.04 409.38 19.28 79.40 6.03 3.06 1.53 160.99 24.38 112.99 139.78 33.88 75.71
blocksworld/p08_16 147.57 62.01 4.22 175.65 62.69 6.68 19.51 5.85 11.04 412.73 47.80 316.08
blocksworld/p09_16 170.36 10.48 13.71 178.27 10.79 18.39 memout 36.30 10.62 21.14 memout memout
blocksworld/p10_17 219.59 14.80 43.78 298.29 15.17 58.35 memout 67.32 14.47 46.79 memout memout
ferryman/p01_09 5.31 1.24 0.15 5.54 1.29 0.26 14.39 1.31 0.60 0.75 0.34 0.26 3.70 1.65 0.72 5.22 2.51 0.80
ferryman/p02_13 41.19 8.08 2.20 41.57 8.25 2.03 82.72 5.03 5.33 2.90 1.24 1.29 14.93 6.49 2.74 30.95 10.16 12.32
ferryman/p03_15 97.20 17.92 4.15 73.95 17.90 5.88 173.13 8.68 12.36 5.66 2.08 2.97 29.02 11.44 6.90 54.84 17.76 21.57
ferryman/p04_17 217.94 36.01 11.89 212.94 36.19 8.20 325.38 14.55 9.49 8.51 3.39 4.21 47.68 18.63 10.78 92.95 29.96 36.18
ferryman/p05_19 426.21 67.99 15.18 447.66 69.22 19.49 556.25 23.38 35.23 19.91 5.35 13.14 92.40 30.33 31.53 178.72 49.16 83.83
ferryman/p06_17 411.66 68.43 27.35 419.37 66.53 47.53 26.89 6.29 18.84 127.91 36.67 54.62 205.24 59.45 91.38
Hanoi/p01_07 0.60 0.13 0.00 0.61 0.13 0.00 0.13 0.05 0.00 0.35 0.17 0.01 0.44 0.23 0.00
Hanoi/p02_15 10.32 1.89 0.05 10.58 1.97 0.04 0.41 0.26 0.04 1.87 0.88 0.15 2.86 1.28 0.54
Hanoi/p03_31 296.08 32.01 2.28 227.85 32.42 4.59 4.61 1.18 3.02 16.82 3.92 8.36 451.81 6.64 437.91
Hanoi/p04_63 197.09 5.23 189.42 384.39 15.85 341.18
yale/y01_12 0.52 0.02 0.01 0.57 0.03 0.01 0.25 0.01 0.18 0.29 0.04 0.18 0.44 0.05 0.31
yale/y02_14 1.51 0.04 0.05 1.36 0.04 0.05 0.79 0.03 0.70 1.14 0.04 1.02 3.30 0.06 3.11
yale/y03_16 5.04 0.05 1.27 5.31 0.07 1.31 4.93 0.03 4.83 6.98 0.06 6.81 11.29 0.14 11.01
yale/y04_18 17.09 0.08 0.12 16.11 0.10 0.12 18.94 0.07 18.80 46.45 0.08 46.23 47.88 0.17 47.49

Properties

Properties, One solution, iClasp wrapper

  default_one_wrap (coala 0.9.1) default_negation_one_wrap (coala 0.9.1) encoding_one_wrap (coala 0.9.1) incremental_one (coala 0.9.1) encoding_incremental_one (coala 0.9.1) encoding_reverse_one (coala 0.9.1)
  Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg
blocksworld/p01_05 28.00 21.00 0.00 2514.00 92.30 5.00 30.00 24.00 0.00 2514.00 92.30 5.00 21.00 15.00 0.00 70580.00 76.80 21.80 26.00 19.00 0.00 2134.00 92.00 5.40 14.00 12.00 0.00 57912.00 76.60 22.00 24.00 11.00 0.00 69275.00 75.00 22.10
blocksworld/p02_06 14.00 5.00 0.00 2124.00 90.20 6.50 14.00 5.00 0.00 2124.00 90.20 6.50 18.00 6.00 0.00 45670.00 77.40 20.60 24.00 13.00 0.00 2000.00 89.90 6.80 37.00 21.00 0.00 36814.00 77.20 21.00 15.00 11.00 0.00 41775.00 75.20 21.10
blocksworld/p03_08 69.00 11.00 0.00 6474.00 94.20 4.00 69.00 11.00 0.00 6474.00 94.20 4.00 164.00 71.00 0.00 273144.00 75.60 23.50 120.00 70.00 0.00 5468.00 94.20 4.10 184.00 111.00 0.00 222734.00 75.60 23.60 275.00 149.00 0.00 282974.00 74.20 24.10
blocksworld/p04_09 585.00 255.00 2.00 13110.00 96.20 2.60 541.00 244.00 1.00 13110.00 96.20 2.60 267.00 135.00 1.00 928972.00 74.70 24.80 344.00 202.00 0.00 11906.00 96.20 2.70 465.00 289.00 1.00 757720.00 74.70 24.90 398.00 218.00 1.00 1030335.00 73.60 25.50
blocksworld/p05_11 2094.00 1071.00 4.00 15798.00 96.20 2.70 1360.00 749.00 3.00 15798.00 96.20 2.70 1244.00 469.00 2.00 1104380.00 74.70 24.90 335.00 195.00 1.00 13254.00 96.20 2.70 572.00 298.00 1.00 895012.00 74.70 24.90 215.00 139.00 0.00 1219235.00 73.60 25.50
blocksworld/p06_13 1141.00 480.00 3.00 15438.00 95.60 3.20 2714.00 1181.00 4.00 15438.00 95.60 3.20 2970.00 1616.00 5.00 910448.00 74.90 24.50 4052.00 2399.00 9.00 13219.00 95.50 3.20 4791.00 2791.00 10.00 734546.00 74.90 24.60 4931.00 2832.00 10.00 981093.00 73.70 25.20
blocksworld/p07_14 4028.00 2051.00 5.00 23394.00 96.60 2.40 6139.00 2628.00 6.00 23394.00 96.60 2.40 10806.00 5661.00 8.00 1872088.00 74.40 25.20 3487.00 2050.00 9.00 19538.00 96.60 2.50 11964.00 7155.00 15.00 1507696.00 74.40 25.20 7445.00 4217.00 13.00 2094062.00 73.40 25.90
blocksworld/p08_16 7196.00 3257.00 7.00 33534.00 97.30 1.90 9645.00 5360.00 8.00 33534.00 97.30 1.90 15705.00 9167.00 20.00 27239.00 97.30 1.90 19575.00 11668.00 21.00 2811903.00 74.00 25.70
blocksworld/p09_16 13869.00 8095.00 9.00 46098.00 97.80 1.60 17164.00 10046.00 9.00 46098.00 97.80 1.60 23777.00 13128.00 23.00 36551.00 97.80 1.60
blocksworld/p10_17 41614.00 22134.00 11.00 54912.00 98.00 1.40 45463.00 27510.00 12.00 54912.00 98.00 1.40 42877.00 25071.00 31.00 42698.00 98.00 1.40
ferryman/p01_09 2070.00 798.00 3.00 3902.00 76.30 8.40 2901.00 1280.00 4.00 3902.00 76.30 8.40 3511.00 1305.00 4.00 101518.00 75.70 22.70 3022.00 1739.00 9.00 3882.00 77.50 7.10 4596.00 2502.00 12.00 101518.00 75.30 23.00 2515.00 1436.00 8.00 183121.00 74.50 23.30
ferryman/p02_13 13447.00 6204.00 8.00 8670.00 77.60 6.50 11939.00 5023.00 8.00 8670.00 77.60 6.50 13154.00 5311.00 8.00 348434.00 76.20 22.30 9973.00 4657.00 20.00 8650.00 78.40 5.80 11367.00 5395.00 26.00 348438.00 75.90 22.60 14582.00 10129.00 31.00 644305.00 75.00 23.00
ferryman/p03_15 22054.00 9409.00 9.00 12077.00 78.20 5.90 25250.00 11193.00 9.00 12077.00 78.20 5.90 20142.00 9560.00 9.00 586384.00 76.40 22.10 17857.00 8720.00 33.00 12063.00 78.40 5.90 20985.00 10050.00 35.00 586394.00 76.20 22.30 22198.00 14290.00 40.00 1091578.00 75.20 22.80
ferryman/p04_17 38050.00 15729.00 10.00 16278.00 78.60 5.40 31671.00 12939.00 10.00 16278.00 78.60 5.40 17004.00 7272.00 8.00 940430.00 76.60 21.90 23090.00 11135.00 42.00 16268.00 78.90 5.40 29021.00 14774.00 46.00 940446.00 76.50 22.00 31195.00 19590.00 54.00 1758597.00 75.30 22.80
ferryman/p05_19 49278.00 20207.00 11.00 21357.00 79.00 5.00 58690.00 25396.00 11.00 21357.00 79.00 5.00 42777.00 18987.00 11.00 1449008.00 76.90 21.80 44307.00 22687.00 62.00 21351.00 79.00 5.40 51520.00 28297.00 66.00 1449030.00 76.70 21.90 50817.00 33609.00 72.00 2717956.00 75.50 22.60
ferryman/p06_17 56788.00 26634.00 12.00 22444.00 78.70 5.20 68138.00 31012.00 12.00 22444.00 78.70 5.20 47019.00 24517.00 61.00 22440.00 79.10 5.10 67057.00 35808.00 72.00 1794512.00 76.30 22.60 50052.00 32158.00 68.00 3345529.00 75.50 23.00
Hanoi/p01_07 34.00 20.00 0.00 1839.00 84.60 11.10 34.00 20.00 0.00 1839.00 84.60 11.10 43.00 29.00 0.00 1729.00 83.60 12.00 50.00 30.00 0.00 16956.00 74.50 22.00 13.00 11.00 0.00 20439.00 74.20 18.20
Hanoi/p02_15 1489.00 810.00 3.00 5771.00 86.40 10.10 1217.00 640.00 3.00 5771.00 86.40 10.10 1147.00 714.00 3.00 5337.00 85.40 11.00 1243.00 715.00 4.00 68502.00 74.30 23.20 2394.00 1535.00 8.00 88600.00 74.20 20.00
Hanoi/p03_31 27232.00 12996.00 10.00 16575.00 88.30 8.70 35095.00 18009.00 11.00 16575.00 88.30 8.70 26102.00 15058.00 58.00 15180.00 85.80 11.30 19841.00 12149.00 50.00 247312.00 73.80 24.20 198375.00 140831.00 127.00 338948.00 71.90 23.70
Hanoi/p04_63 257199.00 166725.00 253.00 40858.00 85.00 12.70 233185.00 141193.00 249.00 827960.00 73.20 25.30
yale/y01_12 439.00 268.00 2.00 852.00 76.60 20.10 439.00 268.00 2.00 852.00 76.60 20.10 5427.00 4009.00 14.00 558.00 67.50 29.70 4644.00 3301.00 13.00 3646.00 70.00 24.80 3744.00 2797.00 11.00 4888.00 68.50 17.80
yale/y02_14 1332.00 879.00 4.00 1120.00 77.20 19.60 1332.00 879.00 4.00 1120.00 77.20 19.60 11189.00 8112.00 25.00 720.00 68.20 29.10 12141.00 9057.00 25.00 4869.00 69.00 26.10 16468.00 13518.00 28.00 6752.00 68.10 19.00
yale/y03_16 26122.00 18555.00 11.00 1424.00 77.80 19.10 29477.00 20592.00 11.00 1424.00 77.80 19.10 55183.00 42983.00 53.00 902.00 65.90 31.60 57224.00 43840.00 57.00 6268.00 67.60 27.90 45809.00 36572.00 46.00 8980.00 66.90 20.90
yale/y04_18 3365.00 1776.00 5.00 1764.00 78.40 18.60 3365.00 1776.00 5.00 1764.00 78.40 18.60 174708.00 130263.00 79.00 1104.00 64.10 33.60 254899.00 200517.00 84.00 7843.00 65.90 29.90 153756.00 122861.00 68.00 11596.00 64.20 24.50

Properties ctd.

Only incremental, more properties, One solution

  incremental_one (coala 0.9.1) encoding_incremental_one (coala 0.9.1) encoding_reverse_one (coala 0.9.1)
  Choices - avg Conflicts - avg Restarts - avg Variables - avg Rules - avg Equivalences - avg Constraints - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Variables - avg Rules - avg Equivalences - avg Constraints - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Variables - avg Rules - avg Equivalences - avg Constraints - avg Binary Constraints - avg Ternary Constraints - avg
blocksworld/p01_05 26.00 19.00 0.00 6493.00 13880.00 2134.00 8144.00 92.00 5.40 14.00 12.00 0.00 18211.00 52332.00 57912.00 28733.00 76.60 22.00 24.00 11.00 0.00 19714.00 57487.00 69275.00 23724.00 75.00 22.10
blocksworld/p02_06 24.00 13.00 0.00 4180.00 9262.00 2000.00 5502.00 89.90 6.80 37.00 21.00 0.00 11909.00 32796.00 36814.00 17165.00 77.20 21.00 15.00 11.00 0.00 13252.00 35355.00 41775.00 14966.00 75.20 21.10
blocksworld/p03_08 120.00 70.00 0.00 25617.00 57122.00 5468.00 32223.00 94.20 4.10 184.00 111.00 0.00 76751.00 218875.00 222734.00 129772.00 75.60 23.60 275.00 149.00 0.00 81131.00 243483.00 282974.00 119599.00 74.20 24.10
blocksworld/p04_09 344.00 202.00 0.00 82024.00 196429.00 11906.00 95267.00 96.20 2.70 465.00 289.00 1.00 269454.00 780876.00 757720.00 437787.00 74.70 24.90 398.00 218.00 1.00 279819.00 888909.00 1030335.00 406084.00 73.60 25.50
blocksworld/p05_11 335.00 195.00 1.00 103167.00 239351.00 13254.00 123467.00 96.20 2.70 572.00 298.00 1.00 329222.00 938248.00 895012.00 570434.00 74.70 24.90 215.00 139.00 0.00 341201.00 1065705.00 1219235.00 508401.00 73.60 25.50
blocksworld/p06_13 4052.00 2399.00 9.00 82460.00 200772.00 13219.00 99849.00 95.50 3.20 4791.00 2791.00 10.00 273611.00 770392.00 734546.00 438551.00 74.90 24.60 4931.00 2832.00 10.00 286001.00 867219.00 981093.00 447721.00 73.70 25.20
blocksworld/p07_14 3487.00 2050.00 9.00 176040.00 415946.00 19538.00 209413.00 96.60 2.50 11964.00 7155.00 15.00 575435.00 1623333.00 1507696.00 984551.00 74.40 25.20 7445.00 4217.00 13.00 594173.00 1851835.00 2094062.00 932639.00 73.40 25.90
blocksworld/p08_16 15705.00 9167.00 20.00 333006.00 782812.00 27239.00 387644.00 97.30 1.90 19575.00 11668.00 21.00 1094314.00 3088854.00 2811903.00 1926910.00 74.00 25.70
blocksworld/p09_16 23777.00 13128.00 23.00 584485.00 1369026.00 36551.00 663883.00 97.80 1.60
blocksworld/p10_17 42877.00 25071.00 31.00 789275.00 1823507.00 42698.00 888760.00 98.00 1.40
ferryman/p01_09 3022.00 1739.00 9.00 5531.00 38232.00 3882.00 15640.00 77.50 7.10 4596.00 2502.00 12.00 50023.00 119851.00 101518.00 21211.00 75.30 23.00 2515.00 1436.00 8.00 58391.00 155639.00 183121.00 23155.00 74.50 23.30
ferryman/p02_13 9973.00 4657.00 20.00 10083.00 133222.00 8650.00 30276.00 78.40 5.80 11367.00 5395.00 26.00 177373.00 431653.00 348438.00 37721.00 75.90 22.60 14582.00 10129.00 31.00 201425.00 558879.00 644305.00 43090.00 75.00 23.00
ferryman/p03_15 17857.00 8720.00 33.00 12815.00 224190.00 12063.00 37648.00 78.40 5.90 20985.00 10050.00 35.00 300554.00 737071.00 586394.00 47473.00 76.20 22.30 22198.00 14290.00 40.00 338126.00 952895.00 1091578.00 56034.00 75.20 22.80
ferryman/p04_17 23090.00 11135.00 42.00 15911.00 358772.00 16268.00 47234.00 78.90 5.40 29021.00 14774.00 46.00 484271.00 1195047.00 940446.00 60365.00 76.50 22.00 31195.00 19590.00 54.00 540215.00 1542743.00 1758597.00 70704.00 75.30 22.80
ferryman/p05_19 44307.00 22687.00 62.00 19389.00 550972.00 21351.00 58657.00 79.00 5.40 51520.00 28297.00 66.00 748498.00 1856761.00 1449030.00 75213.00 76.70 21.90 50817.00 33609.00 72.00 828686.00 2393679.00 2717956.00 87403.00 75.50 22.60
ferryman/p06_17 47019.00 24517.00 61.00 25770.00 661792.00 22440.00 78971.00 79.10 5.10 67057.00 35808.00 72.00 905013.00 2277269.00 1794512.00 101618.00 76.30 22.60 50052.00 32158.00 68.00 994729.00 2932295.00 3345529.00 113776.00 75.50 23.00
Hanoi/p01_07 43.00 29.00 0.00 1162.00 5892.00 1729.00 1419.00 83.60 12.00 50.00 30.00 0.00 5395.00 15196.00 16956.00 3633.00 74.50 22.00 13.00 11.00 0.00 6637.00 17223.00 20439.00 4921.00 74.20 18.20
Hanoi/p02_15 1147.00 714.00 3.00 4726.00 28200.00 5337.00 5551.00 85.40 11.00 1243.00 715.00 4.00 24808.00 68160.00 68502.00 16503.00 74.30 23.20 2394.00 1535.00 8.00 29365.00 78109.00 88600.00 22889.00 74.20 20.00
Hanoi/p03_31 26102.00 15058.00 58.00 17234.00 113881.00 15180.00 20768.00 85.80 11.30 19841.00 12149.00 50.00 97665.00 264310.00 247312.00 60994.00 73.80 24.20 198375.00 140831.00 127.00 112226.00 306079.00 338948.00 93659.00 71.90 23.70
Hanoi/p04_63 257199.00 166725.00 253.00 57512.00 410559.00 40858.00 72010.00 85.00 12.70 233185.00 141193.00 249.00 345816.00 928546.00 827960.00 225475.00 73.20 25.30
yale/y01_12 5427.00 4009.00 14.00 1014.00 1614.00 558.00 2579.00 67.50 29.70 4644.00 3301.00 13.00 1542.00 2744.00 3646.00 2994.00 70.00 24.80 3744.00 2797.00 11.00 2916.00 4166.00 4888.00 3306.00 68.50 17.80
yale/y02_14 11189.00 8112.00 25.00 1393.00 2275.00 720.00 3645.00 68.20 29.10 12141.00 9057.00 25.00 2079.00 3782.00 4869.00 4323.00 69.00 26.10 16468.00 13518.00 28.00 4046.00 5812.00 6752.00 4717.00 68.10 19.00
yale/y03_16 55183.00 42983.00 53.00 1832.00 3080.00 902.00 5233.00 65.90 31.60 57224.00 43840.00 57.00 2696.00 5018.00 6268.00 6040.00 67.60 27.90 45809.00 36572.00 46.00 5392.00 7794.00 8980.00 6498.00 66.90 20.90
yale/y04_18 174708.00 130263.00 79.00 2331.00 4041.00 1104.00 7176.00 64.10 33.60 254899.00 200517.00 84.00 3393.00 6464.00 7843.00 8248.00 65.90 29.90 153756.00 122861.00 68.00 6966.00 10136.00 11596.00 8851.00 64.20 24.50

All Solutions

Times

Times, All solutions, iClasp wrapper

  default_all_wrap default_negation_all_wrap encoding_all_wrap incremental_all encoding_incremental_all encoding_reverse_all
  CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg CPU time - avg Grounding Time - avg Solving Time - avg
blocksworld/p01_05 0.62 0.09 0.00 0.60 0.09 0.00 3.46 0.51 0.08 0.19 0.09 0.00 1.19 0.64 0.03 1.44 0.81 0.02
blocksworld/p02_06 0.61 0.14 0.00 0.62 0.15 0.01 2.51 0.32 0.02 0.15 0.07 0.00 0.73 0.38 0.02 0.88 0.48 0.00
blocksworld/p03_08 3.09 0.36 0.09 3.10 0.36 0.10 18.62 2.28 1.41 0.67 0.38 0.07 6.60 2.97 1.24 8.50 3.87 1.68
blocksworld/p04_09 11.48 1.36 0.21 11.52 1.36 0.23 97.07 9.02 3.54 2.19 1.38 0.12 24.19 11.24 3.23 30.56 15.25 3.01
blocksworld/p05_11 17.39 1.61 0.73 17.35 1.65 0.58 148.25 10.63 12.54 2.91 1.70 0.39 33.30 14.01 6.50 42.26 18.82 7.46
blocksworld/p06_13 43.77 10.77 3.08 43.54 10.88 2.50 168.99 8.56 29.38 5.31 1.41 3.21 59.02 11.31 36.88 69.44 15.36 41.14
blocksworld/p07_14 88.33 28.20 4.57 89.37 28.53 4.73 424.01 18.77 94.67 9.69 2.98 5.26 167.17 24.46 118.99 159.74 33.93 95.54
blocksworld/p08_16 114.87 61.64 9.60 186.75 62.58 16.69 26.20 5.88 17.65 487.10 47.39 391.47
blocksworld/p09_16 177.79 10.71 20.64 119.57 10.51 38.70 553.93 41.38 10.45 26.38 144.70 103.62
blocksworld/p10_17 305.86 14.76 60.67 312.95 15.13 70.07 389.91 85.69 14.67 64.80 121.41 97.25
ferryman/p01_09 5.68 1.25 0.47 5.75 1.26 0.52 15.49 1.33 1.48 0.89 0.32 0.42 4.13 1.67 1.14 6.15 2.50 1.75
ferryman/p02_13 41.94 8.16 2.54 41.89 8.23 2.42 84.80 5.06 7.52 3.03 1.23 1.40 17.58 6.46 5.44 35.09 9.97 16.80
ferryman/p03_15 96.68 17.68 4.86 98.44 17.79 6.45 181.43 8.75 19.50 6.06 2.12 3.31 32.59 11.37 10.57 62.35 18.01 28.66
ferryman/p04_17 216.95 35.85 12.41 216.39 36.54 8.97 326.24 14.45 14.86 9.19 3.42 4.85 55.30 18.91 17.88 107.45 29.81 50.92
ferryman/p05_19 421.56 67.04 16.31 453.98 69.38 22.27 564.53 23.57 43.57 21.31 5.40 14.45 101.39 30.45 40.38 198.92 49.12 103.74
ferryman/p06_17 412.97 68.45 28.98 439.23 68.63 51.81 27.74 6.33 19.64 148.05 36.47 75.03 234.29 58.62 121.94
Hanoi/p01_07 0.60 0.13 0.01 0.61 0.13 0.01 0.13 0.06 0.00 0.35 0.17 0.00 0.44 0.22 0.01
Hanoi/p02_15 10.54 1.93 0.05 10.67 1.96 0.05 0.42 0.26 0.05 1.94 0.90 0.22 2.94 1.27 0.61
Hanoi/p03_31 296.06 32.03 3.01 303.78 32.52 4.78 4.80 1.20 3.22 17.35 4.00 8.70 461.14 6.67 447.15
Hanoi/p04_63 220.00 5.33 212.23 399.57 15.71 356.75
yale/y01_12 0.85 0.02 0.27 0.94 0.03 0.31 0.46 0.02 0.37 0.66 0.04 0.56 0.89 0.07 0.73
yale/y02_14 3.73 0.04 1.78 3.86 0.04 1.98 2.80 0.02 2.70 3.93 0.06 3.78 9.04 0.11 8.84
yale/y03_16 23.70 0.05 15.33 24.96 0.06 16.15 23.37 0.03 23.25 39.49 0.06 39.33 58.46 0.12 58.17
yale/y04_18 199.05 0.08 134.07 197.18 0.10 134.82 247.35 0.05 247.20 334.77 0.07 334.54 526.06 0.19 525.67

Properties

Properties, All solutions, iClasp wrapper

  default_all_wrap (coala 0.9.1) default_negation_all_wrap (coala 0.9.1) encoding_all_wrap (coala 0.9.1) incremental_all (coala 0.9.1) encoding_incremental_all (coala 0.9.1) encoding_reverse_all (coala 0.9.1)
  Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg Choices - avg Conflicts - avg Restarts - avg Equivalences - avg Binary Constraints - avg Ternary Constraints - avg
blocksworld/p01_05 29.00 22.00 0.00 2514.00 92.30 5.00 30.00 24.00 0.00 2514.00 92.30 5.00 21.00 16.00 0.00 70580.00 76.80 21.80 26.00 20.00 0.00 2134.00 92.00 5.40 21.00 18.00 0.00 57912.00 76.60 22.00 32.00 22.00 0.00 69275.00 75.00 22.10
blocksworld/p02_06 19.00 11.00 0.00 2124.00 90.20 6.50 19.00 11.00 0.00 2124.00 90.20 6.50 36.00 17.00 0.00 45670.00 77.40 20.60 32.00 23.00 0.00 2000.00 89.90 6.80 37.00 23.00 0.00 36814.00 77.20 21.00 20.00 16.00 0.00 41775.00 75.20 21.10
blocksworld/p03_08 569.00 353.00 0.00 6474.00 94.20 4.00 614.00 397.00 0.00 6474.00 94.20 4.00 545.00 300.00 0.00 273144.00 75.60 23.50 479.00 281.00 0.00 5468.00 94.20 4.10 643.00 402.00 0.00 222734.00 75.60 23.60 707.00 431.00 0.00 282974.00 74.20 24.10
blocksworld/p04_09 609.00 270.00 2.00 13110.00 96.20 2.60 763.00 399.00 1.00 13110.00 96.20 2.60 591.00 332.00 1.00 928972.00 74.70 24.80 397.00 242.00 0.00 11906.00 96.20 2.70 542.00 319.00 1.00 757720.00 74.70 24.90 445.00 261.00 1.00 1030335.00 73.60 25.50
blocksworld/p05_11 2293.00 1208.00 4.00 15798.00 96.20 2.70 1610.00 928.00 3.00 15798.00 96.20 2.70 2172.00 1051.00 2.00 1104380.00 74.70 24.90 1114.00 667.00 1.00 13254.00 96.20 2.70 849.00 484.00 1.00 895012.00 74.70 24.90 1241.00 605.00 0.00 1219235.00 73.60 25.50
blocksworld/p06_13 8781.00 5639.00 3.00 15438.00 95.60 3.20 8124.00 4974.00 4.00 15438.00 95.60 3.20 5294.00 3089.00 5.00 910448.00 74.90 24.50 9981.00 6258.00 9.00 13219.00 95.50 3.20 7859.00 4634.00 10.00 734546.00 74.90 24.60 7656.00 4514.00 10.00 981093.00 73.70 25.20
blocksworld/p07_14 8251.00 4913.00 5.00 23394.00 96.60 2.40 11151.00 6039.00 6.00 23394.00 96.60 2.40 12006.00 6469.00 8.00 1872088.00 74.40 25.20 10726.00 6502.00 9.00 19538.00 96.60 2.50 12417.00 7440.00 15.00 1507696.00 74.40 25.20 9155.00 5299.00 13.00 2094062.00 73.40 25.90
blocksworld/p08_16 13971.00 7722.00 7.00 33534.00 97.30 1.90 19968.00 12871.00 8.00 33534.00 97.30 1.90 23075.00 13886.00 20.00 27239.00 97.30 1.90 24491.00 15165.00 21.00 2811903.00 74.00 25.70
blocksworld/p09_16 19875.00 12156.00 9.00 46098.00 97.80 1.60 30616.00 19562.00 9.00 46098.00 97.80 1.60 27799.00 15665.00 23.00 36551.00 97.80 1.60
blocksworld/p10_17 50788.00 28936.00 11.00 54912.00 98.00 1.40 51659.00 32244.00 12.00 54912.00 98.00 1.40 53289.00 32102.00 31.00 42698.00 98.00 1.40
ferryman/p01_09 5307.00 2541.00 3.00 3902.00 76.30 8.40 4723.00 2395.00 4.00 3902.00 76.30 8.40 5541.00 2637.00 4.00 101518.00 75.70 22.70 4712.00 2737.00 9.00 3882.00 77.50 7.10 5287.00 2888.00 12.00 101518.00 75.30 23.00 3924.00 2322.00 8.00 183121.00 74.50 23.30
ferryman/p02_13 14716.00 6855.00 8.00 8670.00 77.60 6.50 13294.00 5682.00 8.00 8670.00 77.60 6.50 14377.00 5883.00 8.00 348434.00 76.20 22.30 10650.00 4922.00 20.00 8650.00 78.40 5.80 13966.00 6697.00 26.00 348438.00 75.90 22.60 16505.00 11301.00 31.00 644305.00 75.00 23.00
ferryman/p03_15 23382.00 10076.00 9.00 12077.00 78.20 5.90 26723.00 11937.00 9.00 12077.00 78.20 5.90 23988.00 11716.00 9.00 586384.00 76.40 22.10 19513.00 9392.00 33.00 12063.00 78.40 5.90 22685.00 10753.00 35.00 586394.00 76.20 22.30 23817.00 15234.00 40.00 1091578.00 75.20 22.80
ferryman/p04_17 39596.00 16360.00 10.00 16278.00 78.60 5.40 32915.00 13589.00 10.00 16278.00 78.60 5.40 18206.00 7825.00 8.00 940430.00 76.60 21.90 25894.00 12760.00 42.00 16268.00 78.90 5.40 32658.00 16856.00 46.00 940446.00 76.50 22.00 36746.00 22577.00 54.00 1758597.00 75.30 22.80
ferryman/p05_19 51890.00 21706.00 11.00 21357.00 79.00 5.00 61548.00 26954.00 11.00 21357.00 79.00 5.00 44134.00 19530.00 11.00 1449008.00 76.90 21.80 46516.00 23890.00 62.00 21351.00 79.00 5.40 52866.00 28907.00 66.00 1449030.00 76.70 21.90 55045.00 36031.00 72.00 2717956.00 75.50 22.60
ferryman/p06_17 59729.00 27215.00 12.00 22444.00 78.70 5.20 69771.00 31570.00 12.00 22444.00 78.70 5.20 48408.00 25002.00 61.00 22440.00 79.10 5.10 68918.00 36560.00 72.00 1794512.00 76.30 22.60 52945.00 33716.00 68.00 3345529.00 75.50 23.00
Hanoi/p01_07 48.00 33.00 0.00 1839.00 84.60 11.10 48.00 33.00 0.00 1839.00 84.60 11.10 43.00 30.00 0.00 1729.00 83.60 12.00 51.00 35.00 0.00 16956.00 74.50 22.00 53.00 41.00 0.00 20439.00 74.20 18.20
Hanoi/p02_15 1538.00 854.00 3.00 5771.00 86.40 10.10 1386.00 768.00 3.00 5771.00 86.40 10.10 1351.00 891.00 3.00 5337.00 85.40 11.00 1627.00 1015.00 4.00 68502.00 74.30 23.20 2552.00 1669.00 8.00 88600.00 74.20 20.00
Hanoi/p03_31 32219.00 16122.00 10.00 16575.00 88.30 8.70 35438.00 18225.00 11.00 16575.00 88.30 8.70 27134.00 15839.00 58.00 15180.00 85.80 11.30 20089.00 12314.00 50.00 247312.00 73.80 24.20 199990.00 142127.00 127.00 338948.00 71.90 23.70
Hanoi/p04_63 266004.00 172726.00 253.00 40858.00 85.00 12.70 239906.00 145784.00 249.00 827960.00 73.20 25.30
yale/y01_12 5850.00 3832.00 2.00 852.00 76.60 20.10 5850.00 3832.00 2.00 852.00 76.60 20.10 9077.00 6313.00 14.00 558.00 67.50 29.70 10118.00 6981.00 13.00 3646.00 70.00 24.80 7297.00 4950.00 11.00 4888.00 68.50 17.80
yale/y02_14 24228.00 14090.00 4.00 1120.00 77.20 19.60 24405.00 14457.00 4.00 1120.00 77.20 19.60 25242.00 15513.00 25.00 720.00 68.20 29.10 26541.00 16762.00 25.00 4869.00 69.00 26.10 33268.00 23410.00 28.00 6752.00 68.10 19.00
yale/y03_16 110914.00 55034.00 11.00 1424.00 77.80 19.10 106242.00 49706.00 11.00 1424.00 77.80 19.10 132420.00 73798.00 53.00 902.00 65.90 31.60 141200.00 80207.00 57.00 6268.00 67.60 27.90 130466.00 74573.00 46.00 8980.00 66.90 20.90
yale/y04_18 642613.00 210771.00 5.00 1764.00 78.40 18.60 668058.00 236284.00 5.00 1764.00 78.40 18.60 706883.00 278734.00 79.00 1104.00 64.10 33.60 829753.00 382746.00 84.00 7843.00 65.90 29.90 781916.00 351336.00 68.00 11596.00 64.20 24.50