An implementation of on-demand strategy annotations: Experimental Evaluation

Experiments by María Alpuente, Santiago Escobar and Salvador Lucas.


We give some benchmark programs to demonstrate the practicality of our on-demand prototype interpreter OnDemandOBJ and our program transformation tool for removing negative annotations TransformNeg.

Benchmark examples


Experiments

The following three tables show the runtimes and the number of rewrite steps of the benchmarks for the different OBJ-family systems. Each runtime is the average of 10 executions measured in a Pentium III machine running redhat 7.2. The OnDemandOBJ interpreter is our on-demand prototype interpreter. CafeOBJ is developed in Lisp at the Japan Advanced Inst. of Science and Technology (JAIST). OBJ3, also written in Lisp, is maintained by the University of California at San Diego. Maude is developed in C++ and maintained by the Computer Science Lab at SRI International. OBJ3 and Maude provide only computations with positive annotations whereas CafeOBJ and OnDemandOBJ provide computations with negative annotations as well. Note that CafeOBJ and OBJ3 implement sharing of variables whereas Maude and OnDemandOBJ do not.

The following table compares the evaluation of expression pi(square(square(3))) using pi and pi_noneg. It witnesses that negative annotations are extremely useful in practice and that the program transformation enables the execution of negatively annotated programs in all OBJ implementations. This table also evidences that the implementation of the on-demand evaluation strategy of [AEGL02] in other systems is really interesting.
 
OBJ system (ms./rewrites)
pi
pi_noneg
OnDemandOBJ
25/364
45/689
CafeOBJ
30/364
55/689
OBJ3
unavailable
75/689
Maude
unavailable
0/689

On the other hand, the following table illustrates the interest of using negative annotations to improve the behavior of programs. There, for instance, program msquare_neg runs in less time and requires a smaller number of rewrite steps than msquare_eager or msquare_apt, which do not include negative annotations. Moreover, note that the program transformation is also very useful since execution of the expression minus(0,square(square(5))) is improved.
 
OBJ system (ms./rewrites)
msquare_eager
msquare_apt
msquare_neg
msquare_neg_noneg
OnDemandOBJ
33/715
62/1640
0/1
0/1
40/914
78/1992
80/1992
250/5243
CafeOBJ
40/715
50/715
0/1
0/1
50/914
60/914
60/914
180/2601
OBJ3
20/715
overflow
unavailable
0/1
30/914
overflow
unavailable
overflow
Maude
0/715
0/1640
unavailable
0/1
0/914
3/1992
unavailable
5/5243

Finally, the following table compares the execution of typical functional programs with canonical arbitrary strategies in OnDemandOBJ and in CafeOBJ, and demonstrates that there are clear advantages in using our implementation of the on-demand evaluation of [AEGL02].
 
OBJ system (ms./rewrites)
quicksort
minsort
mod
mod'
average
OnDemandOBJ
55/1373
87/1649
540/13661
135/3117
70/1399
CafeOBJ
42/658
overflow
180/3117
175/3117
130/1399


References

[AEGL02] M. Alpuente, S. Escobar, B. Gramlich, and S. Lucas. Improving on-demand strategy annotations. In Matthias Baaz and Andrei Voronkov, editors, Proc. 9th Int. Conf. on Logic for Programming, Artificial Intelligence, and Reasoning (LPAR'02), Lecture Notes in Computer Science, Tbilisi, Georgia, October 2002. Springer-Verlag.

[AG01] T. Arts and J. Giesl. A collection of examples for termination of term rewriting using dependency pairs. Technical Report AIB-2001-09 RWTH Aachen, 2001.


Please, send comments or suggestions to sescobar@dsic.upv.es. Any kind of contribution is welcome.

Last update Feb 2002 # sescobar@dsic.upv.es