An Investigation on Genetic Algorithms for Generic STRIPS Planning

8/17/2019 An Investigation on Genetic Algorithms for Generic STRIPS Planning

1/10

An Investigation on Genetic Algorithms for

Generic STRIPS Planning

Marcos Castilho1, Luis Allan Kunzle2, Edson Lecheta1,Viviane Palodeto1, and Fabiano Silva2

1 Departamento de Informatica, Federal University of Parana,Curitiba, Brazil

{ marcos, lecheta, vp99}@inf.ufpr.brhttp://www.inf.ufpr.br/∼ marcos

2 CPGEI/CEFET-PR,Curitiba, Brazil

{kunzle, fabiano

}@cpgei.cefetpr.br

http://www.cpgei.cefetpr.br/∼fabiano

Abstract. We investigate the technique of genetic algorithms to solvethe class of STRIPS planning problems in Artificial Intelligence. Wedefine a genetic algorithm called AgPlan and we compare it with somerecent planners.

Keywords: Planning, Evolutionary Computation.

1 Introduction

Given a formal description of the behaviour of a set of actions, to find a sequenceof actions which lead from a known state of the world to some desired oneis defined as the Planning Problem in Artificial Intelligence . When the formallanguage used to describe actions is STRIPS [1] we call it STRIPS Planning .

This restricted class of problems was proved to be PSPACE-complete [2].For more than twenty years the classical AI search techniques were not suf-

ficient to find solutions even for simple problems as the Sussman anomaly, asimple problem with three blocks in the well known blocks world domain. Onemain problem is that the search space is really huge. Moreover, there were nogood heuristic functions to guide the process. The proposals failed reaching veryeasily in local extremes.

Two recent approaches changed that picture. One was SATPLAN whosemain feature was to relate planning with satisfiability [3]. The other one was

GRAPHPLAN [4]. It is based on a structure called the plan graph which suc-cessfully reduced the huge search space to a smaller part of it where a solutioncan more easily be found.

Since then research in the planning field has known an extraordinary growth.Several other areas of research contributed with important algorithms and tools.We can cite integer programming [5], constraint satisfaction [6], Petri nets [7],

C. Lemaı̂tre, C.A. Reyes, and J.A. Gonzalez (Eds.): IBERAMIA 2004, LNAI 3315, pp. 185–194, 2004.cSpringer-Verlag Berlin Heidelberg 2004


2/10

186 M. Castilho et al.

among others. Also, ancient techniques were reintroduced, e.g. the system R, anew implementation of the original STRIPS , has manage to solve really hugeinstances of difficult problems as the blocks world [8]. Of particular interest to us,very good heuristic functions were finally found and the heuristic search planners

HSP [9] and FF [10] are among the best planners known today. Although someof these algorithms are very fast in general and are capable of dealing with largeinstances of problems, there are still open research to be done.

Genetic algorithms [11] (GA’s) are usually considered as a possible techniqueto apply when the search space is huge. Surprisingly, little has been done toimprove genetic techniques in domain independent planning. Main contributionsseems to be related with robotics, but in terms of genetic programming [12, 13].However, in the planning research it is usually considered that planners mustbe general purpose [14]. In this sense, as far as we could investigate, there is no

generic planner based on genetic algorithms. In this paper we show the resultsof our research in this direction.

The paper is organised as follows. Section 2 contains some basic points aboutplanning. In section 3 we introduce genetic algorithms in the context of plan-ning and we present the details of our genetic algorithm. In section 4 we showhow it works when applied to few well known domains used in the planningcompetitions.

2 The Planning Problem

The planning problem in Artificial Intelligence is, basically, to determine a se-quence of actions which, when applied into some known initial state, leads tosome desired final state. This problem is considered to be domain independent.The planner must be capable to read a planning problem consisting of a descrip-tion of the behaviour of the actions and a description of the initial and the finalsituation. It must return a sequence of actions that, when performed in the givenorder to the initial situation, transforms it into the final situation. In our case,

the planning problems are described in PDDL [15] that is considered today thecommon generic language to the planning community.Any sequence of actions that do so is called a plan . Unfortunately, there is no

fixed size for the plan. In most situations, it is supposed that the planner mustreturn the optimal plan. On the other hand, as it is difficult, just to find someplan is considered to be enough. Usually, the planner must run with limitedmemory and time. In this paper we consider that any plan which solve theproblem is a solution. We do not consider questions related to the quality of the plan or the number of actions used. In other terms, we do not consider the

planning problem as an optimisation problem.As we have said in the introduction there are several reasons for which wethink genetic algorithms apply well in planning problems, although it is not atrivial task to elaborate the encodings of the GA. The next section describesthis task in detail.


3/10

An Investigation on Genetic Algorithms for Generic STRIPS Planning 187

3 Preliminary Discussion on Genetic Planners

In this section we recall the main points. A more complete description may befound in [16]. In the next section we present alternative definitions.

3.1 Chromosome Encoding

The first basic point is how to encode a chromosome. Chromosomes are candi-dates of solution to the problem being solved, in our case, they are possible plans.In this way each gene will represent a single action of the plan. Consequently, thewhole chromosome is the full plan, i.e., a complete solution. An action is repre-sented by an integer (from 1 to the possible number of actions). Here comes thefirst difficult decision: plans are sequence of actions with no pre-specified lengthand in general chromosomes have fixed size.

Our first decision was to simulate variable length size using fixed-length chro-mosome with the inclusion of a new action in the list of possible actions (“NOP”– no operation). This is a null action that can be applied in any point of a planand has no effect to the current state of the world. With the use of this extraaction a given plan can be extended as wanted, without loosing its original ef-fect. Variable-length plans were then handled by fixed-length chromosomes, bysetting up an upper bound length for plans and filling them with NOP’s. Thisdecision was well suited for our experiments, since we were using a standardpackage called pgapack 1.

But we found in practice that we spent most of the time taking care of thesimulation process and the planner was too slow. The final decision was to definea variable-length chromosome. The impact of this decision was that we decidedto abandon that package and to develop all the code by ourselves.

3.2 The Fitness Function

The fitness function measures the quality of a chromosome, it must evaluate theindividuals which are in fact plans with the high possible value. The question

is: how do we know that a sequence of actions is a plan? The answer is thatthe chromosome must be validated starting with the action represented by itsfirst gene been applied to the initial state and sequentially all other actionsrepresented by the others genes until the final one obtains the goal state of theproblem.

Suppose a chromosome with size n and genes g1, g2, . . . , gn. It may happenthat g1 is not applicable to the initial state, but g2 is. It is possible that g3 is notapplicable in the state resulting for the application of g2, but g4 is. One possibledecision is to not allow this kind of situation. We have however take another

way.Our decision was to define an auxiliary binary vector parallel to the chro-

mosome to give information about executability of the actions represented by

1 http://www-fp.mcs.anl.gov/CCST/research/reports pre1998//comp bio/stalk/pgapack.html


4/10


the genes with relation with the initial state. In this way, let’s consider the firstnon-zero element in this binary auxiliary vector. Let’s say that it is in positioni. This means that all actions associated with the genes in positions 1 to i − 1are not applicable to the initial state, but that on the ith position is. Let’s now

say that the next non-zero element in the auxiliary vector is in position j. Thismeans that all genes representing actions in the chromosome starting in posi-tion i + 1 until position j − 1 are not applicable to the state resulting from theapplication of the previous valid action but that at the associated j position is.

But this is not enough, we need to measure how far a sequence of actions isfrom been a solution, if it is not one. Giving that we have two vectors, one con-taining a candidate to a solution and the other containing information about theexecutability of actions in the first vector, our fitness function is a combinationof informations extracted from both vectors.

Another point of consideration is that it may happen that a chromosome isnot a solution, but it is a good sequence of actions that lead us closely enoughof a solution. They may be considered good beginnings of plans. It may happen,on the other hand, that it a very good sequence of actions which obtains thefinal state, but not from the initial state. They may be considered good final of plans. We then let for the genetic operator of crossover to put the good partstogether forming a complete plan.

Now, our fitness function will be a combination of the following information:

1. the number of executable actions in the progressive direction, that is, thenumber of non-zero elements on the vector of executability when the valida-tion is analysed from the initial state;

2. the number of executable actions in the regressive direction, that is, the num-ber of non-zero elements on the vector of executability when the validationis analysed from the final state;

3. the number of propositions of the goal state reached when the associatedplan is applicable from the initial state;

4. the number of propositions of the initial state reached when the associated

plan is applicable from the final state;

Actually we have tried with several other informations, like, e.g.:

1. the number of consecutive executable actions;2. the number of valid actions;3. the position of the first executable action;4. the position of the last executable action;5. the number of actions of the largest feasible subsequence (fragment of the

plan) from the beginning of it;

6. the number of applicable actions of the largest partially feasible subsequence,from the beginning of it. This factor is different from the previous becauseit allows non-applicable actions being present in an feasible subsequence;

The idea was to obtain high quality index for a candidate solution basedon its number of feasible actions. The larger the number of feasible actions an


5/10


individual, and the more organised they are, the higher the quality it would have.It is in fact difficult to establish a good fitness function. If we try to maximiseexecutability the resulting chromosomes have lots of useless sequences of actions,as unloading the truck after having loaded it. If we try to give high values to

individual with leads to a great number of goals, then we get no solution.

3.3 The Genetic Operators

We define three operators: mutation, crossover and a new one called compression .The classical mutation operator was used to assure an even exploration of the

search space, aiming to avoid the algorithm gets stuck in local extremes. Thisoperator randomly chooses a gene and changes its value to a random integerranging from 0 to the size of the list of possible actions. For every gene of thechromosome, this operator is applied with a given small probability. This isshown in figure 1.

4 2 7 8 1 3 4 5

4 2 4 8 1 3 4 5

Fig. 1. Example of mutation

For the crossover operator, it is in general implemented in such a way thatsome point is randomly chosen in the chromosome and this point is appliedto both parents. But here, for the reason that we had defined variable-lengthsize chromosomes, we decided to implement a different version. In our work werandomly choose two points referring respectively to the recombination point forthe first and second individual. This is shown in figure 2.

4 2 7 8 1 3 4 5

5 9 2 6 3 1 3 4 5

2 6 3 14 2 7 8 1

5 9

Fig. 2. Example of our crossover

Due to the way we had chosen to construct the chromosome, allowing forthem to have non-executable sequences of actions (for this reason we have towork with a parallel binary vector for executability), it was necessary to definea new operator called compression . The reason is that in some moments, ourchromosomes are too big. This operator looks for null elements in the binary


6/10


vector of executability which, when found, are removed from both vectors, theone defining the chromosome and the one of binary executability of actions. Thisis shown in figure 3.

executability executability

4 2 7 8 1 3 59

1 1 1 1 0 0 11 11

59

1 1 1 1

4 2 7 8

Fig. 3. The compression operator

3.4 The Running Parameters

In this section we describe the way we initialise the initial population, the methodof replacement of the population and the rates of application of the geneticoperators.

There are two possible ways to generate the initial population. The first isto let it be chosen randomly. The other way is to have some control on theprocess. In all the experiments we have done with an randomly initialised initialpopulation we get very disappointing results for every possible combination of the genetic parameters. To control the generation of the initial population we

used the plan graph, in its relaxed and full versions. The idea is to choose actionsappearing in the initial layers as the first genes and actions of the last layersappearing in the last part of the chromosomes. As we will show in the nextsection this has a great impact in our approach.

The algorithm is as follows: construct the plan graph and for each layerrandomly choose a number of actions to be randomly chosen in that layer tocompose the chromosome. Do it for all layers. This will give us individuals withdifferent size.

The selection method has a great impact in the convergence of the GA. In

this work we decided to replace the entire population.The genetic operators were applied with the following probabilities: 1% for

mutation, 80% for crossover, which are within the usual range for GA’s, and 30%for compression. These values were obtained after a number of experiments.

For the fitness function we get our best results with values in the range 0.4to 0.45 for the two parameters controlling the number of goals (backwards andforwards) and values in the range 0.05 to 0.1 to the variables controlling theexecutability of actions). In the majority of cases we evaluate both forwards andbackwards.

It must be said however that the most important running parameter is theplan graph based generation of the initial population. The GA finishes when wehave found a solution or a previously specified number of maximal generationwas found.

In the next section we show some experiments running the GA with differentrunning parameters for the size of initial population and number of generations.


7/10


4 Experiments

We have done several experiments with AgPlan and we compare it with some of the best known planners, all of them based on the plan graph construction:

– IPP : implements the GRAPHPLAN algorithm [17];– BLACKBOX : replaces the second phase of GRAPHPLAN by a translation

to a SAT instance and then solve it using walksat[18];– FF : it implements a heuristic search planner and is considered the faster

of the actual planners. The heuristic function is based on the relaxed plangraph [10].

We choose three classical scenarios used in the planning competitions: logis-

tics, gripper [19] and blocks-world [20]. For lack of space we will show only partof our experiments, giving two instances of each problem, including some easyand some difficult ones.

AgPlan implements the GA described in section 3 in this paper. AgPlan -r implements the GA with randomly initialised initial population. AgPlan -mimplements the generation of the initial population using the complete plangraph, i.e., considering the mutex relation.

The implementation was realized in C and C++ languages for the GNU -Linux environment. The tests were run in a dual Xeon with 2.6GHz with 4GB

of RAM by December 2003 [21] as part of a graduate course on Planning in AI.

4.1 Results for Gripper

Figure 4 we show the results for the gripper problem 22 balls (left table) and 42balls (right table). They are considered very difficult problems. In both cases wehave used 30 as the maximum number of generations. We have used populationof size 700 for the problem with 22 balls and 3600 for other. We analyse the sizeof the resulting plan and the time to obtain it.

Only FF and AgPlan have managed to solve these two huge instances of

problems. It is true that FF is faster, but it may be considered a good resultfor AgPlan . Moreover, we can see the importance of using the plan graph toconstruct the initial population, AgPlan -r had no good results. The initialisationconsidering mutex seems to make the algorithm getting slow.

The plans are longer than those of FF , but a careful analysis shows that mostof the actions are of the type “load” followed by “unload”. This can be solvedas a post-processing step, as it is done for some planners, e.g., the system R [8].

4.2 Results for Blocks WorldFigure 5 shows our experiments with the classical Sussman anomaly (left table)and another using 8 blocks to be put in the inverse position (right table). Thepopulation had up to 9000 to solve the latter. They are presented (in order) below.

AgPlan seems to apply well in this domain. Even the version with randomlyinitialised initial population run faster then IPP and BLACKBOX . It has found


8/10


Planner plan size time (ms)IPP – –

BLACKBOX – –FF 65 0.00

AgPlan 81 0.72AgPlan -r – –

AgPlan -m 85 0.57

Planner plan size time (ms)IPP – –

BLACKBOX – –FF 125 2.00

AgPlan 163 5.69AgPlan -r – –

AgPlan -m 187 12.07

Fig. 4. Experiments with gripper

Planner plan size time (ms)IPP 3 0.00

BLACKBOX 3 0.07FF 3 0.00

AgPlan 3 0.00AgPlan -r 3 0.00

AgPlan -m 3 0.00

Planner plan size time (ms)IPP 11 20.00

BLACKBOX 17 88.81FF 11 0.00

AgPlan 10 0.31AgPlan -r 14 8.22

AgPlan -m 10 0.38

Fig. 5. Experiments with blocks-world

a shorter plan than FF . Here the initial population seems to have no otherimportance except that for the random case which are slower and have biggerplans. The results are satisfactory.

4.3 Results for LogisticsThe only planners that managed to solve logistics problems were FF and IPP .This latter solves only two of them. The reason seems to be the number of layersin the relaxed plan graph, which is very small due to the high rates of parallelismof its actions.

5 Conclusion

In this paper, a methodology for dealing with planning problems using Ge-netic Algorithms was presented. The proposed GA has special features speciallydevised for such kind of problem, including a variable-length chromosome forvariable-size plans, and a special genetic operator, namely, compression. Wehave also implemented a special version of crossover with two points.

Our planner is general purpose and works only with typed versions of thescenarios. Previous experiments shows that the non-typed versions give us verypoor results. One possible way to overcome that is to implement automatic typeverification. This problem was investigated by [22] and [16].

This methodology is still under development, but the results are encourag-ing. Although many experiments are yet to be done, we believe that it will beparticularly interesting for problems that cannot be addressed with current tra-ditional methods. In particular, based on hundreds of experiments, we think thatthe generation of the initial population based on the plan graph construction isthe key for the success of our GA.


9/10


Future work includes some experiments with different genetic parameters inorder to improve the algorithm for harder problems, specially those involvingdomains with huge number of parallel actions, as logistics. This may include theuse of local search or island parallel genetic algorithms.

Acknowledgements. This work has its origin in a graduate course on geneticalgorithms on the CPGEI-CEFET/PR and on a graduate course on planning inthe Department of Informatics/UFPR. We are very thankful to all the studentsthat contributed to our results, in particular Daniel Benke, Vinicius Cubas andTiago Lima. We are specially thankful to prof. Heitor Lopes, with whom wehave several and useful discussions in early work on this subject. We also thanksCAPES for the PhD grant for Fabiano and SESU for the PET grant for Viviane.

References

1. Fikes, R., Nilsson, N.: STRIPS: A new approach to the application of theoremproving to problem solving. Journal of Artificial Intelligence 2 (1971)

2. Bylander, T.: The computational complexity of propositional STRIPS planning.Artificial Intelligence 69 (1994) 165–205

3. Kautz, H., Selman, B.: Planning as satisfiability. In: Proc. 10th Eur. Conf. AI,Vienna, Austria, Wiley (1992) 359–363

4. Blum, A., Furst, M.: Fast planning through planning graph analysis. In: Proceed-

ings of IJCAI-95, Montreal (1995) 1636–16425. Vossen, T., Ball, M., Lotem, A., Nau, D.: On the use of integer programming

models in ai planning. In: Proceedings of the IJCAI 99. (1999) 304–3096. Beek, P., Chen, X.: CPlan: A constraint programming approach to planning. In:

Proc. of the Sixteenth National Conference on AI (AAAI-99). (1999) 585–5907. Silva, F., Castilho, M., Künzle, L.: Petriplan: a new algorithm for plan generation

(preliminary report). In: Proc. of IBERAMIA/SBIA-2000, Springer-Verlag (2000)86–95

8. Lin, F.: A planner called r. In: AIPS-2000 Planning Competition. (2000)9. Bonet, B., Geffner, H.: HSP: Planning as heuristic search. In: Entry at the AIPS-98

Planning Competition, Pittsburgh (1998)10. Hoffmann, J., Nebel, B.: The FF planning system: Fast plan generation through

heuristic search. In: AIPS Planning Competition. (2000)11. Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning.

Addison-Wesley (1989)12. Westerberg, C., Levine, J.: GenPlan: Combining genetic programming and plan-

ning. In: Proceedings for the UK Planning and Scheduling Special Interest Group,Milton Keynes UK (2000)

13. Muslea, I.: SINERGY: A linear planner based on genetic programming. In: Proc.European Conference on Planning (ECP-97), Toulouse, France (1997) 314–326

14. Geffner, H.: Perspectives on artificial intelligence planning. In: Proc. 18th Nat.(US) Conference on AI. (2002) 1013–1023

15. McDermott, D.: PDDL - The Planning Domain Definition Language. (1998)http://ftp.cs.yale.edu/pub/mcdermott.

16. Lecheta, E.M.: Algoritmos genticos para planejamento em inteligncia artificial.Master’s thesis, Dinf-UFPR (2004)


10/10


17. Koehler, J.: IPP home page (1997)18. Kautz, H., Selman, B.: Unifying sat-based and graph-based planning. In: Proceed-

ings of IJCAI, Stockholm, Sweden, Morgan Kaufmann (1999) 318–32519. McDermott, D.: AIPS-98 Planning Competition Results. (1998) http://

ftp.cs.yale.edu/pub/mcdermott/aipscomp-results.html.20. : Aips-2000 planning competition (2000)21. Benke, D., Brand, V., Lima, S.: Uma avaliao de desempenho de planejadores.

Technical report, Departamento de Informtica - UFPR, Curitiba, Brasil (2003)22. Long, D., Fox, M.: Automatic synthesis and use of generic types in planning. In:

Artificial Intelligence Planning Systems. (2000) 196–205

An Investigation on Genetic Algorithms for Generic STRIPS Planning

Documents

Transcript of An Investigation on Genetic Algorithms for Generic STRIPS Planning