Copyright (C) by Rafal Salustowicz, 1997
PROBABILISTIC INCREMENTAL PROGRAM EVOLUTION
by
Rafal Salustowicz
December 1997
CONTENTS
0. About this Guide
1. General Comments about PIPE_v1.0
2. Getting Started
3. Compiling and Running Example Applications
4. Setting Up a New Application
4.1 First Steps
4.2 Creating a Trainings Data File
4.3 Implementing Instructions, a Fitness Function,
and a Random Constant Generator
4.4 Designing a Parameter File
4.5 Running the New Application
5. Testing
5.1 Testing an Example Application
5.2 Testing in Details
6. Removing Object Files
7. The Kernels
8. Notes
8.1 Precision of Random Constants
8.2 Preventing Overflows
8.3 Checkpoints
8.4 Running PIPE in the Background
0. ABOUT THIS GUIDE
This is a quick guide that describes how to install and use the Probabilistic Incremental Program Evolution (PIPE) software package PIPE_v1.0. For a detailed description of the algorithm one should consult the following article:
R.P. Salustowicz and J. Schmidhuber
Probabilistic Incremental Program Evolution.
Evolutionary Computation, 5(2):123-141, 1997.
http://www.idsia.ch/~rafal/research.html
Please, read the license (LICENSE.txt), which is located in the same directory as this file carefully before using this software.
- GENERAL COMMENTS ABOUT PIPE_v1.0
PIPE_v1.0 has been written to allow for applying and enhancing the PIPE paradigm. The software is easy to install and easy to use. To setup PIPE_V1.0 for different problems requires only an minimal amount of programming. User-written application-independent program parts can easily be reused for different applications. For efficiency reasons the PIPE_v1.0 software package is written in C. It has been carefully tested and optimized for SunOS 5.5.1. For use under different operating systems some minor changes might be required. Try first the automated installation and compilation. If PIPE_v1.0 does not install correctly, you need to do the installation manually. Section 2 describes what is done during an automated installation. In case the compilation fails you most likely need to include different header files from /usr/include/.
Apart from the main PIPE program, the package also contains a tool (testpipe) to test the performance of evolved programs by using different data sets. The program testpipe reads in saved programs from files, tests their performance on a specified data set, and writes a performance summary to a report file. Since PIPE_v1.0 supports the program save format of the genetic programming package lil-gp1.0 (Copyright (C), 1995 Michigan State University), people who use this package can also use testpipe to read in and test programs evolved by it. They just need to remove the header created by lil-gp1.0, insert a dummy status line (just a single line with anything on it), and add an extension (a counter) to their *.bst files as done by pipe.
2. GETTING STARTED
After having correctly "gunzipped" and "untared" PIPE_v1.0 you should now have the following directory structure:
PIPE_v1.0/
app/ (application folder)
multiplexer/ (Boolean 11-multiplexer problem)
parity/ (6-bit parity problem)
regression/ (function/symbolic regression problem)
builder/ (shell scripts executed by makefiles in
the application folder to compile and
remove kernel object files)
doc/ (documentation and license)
f_kernel/ (kernel for applications that use floating point
arithmetic)
i_kernel/ (kernel for applications that use integer arithmetic)
Execute INSTALL. INSTALL creates the i_kernel by copying all relevant files
from f_kernel. In addition it also makes all shell scripts in the builder/
directory executable.
Note that f_kernel/ and i_kernel/ are identical except for the file
datatype.h which defines the data type (double or integer, respectively)
for the kernel compilation (creation of kernel object files).
3. COMPILING AND RUNNING EXAMPLE APPLICATIONS
There are three example applications in the application directory app/ to choose from:
regression/
The regression application tries to find a function that approximates:
f(x) = x^3 * e^-x * cos(x) * sin(x) * ( sin^2(x) * cos(x) - 1 )
in the interval [0..10] given 101 equidistant points from this
interval. The fitness function is the sum of absolute distances.
See Section 4.1 of the article mentioned in Section 0. for a complete
description of the task and instructions used.
Important notice: Use the parameters in the file f1.par.
Do not use the parameters mentioned in the article. The parameters in
regression.par deliver statistically significantly better results.
They also turned out to be more robust throughout different applications.
parity/
The parity application tries find a function that solves the 6-bit
parity problem. The training data set contains all 64 data points.
Fitness is measured by the number of points which are misclassified.
See Section 4.2 of the article mentioned in Section 0. for a complete
description of the task and instructions used.
Important notice: Use the parameters in the file p6.par.
Do not use the parameters mentioned in the article. The parameters in
regression.par deliver statistically significantly better results.
They also turned out to be more robust throughout different applications.
multiplexer/
The multiplexer application tries evolve a program that implements
the Boolean 11-multiplexer function. Please, refer to the following
reference for a detailed description of the task and instructions used:
Koza, J. R., Genetic Programming - On the Programming of Computers
by Means of Natural Selection, MIT Press, 1992, pp: 170-173.
The training data set contains all 2048 data points. Fitness is measured by the number of points which are misclassified.
Each application (regression/, parity/, and multiplexer/) consists of the following files and directory:
makefile
fitness.c
instruction.c
random.c
*.par
*.tpar
*.dat_in
output/
(* denotes a wild card; there is only one file with the given extension in each application directory, but the filenames differ.)
To compile an application switch to the selected directory and execute:
make
Make will use the makefile to compile the appropriate kernel (if necessary), to compile the three application defining files (fitness.c, instruction.c, and random.c) and will create two executables: pipe and testpipe (how to use testpipe will be explained in Section 5.)
To run PIPE you can now type:
pipe *.par
Program pipe prints a continuously updated status line to the window and creates the files *.prg_bst.[0-4] in the output/ directory. Each file contains the "best program found so far" (elitist) of each evolutionary run (the *.par files are set to successively run five independent experiments) and a status line reporting among other things the generation at which this solution was found, the fitness of the solution, the length (number of nodes) and maximal depth of the program. The entire status line as well as the numerous other output options that, e.g., enable pipe to run multiple experiments, allow for repeating single runs, show the evolution of programs etc. will be explained in Section 4. Additionally pipe will also create *.seeds files containing the seeds of each independent run.
4. SETTING UP A NEW APPLICATION
Here is a step by step description on how to setup a new application.
4.1 FIRST STEPS
Create an application directory in the app/ directory (e.g. your_directory/) and then create a directory output/ in app/your_directory/.
4.2 CREATING A TRAININGS DATA FILE
To set up a new application you then need to create a file with trainings data in app/your_directory/. Let's name this file your_task.dat_in. You want PIPE to find a correct relationship between a n-dimensional input and a single target value. File your_task.dat_in needs to have the following format:
#data_points
0: I1 I2 ... In T1
1: I1 I2 ... In T2
2: I1 I2 ... In T3
.
.
.
data_points-1: I1 I2 ... In Tdata_points
In the first line of your_task.dat_in contains an integer displaying the total number of data points (and succeeding lines) in this file. Each following line contains a single data point. The first row contains a counter and is separated from the remaining columns by a colon. After the colon follow n values - components of the input vector - and finally a single target value. While the input vector can contain an arbitrary number of components, PIPE in this version can only deal with a single target value. For a better understanding you can also compare the *.dat_in files in the example applications directories (regression/, parity/, and multiplexer/).
4.3 IMPLEMENTING INSTRUCTIONS, A FITNESS FUNCTION,
AND A RANDOM CONSTANT GENERATOR
The easiest way to create a new application is to copy files from an existing example application and edit them. Copy from any of the example application directories the following files to your_directory/:
makefile
fitness.c
instruction.c
random.c
Attention: Make sure you do not copy everything and do not overwrite
the you_task.dat_in file you have created before. Now you will need to edit those files to create a new application.
- MAKEFILE: Decide on whether your application will use integer or floating point arithmetic. If you decide on integer arithmetic, make sure that all instructions (functions and terminals) that you will define will only return integers. If you need to use an instruction that returns a floating point number, you must use floating point arithmetic. Now edit the first line in the makefile: For floating point arithmetic set:
KERNEL_DIR = ../../f_kernel
For integer arithmetic set:
KERNEL_DIR = ../../i_kernel
Both kernels are exactly the same apart from their instruction evaluation and I/O routines that either support doubles or integers, respectively.
Note for the succeeding steps: By defining the KERNEL_DIR you set the constant DATATYPE used in fitness.c, instruction.c, and random.c to either double (using f_kernel) or int (using i_kernel).
B. FITNESS.C:
Now you need to define the fitness function. The file fitness.c contains
only the function fitness_function(). This function returns a floating
point value (double). The return value must be non-negative
and smaller values need to indicate better performance.
Think of the fitness value as being a non-negative error value.
To calculate the return value fitness_function() gets an array of
target[] values, an array of output[] values and an integer
containing the number of data points (the data_points you
defined in the your_task.dat_in file). Now for each data point d
0<=d<data_points from the your_task.dat_in file: target[d] is
the desired target value (value Td specified in the your_task.dat_in
file) and output[d] is the corresponding value that has been
calculated by an evolved program.
To use the mean squared error (mse) as the fitness function you
can first sum (target[d]-output[d])^2 over all data points and then
divide the outcome by the number of data points (#data_points)
for a summed absolute difference (sad) you just calculate the
sum |target[d]-output[d]| over all data points as done in
app/regression/fitness.c.
This is also the right place to write a "wrapper". The wrapper transforms the output of program to accommodate the fitness function. If you look, e.g., at app/parity/fitness.c, the wrapper there classifies all program outputs in two categories (cat.I: negative and catII: non-negative) as required by the Boolean nature of the problem.
C. INSTRUCTION.C:
Here you need to first define the number of inputs and instructions. Then you need to implement a function for each instruction and finally define the number of parameters of each instruction and give each instruction a name (to be used when saving programs).
First set NUMBER_OF_INPUTS to the number of inputs defined in the your_task.dat_in file. The NUMBER_OF_INSTRUCTIONS is then the NUMBER_OF_INPUTS + the number of additional instructions you would like to use.
There are three different types of instructions: functions, terminals and a function for the generic random constant.
- FUNCTIONS
- The "frame" of a function implementing a PIPE function instruction always looks as follows:
DATATYPE function_name (DATATYPE *arg) {
.
.
.
return (....);
}
The DATATYPE is either double or int depending on which kernel you have chosen (see MAKEFILE). The *arg pointer points to an array of arguments to the function. Starting with arg[0], arg[1], ... you can access as many arguments as you like. You will have to define the maximal number of arguments to the function below. The return value must be a valid DATATYPE (see, e.g., protected division in app/regression/instruction.c)
- TERMINALS
- The "frame" of a function implementing a PIPE terminal instruction always looks as follows:
DATATYPE function_name (DATATYPE *x) {
return ( x[i] );
}
The *x pointer points to an array of inputs. You just need to read out the input and return it. Therefore 0 <= i < n, where n is the maximal number of inputs as defined in your_task.dat_in. You have to implement a distinct function for each input returning one of: x[0], x[1],..., x[n-1] (see, e.g., app/parity/instruction.c or app/multiplexer/instruction.c).
GENERIC RANDOM CONSTANTS:
If you like to use generic random constants you need to define the
following dummy function:
DATATYPE i_grc () {
return ((DATATYPE) 0);
} /* i_grc */
The dummy function is only used during initialization of the instruction set. To define which random constants you want to use you will have to edit the random.c file (explained below).
After having implemented all instructions you need to specify the number of arguments each instruction has and give each instruction a label that will be used during program saves (see RESULT FILES below) in the build_instructions() procedure. The build_instructions() procedure must be last in the instruction.c file. It has the following structure:
void build_instructions() {
Instruction instruction[NUMBER_OF_INSTRUCTIONS] =
{ { function_name, number_of_arguments, "label" },
{ function_name, number_of_arguments, "label" },
.
.
.
{ function_name, number_of_arguments, "label" }
};
allocate_initialize_instructions(instruction,
NUMBER_OF_INSTRUCTIONS,
NUMBER_OF_INPUTS);
} /* build_instructions */
To use an instruction you have defined (see above) the instruction's function_name must occur in the list above. The number_of_arguments defines then the number of arguments to this instruction. The label can be any string - e.g., you might want to use "+" for a function performing an addition. Terminals (see TERMINALS above) must have number_of_arguments set to 0. Generic random constants (see GENERIC RANDOM CONSTANTS above) must have number_of_arguments set to GRC_ARGS or -1. The label for generic random constants must be set, but it is a dummy label, since instead of the label the actual constant is saved during program saves. See also the different instruction.c files of the example applications.
D. RANDOM.C:
Here you can define how and which random constants are generated. The file contains only a single function:
DATATYPE create_random_constant() {
return(...);
} /* create_random_constant */
The return value must match the DATATYPE (see MAKEFILE for the
definition of DATATYPE). Apart from this you can freely choose
any arbitrary random constant/number generator. When
using drand48() <stdlib.h> needs to be included. Other generators
might require different header files (read the manual pages of the
generator).
IMPORTANT: If you do not want to use generic random constants you
still need to return a dummy value that matches the
selected DATATYPE (see, e.g., app/multiplexer/random.c).
E. Now you can compile your newly created application by typing:
make
Make will first compile the chosen kernel, if necessary, then it will compile the fitness.c, instruction.c, and random.c and finally it will create the programs: pipe and testpipe.
You have now successfully created a new application. To run your application you now only need to adjust a parameter file (see Section 4.4)
4.4 DESIGNING A PARAMETER FILE
The easiest way to create a new parameter file is to copy one from an existing example application and edit it. Copy from any of the example application directories the following file to your_directory/:
*.par
Rename the parameter file to your_task.par.
This is the general syntax of a parameter file: When read in, lines from the parameter file are tokenized into words (A word is a sequence of arbitrary characters, apart from SPACE, TAB, and NEWLINE that serve as dividers between words.) Comment lines need to start with a '#'. Each parameter is identified by its unique name. To assign a value to a parameter you need to write:
Parameter_Name: value
Parameters do not have a special order. A parameter will be recognized everywhere, as long as it appears by itself on a single line in the format mentioned above. If you define a parameter multiple times (you should not), the parameter takes the value set in the last definition.
The parameter file contains three distinct parameter blocks.
The first block contains only the Number_of_Experiments. Number_of_Experiments defines the number of independent experiments to be run. At the beginning of each experiment the PIPE engine is reset and initialized anew. All results of an experiment are saved to distinct files by adding the extension *.experiment_number to each result file. If you do not want to perform a statistical analysis of how PIPE performs on a specific task with a specific parameter setting, you can perform a single PIPE run (experiment) by setting Number_of_Experiments to 1.
The second block contains parameters for the PIPE algorithm. The meaning
and influence of all parameters is described in detail in the article
mentioned in Section 0 (at the end of Section 3 of the article there
is a parameter summary). Here is just some advise regarding the most
crucial parameters:
a.) Initial_Terminal_Probability:
One of the most influential parameters is the Initial_Terminal_Probability. It influences the the size of the initial programs and their growth during evolution. Make sure you set the Initial_Terminal_Probability to a sufficiently high value (<1.0) before running pipe. There are two good reasons to do this. First smaller programs are evaluated significantly faster and the search space is initially smaller (there are fewer short programs than long programs). Second, if you set the Initial_Terminal_Probability too low pipe will attempt to create infinitely long programs which will eventually lead to a memory allocation fault on your machine. Of course, the minimal value strongly depends on the arity of your program trees (the maximal number of arguments to a function). With arity 2 you can set the value to as low as approximately 0.3, but for larger arities this value will be larger. b.) Population_Size and Learning_Rate:
PIPE usually works better with small populations and small learning
rates.
c.) Epsilon:
Epsilon defines the degree of fitness dependent learning. It needs to be set to a positive value!
The third and final parameter block sets all I/O parameters. There are many I/O parameters that can be used to save and display different kinds of information generated during an experiment. You will not always need all the information, but sometimes, when you want to analyze certain experiments more deeply, it is useful to turn on different save options. Note that history files might grow quickly! Below is the complete list of save options.
Each output file you do not want to create, you have to set to /dev/null. Each input file you do not want to use, you have to set to /dev/null.
List of save options:
Print_Interval:
Print a status line to stdout every Print_Interval generations.
Total number of generations = Individual_Evaluations / Population_Size.
If you do not want to print a status line at all, set
Save_Interval > total number of generations.
Save_Interval:
Save programs and/or data to Program_History_File and/or
Data_History_File (see below) every Save_Interval generations.
If you do not want to use the Program_History_File and the
Data_History_File, set Save_Interval > total number of generations.
Input_Data_File: your_task.dat_in
required file containing trainings data for pipe.
Input_Seed_File: your_task.seeds
In case you want to start each experiment with a prefixed
seed, you need to put the filename of a seed file here.
A seed file contains seed values (unsigned ints), each
value being on a single line. The Input_Seed_File has
the same format as the Output_Seed_File (see below).
You can generate an Output_Seed_File to have an example.
Note that the Input_Seed_File needs to have at least
Number_of_Experiments (see above) distinct seeds on
distinct lines.
If you do not want to read in prefixed seeds, but
generate random seeds for each distinct experiment
you have to put /dev/null here. The "random" seeds
are then generated using time(). They can be saved
to a file by setting the Output_Seed_File (see below).
- Start Index
- Each output file (except the Output_Seed_File and the Log_File) will have an extension attached that is the number of the current experiment plus the Start_Index. Normally you can always set the Start_Index to 0. In case, however, you have already done x experiments and you would like to perform more experiments to obtain a statistically more significant result, you want to set the Start_Index to x+1.
- Program Save Type
- Three different program save types exist: true_tree, tree, function. They define the format in which programs are saved to files. Try them out and pick the one you like most. All three formats are supported by testpipe! Users of the lilgp package might notice that the "tree" type saves programs in the same format as the lilgp package (v1.0).
Program_History_File: output/your_task.prg_his Starting with generation 0, every Save_Interval generations the best program of the current generation is saved (appended) to this file along with the status line.
Data_History_File: output/your_task.dat_his Starting with generation 0, every Save_Interval generations the inputs of the trainings data set and corresponding outputs of the best program of the current generation are saved (appended) to this file along with the status line.
Elitist_Program_File: output/your_task.prg_bst The best program of one entire experiment (elitist) is preserved in this file. The file is updated during the run, whenever a new elitist has been found.
Elitist_Data_File: output/your_task.dat_bst The inputs of the trainings data set and corresponding outputs of the best program of one entire experiment (elitist) are preserved in this file. The file is updated during the run, whenever a new elitist has been found.
Elitist_Program_History_File: output/your_task.e_prg_his Whenever a new elitist program has been found, it is saved (appended) to this file along with the status line.
Elitist_Data_History_File: output/your_task.e_dat_his Whenever a new elitist program has been found it the inputs of the trainings data set and corresponding outputs of the new elitist are saved (appended) to this file along with the status line.
Log_File: output/your_task.log
Contains the information printed to the screen.
Output_Seed_File: output/your_task.seeds For each independent experiment (run) the seed value is saved (appended) to this file.
4.5 RUNNING THE NEW APPLICATION
To run the newly created application type:
pipe your_task.par
5. TESTING
Section 5.1 describes how test performance of programs evolved and saved by the example applications. Section 5.2 then gives a detailed description of how program testpipe can be used to test performance of evolved programs.
5.1 TESTING AN EXAMPLE APPLICATION
After running pipe, program testpipe can be used to read in the saved (elitist) program(s) and test it (them) on a different data set (useful for testing generalization). If you have run pipe you can now type:
testpipe *.tpar
The program saves a report file *.rep in the output/ directory, which reports on the performance of the read in program(s) on the specified data set. Note that in the examples test data set and trainings data set are identical leading to equal fitness evaluations.
5.2 TESTING IN DETAILS
To test the evolved programs on a data set, you need to have saved the Elitist_Program_File(s). Program testpipe reads in the programs and runs them on a specified data set (e.g., the test data set). The parameter file for testpipe is called your_task.tpar and contains the following parameters to be set (again it is best to copy and modify an existing *.tpar file from the example applications):
Input_Program_File: output/your_task.prg_bst Specifies the set of programs for testpipe to be run on. Notice that you specify here only the filename stem, without the additional extension (without the experiment number that is appended to each of the your_task.prg_bst files).
Start_Index:
Specifies the first additional file extension (number of the first
experiment).
- Number Of Experiments
- Number_Of_Experiments saved programs (files) are processed. Starting with your_task.prg_bst.Start_Index up to your_task.prg_bst.Start_Index+Number_Of_Experiments-1.
Input_Data_File: your_task_testdata.dat_in This file contains the test data set. The file format is the same as for the trainings data set.
Output_Data_File: output/your_task.dat_out To this/these file(s) the input part of the data in your_task_testdata.dat_in is saved along with the corresponding outputs of the tested program(s). A file with experiment number as extension is created for each tested program.
- Report File
- The report of the performances of all tested programs is saved to this file. This file is a required output.
After creating the parameter file run:
testpipe your_task.tpar
6. REMOVING OBJECT FILES
You can remove all object files directly from each application directory using the makefile (look at the last line in the makefile).
To remove object files of the application you defined type:
make clean
(removes instruction.o, fitness.o, and random.o)
To remove the object files of the pipe kernel your application uses type:
make clean_pipekernel
Note: A kernel can be used by several applications. If you remove
all object files of a kernel, they will be recompiled when
compiling another application that uses this kernel.
To remove the object files of the testpipe kernel your application uses type:
make clean_testpipekernel
This will only remove the additional object files needed by the testpipe kernel. It will leave the pipe kernel object files untouched. Note: A testpipe kernel can be used by several applications. If you remove
all object files of a testpipe kernel, they will be recompiled when
compiling another application that uses this testpipe kernel.
To remove the object files of the pipe and testpipe kernels your application uses type:
make clean_kernels
Note: see clean_pipekernel and clean_testpipekernel.
To remove all object files that were created during compilation of you application type:
make clean_all
7. THE KERNELS
This section is only interesting to someone who wants to actually change the kernels and implement things like multiple outputs, enhanced learning algorithms, different program structures etc. So if you really want to do this, continue reading and I will try to aid you in this endeavor by giving an overview in what is implemented where.
The files one needs to compile the pipe kernel are the following:
makefile
datatype.h
pipe.h
protos.h
types.h
utils.h
alloc.c
create.c
elitist.c
evaluate.c
get.c
io.c
learn.c
mutate.c
pipe.c
prune.c
utils.c
To compile the testpipe kernel one additionally needs:
testpipe.h
testprotos.h
testio.c
testpipe.c
testutils.c
Note that when you change kernel source files and compile directly from an application directory, you have to first execute: make clean_all. The makefile in an application directory does not "notice" changes done to kernel source files.
Here is a short description of what is in each of the kernel files:
makefile:
The makefile defines how the files are compiled and linked.
See make under Unix.
datatype.h:
This is the only file which differs in the i_kernel and f_kernel.
It defines on which data types your PIPE program instructions
work -- i_kernel: integer and f_kernel: double.
pipe.h:
Contains variables for all major PIPE structures, such as programs, the
PPT (probability distribution), etc. Contains also all variable definitions
for variables set in the your_task.par.
protos.h:
Defines which functions/procedures are exported from which file to which
other file. Format of comments:
/* file where function is defined -> file(s) where function is used */
types.h:
Contains all major PIPE data structures, such as the data structures
for programs, the PPT, and the data set.
utils.h:
Header file used solely in connection with utils.c. Defines the word list
data structure and does the prototyping for utils.c. See utils.c for
details.
alloc.c:
Contains memory allocation procedures for all kernel modules (files).
Except for utils.c which uses its own memory allocation procedures.
Note that program testpipe also has additional memory allocation
procedures.
create.c:
Contains two procedures: One to create a program from the PPT. The second
to create an entire population of programs.
elitist.c:
Contains two procedures: One to copy a program. The other to decide
when to preserve a program as the elitist.
evaluate.c:
Contains two procedures: One to evaluate a program. The other to
evaluate an entire population of programs.
get.c:
Contains functions to determine the number of nodes in a PPT and a program,
as well as the depth of both. It also contains a procedure that finds
the best program of a generation (after all programs have been evaluated).
io.c:
Contains all major I/O routines, such as routines for reading in the
parameter file, saving data, evolved programs, etc.
learn.c:
Contains all major procedures for updating the PPT according to the
learning rule.
mutate.c:
Contains a single procedure that implements the mutation operator
applied to the PPT.
pipe.c:
Contains main(). (The main loop.)
prune.c
Contains a single procedure that implements pruning of the PPT.
utils.c
Contains a utility procedure that takes a string (line) and tokenizes
it into words.
Here is a short description of what is in all the additional kernel files needed to compile the testpipe kernel:
testpipe.h:
Contains variables for all major structures used by testpipe, such as the
program. Contains also all variable definitions for variables set in the
your_task.tpar. Defines also the maximal line length in bytes a program
can occupy as a saved string.
testprotos.h:
Defines which functions/procedures are exported from which file to which
which other file. Format of comments:
/* file where function is defined -> file(s) where function is used */
testio.c:
Contains all major I/O routines, such as routines for reading in the
parameter file, programs, saving data, and the report.
testpipe.c:
Contains main(). (The outermost function -- the program.)
testutils.c:
Contains procedures to convert a program saved in a string to a
program stored in a data structure as used by the pipe and testpipe
programs.
8. NOTES
Here are some useful hints.
8.1 PRECISION OF RANDOM CONSTANTS
Floating point constants in saved programs have a high precision. You can
influence this precision by editing DATATYPE_FORMAT and DATATYPE_DOUBLE
in the PIPE_v1.0/f_kernel/datatype.h file.
If pipe and testpipe deliver different results when run on the same input
data set, then you should increase the precision of the floating point
constants.
If you do not like those long constants you can also decrease their
precision. But be aware, pipe will save constants with the specified
precision, but it will calculate with full precision double floating point
constants. Therefore evolved programs may perform better or worse on the
same input data set depending on whether they have been evaluated
during a pipe run or a testpipe run (after having been read in with the
limited precision). This, of course, is only true for applications that use
floating point arithmetic. Integer arithmetic is not affected by this.
8.2 PREVENTING OVERFLOWS
Note that to ensure proper functioning all functions (see FUNCTIONS) should be protected from overflow. The i_multiply() function in PIPE_v1.0/app/regression/instruction.c, e.g., is not. Theoretically this could lead to an overflow when two very large numbers are multiplied. This problem was never encountered during excessive testing, but according to Murphy's law it could turn up some day.
8.3 CHECKPOINTS
No checkpointing within a single experiment! The probability distribution is never saved to a file. Thus if you have single very long experiments, make sure you run them on a computer that is not rebooted before they finish.
8.4 RUNNING PIPE IN THE BACKGROUND
If you want to run pipe as a background process, set Print_Interval > (Individual_Evaluations / Population_Size) and try:
(pipe *.par >& /dev/null)&