Executors

---

During the process of evolution, various operations are pushed into an executor to be run. Things like evaluating fitness, dispatching events, etc. Executors are responsible for managing how these operations are run, whether that be in a single thread or multiple threads and how those threads are managed.

Currently, radiate supports three executors:

• Serial: Runs all operations in the main thread, one at a time.
  • This is the default executor if none is specified.
• WorkerPool: Uses a rayon thread pool to run operations concurrently.
  • note that in rust this requires the rayon feature to be enabled in Cargo.toml. Python includes this by default.
• FixedSizedWorkerPool(num_threads): Uses an internal thread pool with a fixed number of threads to run operations concurrently.

Example

Continuing with our example from the previous sections - evolving a simple function: finding the best values for y = ax + b where we want to find optimal values for a and b. We'll keep the previous inputs the same as before, but now we add an executor to the GeneticEngine.

Python Rust

Python concurrency

The WorkerPool and FixedSizedWorkerPool executors use multiple threads to run the fitness function concurrently. If you are not using a free-threaded interpreter (ie: python3.13t/3.14t) or the GIL is enabled, the engine will raise an exception. There are a few caveats to this with genetic programming problems - see the GP regression section for more details.

If you are in fact using a free-threaded interpreter, your engine can take advantage of multiple threads to evaluate fitness concurrently. This can significantly speed up evolution, especially if your fitness function is computationally expensive. However, your fitness function must be thread-safe.

import radiate as rd


# Define a fitness function that uses the decoded values
def fit(individual: list[float]) -> float:
    a = individual[0]
    b = individual[1]
    return calculate_error(a, b)  # Your error calculation here


# Here we can see how to use the parallel executors in python.
# Radiate will throw an error if you try to use a parallel executor
# in a non-free-threaded interpreter or if the GIL is enabled.
# a good way to check is by using radiate's utlity function, rd._GIL_ENABLED,
# which will be True if the GIL is enabled and False if it is not.
print(f"GIL enabled: {rd._GIL_ENABLED}")

# Building the engine dynamically can do something like this:
# .parallel(num_workers: int | None = None) method.
# If num_workers is None, it will use rayon's global thread pool, otherwise
# it will use a fixed sized worker pool with the specified number of workers.
engine = (
    rd.Engine.float(2, init_range=(-1.0, 1.0), bounds=(-10.0, 10.0), dtype=rd.Float32)
    .fitness(fit)
    .select(
        offspring=rd.Select.boltzmann(temp=4),
        survivor=rd.Select.tournament(k=3),
        frac=0.5,
    )
    .alters(
        rd.Mutate.gaussian(rate=0.1),
        rd.Cross.blend(rate=0.8, alpha=0.5),
    )
    .limit(rd.Limit.score(0.01), rd.Limit.generations(1000))
    # ... other parameters ...
)

if rd._GIL_ENABLED:
    print("GIL is enabled - have to use Serial Executor.")
else:
    print("GIL is not enabled, using parallel executor.")
    engine = engine.parallel(
        num_workers=4
    )  # Use a fixed sized worker pool with 4 workers
    # --- or ----
    # engine = engine.parallel()  # Use a worker pool with rayon's global thread pool

# Run the engine
result = engine.run()

To use the WorkerPool executor in rust (which uses rayon), ensure you have the rayon feature enabled in your Cargo.toml:

[dependencies]
radiate = { version = "x", features = ["rayon"] }

// Define a fitness function that uses the decoded values
fn fit(individual: Vec<f32>) -> f32 {
    let a = individual[0];
    let b = individual[1];
    calculate_error(a, b) // Your error calculation here
}

// This will produce a Genotype<FloatChromosome> with 1 FloatChromosome which
// holds 2 FloatGenes (a and b), each with a value between -1.0 and 1.0 and a bound between -10.0 and 10.0
let codec = FloatCodec::vector(2, -1.0..1.0).with_bounds(-10.0..10.0);

// Define the executor - here we use a fixed size worker pool with 4 threads
let executor = Executor::FixedSizedWorkerPool(4);
// Alternatively, you can use a WorkerPool (which uses rayon's global thread pool).
// Requires the `rayon` feature to be enabled in your Cargo.toml.
let executor = Executor::WorkerPool;
// Or for single-threaded execution, use Serial - this is the default if none is specified
let executor = Executor::Serial;

let engine = GeneticEngine::builder()
    .codec(codec)
    .offspring_selector(BoltzmannSelector::new(4.0))
    .survivor_selector(TournamentSelector::new(3))
    .fitness_fn(fit)
    .alter(alters!(
        GaussianMutator::new(0.1),
        BlendCrossover::new(0.8, 0.5),
    ))
    .executor(executor) // Set the executor here
    // ... other parameters ...
    .build();

// Run the engine: stop after 1000 generations
let result = engine.iter().take(1000).run();

You can also use the convenient .parallel() method on the engine builder. If rayon is enabled, this will use rayon's global thread pool, otherwise it will use radiate's internal thread pool with # cpu threads. The performance difference between the two is negligible for our use cases.

let engine = GeneticEngine::builder()
    .codec(FloatCodec::vector(2, -1.0..1.0).with_bounds(-10.0..10.0))
    .fitness_fn(fit)
    // ... other builder methods ...
    .parallel()
    .build();