Estimator

This file is part of the TPOT library.

The current version of TPOT was developed at Cedars-Sinai by: - Pedro Henrique Ribeiro (https://github.com/perib, https://www.linkedin.com/in/pedro-ribeiro/) - Anil Saini (anil.saini@cshs.org) - Jose Hernandez (jgh9094@gmail.com) - Jay Moran (jay.moran@cshs.org) - Nicholas Matsumoto (nicholas.matsumoto@cshs.org) - Hyunjun Choi (hyunjun.choi@cshs.org) - Miguel E. Hernandez (miguel.e.hernandez@cshs.org) - Jason Moore (moorejh28@gmail.com)

The original version of TPOT was primarily developed at the University of Pennsylvania by: - Randal S. Olson (rso@randalolson.com) - Weixuan Fu (weixuanf@upenn.edu) - Daniel Angell (dpa34@drexel.edu) - Jason Moore (moorejh28@gmail.com) - and many more generous open-source contributors

TPOT is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

TPOT is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with TPOT. If not, see http://www.gnu.org/licenses/.

TPOTEstimator

Bases: BaseEstimator

Source code in tpot2/tpot_estimator/estimator.py

class TPOTEstimator(BaseEstimator):
    def __init__(self,  
                        search_space,
                        scorers,
                        scorers_weights,
                        classification,
                        cv = 10,
                        other_objective_functions=[],
                        other_objective_functions_weights = [],
                        objective_function_names = None,
                        bigger_is_better = True,

                        export_graphpipeline = False,
                        memory = None,

                        categorical_features = None,
                        preprocessing = False,
                        population_size = 50,
                        initial_population_size = None,
                        population_scaling = .5,
                        generations_until_end_population = 1,
                        generations = None,
                        max_time_mins=60,
                        max_eval_time_mins=10,
                        validation_strategy = "none",
                        validation_fraction = .2,
                        disable_label_encoder = False,

                        #early stopping parameters
                        early_stop = None,
                        scorers_early_stop_tol = 0.001,
                        other_objectives_early_stop_tol =None,
                        threshold_evaluation_pruning = None,
                        threshold_evaluation_scaling = .5,
                        selection_evaluation_pruning = None,
                        selection_evaluation_scaling = .5,
                        min_history_threshold = 20,

                        #evolver parameters
                        survival_percentage = 1,
                        crossover_probability=.2,
                        mutate_probability=.7,
                        mutate_then_crossover_probability=.05,
                        crossover_then_mutate_probability=.05,
                        survival_selector = survival_select_NSGA2,
                        parent_selector = tournament_selection_dominated,

                        #budget parameters
                        budget_range = None,
                        budget_scaling = .5,
                        generations_until_end_budget = 1,
                        stepwise_steps = 5,

                        #dask parameters
                        n_jobs=1,
                        memory_limit = None,
                        client = None,
                        processes = True,

                        #debugging and logging parameters
                        warm_start = False,
                        periodic_checkpoint_folder = None,
                        callback = None,

                        verbose = 0,
                        scatter = True,

                         # random seed for random number generator (rng)
                        random_state = None,

                        ):

        '''
        An sklearn baseestimator that uses genetic programming to optimize a pipeline.

        Parameters
        ----------
        search_space : (String, tpot2.search_spaces.SearchSpace)
            - String : The default search space to use for the optimization.
            | String     | Description      |
            | :---        |    :----:   |
            | linear  | A linear pipeline with the structure of "Selector->(transformers+Passthrough)->(classifiers/regressors+Passthrough)->final classifier/regressor." For both the transformer and inner estimator layers, TPOT may choose one or more transformers/classifiers, or it may choose none. The inner classifier/regressor layer is optional. |
            | linear-light | Same search space as linear, but without the inner classifier/regressor layer and with a reduced set of faster running estimators. |
            | graph | TPOT will optimize a pipeline in the shape of a directed acyclic graph. The nodes of the graph can include selectors, scalers, transformers, or classifiers/regressors (inner classifiers/regressors can optionally be not included). This will return a custom GraphPipeline rather than an sklearn Pipeline. More details in Tutorial 6. |
            | graph-light | Same as graph search space, but without the inner classifier/regressors and with a reduced set of faster running estimators. |
            | mdr |TPOT will search over a series of feature selectors and Multifactor Dimensionality Reduction models to find a series of operators that maximize prediction accuracy. The TPOT MDR configuration is specialized for genome-wide association studies (GWAS), and is described in detail online here.

            Note that TPOT MDR may be slow to run because the feature selection routines are computationally expensive, especially on large datasets. |


            - SearchSpace : The search space to use for the optimization. This should be an instance of a SearchSpace.
                The search space to use for the optimization. This should be an instance of a SearchSpace.
                TPOT2 has groups of search spaces found in the following folders, tpot2.search_spaces.nodes for the nodes in the pipeline and tpot2.search_spaces.pipelines for the pipeline structure.

        scorers : (list, scorer)
            A scorer or list of scorers to be used in the cross-validation process.
            see https://scikit-learn.org/stable/modules/model_evaluation.html

        scorers_weights : list
            A list of weights to be applied to the scorers during the optimization process.

        classification : bool
            If True, the problem is treated as a classification problem. If False, the problem is treated as a regression problem.
            Used to determine the CV strategy.

        cv : int, cross-validator
            - (int): Number of folds to use in the cross-validation process. By uses the sklearn.model_selection.KFold cross-validator for regression and StratifiedKFold for classification. In both cases, shuffled is set to True.
            - (sklearn.model_selection.BaseCrossValidator): A cross-validator to use in the cross-validation process.
                - max_depth (int): The maximum depth from any node to the root of the pipelines to be generated.

        other_objective_functions : list, default=[]
            A list of other objective functions to apply to the pipeline. The function takes a single parameter for the graphpipeline estimator and returns either a single score or a list of scores.

        other_objective_functions_weights : list, default=[]
            A list of weights to be applied to the other objective functions.

        objective_function_names : list, default=None
            A list of names to be applied to the objective functions. If None, will use the names of the objective functions.

        bigger_is_better : bool, default=True
            If True, the objective function is maximized. If False, the objective function is minimized. Use negative weights to reverse the direction.

        memory: Memory object or string, default=None
            If supplied, pipeline will cache each transformer after calling fit with joblib.Memory. This feature
            is used to avoid computing the fit transformers within a pipeline if the parameters
            and input data are identical with another fitted pipeline during optimization process.
            - String 'auto':
                TPOT uses memory caching with a temporary directory and cleans it up upon shutdown.
            - String path of a caching directory
                TPOT uses memory caching with the provided directory and TPOT does NOT clean
                the caching directory up upon shutdown. If the directory does not exist, TPOT will
                create it.
            - Memory object:
                TPOT uses the instance of joblib.Memory for memory caching,
                and TPOT does NOT clean the caching directory up upon shutdown.
            - None:
                TPOT does not use memory caching.              

        categorical_features: list or None
            Categorical columns to inpute and/or one hot encode during the preprocessing step. Used only if preprocessing is not False.
            - None : If None, TPOT2 will automatically use object columns in pandas dataframes as objects for one hot encoding in preprocessing.
            - List of categorical features. If X is a dataframe, this should be a list of column names. If X is a numpy array, this should be a list of column indices

        preprocessing : bool or BaseEstimator/Pipeline,
            EXPERIMENTAL - will be changed in future versions
            A pipeline that will be used to preprocess the data before CV. Note that the parameters for these steps are not optimized. Add them to the search space to be optimized.
            - bool : If True, will use a default preprocessing pipeline which includes imputation followed by one hot encoding.
            - Pipeline : If an instance of a pipeline is given, will use that pipeline as the preprocessing pipeline.

        population_size : int, default=50
            Size of the population

        initial_population_size : int, default=None
            Size of the initial population. If None, population_size will be used.

        population_scaling : int, default=0.5
            Scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.

        generations_until_end_population : int, default=1
            Number of generations until the population size reaches population_size

        generations : int, default=50
            Number of generations to run

        max_time_mins : float, default=float("inf")
            Maximum time to run the optimization. If none or inf, will run until the end of the generations.

        max_eval_time_mins : float, default=5
            Maximum time to evaluate a single individual. If none or inf, there will be no time limit per evaluation.

        validation_strategy : str, default='none'
            EXPERIMENTAL The validation strategy to use for selecting the final pipeline from the population. TPOT2 may overfit the cross validation score. A second validation set can be used to select the final pipeline.
            - 'auto' : Automatically determine the validation strategy based on the dataset shape.
            - 'reshuffled' : Use the same data for cross validation and final validation, but with different splits for the folds. This is the default for small datasets.
            - 'split' : Use a separate validation set for final validation. Data will be split according to validation_fraction. This is the default for medium datasets.
            - 'none' : Do not use a separate validation set for final validation. Select based on the original cross-validation score. This is the default for large datasets.

        validation_fraction : float, default=0.2
          EXPERIMENTAL The fraction of the dataset to use for the validation set when validation_strategy is 'split'. Must be between 0 and 1.

        disable_label_encoder : bool, default=False
            If True, TPOT will check if the target needs to be relabeled to be sequential ints from 0 to N. This is necessary for XGBoost compatibility. If the labels need to be encoded, TPOT2 will use sklearn.preprocessing.LabelEncoder to encode the labels. The encoder can be accessed via the self.label_encoder_ attribute.
            If False, no additional label encoders will be used.

        early_stop : int, default=None
            Number of generations without improvement before early stopping. All objectives must have converged within the tolerance for this to be triggered. In general a value of around 5-20 is good.

        scorers_early_stop_tol :
            -list of floats
                list of tolerances for each scorer. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged
                If an index of the list is None, that item will not be used for early stopping
            -int
                If an int is given, it will be used as the tolerance for all objectives

        other_objectives_early_stop_tol :
            -list of floats
                list of tolerances for each of the other objective function. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged
                If an index of the list is None, that item will not be used for early stopping
            -int
                If an int is given, it will be used as the tolerance for all objectives

        threshold_evaluation_pruning : list [start, end], default=None
            starting and ending percentile to use as a threshold for the evaluation early stopping.
            Values between 0 and 100.

        threshold_evaluation_scaling : float [0,inf), default=0.5
            A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.
            Must be greater than zero. Higher numbers will move the threshold to the end faster.

        selection_evaluation_pruning : list, default=None
            A lower and upper percent of the population size to select each round of CV.
            Values between 0 and 1.

        selection_evaluation_scaling : float, default=0.5
            A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.
            Must be greater than zero. Higher numbers will move the threshold to the end faster.

        min_history_threshold : int, default=0
            The minimum number of previous scores needed before using threshold early stopping.

        survival_percentage : float, default=1
            Percentage of the population size to utilize for mutation and crossover at the beginning of the generation. The rest are discarded. Individuals are selected with the selector passed into survival_selector. The value of this parameter must be between 0 and 1, inclusive.
            For example, if the population size is 100 and the survival percentage is .5, 50 individuals will be selected with NSGA2 from the existing population. These will be used for mutation and crossover to generate the next 100 individuals for the next generation. The remainder are discarded from the live population. In the next generation, there will now be the 50 parents + the 100 individuals for a total of 150. Surivival percentage is based of the population size parameter and not the existing population size (current population size when using successive halving). Therefore, in the next generation we will still select 50 individuals from the currently existing 150.

        crossover_probability : float, default=.2
            Probability of generating a new individual by crossover between two individuals.

        mutate_probability : float, default=.7
            Probability of generating a new individual by crossover between one individuals.

        mutate_then_crossover_probability : float, default=.05
            Probability of generating a new individual by mutating two individuals followed by crossover.

        crossover_then_mutate_probability : float, default=.05
            Probability of generating a new individual by crossover between two individuals followed by a mutation of the resulting individual.

        survival_selector : function, default=survival_select_NSGA2
            Function to use to select individuals for survival. Must take a matrix of scores and return selected indexes.
            Used to selected population_size * survival_percentage individuals at the start of each generation to use for mutation and crossover.

        parent_selector : function, default=parent_select_NSGA2
            Function to use to select pairs parents for crossover and individuals for mutation. Must take a matrix of scores and return selected indexes.

        budget_range : list [start, end], default=None
            A starting and ending budget to use for the budget scaling.

        budget_scaling float : [0,1], default=0.5
            A scaling factor to use when determining how fast we move the budget from the start to end budget.

        generations_until_end_budget : int, default=1
            The number of generations to run before reaching the max budget.

        stepwise_steps : int, default=1
            The number of staircase steps to take when scaling the budget and population size.

        n_jobs : int, default=1
            Number of processes to run in parallel.

        memory_limit : str, default=None
            Memory limit for each job. See Dask [LocalCluster documentation](https://distributed.dask.org/en/stable/api.html#distributed.Client) for more information.

        client : dask.distributed.Client, default=None
            A dask client to use for parallelization. If not None, this will override the n_jobs and memory_limit parameters. If None, will create a new client with num_workers=n_jobs and memory_limit=memory_limit.

        processes : bool, default=True
            If True, will use multiprocessing to parallelize the optimization process. If False, will use threading.
            True seems to perform better. However, False is required for interactive debugging.

        warm_start : bool, default=False
            If True, will use the continue the evolutionary algorithm from the last generation of the previous run.

        periodic_checkpoint_folder : str, default=None
            Folder to save the population to periodically. If None, no periodic saving will be done.
            If provided, training will resume from this checkpoint.

        callback : tpot2.CallBackInterface, default=None
            Callback object. Not implemented

        verbose : int, default=1
            How much information to print during the optimization process. Higher values include the information from lower values.
            0. nothing
            1. progress bar

            3. best individual
            4. warnings
            >=5. full warnings trace
            6. evaluations progress bar. (Temporary: This used to be 2. Currently, using evaluation progress bar may prevent some instances were we terminate a generation early due to it reaching max_time_mins in the middle of a generation OR a pipeline failed to be terminated normally and we need to manually terminate it.)

        scatter : bool, default=True
            If True, will scatter the data to the dask workers. If False, will not scatter the data. This can be useful for debugging.

        random_state : int, None, default=None
            A seed for reproducability of experiments. This value will be passed to numpy.random.default_rng() to create an instnce of the genrator to pass to other classes

            - int
                Will be used to create and lock in Generator instance with 'numpy.random.default_rng()'
            - None
                Will be used to create Generator for 'numpy.random.default_rng()' where a fresh, unpredictable entropy will be pulled from the OS

        Attributes
        ----------

        fitted_pipeline_ : GraphPipeline
            A fitted instance of the GraphPipeline that inherits from sklearn BaseEstimator. This is fitted on the full X, y passed to fit.

        evaluated_individuals : A pandas data frame containing data for all evaluated individuals in the run.
            Columns:
            - *objective functions : The first few columns correspond to the passed in scorers and objective functions
            - Parents : A tuple containing the indexes of the pipelines used to generate the pipeline of that row. If NaN, this pipeline was generated randomly in the initial population.
            - Variation_Function : Which variation function was used to mutate or crossover the parents. If NaN, this pipeline was generated randomly in the initial population.
            - Individual : The internal representation of the individual that is used during the evolutionary algorithm. This is not an sklearn BaseEstimator.
            - Generation : The generation the pipeline first appeared.
            - Pareto_Front	: The nondominated front that this pipeline belongs to. 0 means that its scores is not strictly dominated by any other individual.
                            To save on computational time, the best frontier is updated iteratively each generation.
                            The pipelines with the 0th pareto front do represent the exact best frontier. However, the pipelines with pareto front >= 1 are only in reference to the other pipelines in the final population.
                            All other pipelines are set to NaN.
            - Instance	: The unfitted GraphPipeline BaseEstimator.
            - *validation objective functions : Objective function scores evaluated on the validation set.
            - Validation_Pareto_Front : The full pareto front calculated on the validation set. This is calculated for all pipelines with Pareto_Front equal to 0. Unlike the Pareto_Front which only calculates the frontier and the final population, the Validation Pareto Front is calculated for all pipelines tested on the validation set.

        pareto_front : The same pandas dataframe as evaluated individuals, but containing only the frontier pareto front pipelines.
        '''

        # sklearn BaseEstimator must have a corresponding attribute for each parameter.
        # These should not be modified once set.

        self.scorers = scorers
        self.scorers_weights = scorers_weights
        self.classification = classification
        self.cv = cv
        self.other_objective_functions = other_objective_functions
        self.other_objective_functions_weights = other_objective_functions_weights
        self.objective_function_names = objective_function_names
        self.bigger_is_better = bigger_is_better

        self.search_space = search_space

        self.export_graphpipeline = export_graphpipeline
        self.memory = memory

        self.categorical_features = categorical_features

        self.preprocessing = preprocessing
        self.validation_strategy = validation_strategy
        self.validation_fraction = validation_fraction
        self.disable_label_encoder = disable_label_encoder
        self.population_size = population_size
        self.initial_population_size = initial_population_size
        self.population_scaling = population_scaling
        self.generations_until_end_population = generations_until_end_population
        self.generations = generations
        self.early_stop = early_stop
        self.scorers_early_stop_tol = scorers_early_stop_tol
        self.other_objectives_early_stop_tol = other_objectives_early_stop_tol
        self.max_time_mins = max_time_mins
        self.max_eval_time_mins = max_eval_time_mins
        self.n_jobs= n_jobs
        self.memory_limit = memory_limit
        self.client = client
        self.survival_percentage = survival_percentage
        self.crossover_probability = crossover_probability
        self.mutate_probability = mutate_probability
        self.mutate_then_crossover_probability= mutate_then_crossover_probability
        self.crossover_then_mutate_probability= crossover_then_mutate_probability
        self.survival_selector=survival_selector
        self.parent_selector=parent_selector
        self.budget_range = budget_range
        self.budget_scaling = budget_scaling
        self.generations_until_end_budget = generations_until_end_budget
        self.stepwise_steps = stepwise_steps
        self.threshold_evaluation_pruning =threshold_evaluation_pruning
        self.threshold_evaluation_scaling =  threshold_evaluation_scaling
        self.min_history_threshold = min_history_threshold
        self.selection_evaluation_pruning = selection_evaluation_pruning
        self.selection_evaluation_scaling =  selection_evaluation_scaling
        self.warm_start = warm_start
        self.verbose = verbose
        self.periodic_checkpoint_folder = periodic_checkpoint_folder
        self.callback = callback
        self.processes = processes


        self.scatter = scatter


        timer_set = self.max_time_mins != float("inf") and self.max_time_mins is not None
        if self.generations is not None and timer_set:
            warnings.warn("Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.")

        # create random number generator based on rngseed
        self.rng = np.random.default_rng(random_state)
        # save random state passed to us for other functions that use random_state
        self.random_state = random_state

        #Initialize other used params


        if self.initial_population_size is None:
            self._initial_population_size = self.population_size
        else:
            self._initial_population_size = self.initial_population_size

        if isinstance(self.scorers, str):
            self._scorers = [self.scorers]

        elif callable(self.scorers):
            self._scorers = [self.scorers]
        else:
            self._scorers = self.scorers

        self._scorers = [sklearn.metrics.get_scorer(scoring) for scoring in self._scorers]
        self._scorers_early_stop_tol = self.scorers_early_stop_tol

        self._evolver = tpot2.evolvers.BaseEvolver

        self.objective_function_weights = [*scorers_weights, *other_objective_functions_weights]


        if self.objective_function_names is None:
            obj_names = [f.__name__ for f in other_objective_functions]
        else:
            obj_names = self.objective_function_names
        self.objective_names = [f._score_func.__name__ if hasattr(f,"_score_func") else f.__name__ for f in self._scorers] + obj_names


        if not isinstance(self.other_objectives_early_stop_tol, list):
            self._other_objectives_early_stop_tol = [self.other_objectives_early_stop_tol for _ in range(len(self.other_objective_functions))]
        else:
            self._other_objectives_early_stop_tol = self.other_objectives_early_stop_tol

        if not isinstance(self._scorers_early_stop_tol, list):
            self._scorers_early_stop_tol = [self._scorers_early_stop_tol for _ in range(len(self._scorers))]
        else:
            self._scorers_early_stop_tol = self._scorers_early_stop_tol

        self.early_stop_tol = [*self._scorers_early_stop_tol, *self._other_objectives_early_stop_tol]

        self._evolver_instance = None
        self.evaluated_individuals = None


        self.label_encoder_ = None


        set_dask_settings()


    def fit(self, X, y):
        if self.client is not None: #If user passed in a client manually
           _client = self.client
        else:

            if self.verbose >= 4:
                silence_logs = 30
            elif self.verbose >=5:
                silence_logs = 40
            else:
                silence_logs = 50
            cluster = LocalCluster(n_workers=self.n_jobs, #if no client is passed in and no global client exists, create our own
                    threads_per_worker=1,
                    processes=self.processes,
                    silence_logs=silence_logs,
                    memory_limit=self.memory_limit)
            _client = Client(cluster)

        if self.classification and not self.disable_label_encoder and not check_if_y_is_encoded(y):
            warnings.warn("Labels are not encoded as ints from 0 to N. For compatibility with some classifiers such as sklearn, TPOT has encoded y with the sklearn LabelEncoder. When using pipelines outside the main TPOT estimator class, you can encode the labels with est.label_encoder_")
            self.label_encoder_ = LabelEncoder()
            y = self.label_encoder_.fit_transform(y)

        self.evaluated_individuals = None
        #determine validation strategy
        if self.validation_strategy == 'auto':
            nrows = X.shape[0]
            ncols = X.shape[1]

            if nrows/ncols < 20:
                validation_strategy = 'reshuffled'
            elif nrows/ncols < 100:
                validation_strategy = 'split'
            else:
                validation_strategy = 'none'
        else:
            validation_strategy = self.validation_strategy

        if validation_strategy == 'split':
            if self.classification:
                X, X_val, y, y_val = train_test_split(X, y, test_size=self.validation_fraction, stratify=y, random_state=self.random_state)
            else:
                X, X_val, y, y_val = train_test_split(X, y, test_size=self.validation_fraction, random_state=self.random_state)


        X_original = X
        y_original = y
        if isinstance(self.cv, int) or isinstance(self.cv, float):
            n_folds = self.cv
        else:
            n_folds = self.cv.get_n_splits(X, y)

        if self.classification:
            X, y = remove_underrepresented_classes(X, y, n_folds)

        if self.preprocessing:
            #X = pd.DataFrame(X)

            if not isinstance(self.preprocessing, bool) and isinstance(self.preprocessing, sklearn.base.BaseEstimator):
                self._preprocessing_pipeline = sklearn.base.clone(self.preprocessing)

            #TODO: check if there are missing values in X before imputation. If not, don't include imputation in pipeline. Check if there are categorical columns. If not, don't include one hot encoding in pipeline
            else: #if self.preprocessing is True or not a sklearn estimator

                pipeline_steps = []

                if self.categorical_features is not None: #if categorical features are specified, use those
                    pipeline_steps.append(("impute_categorical", tpot2.builtin_modules.ColumnSimpleImputer(self.categorical_features, strategy='most_frequent')))
                    pipeline_steps.append(("impute_numeric", tpot2.builtin_modules.ColumnSimpleImputer("numeric", strategy='mean')))
                    pipeline_steps.append(("ColumnOneHotEncoder", tpot2.builtin_modules.ColumnOneHotEncoder(self.categorical_features, strategy='most_frequent')))

                else:
                    if isinstance(X, pd.DataFrame):
                        categorical_columns = X.select_dtypes(include=['object']).columns
                        if len(categorical_columns) > 0:
                            pipeline_steps.append(("impute_categorical", tpot2.builtin_modules.ColumnSimpleImputer("categorical", strategy='most_frequent')))
                            pipeline_steps.append(("impute_numeric", tpot2.builtin_modules.ColumnSimpleImputer("numeric", strategy='mean')))
                            pipeline_steps.append(("ColumnOneHotEncoder", tpot2.builtin_modules.ColumnOneHotEncoder("categorical", strategy='most_frequent')))
                        else:
                            pipeline_steps.append(("impute_numeric", tpot2.builtin_modules.ColumnSimpleImputer("all", strategy='mean')))
                    else:
                        pipeline_steps.append(("impute_numeric", tpot2.builtin_modules.ColumnSimpleImputer("all", strategy='mean')))

                self._preprocessing_pipeline = sklearn.pipeline.Pipeline(pipeline_steps)

            X = self._preprocessing_pipeline.fit_transform(X, y)

        else:
            self._preprocessing_pipeline = None

        #_, y = sklearn.utils.check_X_y(X, y, y_numeric=True)

        #Set up the configuation dictionaries and the search spaces

        #check if self.cv is a number
        if isinstance(self.cv, int) or isinstance(self.cv, float):
            if self.classification:
                self.cv_gen = sklearn.model_selection.StratifiedKFold(n_splits=self.cv, shuffle=True, random_state=self.random_state)
            else:
                self.cv_gen = sklearn.model_selection.KFold(n_splits=self.cv, shuffle=True, random_state=self.random_state)

        else:
            self.cv_gen = sklearn.model_selection.check_cv(self.cv, y, classifier=self.classification)



        n_samples= int(math.floor(X.shape[0]/n_folds))
        n_features=X.shape[1]

        if isinstance(X, pd.DataFrame):
            self.feature_names = X.columns
        else:
            self.feature_names = None



        def objective_function(pipeline_individual,
                                            X,
                                            y,
                                            is_classification=self.classification,
                                            scorers= self._scorers,
                                            cv=self.cv_gen,
                                            other_objective_functions=self.other_objective_functions,
                                            export_graphpipeline=self.export_graphpipeline,
                                            memory=self.memory,
                                            **kwargs):
            return objective_function_generator(
                pipeline_individual,
                X,
                y,
                is_classification=is_classification,
                scorers= scorers,
                cv=cv,
                other_objective_functions=other_objective_functions,
                export_graphpipeline=export_graphpipeline,
                memory=memory,
                **kwargs,
            )



        if self.threshold_evaluation_pruning is not None or self.selection_evaluation_pruning is not None:
            evaluation_early_stop_steps = self.cv
        else:
            evaluation_early_stop_steps = None

        if self.scatter:
            X_future = _client.scatter(X)
            y_future = _client.scatter(y)
        else:
            X_future = X
            y_future = y

        if self.classification:
            n_classes = len(np.unique(y))
        else:
            n_classes = None

        get_search_space_params = {"n_classes": n_classes, 
                        "n_samples":len(y), 
                        "n_features":X.shape[1], 
                        "random_state":self.random_state}

        self._search_space = get_template_search_spaces(self.search_space, classification=self.classification, inner_predictors=True, **get_search_space_params)


        # TODO : Add check for empty values in X and if so, add imputation to the search space
        # make this depend on self.preprocessing
        # if check_empty_values(X):
        #     from sklearn.experimental import enable_iterative_imputer

        #     from ConfigSpace import ConfigurationSpace
        #     from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
        #     iterative_imputer_cs = ConfigurationSpace(
        #         space = {
        #             'n_nearest_features' : Categorical('n_nearest_features', [100]),
        #             'initial_strategy' : Categorical('initial_strategy', ['mean','median', 'most_frequent', ]),
        #             'add_indicator' : Categorical('add_indicator', [True, False]),
        #         }
        #     )

        #     imputation_search = tpot2.search_spaces.pipelines.ChoicePipeline([
        #         tpot2.config.get_search_space("SimpleImputer"),
        #         tpot2.search_spaces.nodes.EstimatorNode(sklearn.impute.IterativeImputer, iterative_imputer_cs)
        #     ])




        #     self.search_space_final = tpot2.search_spaces.pipelines.SequentialPipeline(search_spaces=[ imputation_search, self._search_space], memory="sklearn_pipeline_memory")
        # else:
        #     self.search_space_final = self._search_space

        self.search_space_final = self._search_space

        def ind_generator(rng):
            rng = np.random.default_rng(rng)
            while True:
                yield self.search_space_final.generate(rng)

        #If warm start and we have an evolver instance, use the existing one
        if not(self.warm_start and self._evolver_instance is not None):
            self._evolver_instance = self._evolver(   individual_generator=ind_generator(self.rng),
                                            objective_functions= [objective_function],
                                            objective_function_weights = self.objective_function_weights,
                                            objective_names=self.objective_names,
                                            bigger_is_better = self.bigger_is_better,
                                            population_size= self.population_size,
                                            generations=self.generations,
                                            initial_population_size = self._initial_population_size,
                                            n_jobs=self.n_jobs,
                                            verbose = self.verbose,
                                            max_time_mins =      self.max_time_mins ,
                                            max_eval_time_mins = self.max_eval_time_mins,

                                            periodic_checkpoint_folder = self.periodic_checkpoint_folder,
                                            threshold_evaluation_pruning = self.threshold_evaluation_pruning,
                                            threshold_evaluation_scaling =  self.threshold_evaluation_scaling,
                                            min_history_threshold = self.min_history_threshold,

                                            selection_evaluation_pruning = self.selection_evaluation_pruning,
                                            selection_evaluation_scaling =  self.selection_evaluation_scaling,
                                            evaluation_early_stop_steps = evaluation_early_stop_steps,

                                            early_stop_tol = self.early_stop_tol,
                                            early_stop= self.early_stop,

                                            budget_range = self.budget_range,
                                            budget_scaling = self.budget_scaling,
                                            generations_until_end_budget = self.generations_until_end_budget,

                                            population_scaling = self.population_scaling,
                                            generations_until_end_population = self.generations_until_end_population,
                                            stepwise_steps = self.stepwise_steps,
                                            client = _client,
                                            objective_kwargs = {"X": X_future, "y": y_future},
                                            survival_selector=self.survival_selector,
                                            parent_selector=self.parent_selector,
                                            survival_percentage = self.survival_percentage,
                                            crossover_probability = self.crossover_probability,
                                            mutate_probability = self.mutate_probability,
                                            mutate_then_crossover_probability= self.mutate_then_crossover_probability,
                                            crossover_then_mutate_probability= self.crossover_then_mutate_probability,

                                            rng=self.rng,
                                            )


        self._evolver_instance.optimize()
        #self._evolver_instance.population.update_pareto_fronts(self.objective_names, self.objective_function_weights)
        self.make_evaluated_individuals()




        tpot2.utils.get_pareto_frontier(self.evaluated_individuals, column_names=self.objective_names, weights=self.objective_function_weights)

        if validation_strategy == 'reshuffled':
            best_pareto_front_idx = list(self.pareto_front.index)
            best_pareto_front = list(self.pareto_front.loc[best_pareto_front_idx]['Individual'])

            #reshuffle rows
            X, y = sklearn.utils.shuffle(X, y, random_state=self.random_state)

            if self.scatter:
                X_future = _client.scatter(X)
                y_future = _client.scatter(y)
            else:
                X_future = X
                y_future = y

            val_objective_function_list = [lambda   ind,
                                                    X,
                                                    y,
                                                    is_classification=self.classification,
                                                    scorers= self._scorers,
                                                    cv=self.cv_gen,
                                                    other_objective_functions=self.other_objective_functions,
                                                    export_graphpipeline=self.export_graphpipeline,
                                                    memory=self.memory,
                                                    **kwargs: objective_function_generator(
                                                                                                ind,
                                                                                                X,
                                                                                                y,
                                                                                                is_classification=is_classification,
                                                                                                scorers= scorers,
                                                                                                cv=cv,
                                                                                                other_objective_functions=other_objective_functions,
                                                                                                export_graphpipeline=export_graphpipeline,
                                                                                                memory=memory,
                                                                                                **kwargs,
                                                                                                )]

            objective_kwargs = {"X": X_future, "y": y_future}
            val_scores, start_times, end_times, eval_errors = tpot2.utils.eval_utils.parallel_eval_objective_list(best_pareto_front, val_objective_function_list, verbose=self.verbose, max_eval_time_mins=self.max_eval_time_mins, n_expected_columns=len(self.objective_names), client=_client, **objective_kwargs)



            val_objective_names = ['validation_'+name for name in self.objective_names]
            self.objective_names_for_selection = val_objective_names
            self.evaluated_individuals.loc[best_pareto_front_idx,val_objective_names] = val_scores
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_start_times'] = start_times
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_end_times'] = end_times
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_eval_errors'] = eval_errors

            self.evaluated_individuals["Validation_Pareto_Front"] = tpot2.utils.get_pareto_frontier(self.evaluated_individuals, column_names=val_objective_names, weights=self.objective_function_weights)


        elif validation_strategy == 'split':


            if self.scatter:
                X_future = _client.scatter(X)
                y_future = _client.scatter(y)
                X_val_future = _client.scatter(X_val)
                y_val_future = _client.scatter(y_val)
            else:
                X_future = X
                y_future = y
                X_val_future = X_val
                y_val_future = y_val

            objective_kwargs = {"X": X_future, "y": y_future, "X_val" : X_val_future, "y_val":y_val_future }

            best_pareto_front_idx = list(self.pareto_front.index)
            best_pareto_front = list(self.pareto_front.loc[best_pareto_front_idx]['Individual'])
            val_objective_function_list = [lambda   ind,
                                                    X,
                                                    y,
                                                    X_val,
                                                    y_val,
                                                    scorers= self._scorers,
                                                    other_objective_functions=self.other_objective_functions,
                                                    export_graphpipeline=self.export_graphpipeline,
                                                    memory=self.memory,
                                                    **kwargs: val_objective_function_generator(
                                                        ind,
                                                        X,
                                                        y,
                                                        X_val,
                                                        y_val,
                                                        scorers= scorers,
                                                        other_objective_functions=other_objective_functions,
                                                        export_graphpipeline=export_graphpipeline,
                                                        memory=memory,
                                                        **kwargs,
                                                        )]

            val_scores, start_times, end_times, eval_errors = tpot2.utils.eval_utils.parallel_eval_objective_list(best_pareto_front, val_objective_function_list, verbose=self.verbose, max_eval_time_mins=self.max_eval_time_mins, n_expected_columns=len(self.objective_names), client=_client, **objective_kwargs)



            val_objective_names = ['validation_'+name for name in self.objective_names]
            self.objective_names_for_selection = val_objective_names
            self.evaluated_individuals.loc[best_pareto_front_idx,val_objective_names] = val_scores
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_start_times'] = start_times
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_end_times'] = end_times
            self.evaluated_individuals.loc[best_pareto_front_idx,'validation_eval_errors'] = eval_errors

            self.evaluated_individuals["Validation_Pareto_Front"] = tpot2.utils.get_pareto_frontier(self.evaluated_individuals, column_names=val_objective_names, weights=self.objective_function_weights)

        else:
            self.objective_names_for_selection = self.objective_names

        val_scores = self.evaluated_individuals[self.evaluated_individuals[self.objective_names_for_selection].isna().all(1).ne(True)][self.objective_names_for_selection]
        weighted_scores = val_scores*self.objective_function_weights

        if self.bigger_is_better:
            best_indices = list(weighted_scores.sort_values(by=self.objective_names_for_selection, ascending=False).index)
        else:
            best_indices = list(weighted_scores.sort_values(by=self.objective_names_for_selection, ascending=True).index)

        for best_idx in best_indices:

            best_individual = self.evaluated_individuals.loc[best_idx]['Individual']
            self.selected_best_score =  self.evaluated_individuals.loc[best_idx]


            #TODO
            #best_individual_pipeline = best_individual.export_pipeline(memory=self.memory, cross_val_predict_cv=self.cross_val_predict_cv)
            if self.export_graphpipeline:
                best_individual_pipeline = best_individual.export_flattened_graphpipeline(memory=self.memory)
            else:
                best_individual_pipeline = best_individual.export_pipeline(memory=self.memory)

            if self.preprocessing:
                self.fitted_pipeline_ = sklearn.pipeline.make_pipeline(sklearn.base.clone(self._preprocessing_pipeline), best_individual_pipeline )
            else:
                self.fitted_pipeline_ = best_individual_pipeline

            try:
                self.fitted_pipeline_.fit(X_original,y_original) #TODO use y_original as well?
                break
            except Exception as e:
                if self.verbose >= 4:
                    warnings.warn("Final pipeline failed to fit. Rarely, the pipeline might work on the objective function but fail on the full dataset. Generally due to interactions with different features being selected or transformations having different properties. Trying next pipeline")
                    print(e)
                continue


        if self.client is None: #no client was passed in
            #close cluster and client
            # _client.close()
            # cluster.close()
            try:
                _client.shutdown()
                cluster.close()
            #catch exception
            except Exception as e:
                print("Error shutting down client and cluster")
                Warning(e)

        return self

    def _estimator_has(attr):
        '''Check if we can delegate a method to the underlying estimator.
        First, we check the first fitted final estimator if available, otherwise we
        check the unfitted final estimator.
        '''
        return  lambda self: (self.fitted_pipeline_ is not None and
            hasattr(self.fitted_pipeline_, attr)
        )






    @available_if(_estimator_has('predict'))
    def predict(self, X, **predict_params):
        check_is_fitted(self)
        #X = check_array(X)

        preds = self.fitted_pipeline_.predict(X,**predict_params)
        if self.classification and self.label_encoder_:
            preds = self.label_encoder_.inverse_transform(preds)

        return preds

    @available_if(_estimator_has('predict_proba'))
    def predict_proba(self, X, **predict_params):
        check_is_fitted(self)
        #X = check_array(X)
        return self.fitted_pipeline_.predict_proba(X,**predict_params)

    @available_if(_estimator_has('decision_function'))
    def decision_function(self, X, **predict_params):
        check_is_fitted(self)
        #X = check_array(X)
        return self.fitted_pipeline_.decision_function(X,**predict_params)

    @available_if(_estimator_has('transform'))
    def transform(self, X, **predict_params):
        check_is_fitted(self)
        #X = check_array(X)
        return self.fitted_pipeline_.transform(X,**predict_params)

    @property
    def classes_(self):
        """The classes labels. Only exist if the last step is a classifier."""
        if self.label_encoder_:
            return self.label_encoder_.classes_
        else:
            return self.fitted_pipeline_.classes_


    @property
    def _estimator_type(self):
        return self.fitted_pipeline_._estimator_type


    def make_evaluated_individuals(self):
        #check if _evolver_instance exists
        if self.evaluated_individuals is None:
            self.evaluated_individuals  =  self._evolver_instance.population.evaluated_individuals.copy()
            objects = list(self.evaluated_individuals.index)
            object_to_int = dict(zip(objects, range(len(objects))))
            self.evaluated_individuals = self.evaluated_individuals.set_index(self.evaluated_individuals.index.map(object_to_int))
            self.evaluated_individuals['Parents'] = self.evaluated_individuals['Parents'].apply(lambda row: convert_parents_tuples_to_integers(row, object_to_int))

            self.evaluated_individuals["Instance"] = self.evaluated_individuals["Individual"].apply(lambda ind: apply_make_pipeline(ind, preprocessing_pipeline=self._preprocessing_pipeline, export_graphpipeline=self.export_graphpipeline, memory=self.memory))

        return self.evaluated_individuals

    @property
    def pareto_front(self):
        #check if _evolver_instance exists
        if self.evaluated_individuals is None:
            return None
        else:
            if "Pareto_Front" not in self.evaluated_individuals:
                return self.evaluated_individuals
            else:
                return self.evaluated_individuals[self.evaluated_individuals["Pareto_Front"]==1]

classes_ property

The classes labels. Only exist if the last step is a classifier.

__init__(search_space, scorers, scorers_weights, classification, cv=10, other_objective_functions=[], other_objective_functions_weights=[], objective_function_names=None, bigger_is_better=True, export_graphpipeline=False, memory=None, categorical_features=None, preprocessing=False, population_size=50, initial_population_size=None, population_scaling=0.5, generations_until_end_population=1, generations=None, max_time_mins=60, max_eval_time_mins=10, validation_strategy='none', validation_fraction=0.2, disable_label_encoder=False, early_stop=None, scorers_early_stop_tol=0.001, other_objectives_early_stop_tol=None, threshold_evaluation_pruning=None, threshold_evaluation_scaling=0.5, selection_evaluation_pruning=None, selection_evaluation_scaling=0.5, min_history_threshold=20, survival_percentage=1, crossover_probability=0.2, mutate_probability=0.7, mutate_then_crossover_probability=0.05, crossover_then_mutate_probability=0.05, survival_selector=survival_select_NSGA2, parent_selector=tournament_selection_dominated, budget_range=None, budget_scaling=0.5, generations_until_end_budget=1, stepwise_steps=5, n_jobs=1, memory_limit=None, client=None, processes=True, warm_start=False, periodic_checkpoint_folder=None, callback=None, verbose=0, scatter=True, random_state=None)

An sklearn baseestimator that uses genetic programming to optimize a pipeline.

Parameters:

Name	Type	Description	Default
search_space	(String, SearchSpace)	String : The default search space to use for the optimization. | String | Description | | :--- | :----: | | linear | A linear pipeline with the structure of "Selector->(transformers+Passthrough)->(classifiers/regressors+Passthrough)->final classifier/regressor." For both the transformer and inner estimator layers, TPOT may choose one or more transformers/classifiers, or it may choose none. The inner classifier/regressor layer is optional. | | linear-light | Same search space as linear, but without the inner classifier/regressor layer and with a reduced set of faster running estimators. | | graph | TPOT will optimize a pipeline in the shape of a directed acyclic graph. The nodes of the graph can include selectors, scalers, transformers, or classifiers/regressors (inner classifiers/regressors can optionally be not included). This will return a custom GraphPipeline rather than an sklearn Pipeline. More details in Tutorial 6. | | graph-light | Same as graph search space, but without the inner classifier/regressors and with a reduced set of faster running estimators. | | mdr |TPOT will search over a series of feature selectors and Multifactor Dimensionality Reduction models to find a series of operators that maximize prediction accuracy. The TPOT MDR configuration is specialized for genome-wide association studies (GWAS), and is described in detail online here. Note that TPOT MDR may be slow to run because the feature selection routines are computationally expensive, especially on large datasets. | SearchSpace : The search space to use for the optimization. This should be an instance of a SearchSpace. The search space to use for the optimization. This should be an instance of a SearchSpace. TPOT2 has groups of search spaces found in the following folders, tpot2.search_spaces.nodes for the nodes in the pipeline and tpot2.search_spaces.pipelines for the pipeline structure.	required
scorers	(list, scorer)	A scorer or list of scorers to be used in the cross-validation process. see https://scikit-learn.org/stable/modules/model_evaluation.html	required
scorers_weights	list	A list of weights to be applied to the scorers during the optimization process.	required
classification	bool	If True, the problem is treated as a classification problem. If False, the problem is treated as a regression problem. Used to determine the CV strategy.	required
cv	(int, cross - validator)	(int): Number of folds to use in the cross-validation process. By uses the sklearn.model_selection.KFold cross-validator for regression and StratifiedKFold for classification. In both cases, shuffled is set to True. (sklearn.model_selection.BaseCrossValidator): A cross-validator to use in the cross-validation process. max_depth (int): The maximum depth from any node to the root of the pipelines to be generated.	10
other_objective_functions	list	A list of other objective functions to apply to the pipeline. The function takes a single parameter for the graphpipeline estimator and returns either a single score or a list of scores.	[]
other_objective_functions_weights	list	A list of weights to be applied to the other objective functions.	[]
objective_function_names	list	A list of names to be applied to the objective functions. If None, will use the names of the objective functions.	None
bigger_is_better	bool	If True, the objective function is maximized. If False, the objective function is minimized. Use negative weights to reverse the direction.	True
memory		If supplied, pipeline will cache each transformer after calling fit with joblib.Memory. This feature is used to avoid computing the fit transformers within a pipeline if the parameters and input data are identical with another fitted pipeline during optimization process. - String 'auto': TPOT uses memory caching with a temporary directory and cleans it up upon shutdown. - String path of a caching directory TPOT uses memory caching with the provided directory and TPOT does NOT clean the caching directory up upon shutdown. If the directory does not exist, TPOT will create it. - Memory object: TPOT uses the instance of joblib.Memory for memory caching, and TPOT does NOT clean the caching directory up upon shutdown. - None: TPOT does not use memory caching.	None
categorical_features		Categorical columns to inpute and/or one hot encode during the preprocessing step. Used only if preprocessing is not False. - None : If None, TPOT2 will automatically use object columns in pandas dataframes as objects for one hot encoding in preprocessing. - List of categorical features. If X is a dataframe, this should be a list of column names. If X is a numpy array, this should be a list of column indices	None
preprocessing	(bool or BaseEstimator / Pipeline)	EXPERIMENTAL - will be changed in future versions A pipeline that will be used to preprocess the data before CV. Note that the parameters for these steps are not optimized. Add them to the search space to be optimized. - bool : If True, will use a default preprocessing pipeline which includes imputation followed by one hot encoding. - Pipeline : If an instance of a pipeline is given, will use that pipeline as the preprocessing pipeline.	False
population_size	int	Size of the population	50
initial_population_size	int	Size of the initial population. If None, population_size will be used.	None
population_scaling	int	Scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.	0.5
generations_until_end_population	int	Number of generations until the population size reaches population_size	1
generations	int	Number of generations to run	50
max_time_mins	float	Maximum time to run the optimization. If none or inf, will run until the end of the generations.	float("inf")
max_eval_time_mins	float	Maximum time to evaluate a single individual. If none or inf, there will be no time limit per evaluation.	5
validation_strategy	str	EXPERIMENTAL The validation strategy to use for selecting the final pipeline from the population. TPOT2 may overfit the cross validation score. A second validation set can be used to select the final pipeline. - 'auto' : Automatically determine the validation strategy based on the dataset shape. - 'reshuffled' : Use the same data for cross validation and final validation, but with different splits for the folds. This is the default for small datasets. - 'split' : Use a separate validation set for final validation. Data will be split according to validation_fraction. This is the default for medium datasets. - 'none' : Do not use a separate validation set for final validation. Select based on the original cross-validation score. This is the default for large datasets.	'none'
validation_fraction	float	EXPERIMENTAL The fraction of the dataset to use for the validation set when validation_strategy is 'split'. Must be between 0 and 1.	0.2
disable_label_encoder	bool	If True, TPOT will check if the target needs to be relabeled to be sequential ints from 0 to N. This is necessary for XGBoost compatibility. If the labels need to be encoded, TPOT2 will use sklearn.preprocessing.LabelEncoder to encode the labels. The encoder can be accessed via the self.label_encoder_ attribute. If False, no additional label encoders will be used.	False
early_stop	int	Number of generations without improvement before early stopping. All objectives must have converged within the tolerance for this to be triggered. In general a value of around 5-20 is good.	None
scorers_early_stop_tol		-list of floats list of tolerances for each scorer. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged If an index of the list is None, that item will not be used for early stopping -int If an int is given, it will be used as the tolerance for all objectives	0.001
other_objectives_early_stop_tol		-list of floats list of tolerances for each of the other objective function. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged If an index of the list is None, that item will not be used for early stopping -int If an int is given, it will be used as the tolerance for all objectives	None
threshold_evaluation_pruning	list[start, end]	starting and ending percentile to use as a threshold for the evaluation early stopping. Values between 0 and 100.	None
threshold_evaluation_scaling	float [0,inf)	A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile. Must be greater than zero. Higher numbers will move the threshold to the end faster.	0.5
selection_evaluation_pruning	list	A lower and upper percent of the population size to select each round of CV. Values between 0 and 1.	None
selection_evaluation_scaling	float	A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile. Must be greater than zero. Higher numbers will move the threshold to the end faster.	0.5
min_history_threshold	int	The minimum number of previous scores needed before using threshold early stopping.	0
survival_percentage	float	Percentage of the population size to utilize for mutation and crossover at the beginning of the generation. The rest are discarded. Individuals are selected with the selector passed into survival_selector. The value of this parameter must be between 0 and 1, inclusive. For example, if the population size is 100 and the survival percentage is .5, 50 individuals will be selected with NSGA2 from the existing population. These will be used for mutation and crossover to generate the next 100 individuals for the next generation. The remainder are discarded from the live population. In the next generation, there will now be the 50 parents + the 100 individuals for a total of 150. Surivival percentage is based of the population size parameter and not the existing population size (current population size when using successive halving). Therefore, in the next generation we will still select 50 individuals from the currently existing 150.	1
crossover_probability	float	Probability of generating a new individual by crossover between two individuals.	.2
mutate_probability	float	Probability of generating a new individual by crossover between one individuals.	.7
mutate_then_crossover_probability	float	Probability of generating a new individual by mutating two individuals followed by crossover.	.05
crossover_then_mutate_probability	float	Probability of generating a new individual by crossover between two individuals followed by a mutation of the resulting individual.	.05
survival_selector	function	Function to use to select individuals for survival. Must take a matrix of scores and return selected indexes. Used to selected population_size * survival_percentage individuals at the start of each generation to use for mutation and crossover.	survival_select_NSGA2
parent_selector	function	Function to use to select pairs parents for crossover and individuals for mutation. Must take a matrix of scores and return selected indexes.	parent_select_NSGA2
budget_range	list[start, end]	A starting and ending budget to use for the budget scaling.	None
budget_scaling		A scaling factor to use when determining how fast we move the budget from the start to end budget.	0.5
generations_until_end_budget	int	The number of generations to run before reaching the max budget.	1
stepwise_steps	int	The number of staircase steps to take when scaling the budget and population size.	1
n_jobs	int	Number of processes to run in parallel.	1
memory_limit	str	Memory limit for each job. See Dask LocalCluster documentation for more information.	None
client	Client	A dask client to use for parallelization. If not None, this will override the n_jobs and memory_limit parameters. If None, will create a new client with num_workers=n_jobs and memory_limit=memory_limit.	None
processes	bool	If True, will use multiprocessing to parallelize the optimization process. If False, will use threading. True seems to perform better. However, False is required for interactive debugging.	True
warm_start	bool	If True, will use the continue the evolutionary algorithm from the last generation of the previous run.	False
periodic_checkpoint_folder	str	Folder to save the population to periodically. If None, no periodic saving will be done. If provided, training will resume from this checkpoint.	None
callback	CallBackInterface	Callback object. Not implemented	None
verbose	int	How much information to print during the optimization process. Higher values include the information from lower values. 0. nothing 1. progress bar best individual warnings =5. full warnings trace evaluations progress bar. (Temporary: This used to be 2. Currently, using evaluation progress bar may prevent some instances were we terminate a generation early due to it reaching max_time_mins in the middle of a generation OR a pipeline failed to be terminated normally and we need to manually terminate it.)	1
scatter	bool	If True, will scatter the data to the dask workers. If False, will not scatter the data. This can be useful for debugging.	True
random_state	(int, None)	A seed for reproducability of experiments. This value will be passed to numpy.random.default_rng() to create an instnce of the genrator to pass to other classes int Will be used to create and lock in Generator instance with 'numpy.random.default_rng()' None Will be used to create Generator for 'numpy.random.default_rng()' where a fresh, unpredictable entropy will be pulled from the OS	None

Attributes:

Name	Type	Description
fitted_pipeline_	GraphPipeline	A fitted instance of the GraphPipeline that inherits from sklearn BaseEstimator. This is fitted on the full X, y passed to fit.
evaluated_individuals	A pandas data frame containing data for all evaluated individuals in the run.	Columns: - objective functions : The first few columns correspond to the passed in scorers and objective functions - Parents : A tuple containing the indexes of the pipelines used to generate the pipeline of that row. If NaN, this pipeline was generated randomly in the initial population. - Variation_Function : Which variation function was used to mutate or crossover the parents. If NaN, this pipeline was generated randomly in the initial population. - Individual : The internal representation of the individual that is used during the evolutionary algorithm. This is not an sklearn BaseEstimator. - Generation : The generation the pipeline first appeared. - Pareto_Front : The nondominated front that this pipeline belongs to. 0 means that its scores is not strictly dominated by any other individual. To save on computational time, the best frontier is updated iteratively each generation. The pipelines with the 0th pareto front do represent the exact best frontier. However, the pipelines with pareto front >= 1 are only in reference to the other pipelines in the final population. All other pipelines are set to NaN. - Instance : The unfitted GraphPipeline BaseEstimator. - validation objective functions : Objective function scores evaluated on the validation set. - Validation_Pareto_Front : The full pareto front calculated on the validation set. This is calculated for all pipelines with Pareto_Front equal to 0. Unlike the Pareto_Front which only calculates the frontier and the final population, the Validation Pareto Front is calculated for all pipelines tested on the validation set.
pareto_front	The same pandas dataframe as evaluated individuals, but containing only the frontier pareto front pipelines.

Source code in tpot2/tpot_estimator/estimator.py

def __init__(self,  
                    search_space,
                    scorers,
                    scorers_weights,
                    classification,
                    cv = 10,
                    other_objective_functions=[],
                    other_objective_functions_weights = [],
                    objective_function_names = None,
                    bigger_is_better = True,

                    export_graphpipeline = False,
                    memory = None,

                    categorical_features = None,
                    preprocessing = False,
                    population_size = 50,
                    initial_population_size = None,
                    population_scaling = .5,
                    generations_until_end_population = 1,
                    generations = None,
                    max_time_mins=60,
                    max_eval_time_mins=10,
                    validation_strategy = "none",
                    validation_fraction = .2,
                    disable_label_encoder = False,

                    #early stopping parameters
                    early_stop = None,
                    scorers_early_stop_tol = 0.001,
                    other_objectives_early_stop_tol =None,
                    threshold_evaluation_pruning = None,
                    threshold_evaluation_scaling = .5,
                    selection_evaluation_pruning = None,
                    selection_evaluation_scaling = .5,
                    min_history_threshold = 20,

                    #evolver parameters
                    survival_percentage = 1,
                    crossover_probability=.2,
                    mutate_probability=.7,
                    mutate_then_crossover_probability=.05,
                    crossover_then_mutate_probability=.05,
                    survival_selector = survival_select_NSGA2,
                    parent_selector = tournament_selection_dominated,

                    #budget parameters
                    budget_range = None,
                    budget_scaling = .5,
                    generations_until_end_budget = 1,
                    stepwise_steps = 5,

                    #dask parameters
                    n_jobs=1,
                    memory_limit = None,
                    client = None,
                    processes = True,

                    #debugging and logging parameters
                    warm_start = False,
                    periodic_checkpoint_folder = None,
                    callback = None,

                    verbose = 0,
                    scatter = True,

                     # random seed for random number generator (rng)
                    random_state = None,

                    ):

    '''
    An sklearn baseestimator that uses genetic programming to optimize a pipeline.

    Parameters
    ----------
    search_space : (String, tpot2.search_spaces.SearchSpace)
        - String : The default search space to use for the optimization.
        | String     | Description      |
        | :---        |    :----:   |
        | linear  | A linear pipeline with the structure of "Selector->(transformers+Passthrough)->(classifiers/regressors+Passthrough)->final classifier/regressor." For both the transformer and inner estimator layers, TPOT may choose one or more transformers/classifiers, or it may choose none. The inner classifier/regressor layer is optional. |
        | linear-light | Same search space as linear, but without the inner classifier/regressor layer and with a reduced set of faster running estimators. |
        | graph | TPOT will optimize a pipeline in the shape of a directed acyclic graph. The nodes of the graph can include selectors, scalers, transformers, or classifiers/regressors (inner classifiers/regressors can optionally be not included). This will return a custom GraphPipeline rather than an sklearn Pipeline. More details in Tutorial 6. |
        | graph-light | Same as graph search space, but without the inner classifier/regressors and with a reduced set of faster running estimators. |
        | mdr |TPOT will search over a series of feature selectors and Multifactor Dimensionality Reduction models to find a series of operators that maximize prediction accuracy. The TPOT MDR configuration is specialized for genome-wide association studies (GWAS), and is described in detail online here.

        Note that TPOT MDR may be slow to run because the feature selection routines are computationally expensive, especially on large datasets. |


        - SearchSpace : The search space to use for the optimization. This should be an instance of a SearchSpace.
            The search space to use for the optimization. This should be an instance of a SearchSpace.
            TPOT2 has groups of search spaces found in the following folders, tpot2.search_spaces.nodes for the nodes in the pipeline and tpot2.search_spaces.pipelines for the pipeline structure.

    scorers : (list, scorer)
        A scorer or list of scorers to be used in the cross-validation process.
        see https://scikit-learn.org/stable/modules/model_evaluation.html

    scorers_weights : list
        A list of weights to be applied to the scorers during the optimization process.

    classification : bool
        If True, the problem is treated as a classification problem. If False, the problem is treated as a regression problem.
        Used to determine the CV strategy.

    cv : int, cross-validator
        - (int): Number of folds to use in the cross-validation process. By uses the sklearn.model_selection.KFold cross-validator for regression and StratifiedKFold for classification. In both cases, shuffled is set to True.
        - (sklearn.model_selection.BaseCrossValidator): A cross-validator to use in the cross-validation process.
            - max_depth (int): The maximum depth from any node to the root of the pipelines to be generated.

    other_objective_functions : list, default=[]
        A list of other objective functions to apply to the pipeline. The function takes a single parameter for the graphpipeline estimator and returns either a single score or a list of scores.

    other_objective_functions_weights : list, default=[]
        A list of weights to be applied to the other objective functions.

    objective_function_names : list, default=None
        A list of names to be applied to the objective functions. If None, will use the names of the objective functions.

    bigger_is_better : bool, default=True
        If True, the objective function is maximized. If False, the objective function is minimized. Use negative weights to reverse the direction.

    memory: Memory object or string, default=None
        If supplied, pipeline will cache each transformer after calling fit with joblib.Memory. This feature
        is used to avoid computing the fit transformers within a pipeline if the parameters
        and input data are identical with another fitted pipeline during optimization process.
        - String 'auto':
            TPOT uses memory caching with a temporary directory and cleans it up upon shutdown.
        - String path of a caching directory
            TPOT uses memory caching with the provided directory and TPOT does NOT clean
            the caching directory up upon shutdown. If the directory does not exist, TPOT will
            create it.
        - Memory object:
            TPOT uses the instance of joblib.Memory for memory caching,
            and TPOT does NOT clean the caching directory up upon shutdown.
        - None:
            TPOT does not use memory caching.              

    categorical_features: list or None
        Categorical columns to inpute and/or one hot encode during the preprocessing step. Used only if preprocessing is not False.
        - None : If None, TPOT2 will automatically use object columns in pandas dataframes as objects for one hot encoding in preprocessing.
        - List of categorical features. If X is a dataframe, this should be a list of column names. If X is a numpy array, this should be a list of column indices

    preprocessing : bool or BaseEstimator/Pipeline,
        EXPERIMENTAL - will be changed in future versions
        A pipeline that will be used to preprocess the data before CV. Note that the parameters for these steps are not optimized. Add them to the search space to be optimized.
        - bool : If True, will use a default preprocessing pipeline which includes imputation followed by one hot encoding.
        - Pipeline : If an instance of a pipeline is given, will use that pipeline as the preprocessing pipeline.

    population_size : int, default=50
        Size of the population

    initial_population_size : int, default=None
        Size of the initial population. If None, population_size will be used.

    population_scaling : int, default=0.5
        Scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.

    generations_until_end_population : int, default=1
        Number of generations until the population size reaches population_size

    generations : int, default=50
        Number of generations to run

    max_time_mins : float, default=float("inf")
        Maximum time to run the optimization. If none or inf, will run until the end of the generations.

    max_eval_time_mins : float, default=5
        Maximum time to evaluate a single individual. If none or inf, there will be no time limit per evaluation.

    validation_strategy : str, default='none'
        EXPERIMENTAL The validation strategy to use for selecting the final pipeline from the population. TPOT2 may overfit the cross validation score. A second validation set can be used to select the final pipeline.
        - 'auto' : Automatically determine the validation strategy based on the dataset shape.
        - 'reshuffled' : Use the same data for cross validation and final validation, but with different splits for the folds. This is the default for small datasets.
        - 'split' : Use a separate validation set for final validation. Data will be split according to validation_fraction. This is the default for medium datasets.
        - 'none' : Do not use a separate validation set for final validation. Select based on the original cross-validation score. This is the default for large datasets.

    validation_fraction : float, default=0.2
      EXPERIMENTAL The fraction of the dataset to use for the validation set when validation_strategy is 'split'. Must be between 0 and 1.

    disable_label_encoder : bool, default=False
        If True, TPOT will check if the target needs to be relabeled to be sequential ints from 0 to N. This is necessary for XGBoost compatibility. If the labels need to be encoded, TPOT2 will use sklearn.preprocessing.LabelEncoder to encode the labels. The encoder can be accessed via the self.label_encoder_ attribute.
        If False, no additional label encoders will be used.

    early_stop : int, default=None
        Number of generations without improvement before early stopping. All objectives must have converged within the tolerance for this to be triggered. In general a value of around 5-20 is good.

    scorers_early_stop_tol :
        -list of floats
            list of tolerances for each scorer. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged
            If an index of the list is None, that item will not be used for early stopping
        -int
            If an int is given, it will be used as the tolerance for all objectives

    other_objectives_early_stop_tol :
        -list of floats
            list of tolerances for each of the other objective function. If the difference between the best score and the current score is less than the tolerance, the individual is considered to have converged
            If an index of the list is None, that item will not be used for early stopping
        -int
            If an int is given, it will be used as the tolerance for all objectives

    threshold_evaluation_pruning : list [start, end], default=None
        starting and ending percentile to use as a threshold for the evaluation early stopping.
        Values between 0 and 100.

    threshold_evaluation_scaling : float [0,inf), default=0.5
        A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.
        Must be greater than zero. Higher numbers will move the threshold to the end faster.

    selection_evaluation_pruning : list, default=None
        A lower and upper percent of the population size to select each round of CV.
        Values between 0 and 1.

    selection_evaluation_scaling : float, default=0.5
        A scaling factor to use when determining how fast we move the threshold moves from the start to end percentile.
        Must be greater than zero. Higher numbers will move the threshold to the end faster.

    min_history_threshold : int, default=0
        The minimum number of previous scores needed before using threshold early stopping.

    survival_percentage : float, default=1
        Percentage of the population size to utilize for mutation and crossover at the beginning of the generation. The rest are discarded. Individuals are selected with the selector passed into survival_selector. The value of this parameter must be between 0 and 1, inclusive.
        For example, if the population size is 100 and the survival percentage is .5, 50 individuals will be selected with NSGA2 from the existing population. These will be used for mutation and crossover to generate the next 100 individuals for the next generation. The remainder are discarded from the live population. In the next generation, there will now be the 50 parents + the 100 individuals for a total of 150. Surivival percentage is based of the population size parameter and not the existing population size (current population size when using successive halving). Therefore, in the next generation we will still select 50 individuals from the currently existing 150.

    crossover_probability : float, default=.2
        Probability of generating a new individual by crossover between two individuals.

    mutate_probability : float, default=.7
        Probability of generating a new individual by crossover between one individuals.

    mutate_then_crossover_probability : float, default=.05
        Probability of generating a new individual by mutating two individuals followed by crossover.

    crossover_then_mutate_probability : float, default=.05
        Probability of generating a new individual by crossover between two individuals followed by a mutation of the resulting individual.

    survival_selector : function, default=survival_select_NSGA2
        Function to use to select individuals for survival. Must take a matrix of scores and return selected indexes.
        Used to selected population_size * survival_percentage individuals at the start of each generation to use for mutation and crossover.

    parent_selector : function, default=parent_select_NSGA2
        Function to use to select pairs parents for crossover and individuals for mutation. Must take a matrix of scores and return selected indexes.

    budget_range : list [start, end], default=None
        A starting and ending budget to use for the budget scaling.

    budget_scaling float : [0,1], default=0.5
        A scaling factor to use when determining how fast we move the budget from the start to end budget.

    generations_until_end_budget : int, default=1
        The number of generations to run before reaching the max budget.

    stepwise_steps : int, default=1
        The number of staircase steps to take when scaling the budget and population size.

    n_jobs : int, default=1
        Number of processes to run in parallel.

    memory_limit : str, default=None
        Memory limit for each job. See Dask [LocalCluster documentation](https://distributed.dask.org/en/stable/api.html#distributed.Client) for more information.

    client : dask.distributed.Client, default=None
        A dask client to use for parallelization. If not None, this will override the n_jobs and memory_limit parameters. If None, will create a new client with num_workers=n_jobs and memory_limit=memory_limit.

    processes : bool, default=True
        If True, will use multiprocessing to parallelize the optimization process. If False, will use threading.
        True seems to perform better. However, False is required for interactive debugging.

    warm_start : bool, default=False
        If True, will use the continue the evolutionary algorithm from the last generation of the previous run.

    periodic_checkpoint_folder : str, default=None
        Folder to save the population to periodically. If None, no periodic saving will be done.
        If provided, training will resume from this checkpoint.

    callback : tpot2.CallBackInterface, default=None
        Callback object. Not implemented

    verbose : int, default=1
        How much information to print during the optimization process. Higher values include the information from lower values.
        0. nothing
        1. progress bar

        3. best individual
        4. warnings
        >=5. full warnings trace
        6. evaluations progress bar. (Temporary: This used to be 2. Currently, using evaluation progress bar may prevent some instances were we terminate a generation early due to it reaching max_time_mins in the middle of a generation OR a pipeline failed to be terminated normally and we need to manually terminate it.)

    scatter : bool, default=True
        If True, will scatter the data to the dask workers. If False, will not scatter the data. This can be useful for debugging.

    random_state : int, None, default=None
        A seed for reproducability of experiments. This value will be passed to numpy.random.default_rng() to create an instnce of the genrator to pass to other classes

        - int
            Will be used to create and lock in Generator instance with 'numpy.random.default_rng()'
        - None
            Will be used to create Generator for 'numpy.random.default_rng()' where a fresh, unpredictable entropy will be pulled from the OS

    Attributes
    ----------

    fitted_pipeline_ : GraphPipeline
        A fitted instance of the GraphPipeline that inherits from sklearn BaseEstimator. This is fitted on the full X, y passed to fit.

    evaluated_individuals : A pandas data frame containing data for all evaluated individuals in the run.
        Columns:
        - *objective functions : The first few columns correspond to the passed in scorers and objective functions
        - Parents : A tuple containing the indexes of the pipelines used to generate the pipeline of that row. If NaN, this pipeline was generated randomly in the initial population.
        - Variation_Function : Which variation function was used to mutate or crossover the parents. If NaN, this pipeline was generated randomly in the initial population.
        - Individual : The internal representation of the individual that is used during the evolutionary algorithm. This is not an sklearn BaseEstimator.
        - Generation : The generation the pipeline first appeared.
        - Pareto_Front	: The nondominated front that this pipeline belongs to. 0 means that its scores is not strictly dominated by any other individual.
                        To save on computational time, the best frontier is updated iteratively each generation.
                        The pipelines with the 0th pareto front do represent the exact best frontier. However, the pipelines with pareto front >= 1 are only in reference to the other pipelines in the final population.
                        All other pipelines are set to NaN.
        - Instance	: The unfitted GraphPipeline BaseEstimator.
        - *validation objective functions : Objective function scores evaluated on the validation set.
        - Validation_Pareto_Front : The full pareto front calculated on the validation set. This is calculated for all pipelines with Pareto_Front equal to 0. Unlike the Pareto_Front which only calculates the frontier and the final population, the Validation Pareto Front is calculated for all pipelines tested on the validation set.

    pareto_front : The same pandas dataframe as evaluated individuals, but containing only the frontier pareto front pipelines.
    '''

    # sklearn BaseEstimator must have a corresponding attribute for each parameter.
    # These should not be modified once set.

    self.scorers = scorers
    self.scorers_weights = scorers_weights
    self.classification = classification
    self.cv = cv
    self.other_objective_functions = other_objective_functions
    self.other_objective_functions_weights = other_objective_functions_weights
    self.objective_function_names = objective_function_names
    self.bigger_is_better = bigger_is_better

    self.search_space = search_space

    self.export_graphpipeline = export_graphpipeline
    self.memory = memory

    self.categorical_features = categorical_features

    self.preprocessing = preprocessing
    self.validation_strategy = validation_strategy
    self.validation_fraction = validation_fraction
    self.disable_label_encoder = disable_label_encoder
    self.population_size = population_size
    self.initial_population_size = initial_population_size
    self.population_scaling = population_scaling
    self.generations_until_end_population = generations_until_end_population
    self.generations = generations
    self.early_stop = early_stop
    self.scorers_early_stop_tol = scorers_early_stop_tol
    self.other_objectives_early_stop_tol = other_objectives_early_stop_tol
    self.max_time_mins = max_time_mins
    self.max_eval_time_mins = max_eval_time_mins
    self.n_jobs= n_jobs
    self.memory_limit = memory_limit
    self.client = client
    self.survival_percentage = survival_percentage
    self.crossover_probability = crossover_probability
    self.mutate_probability = mutate_probability
    self.mutate_then_crossover_probability= mutate_then_crossover_probability
    self.crossover_then_mutate_probability= crossover_then_mutate_probability
    self.survival_selector=survival_selector
    self.parent_selector=parent_selector
    self.budget_range = budget_range
    self.budget_scaling = budget_scaling
    self.generations_until_end_budget = generations_until_end_budget
    self.stepwise_steps = stepwise_steps
    self.threshold_evaluation_pruning =threshold_evaluation_pruning
    self.threshold_evaluation_scaling =  threshold_evaluation_scaling
    self.min_history_threshold = min_history_threshold
    self.selection_evaluation_pruning = selection_evaluation_pruning
    self.selection_evaluation_scaling =  selection_evaluation_scaling
    self.warm_start = warm_start
    self.verbose = verbose
    self.periodic_checkpoint_folder = periodic_checkpoint_folder
    self.callback = callback
    self.processes = processes


    self.scatter = scatter


    timer_set = self.max_time_mins != float("inf") and self.max_time_mins is not None
    if self.generations is not None and timer_set:
        warnings.warn("Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.")

    # create random number generator based on rngseed
    self.rng = np.random.default_rng(random_state)
    # save random state passed to us for other functions that use random_state
    self.random_state = random_state

    #Initialize other used params


    if self.initial_population_size is None:
        self._initial_population_size = self.population_size
    else:
        self._initial_population_size = self.initial_population_size

    if isinstance(self.scorers, str):
        self._scorers = [self.scorers]

    elif callable(self.scorers):
        self._scorers = [self.scorers]
    else:
        self._scorers = self.scorers

    self._scorers = [sklearn.metrics.get_scorer(scoring) for scoring in self._scorers]
    self._scorers_early_stop_tol = self.scorers_early_stop_tol

    self._evolver = tpot2.evolvers.BaseEvolver

    self.objective_function_weights = [*scorers_weights, *other_objective_functions_weights]


    if self.objective_function_names is None:
        obj_names = [f.__name__ for f in other_objective_functions]
    else:
        obj_names = self.objective_function_names
    self.objective_names = [f._score_func.__name__ if hasattr(f,"_score_func") else f.__name__ for f in self._scorers] + obj_names


    if not isinstance(self.other_objectives_early_stop_tol, list):
        self._other_objectives_early_stop_tol = [self.other_objectives_early_stop_tol for _ in range(len(self.other_objective_functions))]
    else:
        self._other_objectives_early_stop_tol = self.other_objectives_early_stop_tol

    if not isinstance(self._scorers_early_stop_tol, list):
        self._scorers_early_stop_tol = [self._scorers_early_stop_tol for _ in range(len(self._scorers))]
    else:
        self._scorers_early_stop_tol = self._scorers_early_stop_tol

    self.early_stop_tol = [*self._scorers_early_stop_tol, *self._other_objectives_early_stop_tol]

    self._evolver_instance = None
    self.evaluated_individuals = None


    self.label_encoder_ = None


    set_dask_settings()

apply_make_pipeline(ind, preprocessing_pipeline=None, export_graphpipeline=False, **pipeline_kwargs)

Helper function to create a column of sklearn pipelines from the tpot2 individual class.

Parameters:

Name	Type	Description	Default
ind		The individual to convert to a pipeline.	required
preprocessing_pipeline		The preprocessing pipeline to include before the individual's pipeline.	None
export_graphpipeline		Force the pipeline to be exported as a graph pipeline. Flattens all nested pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.	False
pipeline_kwargs		Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.	{}

Returns:

Type	Description
sklearn estimator

Source code in tpot2/tpot_estimator/estimator_utils.py

def apply_make_pipeline(ind, preprocessing_pipeline=None, export_graphpipeline=False, **pipeline_kwargs):
    """
    Helper function to create a column of sklearn pipelines from the tpot2 individual class.

    Parameters
    ----------
    ind: tpot2.SklearnIndividual
        The individual to convert to a pipeline.
    preprocessing_pipeline: sklearn.pipeline.Pipeline, optional
        The preprocessing pipeline to include before the individual's pipeline.
    export_graphpipeline: bool, default=False
        Force the pipeline to be exported as a graph pipeline. Flattens all nested pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.
    pipeline_kwargs: dict
        Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.

    Returns
    -------
    sklearn estimator
    """

    try:

        if export_graphpipeline:
            est = ind.export_flattened_graphpipeline(**pipeline_kwargs)
        else:
            est = ind.export_pipeline(**pipeline_kwargs)


        if preprocessing_pipeline is None:
            return est
        else:
            return sklearn.pipeline.make_pipeline(sklearn.base.clone(preprocessing_pipeline), est)
    except:
        return None

check_empty_values(data)

Checks for empty values in a dataset.

Args: data (numpy.ndarray or pandas.DataFrame): The dataset to check.

Returns: bool: True if the dataset contains empty values, False otherwise.

Source code in tpot2/tpot_estimator/estimator.py

def check_empty_values(data):
    """
    Checks for empty values in a dataset.

    Args:
        data (numpy.ndarray or pandas.DataFrame): The dataset to check.

    Returns:
        bool: True if the dataset contains empty values, False otherwise.
    """
    if isinstance(data, pd.DataFrame):
        return data.isnull().values.any()
    elif isinstance(data, np.ndarray):
        return np.isnan(data).any()
    else:
        raise ValueError("Unsupported data type")

check_if_y_is_encoded(y)

Checks if the target y is composed of sequential ints from 0 to N. XGBoost requires the target to be encoded in this way.

Parameters:

Name	Type	Description	Default
y		The target vector.	required

Returns:

Type	Description
bool	True if the target is encoded as sequential ints from 0 to N, False otherwise

Source code in tpot2/tpot_estimator/estimator_utils.py

def check_if_y_is_encoded(y):
    '''
    Checks if the target y is composed of sequential ints from 0 to N.
    XGBoost requires the target to be encoded in this way.

    Parameters
    ----------
    y: np.ndarray
        The target vector.

    Returns
    -------
    bool
        True if the target is encoded as sequential ints from 0 to N, False otherwise
    '''
    y = sorted(set(y))
    return all(i == j for i, j in enumerate(y))

convert_parents_tuples_to_integers(row, object_to_int)

Helper function to convert the parent rows into integers representing the index of the parent in the population.

Original pandas dataframe using a custom index for the parents. This function converts the custom index to an integer index for easier manipulation by end users.

Parameters:

Name	Type	Description	Default
row		The row to convert.	required
object_to_int		A dictionary mapping the object to an integer index.	required

Returns

tuple The row with the custom index converted to an integer index.

Source code in tpot2/tpot_estimator/estimator_utils.py

def convert_parents_tuples_to_integers(row, object_to_int):
    """
    Helper function to convert the parent rows into integers representing the index of the parent in the population.

    Original pandas dataframe using a custom index for the parents. This function converts the custom index to an integer index for easier manipulation by end users.

    Parameters
    ----------
    row: list, np.ndarray, tuple
        The row to convert.
    object_to_int: dict
        A dictionary mapping the object to an integer index.

    Returns 
    -------
    tuple
        The row with the custom index converted to an integer index.
    """
    if type(row) == list or type(row) == np.ndarray or type(row) == tuple:
        return tuple(object_to_int[obj] for obj in row)
    else:
        return np.nan

cross_val_score_objective(estimator, X, y, scorers, cv, fold=None)

Compute the cross validated scores for a estimator. Only fits the estimator once per fold, and loops over the scorers to evaluate the estimator.

Parameters:

Name	Type	Description	Default
estimator		The estimator to fit and score.	required
X		The feature matrix.	required
y		The target vector.	required
scorers		The scorers to use. If a list, will loop over the scorers and return a list of scorers. If a single scorer, will return a single score.	required
cv		The cross-validator to use. For example, sklearn.model_selection.KFold or sklearn.model_selection.StratifiedKFold.	required
fold		The fold to return the scores for. If None, will return the mean of all the scores (per scorer). Default is None.	None

Returns:

Name	Type	Description
scores	ndarray or float	The scores for the estimator per scorer. If fold is None, will return the mean of all the scores (per scorer). Returns a list if multiple scorers are used, otherwise returns a float for the single scorer.

Source code in tpot2/tpot_estimator/cross_val_utils.py

def cross_val_score_objective(estimator, X, y, scorers, cv, fold=None):
    """
    Compute the cross validated scores for a estimator. Only fits the estimator once per fold, and loops over the scorers to evaluate the estimator.

    Parameters
    ----------
    estimator: sklearn.base.BaseEstimator
        The estimator to fit and score.
    X: np.ndarray or pd.DataFrame
        The feature matrix.
    y: np.ndarray or pd.Series
        The target vector.
    scorers: list or scorer
        The scorers to use. 
        If a list, will loop over the scorers and return a list of scorers.
        If a single scorer, will return a single score.
    cv: sklearn cross-validator
        The cross-validator to use. For example, sklearn.model_selection.KFold or sklearn.model_selection.StratifiedKFold.
    fold: int, optional
        The fold to return the scores for. If None, will return the mean of all the scores (per scorer). Default is None.

    Returns
    -------
    scores: np.ndarray or float
        The scores for the estimator per scorer. If fold is None, will return the mean of all the scores (per scorer).
        Returns a list if multiple scorers are used, otherwise returns a float for the single scorer.

    """

    #check if scores is not iterable
    if not isinstance(scorers, Iterable): 
        scorers = [scorers]
    scores = []
    if fold is None:
        for train_index, test_index in cv.split(X, y):
            this_fold_estimator = sklearn.base.clone(estimator)
            if isinstance(X, pd.DataFrame) or isinstance(X, pd.Series):
                X_train, X_test = X.iloc[train_index], X.iloc[test_index]
            else:
                X_train, X_test = X[train_index], X[test_index]

            if isinstance(y, pd.DataFrame) or isinstance(y, pd.Series):
                y_train, y_test = y.iloc[train_index], y.iloc[test_index]
            else:
                y_train, y_test = y[train_index], y[test_index]


            start = time.time()
            this_fold_estimator.fit(X_train,y_train)
            duration = time.time() - start

            this_fold_scores = [sklearn.metrics.get_scorer(scorer)(this_fold_estimator, X_test, y_test) for scorer in scorers] 
            scores.append(this_fold_scores)
            del this_fold_estimator
            del X_train
            del X_test
            del y_train
            del y_test


        return np.mean(scores,0)
    else:
        this_fold_estimator = sklearn.base.clone(estimator)
        train_index, test_index = list(cv.split(X, y))[fold]
        if isinstance(X, pd.DataFrame) or isinstance(X, pd.Series):
            X_train, X_test = X.iloc[train_index], X.iloc[test_index]
        else:
            X_train, X_test = X[train_index], X[test_index]

        if isinstance(y, pd.DataFrame) or isinstance(y, pd.Series):
            y_train, y_test = y.iloc[train_index], y.iloc[test_index]
        else:
            y_train, y_test = y[train_index], y[test_index]

        start = time.time()
        this_fold_estimator.fit(X_train,y_train)
        duration = time.time() - start
        this_fold_scores = [sklearn.metrics.get_scorer(scorer)(this_fold_estimator, X_test, y_test) for scorer in scorers] 
        return this_fold_scores

objective_function_generator(pipeline, x, y, scorers, cv, other_objective_functions, step=None, budget=None, is_classification=True, export_graphpipeline=False, **pipeline_kwargs)

Uses cross validation to evaluate the pipeline using the scorers, and concatenates results with scores from standalone other objective functions.

Parameters:

Name	Type	Description	Default
pipeline		The individual to evaluate.	required
x		The feature matrix.	required
y		The target vector.	required
scorers		The scorers to use for cross validation.	required
cv		The cross-validator to use. For example, sklearn.model_selection.KFold or sklearn.model_selection.StratifiedKFold. If an int, will use sklearn.model_selection.KFold with n_splits=cv.	required
other_objective_functions		A list of standalone objective functions to evaluate the pipeline. With signature obj(pipeline) -> float. or obj(pipeline) -> np.ndarray These functions take in the unfitted estimator.	required
step		The fold to return the scores for. If None, will return the mean of all the scores (per scorer). Default is None.	None
budget		The budget to subsample the data. If None, will use the full dataset. Default is None. Will subsample budget*len(x) samples.	None
is_classification		If True, will stratify the subsampling. Default is True.	True
export_graphpipeline		Force the pipeline to be exported as a graph pipeline. Flattens all nested sklearn pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.	False
pipeline_kwargs		Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.	{}

Returns:

Type	Description
ndarray	The concatenated scores for the pipeline. The first len(scorers) elements are the cross validation scores, and the remaining elements are the standalone objective functions.

Source code in tpot2/tpot_estimator/estimator_utils.py

def objective_function_generator(pipeline, x,y, scorers, cv, other_objective_functions, step=None, budget=None, is_classification=True, export_graphpipeline=False, **pipeline_kwargs):
    """
    Uses cross validation to evaluate the pipeline using the scorers, and concatenates results with scores from standalone other objective functions.

    Parameters
    ----------
    pipeline: tpot2.SklearnIndividual
        The individual to evaluate.
    x: np.ndarray
        The feature matrix.
    y: np.ndarray
        The target vector.
    scorers: list
        The scorers to use for cross validation. 
    cv: int, float, or sklearn cross-validator
        The cross-validator to use. For example, sklearn.model_selection.KFold or sklearn.model_selection.StratifiedKFold.
        If an int, will use sklearn.model_selection.KFold with n_splits=cv.
    other_objective_functions: list
        A list of standalone objective functions to evaluate the pipeline. With signature obj(pipeline) -> float. or obj(pipeline) -> np.ndarray
        These functions take in the unfitted estimator.
    step: int, optional
        The fold to return the scores for. If None, will return the mean of all the scores (per scorer). Default is None.
    budget: float, optional
        The budget to subsample the data. If None, will use the full dataset. Default is None.
        Will subsample budget*len(x) samples.
    is_classification: bool, default=True
        If True, will stratify the subsampling. Default is True.
    export_graphpipeline: bool, default=False
        Force the pipeline to be exported as a graph pipeline. Flattens all nested sklearn pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.
    pipeline_kwargs: dict
        Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.

    Returns
    -------
    np.ndarray
        The concatenated scores for the pipeline. The first len(scorers) elements are the cross validation scores, and the remaining elements are the standalone objective functions.

    """

    if export_graphpipeline:
        pipeline = pipeline.export_flattened_graphpipeline(**pipeline_kwargs)
    else:
        pipeline = pipeline.export_pipeline(**pipeline_kwargs)

    if budget is not None and budget < 1:
        if is_classification:
            x,y = sklearn.utils.resample(x,y, stratify=y, n_samples=int(budget*len(x)), replace=False, random_state=1)
        else:
            x,y = sklearn.utils.resample(x,y, n_samples=int(budget*len(x)), replace=False, random_state=1)

        if isinstance(cv, int) or isinstance(cv, float):
            n_splits = cv
        else:
            n_splits = cv.n_splits

    if len(scorers) > 0:
        cv_obj_scores = cross_val_score_objective(sklearn.base.clone(pipeline),x,y,scorers=scorers, cv=cv , fold=step)
    else:
        cv_obj_scores = []

    if other_objective_functions is not None and len(other_objective_functions) >0:
        other_scores = [obj(sklearn.base.clone(pipeline)) for obj in other_objective_functions]
        #flatten
        other_scores = np.array(other_scores).flatten().tolist()
    else:
        other_scores = []

    return np.concatenate([cv_obj_scores,other_scores])

remove_underrepresented_classes(x, y, min_count)

Helper function to remove classes with less than min_count samples from the dataset.

Parameters:

Name	Type	Description	Default
x		The feature matrix.	required
y		The target vector.	required
min_count		The minimum number of samples to keep a class.	required

Returns:

Type	Description
(ndarray, ndarray)	The feature matrix and target vector with rows from classes with less than min_count samples removed.

Source code in tpot2/tpot_estimator/estimator_utils.py

def remove_underrepresented_classes(x, y, min_count):
    """
    Helper function to remove classes with less than min_count samples from the dataset.

    Parameters
    ----------
    x: np.ndarray or pd.DataFrame
        The feature matrix.
    y: np.ndarray or pd.Series
        The target vector.
    min_count: int
        The minimum number of samples to keep a class.

    Returns
    -------
    np.ndarray, np.ndarray
        The feature matrix and target vector with rows from classes with less than min_count samples removed.
    """
    if isinstance(y, (np.ndarray, pd.Series)):
        unique, counts = np.unique(y, return_counts=True)
        if min(counts) >= min_count:
            return x, y
        keep_classes = unique[counts >= min_count]
        mask = np.isin(y, keep_classes)
        x = x[mask]
        y = y[mask]
    elif isinstance(y, pd.DataFrame):
        counts = y.apply(pd.Series.value_counts)
        if min(counts) >= min_count:
            return x, y
        keep_classes = counts.index[counts >= min_count].tolist()
        mask = y.isin(keep_classes).all(axis=1)
        x = x[mask]
        y = y[mask]
    else:
        raise TypeError("y must be a numpy array or a pandas Series/DataFrame")
    return x, y

val_objective_function_generator(pipeline, X_train, y_train, X_test, y_test, scorers, other_objective_functions, export_graphpipeline=False, **pipeline_kwargs)

Trains a pipeline on a training set and evaluates it on a test set using the scorers and other objective functions.

Parameters:

Name	Type	Description	Default
pipeline		The individual to evaluate.	required
X_train		The feature matrix of the training set.	required
y_train		The target vector of the training set.	required
X_test		The feature matrix of the test set.	required
y_test		The target vector of the test set.	required
scorers		The scorers to use for cross validation.	required
other_objective_functions		A list of standalone objective functions to evaluate the pipeline. With signature obj(pipeline) -> float. or obj(pipeline) -> np.ndarray These functions take in the unfitted estimator.	required
export_graphpipeline		Force the pipeline to be exported as a graph pipeline. Flattens all nested sklearn pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.	False
pipeline_kwargs		Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.	{}

Returns:

Type	Description
ndarray	The concatenated scores for the pipeline. The first len(scorers) elements are the cross validation scores, and the remaining elements are the standalone objective functions.

Source code in tpot2/tpot_estimator/estimator_utils.py

def val_objective_function_generator(pipeline, X_train, y_train, X_test, y_test, scorers, other_objective_functions, export_graphpipeline=False, **pipeline_kwargs):
    """
    Trains a pipeline on a training set and evaluates it on a test set using the scorers and other objective functions.

    Parameters
    ----------

    pipeline: tpot2.SklearnIndividual
        The individual to evaluate.
    X_train: np.ndarray
        The feature matrix of the training set.
    y_train: np.ndarray
        The target vector of the training set.
    X_test: np.ndarray
        The feature matrix of the test set.
    y_test: np.ndarray
        The target vector of the test set.
    scorers: list
        The scorers to use for cross validation.
    other_objective_functions: list
        A list of standalone objective functions to evaluate the pipeline. With signature obj(pipeline) -> float. or obj(pipeline) -> np.ndarray
        These functions take in the unfitted estimator.
    export_graphpipeline: bool, default=False
        Force the pipeline to be exported as a graph pipeline. Flattens all nested sklearn pipelines, FeatureUnions, and GraphPipelines into a single GraphPipeline.
    pipeline_kwargs: dict
        Keyword arguments to pass to the export_pipeline or export_flattened_graphpipeline method.

    Returns
    -------
    np.ndarray
        The concatenated scores for the pipeline. The first len(scorers) elements are the cross validation scores, and the remaining elements are the standalone objective functions.


    """

    #subsample the data
    if export_graphpipeline:
        pipeline = pipeline.export_flattened_graphpipeline(**pipeline_kwargs)
    else:
        pipeline = pipeline.export_pipeline(**pipeline_kwargs)

    fitted_pipeline = sklearn.base.clone(pipeline)
    fitted_pipeline.fit(X_train, y_train)

    if len(scorers) > 0:
        scores =[sklearn.metrics.get_scorer(scorer)(fitted_pipeline, X_test, y_test) for scorer in scorers]

    other_scores = []
    if other_objective_functions is not None and len(other_objective_functions) >0:
        other_scores = [obj(sklearn.base.clone(pipeline)) for obj in other_objective_functions]

    return np.concatenate([scores,other_scores])