deforce.model package

deforce.model.base_cfn_numpy module

class deforce.model.base_cfn_numpy.BaseDfoCfn(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name=None, optimizer='OriginalWOA', optimizer_paras=None, verbose=True, seed=None)

Bases: sklearn.base.BaseEstimator

Defines the most general class for DFO-based CFN model that inherits the BaseCfnNumpy class

Parameters

• hidden_size (int, default=50) – The number of hidden nodes

• act1_name (str, default='tanh') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• act2_name (str, default='sigmoid') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• obj_name (None or str, default=None) – The name of objective for the problem, also depend on the problem is classification and regression.

• optimizer (str or instance of Optimizer class (from Mealpy library), default = "OriginalWOA") – The Metaheuristic Algorithm that use to solve the feature selection problem. Current supported list, please check it here: https://github.com/thieu1995/mealpy. If a custom optimizer is passed, make sure it is an instance of Optimizer class.

• optimizer_paras (None or dict of parameter, default=None) – The parameter for the optimizer object. If None, the default parameters of optimizer is used (defined in https://github.com/thieu1995/mealpy.) If dict is passed, make sure it has at least epoch and pop_size parameters.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

CLS_OBJ_LOSSES = None

SUPPORTED_ACTIVATIONS = ['none', 'relu', 'leaky_relu', 'celu', 'prelu', 'gelu', 'elu', 'selu', 'rrelu', 'tanh', 'hard_tanh', 'sigmoid', 'hard_sigmoid', 'log_sigmoid', 'swish', 'hard_swish', 'soft_plus', 'mish', 'soft_sign', 'tanh_shrink', 'soft_shrink', 'hard_shrink', 'softmin', 'softmax', 'log_softmax', 'silu']

SUPPORTED_CLS_METRICS = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_CLS_OBJECTIVES = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_OPTIMIZERS = ['OriginalABC', 'OriginalACOR', 'AugmentedAEO', 'EnhancedAEO', 'ImprovedAEO', 'ModifiedAEO', 'OriginalAEO', 'MGTO', 'OriginalAGTO', 'DevALO', 'OriginalALO', 'OriginalAO', 'OriginalAOA', 'IARO', 'LARO', 'OriginalARO', 'OriginalASO', 'OriginalAVOA', 'OriginalArchOA', 'AdaptiveBA', 'DevBA', 'OriginalBA', 'DevBBO', 'OriginalBBO', 'OriginalBBOA', 'OriginalBES', 'ABFO', 'OriginalBFO', 'OriginalBMO', 'DevBRO', 'OriginalBRO', 'OriginalBSA', 'ImprovedBSO', 'OriginalBSO', 'CleverBookBeesA', 'OriginalBeesA', 'ProbBeesA', 'OriginalCA', 'OriginalCDO', 'OriginalCEM', 'OriginalCGO', 'DevCHIO', 'OriginalCHIO', 'OriginalCOA', 'OCRO', 'OriginalCRO', 'OriginalCSA', 'OriginalCSO', 'OriginalCircleSA', 'OriginalCoatiOA', 'JADE', 'OriginalDE', 'SADE', 'SAP_DE', 'DevDMOA', 'OriginalDMOA', 'OriginalDO', 'DevEFO', 'OriginalEFO', 'OriginalEHO', 'AdaptiveEO', 'ModifiedEO', 'OriginalEO', 'OriginalEOA', 'LevyEP', 'OriginalEP', 'CMA_ES', 'LevyES', 'OriginalES', 'Simple_CMA_ES', 'OriginalESOA', 'OriginalEVO', 'OriginalFA', 'DevFBIO', 'OriginalFBIO', 'OriginalFFA', 'OriginalFFO', 'OriginalFLA', 'DevFOA', 'OriginalFOA', 'WhaleFOA', 'OriginalFOX', 'OriginalFPA', 'BaseGA', 'EliteMultiGA', 'EliteSingleGA', 'MultiGA', 'SingleGA', 'OriginalGBO', 'DevGCO', 'OriginalGCO', 'OriginalGJO', 'OriginalGOA', 'DevGSKA', 'OriginalGSKA', 'Matlab101GTO', 'Matlab102GTO', 'OriginalGTO', 'GWO_WOA', 'IGWO', 'OriginalGWO', 'RW_GWO', 'OriginalHBA', 'OriginalHBO', 'OriginalHC', 'SwarmHC', 'OriginalHCO', 'OriginalHGS', 'OriginalHGSO', 'OriginalHHO', 'DevHS', 'OriginalHS', 'OriginalICA', 'OriginalINFO', 'OriginalIWO', 'DevJA', 'LevyJA', 'OriginalJA', 'DevLCO', 'ImprovedLCO', 'OriginalLCO', 'OriginalMA', 'OriginalMFO', 'OriginalMGO', 'OriginalMPA', 'OriginalMRFO', 'WMQIMRFO', 'OriginalMSA', 'DevMVO', 'OriginalMVO', 'OriginalNGO', 'ImprovedNMRA', 'OriginalNMRA', 'OriginalNRO', 'OriginalOOA', 'OriginalPFA', 'OriginalPOA', 'AIW_PSO', 'CL_PSO', 'C_PSO', 'HPSO_TVAC', 'LDW_PSO', 'OriginalPSO', 'P_PSO', 'OriginalPSS', 'DevQSA', 'ImprovedQSA', 'LevyQSA', 'OppoQSA', 'OriginalQSA', 'OriginalRIME', 'OriginalRUN', 'GaussianSA', 'OriginalSA', 'SwarmSA', 'DevSARO', 'OriginalSARO', 'DevSBO', 'OriginalSBO', 'DevSCA', 'OriginalSCA', 'QleSCA', 'OriginalSCSO', 'ImprovedSFO', 'OriginalSFO', 'L_SHADE', 'OriginalSHADE', 'OriginalSHIO', 'OriginalSHO', 'ImprovedSLO', 'ModifiedSLO', 'OriginalSLO', 'DevSMA', 'OriginalSMA', 'DevSOA', 'OriginalSOA', 'OriginalSOS', 'DevSPBO', 'OriginalSPBO', 'OriginalSRSR', 'DevSSA', 'OriginalSSA', 'OriginalSSDO', 'OriginalSSO', 'OriginalSSpiderA', 'OriginalSSpiderO', 'OriginalSTO', 'OriginalSeaHO', 'OriginalServalOA', 'OriginalTDO', 'DevTLO', 'ImprovedTLO', 'OriginalTLO', 'OriginalTOA', 'DevTPO', 'OriginalTS', 'OriginalTSA', 'OriginalTSO', 'EnhancedTWO', 'LevyTWO', 'OppoTWO', 'OriginalTWO', 'DevVCS', 'OriginalVCS', 'OriginalWCA', 'OriginalWDO', 'OriginalWHO', 'HI_WOA', 'OriginalWOA', 'OriginalWaOA', 'OriginalWarSO', 'OriginalZOA']

SUPPORTED_REG_METRICS = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

SUPPORTED_REG_OBJECTIVES = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

create_network(X, y)

evaluate(y_true, y_pred, list_metrics=None)

Return the list of performance metrics of the prediction.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

fit(X, y, lb=(- 1.0,), ub=(1.0,), save_population=False, obj_weights=None)

static load_model(load_path='history', filename='model.pkl')

objective_function(solution=None)

predict(X, return_prob=False)

Inherit the predict function from BaseCFNNumpy class, with 1 more parameter return_prob.

Parameters

• X ({array-like, sparse matrix} of shape (n_samples, n_features)) – The input data.

• return_prob (bool, default=False) –

  It is used for classification problem:

  • If True, the returned results are the probability for each sample

  • If False, the returned results are the predicted labels

save_evaluation_metrics(y_true, y_pred, list_metrics=('RMSE', 'MAE'), save_path='history', filename='metrics.csv')

Save evaluation metrics to csv file

Parameters

• y_true (ground truth data) –

• y_pred (predicted output) –

• list_metrics (list of evaluation metrics) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_model(save_path='history', filename='model.pkl')

Save model to pickle file

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".pkl" extension) –

save_training_loss(save_path='history', filename='loss.csv')

Save the loss (convergence) during the training process to csv file.

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_y_predicted(X, y_true, save_path='history', filename='y_predicted.csv')

Save the predicted results to csv file

Parameters

• X (The features data, nd.ndarray) –

• y_true (The ground truth data) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

score(X, y, method=None)

Return the metric of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• method (str, default="RMSE") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=None)

Return the list of metrics of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• list_methods (list, default=("MSE", "MAE")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_fit_request(*, lb: Union[bool, None, str] = '$UNCHANGED$', obj_weights: Union[bool, None, str] = '$UNCHANGED$', save_population: Union[bool, None, str] = '$UNCHANGED$', ub: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.base_cfn_numpy.BaseDfoCfn

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

• lb (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for lb parameter in fit.

• obj_weights (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for obj_weights parameter in fit.

• save_population (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for save_population parameter in fit.

• ub (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for ub parameter in fit.

Returns

self – The updated object.

Return type

object

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.base_cfn_numpy.BaseDfoCfn

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.base_cfn_numpy.BaseDfoCfn

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object

class deforce.model.base_cfn_numpy.CfnNumpy(input_size=5, hidden_size=10, output_size=1, act1_name='tanh', act2_name='sigmoid')

Bases: object

This class defines the general Cascade Forward Neural Networks (CFN) model using Numpy

Parameters

• input_size (int, default=5) – The number of input nodes

• hidden_size (int, default=10) – The number of hidden nodes

• output_size (int, default=1) – The number of output nodes

• act1_name (str, default='tanh') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• act2_name (str, default='sigmoid') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

fit(X, y)

Fit the model to data matrix X and target(s) y.

Parameters

• X (ndarray or sparse matrix of shape (n_samples, n_features)) – The input data.

• y (ndarray of shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels in classification, real numbers in regression).

Returns

self – Returns a trained CFN model.

Return type

object

forward(inputs)

get_weights()

get_weights_size()

predict(X)

Predict using the Extreme Learning Machine model.

Parameters

X ({array-like, sparse matrix} of shape (n_samples, n_features)) – The input data.

Returns

y – The predicted values.

Return type

ndarray of shape (n_samples, n_outputs)

set_weights(weights)

update_weights_from_solution(solution)

deforce.model.base_cfn_torch module

class deforce.model.base_cfn_torch.BaseCfnTorch(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name=None, max_epochs=1000, batch_size=32, optimizer='SGD', optimizer_paras=None, verbose=False, seed=None)

Bases: sklearn.base.BaseEstimator

Defines the most general class for traditional CFN models that inherits the BaseEstimator class of Scikit-Learn library.

Parameters

• hidden_size (int, default=50) – The hidden size of CFN network (This network only has single hidden layer).

• act1_name (str, defeault="tanh") – This is activation for hidden layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• act2_name (str, defeault="sigmoid") – This is activation for output layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• obj_name (str, default=None) – The name of objective for the problem, also depend on the problem is classification and regression.

• max_epochs (int, default=1000) – Maximum number of epochs / iterations / generations

• batch_size (int, default=32) – The batch size

• optimizer (str, default = "SGD") – The gradient-based optimizer from Pytorch. List of supported optimizer is: [“Adadelta”, “Adagrad”, “Adam”, “Adamax”, “AdamW”, “ASGD”, “LBFGS”, “NAdam”, “RAdam”, “RMSprop”, “Rprop”, “SGD”]

• optimizer_paras (dict or None, default=None) – The dictionary parameters of the selected optimizer.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

CLS_OBJ_LOSSES = None

SUPPORTED_CLS_METRICS = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_LOSSES = {'MAE': <class 'torch.nn.modules.loss.L1Loss'>, 'MSE': <class 'torch.nn.modules.loss.MSELoss'>}

SUPPORTED_OPTIMIZERS = ['Adadelta', 'Adagrad', 'Adam', 'Adamax', 'AdamW', 'ASGD', 'LBFGS', 'NAdam', 'RAdam', 'RMSprop', 'Rprop', 'SGD']

SUPPORTED_REG_METRICS = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

create_network(X, y)

evaluate(y_true, y_pred, list_metrics=None)

Return the list of performance metrics of the prediction.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

fit(X, y)

static load_model(load_path='history', filename='model.pkl')

predict(X, return_prob=False)

Inherit the predict function from BaseCFN class, with 1 more parameter return_prob.

Parameters

• X ({array-like, sparse matrix} of shape (n_samples, n_features)) – The input data.

• return_prob (bool, default=False) –

  It is used for classification problem:

  • If True, the returned results are the probability for each sample

  • If False, the returned results are the predicted labels

save_evaluation_metrics(y_true, y_pred, list_metrics=('RMSE', 'MAE'), save_path='history', filename='metrics.csv')

Save evaluation metrics to csv file

Parameters

• y_true (ground truth data) –

• y_pred (predicted output) –

• list_metrics (list of evaluation metrics) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_model(save_path='history', filename='model.pkl')

Save model to pickle file

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".pkl" extension) –

save_training_loss(save_path='history', filename='loss.csv')

Save the loss (convergence) during the training process to csv file.

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_y_predicted(X, y_true, save_path='history', filename='y_predicted.csv')

Save the predicted results to csv file

Parameters

• X (The features data, nd.ndarray) –

• y_true (The ground truth data) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

score(X, y, method=None)

Return the metric of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• method (str, default="RMSE") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=None)

Return the list of metrics of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• list_methods (list, default=("MSE", "MAE")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.base_cfn_torch.BaseCfnTorch

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.base_cfn_torch.BaseCfnTorch

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object

class deforce.model.base_cfn_torch.CfnTorch(input_size, hidden_size, output_size, act1_name='tanh', act2_name='sigmoid')

Bases: torch.nn.modules.module.Module

Define the CFN model

SUPPORTED_ACTIVATIONS = ['none', 'threshold', 'relu', 'rrelu', 'hardtanh', 'relu6', 'sigmoid', 'hardsigmoid', 'tanh', 'silu', 'mish', 'hardswish', 'elu', 'celu', 'selu', 'glu', 'gelu', 'hardshrink', 'leakyrelu', 'logsigmoid', 'softplus', 'softshrink', 'multiheadattention', 'prelu', 'softsign', 'tanhshrink', 'softmin', 'softmax', 'logsoftmax']

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

deforce.model.dfo_cfn module

class deforce.model.dfo_cfn.DfoCfnClassifier(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name='CEL', optimizer='OriginalWOA', optimizer_paras=None, verbose=True, seed=None)

Bases: deforce.model.base_cfn_numpy.BaseDfoCfn, sklearn.base.ClassifierMixin

Defines the general class of Metaheuristic-based CFN model for Classification problems that inherit the BaseDfoCfn and ClassifierMixin classes.

Parameters

• hidden_size (int, default=50) – The number of hidden nodes

• act1_name (str, default='tanh') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• act2_name (str, default='sigmoid') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• obj_name (str, default="MSE") – Current supported objective functions, please check it here: https://github.com/thieu1995/permetrics

• optimizer (str or instance of Optimizer class (from Mealpy library), default = "OriginalWOA") – The Metaheuristic Algorithm that use to solve the feature selection problem. Current supported list, please check it here: https://github.com/thieu1995/mealpy. If a custom optimizer is passed, make sure it is an instance of Optimizer class.

• optimizer_paras (None or dict of parameter, default=None) – The parameter for the optimizer object. If None, the default parameters of optimizer is used (defined in https://github.com/thieu1995/mealpy.) If dict is passed, make sure it has at least epoch and pop_size parameters.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

Examples

>>> from deforce import Data, DfoCfnClassifier
>>> from sklearn.datasets import make_classification
>>> X, y = make_classification(n_samples=100, random_state=1)
>>> data = Data(X, y)
>>> data.split_train_test(test_size=0.2, random_state=1)
>>> data.X_train_scaled, scaler = data.scale(data.X_train, method="MinMaxScaler")
>>> data.X_test_scaled = scaler.transform(data.X_test)
>>> opt_paras = {"name": "GA", "epoch": 10, "pop_size": 30}
>>> print(DfoCfnClassifier.SUPPORTED_CLS_OBJECTIVES)
{'PS': 'max', 'NPV': 'max', 'RS': 'max', ...., 'KLDL': 'min', 'BSL': 'min'}
>>> model = DfoCfnClassifier(hidden_size=50, act1_name="tanh", act2_name="sigmoid",
>>>             obj_name="NPV", optimizer="OriginalWOA", optimizer_paras=opt_paras, verbose=True, seed=42)
>>> model.fit(data.X_train_scaled, data.y_train)
>>> pred = model.predict(data.X_test_scaled)
>>> print(pred)
array([1, 0, 1, 0, 1])

CLS_OBJ_LOSSES = ['CEL', 'HL', 'KLDL', 'BSL']

create_network(X, y) → Tuple[deforce.model.base_cfn_numpy.CfnNumpy, deforce.toolkit.scalers.ObjectiveScaler]

evaluate(y_true, y_pred, list_metrics=('AS', 'RS'))

Return the list of performance metrics on the given test data and labels.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("AS", "RS")) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

objective_function(solution=None)

Evaluates the fitness function for classification metric

Parameters

solution (np.ndarray, default=None) –

Returns

result – The fitness value

Return type

float

score(X, y, method='AS')

Return the metric on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

• method (str, default="AS") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=('AS', 'RS'))

Return the list of metrics on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

• list_methods (list, default=("AS", "RS")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_fit_request(*, lb: Union[bool, None, str] = '$UNCHANGED$', obj_weights: Union[bool, None, str] = '$UNCHANGED$', save_population: Union[bool, None, str] = '$UNCHANGED$', ub: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

• lb (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for lb parameter in fit.

• obj_weights (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for obj_weights parameter in fit.

• save_population (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for save_population parameter in fit.

• ub (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for ub parameter in fit.

Returns

self – The updated object.

Return type

object

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnClassifier

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object

class deforce.model.dfo_cfn.DfoCfnRegressor(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name='MSE', optimizer='OriginalWOA', optimizer_paras=None, verbose=True, seed=None)

Bases: deforce.model.base_cfn_numpy.BaseDfoCfn, sklearn.base.RegressorMixin

Defines the general class of Metaheuristic-based CFN model for Regression problems that inherit the BaseDfoCfn and RegressorMixin classes.

Parameters

• hidden_size (int, default=50) – The number of hidden nodes

• act1_name (str, default='tanh') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• act2_name (str, default='sigmoid') – Activation function for the hidden layer. The supported activation functions are: [“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”, “silu”]

• obj_name (str, default="MSE") – Current supported objective functions, please check it here: https://github.com/thieu1995/permetrics

• optimizer (str or instance of Optimizer class (from Mealpy library), default = "OriginalWOA") – The Metaheuristic Algorithm that use to solve the feature selection problem. Current supported list, please check it here: https://github.com/thieu1995/mealpy. If a custom optimizer is passed, make sure it is an instance of Optimizer class.

• optimizer_paras (None or dict of parameter, default=None) – The parameter for the optimizer object. If None, the default parameters of optimizer is used (defined in https://github.com/thieu1995/mealpy.) If dict is passed, make sure it has at least epoch and pop_size parameters.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

Examples

>>> from deforce import DfoCfnRegressor, Data
>>> from sklearn.datasets import make_regression
>>> X, y = make_regression(n_samples=200, random_state=1)
>>> data = Data(X, y)
>>> data.split_train_test(test_size=0.2, random_state=1)
>>> data.X_train_scaled, scaler = data.scale(data.X_train, method="MinMaxScaler")
>>> data.X_test_scaled = scaler.transform(data.X_test)
>>> opt_paras = {"name": "GA", "epoch": 10, "pop_size": 30}
>>> model = DfoCfnRegressor(hidden_size=15, act1_name="relu", act2_name="sigmoid",
>>>             obj_name="MSE", optimizer="BaseGA", optimizer_paras=opt_paras, verbose=True, seed=42)
>>> model.fit(data.X_train_scaled, data.y_train)
>>> pred = model.predict(data.X_test_scaled)
>>> print(pred)

create_network(X, y) → Tuple[deforce.model.base_cfn_numpy.CfnNumpy, deforce.toolkit.scalers.ObjectiveScaler]

Returns

• network (CFN, an instance of CFN network)

• obj_scaler (ObjectiveScaler, the objective scaler that used to scale output)

evaluate(y_true, y_pred, list_metrics=('MSE', 'MAE'))

Return the list of performance metrics of the prediction.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("MSE", "MAE")) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

objective_function(solution=None)

Evaluates the fitness function for regression metric

Parameters

solution (np.ndarray, default=None) –

Returns

result – The fitness value

Return type

float

score(X, y, method='RMSE')

Return the metric of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• method (str, default="RMSE") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=('MSE', 'MAE'))

Return the list of metrics of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• list_methods (list, default=("MSE", "MAE")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_fit_request(*, lb: Union[bool, None, str] = '$UNCHANGED$', obj_weights: Union[bool, None, str] = '$UNCHANGED$', save_population: Union[bool, None, str] = '$UNCHANGED$', ub: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnRegressor

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

• lb (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for lb parameter in fit.

• obj_weights (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for obj_weights parameter in fit.

• save_population (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for save_population parameter in fit.

• ub (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for ub parameter in fit.

Returns

self – The updated object.

Return type

object

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnRegressor

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.dfo_cfn.DfoCfnRegressor

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object

deforce.model.dfo_tune_cfn module

class deforce.model.dfo_tune_cfn.DfoTuneCfn(problem_type='regression', bounds=None, cv=5, scoring='MSE', optimizer='OriginalWOA', optimizer_paras=None, verbose=True, seed=None)

Bases: object

SUPPORTED_CLS_METRICS = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_REG_METRICS = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

fit(X, y)

static load_model(load_path='history', filename='network.pkl')

predict(X)

save_convergence(save_path='history', filename='convergence.csv')

Save the convergence (fitness value) during the training process to csv file.

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_model(save_path='history', filename='network.pkl')

Save network to pickle file

Parameters

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".pkl" extension) –

save_performance_metrics(y_true, y_pred, list_metrics=('RMSE', 'MAE'), save_path='history', filename='metrics.csv')

Save evaluation metrics to csv file

Parameters

• y_true (ground truth data) –

• y_pred (predicted output) –

• list_metrics (list of evaluation metrics) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

save_y_predicted(X, y_true, save_path='history', filename='y_predicted.csv')

Save the predicted results to csv file

Parameters

• X (The features data, nd.ndarray) –

• y_true (The ground truth data) –

• save_path (saved path (relative path, consider from current executed script path)) –

• filename (name of the file, needs to have ".csv" extension) –

class deforce.model.dfo_tune_cfn.HyperparameterProblem(bounds=None, minmax='max', X=None, y=None, model_class=None, metric_class=None, obj_name=None, cv=5, seed=None, **kwargs)

Bases: mealpy.utils.problem.Problem

obj_func(x)

Objective function

Parameters

x (numpy.ndarray) – Solution.

Returns

Function value of x.

Return type

float

deforce.model.gd_cfn module

class deforce.model.gd_cfn.CfnClassifier(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name='NLLL', max_epochs=1000, batch_size=32, optimizer='SGD', optimizer_paras=None, verbose=False, seed=None, **kwargs)

Bases: deforce.model.base_cfn_torch.BaseCfnTorch

Defines the class for traditional CFN network for Classification problems that inherit the BaseCfnTorch class

Parameters

• hidden_size (int, default=50) – The hidden size of CFN network (This network only has single hidden layer).

• act1_name (str, defeault="tanh") – This is activation for hidden layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• act2_name (str, defeault="sigmoid") – This is activation for output layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• obj_name (str, default="NLLL") – The name of objective for classification problem (binary and multi-class classification)

• max_epochs (int, default=1000) – Maximum number of epochs / iterations / generations

• batch_size (int, default=32) – The batch size

• optimizer (str, default = "SGD") – The gradient-based optimizer from Pytorch. List of supported optimizer is: [“Adadelta”, “Adagrad”, “Adam”, “Adamax”, “AdamW”, “ASGD”, “LBFGS”, “NAdam”, “RAdam”, “RMSprop”, “Rprop”, “SGD”]

• optimizer_paras (dict or None, default=None) – The dictionary parameters of the selected optimizer.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

Examples

>>> from deforce import CfnClassifier, Data
>>> from sklearn.datasets import make_regression
>>>
>>> ## Make dataset
>>> X, y = make_regression(n_samples=200, n_features=10, random_state=1)
>>> ## Load data object
>>> data = Data(X, y)
>>> ## Split train and test
>>> data.split_train_test(test_size=0.2, random_state=1, inplace=True)
>>> ## Scale dataset
>>> data.X_train, scaler = data.scale(data.X_train, scaling_methods=("minmax"))
>>> data.X_test = scaler.transform(data.X_test)
>>> ## Create model
>>> model = CfnClassifier(hidden_size=25, act1_name="tanh", act2_name="sigmoid", obj_name="BCEL",
>>>                 max_epochs=10, batch_size=32, optimizer="SGD", optimizer_paras=None, verbose=True, seed=42)
>>> ## Train the model
>>> model.fit(data.X_train, data.y_train)
>>> ## Test the model
>>> y_pred = model.predict(data.X_test)
>>> ## Calculate some metrics
>>> print(model.score(X=data.X_test, y=data.y_test, method="RMSE"))
>>> print(model.scores(X=data.X_test, y=data.y_test, list_methods=["R2", "NSE", "MAPE"]))
>>> print(model.evaluate(y_true=data.y_test, y_pred=y_pred, list_metrics=["R2", "NSE", "MAPE", "NNSE"]))

CLS_OBJ_BINARY_1 = ['PNLLL', 'HEL', 'BCEL', 'CEL', 'BCELL']

CLS_OBJ_BINARY_2 = ['NLLL']

CLS_OBJ_LOSSES = ['CEL', 'HEL', 'KLDL']

CLS_OBJ_MULTI = ['NLLL', 'CEL']

SUPPORTED_LOSSES = {'BCEL': <class 'torch.nn.modules.loss.BCELoss'>, 'BCELL': <class 'torch.nn.modules.loss.BCEWithLogitsLoss'>, 'CEL': <class 'torch.nn.modules.loss.CrossEntropyLoss'>, 'GNLLL': <class 'torch.nn.modules.loss.GaussianNLLLoss'>, 'HEL': <class 'torch.nn.modules.loss.HingeEmbeddingLoss'>, 'KLDL': <class 'torch.nn.modules.loss.KLDivLoss'>, 'NLLL': <class 'torch.nn.modules.loss.NLLLoss'>, 'PNLLL': <class 'torch.nn.modules.loss.PoissonNLLLoss'>}

create_network(X, y) → Tuple[skorch.classifier.NeuralNetClassifier, deforce.toolkit.scalers.ObjectiveScaler]

Returns

• network (CFN, an instance of CFN network)

• obj_scaler (ObjectiveScaler, the objective scaler that used to scale output)

evaluate(y_true, y_pred, list_metrics=('AS', 'RS'))

Return the list of performance metrics on the given test data and labels.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("AS", "RS")) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

fit(X, y)

score(X, y, method='AS')

Return the metric on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

• method (str, default="AS") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=('AS', 'RS'))

Return the list of metrics on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

• list_methods (list, default=("AS", "RS")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.gd_cfn.CfnClassifier

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.gd_cfn.CfnClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object

class deforce.model.gd_cfn.CfnRegressor(hidden_size=50, act1_name='tanh', act2_name='sigmoid', obj_name='MSE', max_epochs=1000, batch_size=32, optimizer='SGD', optimizer_paras=None, verbose=False, seed=None, **kwargs)

Bases: deforce.model.base_cfn_torch.BaseCfnTorch

Defines the class for traditional CFN network for Regression problems that inherit the BaseCfnTorch and RegressorMixin classes.

Parameters

• hidden_size (int, default=50) – The hidden size of CFN network (This network only has single hidden layer).

• act1_name (str, defeault="tanh") – This is activation for hidden layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• act2_name (str, defeault="sigmoid") – This is activation for output layer. The supported activation are: {“none”, “relu”, “leaky_relu”, “celu”, “prelu”, “gelu”, “elu”, “selu”, “rrelu”, “tanh”, “hard_tanh”, “sigmoid”, “hard_sigmoid”, “log_sigmoid”, “silu”, “swish”, “hard_swish”, “soft_plus”, “mish”, “soft_sign”, “tanh_shrink”, “soft_shrink”, “hard_shrink”, “softmin”, “softmax”, “log_softmax”}.

• obj_name (str, default="MSE") – The name of loss function for the network.

• max_epochs (int, default=1000) – Maximum number of epochs / iterations / generations

• batch_size (int, default=32) – The batch size

• optimizer (str, default = "SGD") – The gradient-based optimizer from Pytorch. List of supported optimizer is: [“Adadelta”, “Adagrad”, “Adam”, “Adamax”, “AdamW”, “ASGD”, “LBFGS”, “NAdam”, “RAdam”, “RMSprop”, “Rprop”, “SGD”]

• optimizer_paras (dict or None, default=None) – The dictionary parameters of the selected optimizer.

• verbose (bool, default=True) – Whether to print progress messages to stdout.

• seed (int, default=None) – Seed value to reproduce the results.

Examples

>>> from deforce import CfnRegressor, Data
>>> from sklearn.datasets import make_regression
>>>
>>> ## Make dataset
>>> X, y = make_regression(n_samples=200, n_features=10, random_state=1)
>>> ## Load data object
>>> data = Data(X, y)
>>> ## Split train and test
>>> data.split_train_test(test_size=0.2, random_state=1, inplace=True)
>>> ## Scale dataset
>>> data.X_train, scaler = data.scale(data.X_train, scaling_methods=("minmax"))
>>> data.X_test = scaler.transform(data.X_test)
>>> ## Create model
>>> model = CfnRegressor(hidden_size=25, act1_name="tanh", act2_name="sigmoid", obj_name="MSE",
>>>             max_epochs=10, batch_size=32, optimizer="SGD", optimizer_paras=None, verbose=True, seed=42)
>>> ## Train the model
>>> model.fit(data.X_train, data.y_train)
>>> ## Test the model
>>> y_pred = model.predict(data.X_test)
>>> ## Calculate some metrics
>>> print(model.score(X=data.X_test, y=data.y_test, method="RMSE"))
>>> print(model.scores(X=data.X_test, y=data.y_test, list_methods=["R2", "NSE", "MAPE"]))
>>> print(model.evaluate(y_true=data.y_test, y_pred=y_pred, list_metrics=["R2", "NSE", "MAPE", "NNSE"]))

SUPPORTED_LOSSES = {'MAE': <class 'torch.nn.modules.loss.L1Loss'>, 'MSE': <class 'torch.nn.modules.loss.MSELoss'>}

create_network(X, y)

Returns

• network (CFN, an instance of CFN network)

• obj_scaler (ObjectiveScaler, the objective scaler that used to scale output)

evaluate(y_true, y_pred, list_metrics=('MSE', 'MAE'))

Return the list of performance metrics of the prediction.

Parameters

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("MSE", "MAE")) – You can get metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

fit(X, y)

score(X, y, method='RMSE')

Return the metric of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• method (str, default="RMSE") – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

result – The result of selected metric

Return type

float

scores(X, y, list_methods=('MSE', 'MAE'))

Return the list of metrics of the prediction.

Parameters

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• list_methods (list, default=("MSE", "MAE")) – You can get all metrics from Permetrics library: https://github.com/thieu1995/permetrics

Returns

results – The results of the list metrics

Return type

dict

set_predict_request(*, return_prob: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.gd_cfn.CfnRegressor

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

return_prob (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_prob parameter in predict.

Returns

self – The updated object.

Return type

object

set_score_request(*, method: Union[bool, None, str] = '$UNCHANGED$') → deforce.model.gd_cfn.CfnRegressor

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in score.

Returns

self – The updated object.

Return type

object