onetick.ml.impl.evaluators#

class BaseMethodEvaluator(**kwargs)[source]#

Base class for evaluators using simple loss function with interface (y_test, predict). Override _evaluator_method to set loss function.

_evaluator_method#

loss function with interface (y_test, predict)

Type

function

evaluate(y_test: pandas.core.frame.DataFrame, predict: pandas.core.frame.DataFrame)[source]#

Evaluate loss by comparing y_test and predict.

Parameters

  • y_test (pandas.DataFrame) – Ground truth (correct) target values.

  • predict (pandas.DataFrame) – Estimated target values.

Returns

calculated loss

Return type

float

class MAEEvaluator(**kwargs)[source]#

MAE score evaluator

class MAPEEvaluator(**kwargs)[source]#

MAPE score evaluator.

class MSEEvaluator(**kwargs)[source]#

MSE score evaluator

class MSLEEvaluator(**kwargs)[source]#

MSLE score evaluator

class MdAEEvaluator(**kwargs)[source]#

MdAE score evaluator

class R2Evaluator(**kwargs)[source]#

R2 score evaluator

class RMSEEvaluator(**kwargs)[source]#

RMSE score evaluator

class SMAPEEvaluator(**kwargs)[source]#

SMAPE score evaluator

class OneStepPredictionIntervals(**kwargs)[source]#

Evaluator for one-step prediction intervals: https://otexts.com/fpp3/prediction-intervals.html

evaluate(y, prediction: pandas.core.frame.DataFrame, z_value: float = 1.96)[source]#

Evaluate one-step prediction interval using the standard deviation of the residuals.

Parameters

  • y (pandas.DataFrame) – Ground truth (correct) target values.

  • prediction (pandas.DataFrame) – Estimated target values.

  • z_value (float) – z-value for confidence interval. Default is 1.96 for 95% confidence interval.

Returns

calculated one-step prediction interval for each target column

Return type

pandas.DataFrame

class BootstrapPredictionIntervals[source]#
evaluate(experiment=None, bucket_size: int = 39, resampling_num: int = 5, alpha: float = 0.05)[source]#

Calculate confidence interval with feature sampling

Parameters

  • experiment (Experiment or inherited class) – instance of Experiment or inherited class.

  • bucket_size (int) – Size of block for bootstrapping.

  • resampling_num (int) – Number of resamples.

  • alpha (float) – The prediction uncertainty.

Returns

tuple of calculated mean and standard deviation

Return type

tuple of (float, float)

class AccuracyEvaluator(**kwargs)[source]#

Accuracy score evaluator

class PrecisionEvaluator(**kwargs)[source]#

Precision score evaluator

class RecallEvaluator(**kwargs)[source]#

Recall score evaluator

class F1Evaluator(**kwargs)[source]#

F1 score evaluator

class RocAucEvaluator(**kwargs)[source]#

ROC AUC score evaluator