Customize machine learning service

Level: advanced

Machine learning service basics

The machine learning (lookup value prediction) service uses statistical analysis methods for machine learning based on historical data. For example, a history of customer communications with customer support is considered historical data in Creatio. The message text, the date and the account category are used. The result is the Responsible Group field.

Creatio interaction with the prediction service

There are two stages of model processing in Creatio: training and prediction.

Prediction model is the algorithm which builds predictions and enables the system to automatically make decisions based on historical data.

Training

The service is "trained" at this stage. Main training steps:

• Establishing a session for data transfer and training.
• Sequentially selecting a portion of data for the model and uploading it to the service.
• Requesting to include a model a training queue.
• Training engine processes the queue for model training, trains the model and saves its parameters to the local database.
• Creatio occasionally queries the service to get the model status.
• Once the model status is set to Done, the model is ready for prediction.

Prediction

The prediction task is performed through a call to the cloud service, indicating the Id of the model instance and the data for the prediction. The result of the service operation is a set of values with prediction probabilities, which is stored in Creatio in the MLPrediction table.

If there is a prediction in the MLPrediction table for a particular entity record, the predicted values for the field are automatically displayed on the edit page.

Creatio settings and data types for working with the prediction service

Creatio setup

The following data is provided for working with the prediction service in Creatio.

1. The CloudServicesAPIKey system setting authenticates the Creatio instance in cloud services.
2. The record in the ML problem types (MLProblemType) lookup with the populated ServiceUrl field is the address of the implemented prediction service.
3. The model records in the ML model (MLModel) lookup that contains information about the selected data for the model, the training period, the current training status, etc. For each model, the MLProblemType field must contain a reference to the correct record of the ML problem types lookup.
4. The MLModelTrainingPeriodMinutes system setting determines the frequency of model synchronization launch.

Expanding the training model logic

The above chain of classes calls and creates instances of each other through the IOC of the Terrasoft.Core.Factories.ClassFactory container.

If you need to replace the logic of any component, you need to implement the appropriate interface. When you start the application, you must bind the interface in your own implementation.

Interfaces for logic expansion:

IMLModelTrainerJob – the implementation of this interface will enable you to change the set of models for training.

IMLModelTrainer – responsible for the logic of loading data for training and updating the status of models.

IMLServiceProxy - the implementation of this interface will enable you to execute queries to arbitrary predictive services.

Auxiliary classes for forecasting

Auxiliary (utility) classes for forecasting enable you to implement two basic cases:

1. Prediction at the time of creating or updating an entity record on the server.
2. Prediction when the entity is changed on the edit page.

While predicting on the Creatio server-side, a business process is created that responds to the entity creation/change signal, reads a set of fields, and calls the prediction service. If you get the correct result, it stores the set of field values with probabilities in the MLClassificationResult table. If necessary, the business process records a separate value (for example, with the highest probability) in the corresponding field of the entity.

Creating data queries for the machine learning model

Use the Terrasoft.Core.DB.Select class instance for queries of training data or data for predicting machine learning service. It is dynamically imported by the Terrasoft.Configuration.ML.QueryInterpreter.

Important

The QueryInterpreter interpreter does not allow the use of lambda expressions.

Use the provided userConnection variable as an argument of the Terrasoft.Core.UserConnection type in the Select constructor when building query expression. The column with the "Id" alias (the unique id of the target object instance) is required in the query expression.

The Select expression can be complex. Use the following practices to simplify it:

• Dynamic adding of types for the interpreter.
• Using local variables.
• Using the Terrasoft.Configuration.QueryExtensions utility class.

Dynamic adding of types for the interpreter

You can dynamically add types for the interpreter. For this, the QueryInterpreter class provides the RegisterConfigurationType and RegisterType methods. You can use them directly in the expression. For example, instead of direct using the type id:

new Select(userConnection)
        .Column("Id")
        .Column("Body")
    .From("Activity")
    .Where("TypeId").IsEqual(Column.Parameter("E2831DEC-CFC0-DF11-B00F-001D60E938C6"));

you can use the name of a constant from dynamically registered enumeration:

RegisterConfigurationType("ActivityConsts");
new Select(userConnection)
        .Column("Id")
        .Column("Body")
    .From("Activity")
    .Where("TypeId").IsEqual(Column.Parameter(ActivityConsts.EmailTypeUId));

Using local variables

You can use local variables to avoid code duplication and more convenient structuring. Constraint: the type of the variable must be statically calculated and defined by the var word.

For example, the query with repetitive use of delegates:

new Select(userConnection)
        .Column("Id")
        .Column("Body")
    .From("Activity")
    .Where("CreatedOn").IsGreater(Func.DateAddMonth(-1, Func.CurrentDateTime()))
    .And("StartDate").IsGreater(Func.DateAddMonth(-1, Func.CurrentDateTime()));

you can write in a following way:

var monthAgo = Func.DateAddMonth(-1, Func.CurrentDateTime());
 
new Select(userConnection)
        .Column("Id")
        .Column("Body")
    .From("Activity")
    .Where("StartDate").IsGreater(monthAgo)
    .And("ModifiedOn").IsGreater(monthAgo);