Machine learning service basics
The machine learning (lookup value prediction) service uses statistical analysis methods for machine learning based on historical data. For example, a history of customer communications with customer support is considered historical data in Creatio. The message text, the date and the account category are used. The result is the Responsible Group field.
Creatio interaction with the prediction service
There are two stages of model processing in Creatio: training and prediction.
Prediction model is the algorithm which builds predictions and enables the system to automatically make decisions based on historical data.
The service is “trained” at this stage. Main training steps:
- Establishing a session for data transfer and training.
- Sequentially selecting a portion of data for the model and uploading it to the service.
- Requesting to include a model a training queue.
Training engineprocesses the queue for model training, trains the model and saves its parameters to the local database.
- Creatio occasionally queries the service to get the model status.
- Once the model status is set to
Done, the model is ready for prediction.
The prediction task is performed through a call to the cloud service, indicating the Id of the model instance and the data for the prediction. The result of the service operation is a set of values with prediction probabilities, which is stored in Creatio in the
If there is a prediction in the
MLPrediction table for a particular entity record, the predicted values for the field are automatically displayed on the edit page.
Creatio settings and data types for working with the prediction service
The following data is provided for working with the prediction service in Creatio.
CloudServicesAPIKeysystem setting authenticates the Creatio instance in cloud services.
- The record in the ML problem types (
MLProblemType) lookup with the populated ServiceUrl field is the address of the implemented prediction service.
- The model records in the ML model (
MLModel) lookup that contains information about the selected data for the model, the training period, the current training status, etc. For each model, the
MLProblemTypefield must contain a reference to the correct record of the ML problem types lookup.
MLModelTrainingPeriodMinutessystem setting determines the frequency of model synchronization launch.
Expanding the training model logic
The above chain of classes calls and creates instances of each other through the IOC of the
If you need to replace the logic of any component, you need to implement the appropriate interface. When you start the application, you must bind the interface in your own implementation.
Interfaces for logic expansion:
IMLModelTrainerJob – the implementation of this interface will enable you to change the set of models for training.
IMLModelTrainer – responsible for the logic of loading data for training and updating the status of models.
IMLServiceProxy - the implementation of this interface will enable you to execute queries to arbitrary predictive services.
Auxiliary classes for forecasting
Auxiliary (utility) classes for forecasting enable you to implement two basic cases:
- Prediction at the time of creating or updating an entity record on the server.
- Prediction when the entity is changed on the edit page.
While predicting on the Creatio server-side, a business process is created that responds to the entity creation/change signal, reads a set of fields, and calls the prediction service. If you get the correct result, it stores the set of field values with probabilities in the
MLClassificationResult table. If necessary, the business process records a separate value (for example, with the highest probability) in the corresponding field of the entity.
Creating data queries for the machine learning model
Use the Terrasoft.Core.DB.Select class instance for queries of training data or data for predicting machine learning service (see “Machine learning service”). It is dynamically imported by the
QueryInterpreter interpreter does not allow the use of lambda expressions.
Use the provided
userConnection variable as an argument of the
Terrasoft.Core.UserConnection type in the
Select constructor when building query expression. The column with the “Id” alias (the unique id of the target object instance) is required in the query expression.
Select expression can be complex. Use the following practices to simplify it:
- Dynamic adding of types for the interpreter.
- Using local variables.
- Using the
Dynamic adding of types for the interpreter
You can dynamically add types for the interpreter. For this, the
QueryInterpreter class provides the
RegisterType methods. You can use them directly in the expression. For example, instead of direct using the type id:
you can use the name of a constant from dynamically registered enumeration:
Using local variables
You can use local variables to avoid code duplication and more convenient structuring. Constraint: the type of the variable must be statically calculated and defined by the
For example, the query with repetitive use of delegates:
you can write in a following way:
var monthAgo = Func.DateAddMonth(-1, Func.CurrentDateTime());
Connecting a custom web-service to the machine learning functionality
Creatio 7.16.2 and up supports connecting custom web-services to the machine learning functionality.
You can implement typical machine learning problems (classification, scoring, numerical regression) or other similar problems (for example, customer churn forecast) using a custom web-service. This article covers the procedure for connecting a custom web-service implementation of a prediction model to Creatio.
The main principles of the machine learning service operation are covered in the “Machine learning service” article.
The general procedure of connecting a custom web-service to the machine learning service is as follows:
- Create a machine learning web-service engine.
- Expand the problem type list of the machine learning service.
- Implement a machine learning model.
Create a machine learning web-service engine
A custom web-service must implement a service contract for model training and making forecasts based on an existing prediction model. You can find a sample Swagger service contract of the Creatio machine learning service at https://demo-ml.bpmonline.com/swagger/index.html#/MLService
/session/start– starting a model training session.
/data/upload– uploading data for an active training session.
/session/info/get– getting the status of the training session.
<training start custom method>– a user-defined method to be called by Creatio when the data upload is over. The model training process must not be terminated until the execution of the method is complete. A training session may last for an indefinite period (minutes or even hours). When the training is over, the
/session/info/getmethod will return the training session status: either
Errordepending on the result. Additionally, if the model is trained successfully, the method will return a model instance summary (
ModelSummary): metrics type, metrics value, instance ID, and other data.
<Prediction custom method>– an arbitrary signature method that will make predictions based on a trained prediction model referenced by the ID.
Web-service development with the Microsoft Visual Studio IDE is covered in the “Developing the configuration server code in the user solution” article.
Expanding the problem type list of the machine learning service
To expand the problem type list of the Creatio machine learning service, add a new record to the
MLProblemType lookup. You must specify the following parameters:
Service endpoint Url– the URL endpoint for the online machine learning service.
Training endpoint– the endpoint for the training start method.
Prediction endpoint– the endpoint for the prediction method.
You will need the ID of the new problem type record to proceed. Look up the ID in the dbo.MLProblemType DB table.
Implementing a machine learning model
To configure and display a machine learning model, you may need to extend the
MLModelPage mini-page schema.
Predict method. The method accepts data exported from the system by object (formatted as
Dictionary<string, object>, where
key is the field name and
value is the field value), and returns the prediction value. This method may use a proxy class that implements the
IMLServiceProxy interface to facilitate web-service calls.
ProblemTypeId property with the ID of the new problem type record created in the
MLProblemType lookup. Additionally, implement the following methods:
SaveEntityPredictedValues– the method retrieves the prediction value and saves it for the system
entity, for which the prediction process is run. If the returned value is of the
doubletype or is similar to classification results, you can use the methods provided in the
SavePredictionoptional – the method saves the prediction value with a reference to the trained model instance and the ID of the entity (
entityId). For basic problems, the system provides the
You can extend the existing
IMLServiceProxy interface and the corresponding implementations in the prediction method of the current problem type. In particular, the
MLServiceProxy class provides the
Predict generic method that accepts contracts for input data and for prediction results.
If the web-service is called with a large set of data (500 instances and more), implement the
IMLBatchPredictor interface. You must implement the following methods:
FormatValueForSaving– returns a converted prediction value ready for database storage. In case a batch prediction process is running, the record is updated using the
Updatemethod rather than
Entityinstances to speed up the process.
SavePredictionResult– defines how the system will store the prediction value per entity. For basic ML problems, the system provides