Find and merge duplicates

Duplicate records might appear whenever you add new records to Creatio sections. Find and merge duplicates to maintain the quality of your Creatio data.

Bulk duplicate search is run for the entire database, manually or on schedule. You can also merge any duplicate records manually without running the search.

Creatio uses a set of pre-configured rules to search for duplicates. For example, duplicates can be identified by identical phone numbers or email addresses. Creatio lets you customize these rules in multiple ways:

• Customize out-of-the-box contact, account, and lead duplicate search rules to suit your specific needs.
• Create custom rules for any Creatio section, including custom sections.

To use the duplicate search functionality, the following prerequisites must be met:

• No-code developer must create at least one duplicate search rule. Learn more: Add a duplicate search rule. The rules are available out of the box for Accounts, Contacts, and Leads sections.
• No-code developer must add a button that triggers the "Open duplicates page" action to the list page of the relevant section.

Important

To ensure the correct operation of bulk duplicate search, on-site users must install additional components. Learn more in a separate article: Bulk duplicate search.

Find and process duplicates

To open the duplicates page, you need permissions to read and edit the records of the section where you search for duplicates, as well as permission to the "Duplicates search" ("CanSearchDuplicates" code) system operation. Learn more: System operation permissions.

Creatio displays the duplicate search results on the found duplicates page (Fig. 1). To set up the columns of the duplicate list, click View → List setting in the top right.

Fig. 1 Duplicate search results in the Contacts section

To search for duplicates:

1. Open a section where you want to search for duplicates. For example, the Contacts section.

2. Click the button that has the "Open duplicates page" action. This opens a page. If the duplicate search was performed earlier (e. g., automatically), the page will display its results. You can process the previous duplicate search results before searching again.

   This button can be added by no-code developer. Learn more: Set up Button components.

3. Click Actions → Run duplicate search. Creatio will search for duplicates in the background. In the meantime, you can continue working in the system.

   Once the duplicate search is complete, you will receive a notification on the sidebar of the notification panel (Fig. 2).

   Fig. 2 Notification about the completed duplicate search

   

4. Click the link in the notification to view the duplicate search results.

   The results list displays the duplicate records found based on the active search rules. The records are grouped by similarity (Fig. 3).

   You can merge each group of records into a single record or indicate that the records in the group are not duplicates. In the latter case, Creatio will add the records to the exception list for the next search.

   Fig. 3 Select the duplicates to merge

   

5. Select the necessary records → Merge to merge duplicates.

   Creatio will merge the selected records in the group into one record that contains the unique data from the merged records. If the selected records have different data in the same field, Creatio will prompt you to select the data to save.

6. Click Not duplicates for the group that contains unique records to add records to the exception list (Fig. 4).

   Creatio will not consider the records in the group as duplicates for the next duplicate search.

   Fig. 4 Records that are not duplicates

   

Duplicate search procedure

The duplicate search mechanism is identical to the global search mechanism.

Creatio indexes data to remove all special characters and divide the remaining letters and numbers into sets of two or three characters. The characters are recorded in the index that the search mechanism uses.

Creatio does not modify the section records as part of the indexing.

The duplicate search procedure is as follows:

1. The user runs the duplicate search.
2. Creatio processes new data (removes all special characters, divides the remaining data into sets of two or three characters) and requests Elasticsearch to search for records that contain the characters the user entered.
3. Creatio displays the matches according to at least one active duplicate search rule that has the Use this rule on save checkbox selected. Matches that have a different word order will be displayed as well.

The phone numbers are compared to each other regardless of their field: Business phone, Mobile phone, Home phone, etc.

note

The duplicate list does not display the records excluded via the Not duplicates button earlier.

If the field contains several words, Creatio considers records that have both full and partial matches as duplicates. For example, if you search using the "Contact duplicates, contact name" condition, "John Best" and "John" contacts will be considered as duplicates.

If the search conditions overlap, Creatio searches for duplicates using the stricter rule. For example, you have the following duplicate search rules configured: "Contact duplicates, contact name," "Contact duplicates. Contact name, Phone." Creatio will consider only the records that have matching full names as duplicates since the corresponding rule is stricter.

Set up the duplicate search

Creatio checks for duplicates using the existing search rules. The Accounts, Contacts, and Leads sections include the duplicate search rules out-of-the-box.

You can perform the following actions in Creatio:

• Create new duplicate search rules based on a text or a lookup field in any section.

• Enable or disable individual rules.

• Specify the rules to use while saving a record.

• Remove unused rules.

  note

  Default rules that search for contact and account duplicates in leads cannot be deleted. However, you can disable these rules.

Add a duplicate search rule

Important

Learn more about setting up the service for Creatio on-site in a separate article: Bulk duplicate search.

1. Click to open the System Designer.

2. Go to the System setup block → Setup duplicates rules.

3. Click New rule.

4. Fill out the following fields on the rule setup page (Fig. 5):

   1. Select a section that will use this rule in the Rule type field. For example, "Products." You can create a rule for sections that are indexed in global search. This includes all sections created in Freedom UI from scratch.

   2. Click on the Attributes detail and add one or more columns by which to search for duplicates. The detail accepts only text and lookup fields.

   If you select multiple attributes in a single rule, e. g., "Code" and "Name," the duplicate search will combine them using the "AND" operator. I. e., Creatio will look for records that contain both duplicate code and name. If you create multiple rules that have only one attribute, e.g., "Code" in the first rule and "Name" in the second rule, the duplicate search will combine them using the "OR" operator. I. e., the search will display records that contain either duplicate code or duplicate name.

   Fig. 5 Create a new duplicate search rule

   

5. Select the Active checkbox.

6. Save changes.

As a result, Creatio will notify you that the changes will apply upon your next login. After your next login, Creatio will use the new rule for the duplicate search.

Disable a duplicate search rule

You can deactivate a rule permanently or temporarily. Creatio will not use it to search for duplicates. To do this:

1. Click to open the System Designer.
2. Go to the System setup block → Setup duplicates rules.
3. Select a rule to deactivate → Open.
4. Clear the Active checkbox.
5. Save changes.

As a result, Creatio will not use the rule to search for duplicates. You can reactivate it at any time.

note

If you deactivate a duplicate search rule specified in a Find and merge duplicates business process element, the element will no longer use it. If you deactivate all specified rules, the process that contains the element will fail to complete.

Creatio warns you that a business process might use the duplicate search rule when you deactivate it. However, Creatio does not check whether any Find and merge duplicates element is using it currently.

Schedule the automatic duplicate search

In Creatio, you can schedule an automatic duplicate search. For example, run it three times a week. To do this:

1. Click the to open the System Designer.

2. Go to the System setup block → Setup duplicates rules.

3. Click Actions → Setup automatic duplicate search (Fig. 6). This opens a page.

   Fig. 6 Open the automatic duplicate search setup window

   

4. Set up the parameters of the automatic bulk duplicate search (Fig. 7):

   Fig. 7 Schedule the duplicate search

   

   1. Select a section for which to schedule the search. The list displays only the sections that have at least one duplicate search rule configured.

   2. Select the time to run the search.

   3. Select the day of the week to run the search.

   4. Click Apply.

As a result, Creatio will search for duplicates automatically on the selected days at the set time using the active rules. Keep in mind that Creatio does not merge duplicates automatically. You must process the found duplicates manually.

To disable the automatic duplicate search, clear the time field or the checkboxes next to the days of the week → Apply settings. Both options will disable the automatic duplicate search.

---

See also

Filters

Folders

Tags

Excel data import

Excel data export

Share records

Global search