Keep a webpage source up to date
Agents answer from the content that a webpage knowledge source stored when it last fetched its pages. Keep the source up to date so agents answer from the current version of your website. Re-fetch the source when you know the website has changed, or set a schedule so Creatio AI Studio refreshes it on its own.
Re-fetch a webpage source
Re-fetch a webpage source to pick up changes on its pages. The source keeps serving its current content while it refreshes. To add pages to the source, click Add URLs on the source page, enter one URL per line in the "Add URLs" dialog, and click Add URLs.
- Open the knowledge source from the Knowledge list.
- Go to the URLs tab.
- Click Re-fetch → Re-fetch all. The action is unavailable while the source is fetching or indexing.
- Click Re-fetch all in the "Re-fetch all URLs" dialog.
As a result, Creatio AI Studio reports "Re-fetching all URLs" and fetches every page of the source again. A page whose text has not changed is not indexed again, and the Web Content Fetch Sessions table on the Web fetch tab counts it under Skipped. A page that the website removed is counted under Removed. Each page is replaced only after its new version is fetched, so a page that cannot be fetched keeps serving its current content.
On the URLs tab, Last extracted shows when the served version of a page was extracted, and Last attempt shows when the page was last fetched.
When a website reports a page as removed, Creatio AI Studio deletes its stored copy of the page and stops using it for answers. The row of the page on the URLs tab shows the date of the removal. If the website reports every page of the source as missing at once, the source keeps its stored content.
Refresh a webpage source on a schedule
A schedule keeps a webpage source up to date with no manual work. Creatio AI Studio fetches the pages of the source again at the times you set.
-
Open the knowledge source from the Knowledge list.
-
Go to the Settings tab. The tab is available for webpage sources.
-
Click Set schedule. This opens the "Set schedule" dialog.
-
Select how often to refresh the source in the Repeat field.
Option
What to set
Every few hours
The Interval, from "Every hour" to "Every 7 days." The first refresh runs one interval after you save the schedule.
At a time of day
The Time, the Days, and the Time zone. In the Days field, select "Every day," "Weekdays only," or the days you need.
-
Check the Preview. It shows when the first refresh runs.
-
Click Save.
As a result, Creatio AI Studio refreshes the source on the schedule. The Schedule block on the Settings tab shows the state of the schedule, the last scheduled refresh, and the next refresh. Pages that did not change are not indexed again. When four scheduled refreshes in a row fail because of the website, the schedule pauses.
To manage the schedule, click Change, Pause, Resume, or Remove in the Schedule block. Click View runs to open the Web fetch tab with the history of the refreshes.