Syncing a website
Re-read a site you added, when you ask or on a schedule. New pages are offered, never added.
Re-read a site
In Knowledge, find the site in the list and press Open site on its row. Its tabs are All, Can’t read, Found and Excluded, and Exclude takes a page out of every agent.
Open Manage and press Re-sync now. For a schedule, open Manage › Site settings, switch on Re-read on a schedule and pick Weekly or Monthly under How often, then press Save. It’s off until you switch it on, so nothing is re-read unless you ask.
A re-read also looks for pages the site has gained. A suggestion such as 12 pages found appears on the site’s page. Press Review, tick the pages you want and add them. Nothing is imported until you do.
Early Access re-reads
On Early Access, you can re-read a site yourself once every 7 days. Re-sync now here and Sync and Re-read in Knowledge share that one clock, and the button says when the next one opens. Scheduled re-reads and Try again on a page that failed don’t count; for an earlier one, ask us.
What gets read
The first group in Site settings, and the one worth setting for a big site.
Use the sitemapon by defaultHow hiy finds pages. Turn it off and hiy skips the sitemap and the pages your home page links to. It still reads the site’s feed and anything you list under Also read these.
Respect robots.txton by defaultHonours the site’s
Disallowrules. Leave it on for a site you don’t own.
What gets kept
What survives from a page once it has been fetched.
How pages are read
Most pages are plain HTML. Some sites paint their text with JavaScript, and those come back empty.
Each workspace has a daily allowance of rendered pages. A page over it keeps the plain read and can be tried again tomorrow. Where rendering isn’t available yet, all three read like Fast.
Warning
Delete site and its pages removes the pages and what your agents learned from them, and can’t be undone. Any agent using them stops answering from them at its next train.
Good to know
- A page that fails keeps its last text and shows under Can’t read. Open it and press Try again, or Change the address if it moved (the same site only).
- Stop syncing keeps every page, your settings and your history, so turning it back on is one switch in Site settings.
- Manage › Sync history lists every run, who started it and what it found.
- Add a source › From your website re-reads a site from the Add knowledge sheet and shows how far it has read. The bell says when a re-read you pressed is done, and when a scheduled one found something new.
- A re-read that would take you past your plan’s words keeps that page’s old text.
- A changed page joins your agent’s answers at the next train. What changed shows the old copy against the new, and Keep the old copy puts the earlier text back.