1832 Commits
Author SHA1 Message Date
warm-cloth 1ebee696a7 fix(agent-skill): accept uv for OpenClaw's installer check (#586) 2026-11-14 19:28:42 +02:00
Karim shoair 66b885d811 build: pump version up and deps 2026-11-14 19:24:39 +02:00
mschoi0411 63a040f601 fix(spiders): interpret asctime Retry-After dates as UTC (#561) 2026-11-14 18:27:06 +02:00
Karim shoair 193bd48776 fix(spider)!: make follow function doesn't inherit any request data from parent
Fixes #558 #587
Avoids the whole bug class.
Closes #559
2026-11-14 18:23:25 +02:00
SirTeggunandKarim shoair 822bb5f80c fix: deny file URL schema in LinkExtractor to prevent local file disclosure (#542)
Co-authored-by: Karim shoair <D4Vinci@users.noreply.github.com>
2026-11-14 16:24:31 +02:00
Karim shoair bb223f2f5f ci: add a new workflow to handle stale issues/PRs 2026-11-11 19:04:10 +02:00
Karim shoair c92fd624a7 docs: update a sponsor image 2026-11-11 19:03:06 +02:00
Karim shoair c5121c7c06 docs: add a new sponsor 2026-11-09 12:36:34 +02:00
Karim shoair 30cadb56a4 docs: update a sponsor offer 2026-11-08 15:18:18 +02:00
Karim shoair bc602488b4 docs: add additional video demo 2026-11-05 20:01:44 +02:00
Karim shoair 3a4bb5b0a0 v0.5.1 (#556) v0.5.1 2026-11-04 22:28:59 +02:00
Karim shoair 124bd58df3 docs: Update changelog with the latest changes 2026-11-04 22:27:55 +02:00
Karim shoair 28bdfbb2a1 test: fix repeated issue on GitHub Actions 2026-11-04 18:06:10 +02:00
Karim shoair f53f8b2ed5 Merge branch 'main' into dev 2026-11-04 16:18:18 +02:00
Karim shoair 2e27e16fa9 docs: update the migration guide with RemoteSession addition 2026-11-04 16:17:22 +02:00
W fb26a7ab91 fix(shell): stop dropping cookies whose values SimpleCookie rejects (#541)
* fix(shell): stop dropping cookies whose values SimpleCookie rejects

_CookieParser fed the cookie string to http.cookies.SimpleCookie, which
stops at the first value it considers illegal (a space, a double quote,
...) and silently discards that cookie and everything after it. Cookie
strings copied from DevTools contain such values regularly, so curl
commands parsed by uncurl/curl2fetcher and `extract --cookies` lost
cookies without any warning.

Split the string on ";" instead and keep each value as sent, including
surrounding quotes, which SimpleCookie used to strip. Set-Cookie
attribute names are still skipped, as before.

* fix(shell): keep request cookies named partitioned
2026-11-04 16:08:55 +02:00
Jay Hemnani ead76eadde fix(shell): accept curl's -L flag (#554) 2026-11-04 16:04:36 +02:00
Jay Hemnani 7ac09b6519 fix(parser): return the value of XPath expressions that do not select nodes (#552)
string(), count(), boolean() and the like return one value, not a list of nodes.
xpath() walked it as a list: a string became one Selector per character and failed
with "Type 'str' cannot be serialized" on .getall(), and a number raised
"'float' object is not iterable".
2026-11-04 16:03:06 +02:00
Lykin 12c855d829 fix(spiders): handle CR record separators in CSV feeds (#551) 2026-11-04 15:56:27 +02:00
Karim shoair 6384909076 docs: re-add a sponsor
This reverts commit 8a340fe207.
2026-11-04 11:39:53 +02:00
Wu Shuwen f0a54ac9b5 fix(spiders): skip malformed links during extraction (#548) 2026-11-03 22:21:08 +02:00
ZHANG YU fc6c7e62c7 fix(spiders): isolate temporary files for concurrent cache writes (#546) 2026-11-03 22:18:16 +02:00
kevin c2bd467ad7 fix(fetchers): decode percent-encoded credentials in proxy URLs (#545) 2026-11-03 22:07:15 +02:00
Karim shoair f35fa0b93c test: update tests with recent changes 2026-11-03 18:52:25 +02:00
Karim shoair 3eb3b31641 docs: update docs and skill with recent changes 2026-11-03 18:52:25 +02:00
Karim shoair c83d10bdd1 feat(mcp): replace one mouse drag action with two mouse control actions
Making it able to solve all slider captchas, tested on all datadome and geetest CAPTCHAs I can find
2026-11-03 18:52:25 +02:00
Karim shoair 22318c34a6 test: add tests for the latest changes 2026-11-03 18:52:25 +02:00
Karim shoair c476b73ce0 docs: updates docs and skill with the latest changes 2026-11-03 18:52:25 +02:00
Karim shoair a16d0cf8f0 fix(mcp): rewrite docstrings to save tokens 2026-11-03 18:52:25 +02:00
Karim shoair 37f448d689 feat(mcp): add browser observe tool to record the browser. 2026-11-03 18:52:25 +02:00
Nor.na 104294c2d8 fix(parser): honor zero similarity threshold without reference attributes (#539) 2026-11-02 22:16:22 +02:00
Rayen Mlayeh 49eaff11da fix(spiders): fetch robots.txt once per domain under concurrent requests (#538) 2026-11-02 22:07:57 +02:00
Karim shoair e3005a1d40 feat(spiders): add logging file encoding parameter
Better fix than #538
2026-11-02 21:50:48 +02:00
rushilnimbalker250 254b31322b fix(parser): handle empty HTML roots (#536) 2026-11-02 20:37:23 +02:00
cangen185-sudo 37950a6304 style(spiders): raise error on truncated and invalid gzip in templates (#522) 2026-11-02 20:34:22 +02:00
Maulana 41d88c0696 test: fix flaky Cloudflare readiness test (#515) 2026-11-02 20:29:17 +02:00
Karim shoair be4e412daa Merge branch 'main' into dev 2026-11-02 16:39:54 +02:00
Karim shoair 95cf9019ab docs: add a new sponsor 2026-11-01 19:59:59 +02:00
postoso d0639cddcb fix(cli): make --ai-targeted enable ad blocking on fetch (#527) 2026-10-31 18:11:16 +02:00
Karim shoair 0b74c0f82b docs: make sure the pages are accurate to the changes 2026-10-31 17:51:13 +02:00
Karim shoair 70ded5d253 Merge branch 'main' into dev 2026-10-31 17:04:11 +02:00
Karim shoair fb5417a958 docs: update pages with latest changes including translations 2026-10-31 16:56:42 +02:00
Karim shoair e7d435de87 agent: update skill with the current changes 2026-10-31 16:51:30 +02:00
Karim shoair acc03fcee5 test: update to the current changes 2026-10-31 16:50:39 +02:00
Karim shoair b02a20af68 feat(fetchers)!: move remote browsers sessions to separate locations with more control 2026-10-31 16:39:10 +02:00
Karim shoair 26dc85a4fa docs: adding a new sponsor 2026-10-29 17:00:52 +03:00
Karim shoair 0336400416 docs: adding a new sponsor 2026-10-28 12:31:13 +03:00
Karim shoair c0009dbee8 docs: update a sponsor 2026-10-27 11:27:55 +03:00
Karim shoair 063feb0cd3 docs: update skill instructions 2026-10-26 14:47:34 +03:00
Karim shoair 82180a1b3e docs: update a sponsor url 2026-10-22 23:43:40 +03:00