71 Commits
Author SHA1 Message Date
Karim shoair 0b74c0f82b docs: make sure the pages are accurate to the changes 2026-10-31 17:51:13 +02:00
Karim shoair fb5417a958 docs: update pages with latest changes including translations 2026-10-31 16:56:42 +02:00
Karim shoair e09f595e66 fix(parser)!: remove any left traces of keep_cdata parameter
The parts that didn't get covered in #493
2026-10-16 01:51:32 +03:00
Karim shoair c8568a4149 docs: update all pages to be clearer, fix mistakes, and improve for SEO 2026-10-15 02:52:19 +03:00
Karim shoair 2228316b40 docs: update the docs accordingly with the latest breaking changes 2026-10-08 23:28:12 +03:00
Karim shoair d5b1995ce6 feat(fetchers): add functions to save browser session state and load it 2026-10-08 23:28:12 +03:00
Karim shoair 0d5be50317 docs: mention the default encoding issue for websites with bad default encoding
Fixes the docs issue mentioned at #469
2026-10-07 19:38:04 +03:00
Karim shoair e45311e2c1 docs!: update pages and skill to cover the recent changes 2026-10-04 22:11:08 +03:00
Karim shoair 1d0d62cfee fix(fetchers): add early validation for the capture_xhr's value before starting the browser 2026-10-04 21:09:10 +03:00
Karim shoair 00d5f3da6e docs: update pages and the skill with the latest changes 2026-10-04 20:59:43 +03:00
Karim shoair 5dac176917 docs: pump version up, update docs, and contact info 2026-09-25 02:57:37 +03:00
Karim shoair 5edd203076 feat(browsers): Include open Shadow DOM content in the response HTML.
Solves #440
2026-09-13 23:30:08 +03:00
Karim shoair f164713efa feat(fetchers): keep browser tabs open and reuse them across requests
Browser sessions no longer close a tab after its request is completed. The tab returns to the pool as ready, and the next request reuses it, reapplying its own timeouts, extra headers, and resource/domain routes (after `unroute_all`) so nothing leaks between requests. Tabs that hit an error or got closed by the browser are closed and evicted, and the internal response listener is detached after every request so reused tabs don't stack handlers.

Adds `close_pages()` to all browser sessions to close every open tab; the next request opens a fresh one. `PagePool` gains `get_ready_page`/`remove_page`/`clear` and `PageInfo.mark_ready`, new pages start busy, and the unused `cleanup_error_pages` is removed.

Proxy-rotation contexts keep closing per request. Docs and the agent skill describe the new lifecycle.
2026-08-23 20:28:30 +03:00
Karim shoair 6ca35d0448 feat(spiders): Adding easy interface for building RAG systems 2026-08-23 16:26:15 +03:00
Karim shoair 9ecffdb42f fix(cdp): Make browsers accept CDP urls with http protocol 2026-07-26 16:23:00 +03:00
mehmet turac 35a83d38c7 docs: clarify stealth init script visibility 2026-06-14 07:43:04 +03:00
Karim shoair 835e7ca8c3 docs: update old examples 2026-04-26 02:46:13 +03:00
Karim shoair 9af644ef84 docs: improving the code copy-paste experience and use less tokens for the agent skill 2026-04-22 16:52:59 +02:00
Karim shoair 158ff04c96 docs: add the new page_setup feature 2026-04-13 04:38:23 +02:00
Karim shoair 131b729f22 docs: adding the new features 2026-04-12 18:07:38 +02:00
Karim shoair f756e519f3 docs: update pages with the new changes 2026-04-05 19:40:05 +02:00
Karim shoair 1f1e475772 docs: add the browser path to docs 2026-03-30 03:37:38 +02:00
Karim shoair 61cda587be docs: update pages with the XHR feature 2026-03-29 23:12:56 +02:00
Karim shoair 73832f9dfe docs: update the website to reflect the google referer logic 2026-03-09 00:29:54 +02:00
Karim shoair b8b9a7f864 docs: general correction across all the pages
Automated. I don't know how I missed all of that!
2026-02-15 03:58:51 +02:00
Karim shoair 7b4523af8e docs: update the titles for all files 2026-02-15 02:35:39 +02:00
Karim shoair c3374bce41 docs: add a note about response body 2026-02-15 01:03:36 +02:00
Karim shoair dee0fd00b8 docs: use more accurate naming for proxy rotation strategy 2026-02-14 23:21:08 +02:00
Karim shoair c6a662eb38 docs(dynamic pages): add an important warning 2026-02-13 18:45:44 +02:00
Karim shoair 8902034418 docs: improve wording, style, and use more visual ways 2026-02-11 03:04:35 +02:00
Karim shoair a72886f158 docs: update camoufox instructions for StealthyFetcher 2026-02-10 21:58:08 +02:00
Karim shoair f350800fd7 docs: updating Dynamic websites with protection page and some corrections 2026-02-10 21:48:09 +02:00
Karim shoair dba3e9d53e docs: updating dynamic websites page and some corrections 2026-02-10 21:23:14 +02:00
Karim shoair 8b1704a87e docs: use better import for ProxyRotator class 2026-02-10 21:20:43 +02:00
Karim shoair 8df74dc57b docs: updating http requests page and some corrections 2026-02-10 19:07:09 +02:00
Karim shoair f25fe64d3c docs: updating fetcher choosing page and some corrections 2026-02-10 19:01:50 +02:00
Karim shoair db41a6c4b6 docs: update pages with the new media for testing zensical 2026-02-08 20:46:14 +02:00
Karim shoair 55ff20c238 docs: update replacing AI article 2025-12-31 18:28:28 +02:00
Karim shoair d4ae6c49c0 docs: update the StealthyFetcher page 2025-12-31 00:22:33 +02:00
Karim shoair a2b5628a73 docs: update DynamicFetcher page 2025-12-29 22:02:01 +02:00
Karim shoair 67e8f4d515 docs: update DynamicFetcher page 2025-12-29 20:48:45 +02:00
Karim shoair e1cac0f69f docs: update requests page 2025-12-29 20:48:32 +02:00
Karim shoair 6c25dccb05 docs: updating the Fetcher Basics and the HTTP requests pages 2025-12-28 21:57:21 +02:00
Karim shoair ecb97d3fd6 docs: add docs for timezone_id argument 2025-12-18 01:16:22 +02:00
Karim shoair f84fbe8814 docs: update the renamed argument 2025-11-26 02:12:50 +02:00
Karim shoair 026161c8f8 docs: update docs to add file downloading examples 2025-11-10 03:02:41 +02:00
Karim shoair 372fc1e051 docs: Style updates and a lot of clarifications 2025-11-09 02:15:43 +02:00
Karim shoair 3b5f0470d2 docs: Update docs accordingly and some adjustments 2025-11-08 20:18:58 +02:00
Karim shoair 339ec43341 docs: adding separate doc for external browser 2025-10-27 06:32:31 +03:00
Karim shoair 13b9a4668c feat(DynamicSession): New option to add extra browser flags 2025-10-26 16:53:27 +03:00