mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-10-02 07:44:37 +08:00
7d18cc033c54d91e57bbbb34c28c44f0426c8753
404
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ray
|
7d18cc033c |
docs: drop stray blank lines in the cloud quickstart snippet
Claude-Session: https://claude.ai/code/session_01Wh75QagUhr5DPVATjmGrCe |
||
|
Mingtian Zhang
|
9fdbc7d6ad |
Merge pull request #451 from VectifyAI/docs/query-cost-chart
improve model name |
||
|
张鸣天
|
2d02e8b6e8 | improve model name | ||
|
Mingtian Zhang
|
72469fb9e2 |
Merge pull request #449 from VectifyAI/docs/query-cost-chart
fix agent integration |
||
|
张鸣天
|
aa153ab1e7 | fix agent integration | ||
|
Mingtian Zhang
|
635bca4e65 |
Merge pull request #448 from VectifyAI/docs/query-cost-chart
fix agent integration |
||
|
张鸣天
|
d1b9fbd1b9 | fix agent integration | ||
|
Mingtian Zhang
|
0a3b563d69 |
Merge pull request #447 from VectifyAI/docs/query-cost-chart
Docs/query cost chart |
||
|
张鸣天
|
869c76ad17 | fix agent integration | ||
|
张鸣天
|
70cc912300 | fix agent integration | ||
|
Mingtian Zhang
|
4f8414c328 |
Merge pull request #446 from VectifyAI/docs/query-cost-chart
fix chat |
||
|
张鸣天
|
7cc46303d6 | fix chat | ||
|
Mingtian Zhang
|
75bca66e9b |
Merge pull request #445 from VectifyAI/docs/query-cost-chart
add image to readme |
||
|
张鸣天
|
10e7433ec6 | add image to readme | ||
|
Ray
|
9fee239b17 |
docs: correct what the index model does (#441)
* docs: correct what the index model does The index model does not build the tree structure — Flash extracts it from the document layout without an LLM. The model only summarizes and refines the tree. Claude-Session: https://claude.ai/code/session_01EtDZekHStmxXNexn95aAeD * docs: name PageIndex Flash in the submit_document note Claude-Session: https://claude.ai/code/session_01EtDZekHStmxXNexn95aAeDv0.2.12 |
||
|
Ray
|
21f2b1018e |
docs: FinanceBench chart as local light/dark assets (#440)
* docs: swap the FinanceBench chart for local light/dark assets Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF * docs: replace the FinanceBench chart with light/dark assets Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF * docs: narrow the FinanceBench chart to 60% Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF * docs: set the FinanceBench chart to 65% Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF * docs: set the FinanceBench chart to 70% Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF |
||
|
Ray
|
688ce200fa |
docs: hide the FinanceBench accuracy chart
Claude-Session: https://claude.ai/code/session_012edjBdTjAM24UAZGyq6TfF |
||
|
Ray
|
d947ab6c93 |
docs: README structure and layout pass (#438)
* docs: the benchmark charts render at 70% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the benchmark charts render centered at 80% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the benchmark charts render at 85% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the benchmark charts render at 90% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the collapsed sections drop the spacer <br> Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the header link row drops Discord Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Updates entry comments out the MCP/API pointer Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the one-line summary stands without the Why it works heading Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: TEMP four punchline variants side by side Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the one-line summary reads as a centered pull quote Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: TEMP alert-style variants next to the pull quote Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the pull quote sits under an In one sentence heading Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the pull quote sits under a TL;DR heading Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the TLDR quote bolds the claims and drops the vector DB and chunking Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the TLDR quote reads no vector DBs or chunking Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the TLDR quote leaves retrieval unbolded Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Retrieve step leads with agentically Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the TLDR quote reads left-aligned Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the TLDR quote wraps on its own Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations example uses a plain system-message string Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations example keeps the system prompt inside the code block Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations example inlines the system prompt Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Cloud example drops the doubled blank line Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench case study returns to Benchmarks Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench paragraph speaks of PageIndex directly Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench sentence names the corpus and the margin plainly Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench sentence drops the corpus aside Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench sentence keeps the benchmark description Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Benchmarks splits into the local open-source run and FinanceBench Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: top-level sections use single-hash headings again Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the local benchmark part is headed Local mode Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the local benchmark part is headed PageIndex Local Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the benchmark charts render at 80% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench chart stays at 90% and the benchmark gloss goes in parentheses Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench chart returns to its original 70% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the local benchmark charts render at 70% width Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the local benchmark part is headed Running PageIndex locally, charts at 75% Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations example indents the prompt continuation Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations prompt breaks before the cite tag Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the citations prompt breaks before using Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the collapsed usage guides follow the Quickstart Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: a Usage section holds the two collapsed guides Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage guides sit at h3 with their steps one level down Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage guides sit at h2 so their steps keep their levels Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage guides sit at h3, inner levels untouched Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: each usage step collapses on its own under Usage Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Usage keeps its two guides as headings with collapsed items inside Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the *_config note folds into Other agent frameworks Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Usage and the Detailed Usage Guide open with a line of orientation Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in drops the happy-path idiom Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in names the two guides directly Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the guide and integration lead-ins say what each block holds Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in contrasts direct use with integration Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in lists its two guides; their intros stay general Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in is one line with (i) and (ii) Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in says SDK client and your own agent Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in uses parallel verbs Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in says access rather than call Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in drops the verb in (i) Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the integration lead-in reads as one plain sentence pair Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the client guide is headed Use PageIndex through the SDK client Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in lists its two ways as bullets Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage lead-in returns to one (i)/(ii) line Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Ready to Try It links cover only the noun Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Usage section is headed Usage Guide Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the summary heading reads tl;dr Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: collapsed items use bold summaries so they sit close together Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: expanded items get a line of air under their summary Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: collapsed items try h4 summaries again Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: collapsed items settle on bold summaries with a spacer Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: a line of air between neighbouring collapsed items Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Step 2 anchor lives inside its summary so item gaps match Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the two guide headings rely on their generated anchors Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the SDK client guide opens with a plainer line Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the SDK client guide lead-in drops workflow Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the SDK client guide lead-in names its three steps Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the SDK client guide lead-in, polished Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the SDK client guide lead-in points below Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: a line of air before the integration heading Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: no spacer before the integration heading Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the two usage guides are labelled (a) and (b) Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench closing line names the evaluation Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench closing line links only the results Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench claim links straight to its benchmark subsection Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the indexing-cost line names the model in prose Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench subsection is headed Leading on FinanceBench Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the FinanceBench heading reads Leading accuracy on FinanceBench Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Updates lists the PageIndex File System again Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the File System update entry links in plain weight Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the header link row gains Blog Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the summary heading reads TL;DR Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Model Recommendations points at the Usage Guide Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Model Recommendations names the SDK client usage guide Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the usage-guide pointer promises more than models Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the usage-guide pointer, one word shorter Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the integration lead-in says each example covers one framework Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the integration lead-in says a different framework Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: each framework fold shows the one-call and explicit forms end to end Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the streaming example uses a literal and the quickstart drops trailing spaces Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Flash update entry says where the structure comes from Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Flash update entry, tighter Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Flash update entry says built by an LLM Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Flash update entry, Ray's wording Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: the Flash update entry keeps extracted heuristically Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k * docs: Updates dates read [Aug '26] Claude-Session: https://claude.ai/code/session_01BtbxfFP1wdkFou1FMCMA6k |
||
|
akong
|
afb5e11976 | docs: reduce README banner image size | ||
|
Mingtian Zhang
|
c346056ae9 |
Replace PageIndex banner image
Updated the banner image in the README file. |
||
|
Ray
|
d4ce6ee65f |
docs: the run_messages comment keeps the principle, drops the version numbers
The rationale to preserve is that execution policy belongs to the vendor and only the runner's history is ground truth; the exact 1.0/1.1 switch point lives in the previous commit's message. |
||
|
Ray
|
070293bdca |
test: the messages max_tokens test follows anthropic 1.1's stop policy
anthropic 1.1.0 replaced the runner's refusal special case with an explicit stop-reason table: every non-tool_use stop is terminal and its tool_use blocks are never executed (1.0 executed a max_tokens turn's complete blocks). The envelope already keys on the runner's history, so the product adapts by design; only the test had the 1.0 behavior baked in. It now asserts the version-appropriate shape on both sides, and the run_messages comment describing the old behavior is reworded to name the policy split. Verified: 434 green on anthropic 1.1.0 (CI's failing config) and on 0.120.2 (the pre-1.1 branch); no-frameworks collection stays clean. |
||
|
Ray
|
f47f27b7b3 |
docs: README restructure with quickstart, benchmarks, and a collapsed usage guide (#431)
Quickstart on the v0.2.11 index=/chat= spellings, a citations example, indexing-time benchmarks with new charts, the own-agent integration in its own collapsed block, a rewritten PageIndex Cloud section, the reasoning positioning restored, and prose em dashes replaced with plainer punctuation. Earlier snapshots of this branch landed via #418/#419; this squash carries the tail. |
||
|
Ray
|
31f01910cd |
docs: API-key URLs follow the dashboard move to developer.pageindex.ai
dash.pageindex.ai/api-keys now 307-redirects to developer.pageindex.ai/api-keys, and the README (PR #431) already standardized on the developer domain — the client docstring and the CloudClient keyless error message were the last SDK references to the old host. |
||
|
Ray
|
57252e2686 |
fix: the LiteLLM lanes hide litellm's bridge usage warning (#430)
Streaming chat() on an OpenAI gpt-5.4+ model with function tools makes litellm re-route chat.completions through the Responses API. When the stream ends, litellm's logging stores a chat-shaped usage dict inside a ResponseAPIUsage field and model_dump()s it, so pydantic prints "Expected `ResponseAPIUsage` - serialized value may not be as expected" once per streamed turn. litellm does this on purpose (litellm_logging._get_assembled_streaming_response, 1.97 and 1.98) and the answer is unaffected. Both seams that hand a LiteLLM-routed model to openai-agents — the chat lanes' model builder and openai_agent_config()'s litellm/ lane — register one warnings filter matching exactly that message; every other warning still surfaces. A process-wide filter is the only placement that works: litellm emits the warning from its async success handler on the worker thread, out of catch_warnings' reach. Claude-Session: https://claude.ai/code/session_01APJbbp3jpRxJhvchcU2Aed |
||
|
Ray
|
174f95f35b |
fix: the slots accept any Mapping at runtime; eight docstrings stop calling cloud tool scoping server-side (#429)
fix: the slots accept any Mapping at runtime, as their annotation admits; eight docstrings stop calling cloud tool scoping server-side |
||
|
Ray
|
b9a9a3b5aa |
fix: the .env search ends at the cwd tree; a local client with a blank chat_model refuses at the chat door; storage_path is typed PathLike (#428)
* fix: .env stays unset when the cwd tree has none; a local client with a blank chat_model refuses at the chat door; storage_path is typed PathLike find_dotenv(usecwd=True) returns '' when nothing is reachable from the cwd, and `or None` turned that into load_dotenv's own upward walk from utils.py — the install-dir leak the cwd search was added to replace. A pip-installed SDK could load another project's .env from above site-packages, silently. _local_chat treats a blank chat_model as "managed chat", which a client without an api_key does not have: chat_completions() then reached for LocalAPI.chat_completions and raised a bare AttributeError. The managed branch now refuses as a PageIndexAPIError naming chat_model. py.typed made the annotations authoritative while storage_path was typed str; _ARG_TYPES accepts os.PathLike, so Path(...) ran fine and failed the user's type check. Both signatures and LocalIndexConfig now say so. Claude-Session: https://claude.ai/code/session_017Fd7jVm366S2Xamzhxv6yb * fix: the exported config shapes pass into index=/chat=; a comment and two docstrings stop overclaiming The slots were annotated dict[str, Any]. A TypedDict is consistent with Mapping[str, object], never with dict (PEP 589: a dict-typed receiver could write arbitrary keys through it), so the four shapes types.py exports — and py.typed advertises to installed callers' checkers — could not be passed to the one place they describe. pyright on a probe that does exactly that: 9 errors before, 0 after. The constructor only reads the slot (items(), then a fresh conf dict), so Mapping is the honest bound; a plain dict is a Mapping, and TypedDict instances are plain dicts at runtime, so nothing moves at runtime. The _ARG_TYPES comment said "every value" is shape-checked; api_key is not in the table (its empty check is separate, its type check stays unchecked by ruling), so the comment now speaks for the table only. _local_doc_scope and _require_local_scope still explained the cloud drop as "scoping is server-side" — true of the managed chat, which never reaches either function. What reaches them on a cloud client is own-model chat and the config helpers, whose cloud tools take no allowlist: targeting there is prompt-level only, as the error message between them already said. 434 passed; pyright on pageindex/ unchanged at 235 (0 in the touched files, before and after). Claude-Session: https://claude.ai/code/session_01TxG8u8x29XRnK4yscZVCch * test: the install-dir .env test is named for what it asserts Claude-Session: https://claude.ai/code/session_017Fd7jVm366S2Xamzhxv6yb |
||
|
Ray
|
920db2b1b1 |
feat: the client grows two sides — documents and chat each pick their home (#424)
* feat: the client grows two sides — documents and chat each pick their home
One client, two independent switches: api_key decides where documents
live (the PageIndex cloud, or the local store); a configured chat model
decides who answers (your own model in your process, or the managed
cloud chat). Their free combination opens the bridge — cloud documents,
your model — and the fourth cell stays unspellable.
- index=/chat= slots: string shorthand or grouped dict, 1:1 with the
flat arguments; one spelling per side, sides mix freely
- optional "type" everywhere (top-level and in either dict): always
omittable, checked against the content, meaningful alone —
type="cloud" is a keyless cloud spelling
- PAGEINDEX_API_KEY is read only when the code explicitly says cloud
(PageIndexCloudClient(), type="cloud", "pageindex-cloud",
{"type": "cloud"}); a bare PageIndexClient() stays local
- bare mode words ("cloud", "local", …) are reserved: they error
with the real spellings instead of silently parsing as model names
- bridge chat runs the in-process agent over the live cloud MCP tools
and instructions; doc_id targets at the prompt level; citations stay
managed-only; an auth-shaped backend failure explains whose
credentials run the model
- typed shapes (IndexConfig, ChatConfig) ship as optional annotations
Every previously working program is byte-for-byte unchanged: the only
behavioral deltas are error paths — reworded guidance, and the
api_key+chat_model combination graduating from an error into the
bridge.
* fix: the constructor refuses empty and mistyped values on every spelling
- .env keys reach all four keyless-cloud spellings: utils' import-time
load_dotenv now runs before every PAGEINDEX_API_KEY read
- an empty chat-side value ("", {}) errors instead of silently selecting
own-model chat on the default model; None-valued slot keys mean absent,
exactly like the flat arguments
- _local_chat derives from chat_model, so a post-construction assignment
switches the whole client, never half of it
- model= beside a slot gets the split guidance (index_model=/chat_model=)
instead of "two spellings of the same thing"
- the messages door wraps provider failures through _model_backend_error,
and 401s count as auth-shaped even without "api key" in the text
- keyless-cloud hints name the spelling that actually combines; slot
strings are stripped; wrong-typed values raise PageIndexAPIError
- retrieve_model/chat_backend docs drop the stale "Local mode only";
the local-scope refusal no longer claims bridge tools are server-scoped
* fix: type= cross-checks the index slot; the cloud pinned class frees its chat side
- type= beside index= now does what the docstring promises: agreement
passes, disagreement errors, and a mistyped value reports the
vocabulary error instead of a spelling collision
- PageIndexCloudClient grows the chat-side arguments (chat=, chat_model,
retrieve_model, chat_backend), so "pin the index side" is literally
true and the chat surfaces' construct-with-chat_model guidance is
followable on it
- the four chat doors' doc_id entries carry the enforcement split the
config helpers already state (local: tool-layer allowlist; cloud:
prompt-level / server-side)
- types.py stops claiming slot keys share the flat names — the side
prefix is factored out, index={"model"} is index_model=
* docs: bridge-reachable wording — dependency errors say own-model chat, hints name a chat= model
- the three framework-missing errors said "in local mode", which is
wrong on a bridge client (cloud documents + own model) — they now
explain the dependency the way the surfaces do: your own chat model
- the construct-with guidance reads "(or a chat= model)": a bare
chat="pageindex-cloud" is also chat= but selects the managed side
- the mechanical Local-only → Own-model-chat-only substitution left
orphan fragments and two overlong lines; those paragraphs re-flowed
* fix: managed chat reads None; the bridge stops paying per-turn tool lists
- a managed-chat cloud client stores chat_model/chat_backend as None, so
the documented attribute reads instead of raising AttributeError;
_local_chat derives from "is a chat model configured"
- McpBridge caches tools/list per session — every chat turn rebuilds the
tool set, and the round trip was pure latency; the 404 session-expiry
reset drops the cache with the session
- run_messages builds tools before the transport: on a bridge client
that build is network I/O, and a failure there stranded a per-call
anthropic client ahead of the try/finally
* refactor: the side declaration is spelled mode=, not type=
"type" is Python's own word — a builtin, and "data type" beside the
TypedDict shapes; "mode" is what the SDK already calls the two sides
("local mode", "cloud mode"). Same grammar everywhere the declaration
appears: the top-level argument, the index dict, the chat dict, the
typed shapes. The rename also frees the builtin inside the constructor,
so the shape-check error names the offending class through type() again.
"type" in a slot dict is now an ordinary unknown key.
* fix: the reserved-word errors stop calling "cloud" not a mode word
With the declaration key spelled mode=, 'index="cloud" is not a mode
word' contradicted its own remedy, index={"mode": "cloud"} — "cloud" is
exactly a mode value. The four bare strings are reserved words; the
message now says so.
* fix: a blank tools/list is not cached; the auth note's managed exit is chat-lane only
- McpBridge.list_tools caches only a non-empty list — a transient blank
(a deploy blip, a gate misconfiguration) would otherwise run every later
turn with zero tools while the instructions still name them, and only a
404 session reset could clear it
- the 401 architecture note appends "drop the chat model configuration"
only on the chat lane: responses() and messages() refuse a client
without an own model, so on those lanes the exit sent the caller in a
circle
- CloudIndexConfig says api_key is omittable only while mode: "cloud"
stays — index={} refuses as an empty dict rather than reading the env
* fix: the bridge fetches tools/list per call again; .env resolves from the cwd
- McpBridge.list_tools no longer caches: the tool set is built once per
SDK call (Agent(tools=...) ahead of Runner.run; build_anthropic_tools
ahead of tool_runner), not per model turn, so the cache saved one round
trip per later call while a mid-pagination 404 replayed a dead cursor
into a duplicated (and cached) list, and the list went out by reference
across a lock dropped between miss and store
- utils.load_dotenv searches upward from the cwd: a bare load_dotenv()
walked up from utils.py, which is site-packages for an installed SDK,
so the four keyless-cloud spellings never saw a project-root .env; the
package-relative walk stays as the fallback
- the emptiness guard strips strings: chat_model=" " selected own-model
chat, the silent flip the guard's own comment rules out
- the _local_chat comment stops advertising post-construction assignment
as a full mode switch
* fix: the pinned classes take index=/chat=; "cloud"/"local" are mode words; a blank chat_model stays managed
- PageIndexLocalClient takes index= and chat=, PageIndexCloudClient takes
index= — the grouped spelling of the flat vocabulary each already took;
their refusals name the class and an exit that class can take, and the
mode cross-check runs before any environment read
- "cloud" and "local" are accepted wherever "pageindex-cloud" was (index=,
chat=, mode=, {"mode": ...}), case- and whitespace-insensitive; "hosted"
and "managed" still refuse, pointing at the real word
- every spelling strips its strings, and the slot spellings' type/empty
errors name the slot key (index["model"]), not the flat argument
- _local_chat treats a blank chat_model as managed: the constructor
refuses "", so assignment agrees instead of opening the bridge on a
nameless model; openai_agent_config carries no model then either
- an empty MCP tools/list raises like empty instructions does — a
zero-tool agent would answer from the model's own knowledge silently
- enable_citations names the real gate (managed vs own chat), not
"cloud-only", on a cloud own-model client
- pageindex/py.typed: the exported config TypedDicts reach installed
type-checked callers
* test: the two framework-door tests skip without openai-agents
as_openai_tools() and openai_agent_config() need the agents package, which
the "without frameworks" CI legs do not install — the same importorskip
every other test on those doors already carries.
v0.2.11
|
||
|
Ray
|
416e304f51 |
perf: expand schedules dependency-exact at thirty-two concurrent proposals (#422)
* perf: expand proposes a wave of nodes concurrently The expand loop awaited one propose_children at a time — 20-30 nodes at ~3s each put 1-3 minutes of pure round-trip latency on every default local submit. Nodes waiting in a wave are all frontier leaves whose decisions cannot affect each other, so the model half now runs concurrently (EXPAND_CONCURRENCY = 8) while the apply half stays serial in wave order: decisions, log entries, and child ids land exactly as before, and children attach into the next wave. A fatal classification still aborts the run right after the wave's gather. Benchmarked on real PDFs with a fixed-latency fake model: 408 pages 21.1s -> 3.0s, 758 pages 28.2s -> 3.5s (7-8x); final trees byte-identical to the serial pass on both. The cap stays low on purpose: expand treats an exhausted retry ladder as fatal, and a wide burst on a rate-limited account would trip exactly that — 8 already collapses minutes to seconds. * perf: expand schedules dependency-exact instead of in waves A child's only prerequisite is its own parent's apply, so each kept node gathers its children directly rather than waiting for its whole generation to finish. Same recursive shape as summarize_tree; the semaphore still caps in-flight proposals at 8; trees are unchanged. * perf: expand admits thirty-two concurrent proposals Cap sweeps on six real documents put the speed plateau at 32: the ready frontier tops out at 21-28 nodes on few-hundred-page PDFs, so 64 buys nothing while doubling the burst. Live runs at 32 cut the expand phase 24-30% on the two documents wide enough to feel it, with zero ladder retries anywhere - and summaries already burst twice as wide through the same ladder. |
||
|
Ray
|
8289729aff |
fix: indexing failures surface loud, chat envelopes append verbatim (#421)
Indexing: dead credentials or a missing model fail the run instead of storing a document with blank summaries; a 400 (context_length_exceeded) skips the retry ladder — the prompt will not shrink — and stays a per-prompt failure the run absorbs; all-empty model replies can no longer store a retrieval-ready document; the one-sentence doc description absorbs its own context overflow instead of discarding a fully indexed document; the heading-less flash refusal points at mode='standard'. Chat: messages() output is append-verbatim clean — unset response-only defaults are dropped (no "caller": null the request schema rejects); Claude cache marks follow the wire routing; model_settings and name are openai_agent_config parameters; one Anthropic client per backend; lifted thinking defaults are clamped to the model's output ceiling from LiteLLM's capability map. Store and inputs: lone surrogates are scrubbed from page text and the stored basename, so the returned name is byte-for-byte the stored name and the rename warning fires; NaN/Infinity metadata is rejected at the gate; every cloud error now carries its HTTP status. CLI: the flash lane resolves the summary model through ConfigLoader like the standard and markdown lanes; an empty flash structure errors like the SDK instead of writing "structure": [] with exit 0; --summary-model reaches the markdown lane; the SDK page-spec surface keeps 0.2.10's whitespace tolerance while the tool layer stays strict. pypdfium2 stays on the 5.x line for every install; the 4.x code paths are tested compatibility insurance with their own CI leg; process-pool construction failure falls back to the sequential parse; a py3.10 GC flake in text extraction is fixed. Port of feat/local-chat 0667e3b..1993740 (28 commits); README and assets untouched. |
||
|
Mingtian Zhang
|
5f44d691f3 |
Feat/newreadme (#419)
* improve readme |
||
|
Mingtian Zhang
|
18ea86a643 |
Feat/newreadme (#418)
improve readme |
||
|
Mingtian Zhang
|
8bcd7b7ee0 |
Remove open-source ecosystem section from README
Removed the section about the open-source ecosystem and related projects. |
||
|
Ray
|
418d1549bb | docs: weave the original README's highlights back into the SDK rewrite | ||
|
张鸣天
|
f71afefeb0 | improve readme | ||
|
Ray
|
b628190529 |
ci: dev tags publish to PyPI only — skip the GitHub Release (#414)
Dev builds need an explicit ==pin to install, so their GitHub Releases carry no install value and double the feed next to the same-day stable. |
||
|
Ray
|
ba0ef02d78 |
fix: Aug 18-19 review hardening — five max-effort rounds across tools, chat, and flash (#413)
Squash-port of feat/local-chat's post-dev5 wave (03ffab3..49a24e1): five review rounds of reproduced-then-fixed findings. Highlights: standalone agent_instructions back on the strict shadow check (the *_agent_config bundles keep the relaxed in-set check they can prove); caller-owned http_client survives the per-call backend closes; get_tree keeps key_items under the flash merge default; OpenAI-protocol classification follows litellm's own routing (azure/openrouter/deepseek/groq/xai); thinking-aware max_tokens default shared by messages() and anthropic_runner_config(thinking=); 401/403 re-raise instead of retry-coaching envelopes, bridge errors carry status_code; cloud discovery + instructions ride the ?tools=read endpoint matching the tool gate (live-verified); the chat lane's litellm model resolution deduplicated into utils._litellm_model; explicit optimize= wins over the deprecated optimize_expand (DeprecationWarning added); mcp 2.0 compatibility; spawn-worker and doc-scope guard fixes; the suite is .env-independent and pins the LitellmModel._fetch_response seam. Per-finding rationale in the ported commit messages on feat/local-chat.v0.2.10 v0.2.10.dev6 |
||
|
Ray
|
ae2a5b49b5 |
feat: backend connection overrides; extra_headers on every local door (#411)
* feat: backend connection overrides; extra_headers on every local door
Two clients, two configs — the gap this closes. index_backend /
chat_backend on the constructor (and per-call backend on the chat
doors, mirroring per-call model) carry connection params in each
lane's own vocabulary, verbatim: the indexing gateways take the dict
as LiteLLM call kwargs (a contextvar scopes it per operation, and the
env-var key pre-check yields to it), the chat lane lifts api_key /
base_url into LitellmModel's two pinned constructor slots and rides
the rest as call kwargs, responses() and messages() hand the dict to
their SDK client constructors. Per-call keys win over the client's;
the openai-SDK fast path normalizes LiteLLM's api_base spelling.
Config bundles deliberately don't carry it — you run those in your
own environment (docstring says so).
extra_headers lands on all three protocol doors, each engine merging
caller headers verbatim (anthropic-beta wire-proven on messages).
Wire-probed exception, documented on the chat door: LiteLLM's
anthropic adapter owns the anthropic-beta header and drops the
caller's value — Anthropic beta flags belong on messages(). Cloud
rejects the new knobs like the other local-only params, and the
docstrings now say credentials belong in backend, never extra_body
(on the bare lane they would leak into the JSON body without
touching auth — wire-probed).
* docs: constructor Args list gains index_backend / chat_backend
The class docstring documents every constructor argument; the backend
wave added two without entries. Same phrasing as the per-call docs:
index lane is LiteLLM vocabulary verbatim, chat_backend reaches
whichever door runs (api_key/base_url portable across all three).
* chore: three audit leftovers
The classic pipeline's ThreadPoolExecutor import (unused since the
0.2.9 merge) goes. The bare-model missing-key error now also names
the backend={'api_key': ...} route, which satisfies the same check.
messages() joins the uniform: a bad backend dict wraps as
PageIndexAPIError like the responses door, instead of leaking the
SDK's raw constructor error.
* chore: narrow the anthropic wrap to TypeError; key advice names chat_backend
Anthropic's constructor raises only TypeError in our supported range
(unknown kwargs, conflicting credentials) — a missing key defers to
request time, so catching AnthropicError there guarded an impossible
case. And chat() has no backend parameter, so the missing-key advice
now names chat_backend alongside the per-call route.
v0.2.10.dev5
|
||
|
Ray
|
08ea1d975c |
fix: py3.10 litellm type repair; chat() gains reasoning_effort (#410)
* fix: rebuild litellm's Message/Delta types on Python 3.10 litellm 1.97.0 ships Message and Delta annotations whose nested forward refs (ChatCompletionReasoningSummaryTextBlock et al) do not resolve on 3.10, so every completion() dies constructing its response object — non-stream and stream alike (upstream BerriAI/litellm#36384, open, no patch release; 1.96.2 is clean, so the floor raise surfaced it, and pydantic 2.12/2.13 both reproduce). The repair rebuilds the two models once with their defining modules' namespaces at our three completion gateways; version-gated to <3.11 and best-effort, so it is a no-op on healthy interpreters and future fixed litellm releases. Verified on a 3.10 venv: the previously failing anthropic wire test and the full suite pass (250 green, matching CI's matrix leg). * feat: chat() takes reasoning_effort — the front door's one thinking knob Ruled in as a business-level control alongside model: who answers, and how hard it thinks. Same name, values, and verbatim semantics as chat_completions underneath (LiteLLM's cross-provider tier string); unset sends nothing so each backend's own default behavior applies. Sampling and wire-level knobs deliberately stay off the front door.v0.2.10.dev4 |
||
|
Ray
|
bc1c1740ff |
feat: model knobs, LiteLLM-verbatim chat lane, per-door passthrough params (#409)
* docs: drop the demo's install step — openai-agents ships with the SDK now
* feat: the chat lane routes every model through LiteLLM — bare names included
The direct-OpenAI special case existed to dodge LiteLLM's import cost,
and it made OpenAI's own Responses-first models fail on the front door:
gpt-5.6-sol 400s on chatcmpl+tools while reasoning is on (server-side
policy — wire-captured with no reasoning_effort in our request).
LiteLLM 1.97 translates such calls onto /v1/responses; 1.84 does not,
so the sol-class 400 now carries its two exits (upgrade litellm /
responses()).
Routing after the flip: chat protocol — bare names are OpenAI-compatible
shorthand (wire form openai/<name>; OPENAI_API_KEY / OPENAI_BASE_URL
still select the backend, and the missing key stays a build-time
failure), litellm/ strips, openai/ opts out to the OpenAI SDK directly;
responses protocol unchanged (OpenAI-SDK native, LiteLLM refused).
The import cost is handled instead of dodged: local clients preload
litellm on a background thread (first call then perceives 0.0s), and
pageindex sets LITELLM_LOCAL_MODEL_COST_MAP=True via setdefault —
LiteLLM's import otherwise blocks on a network fetch of its price map
(fresh venv: 5.6s -> 1.3s; offline it hangs to the timeout).
Also restores prompt_cache_key delivery, found dead during the flip's
gating verification: openai-agents 0.20 no longer derives it from
RunConfig.group_id, so both lanes sent nothing. ModelSettings.extra_body
is the one channel all three model classes put on the wire (the bare
kwarg is dropped by LiteLLM; extra_args[extra_body] collides with the
responses model's own parameter — both wire-verified), and it is scoped
to OpenAI destinations: LiteLLM plants extra_body as a literal field in
other providers' bodies, and Anthropic rejects unknown fields — the
anthropic wire test now pins the absence.
Verified before landing: mock-server matrix (OPENAI_BASE_URL + bare
name works through LiteLLM; gpt-named self-hosted models are NOT
bridged off a custom base_url; prompt_cache_key on the wire in every
OpenAI lane with distinct per-conversation keys; anthropic body clean)
and live (sol answers through chat(), gpt-5.4 unchanged, responses()
bare unchanged with the key on its wire).
* refactor: no prefix-triggered direct lane — chat model names are LiteLLM's, verbatim
Ray's ruling on the flip's remaining carve-out: a routing decision must
never hide in a model-name prefix. openai/ now means what LiteLLM says
it means (its openai provider), like every other name on the chat lane —
the grammar is LiteLLM's with zero exceptions.
The two defenses for keeping a direct carve-out had no concrete victim:
debugging isolation (litellm is unavoidable in indexing anyway, and
responses() IS the OpenAI-SDK-native door), and endpoint determinism
(litellm sends chatcmpl for openai-provider models except the gpt-5
bridge, which never fires against a custom base_url — wire-verified).
If a direct escape is ever needed, it will be a declared parameter,
never name grammar.
openai/-prefixed names keep the build-time OPENAI_API_KEY check for
parity with bare names; responses() is untouched (bare and openai/
still drive the OpenAI SDK — LiteLLM cannot speak that protocol).
* fix: third-audit findings — extra_body naming drift, litellm floor hint
The prompt_cache_key delivery channel went through three iterations and
settled on ModelSettings.extra_body; the _conversation_cache_key
docstring still named extra_args from the middle iteration.
The litellm install hint said >=1.30, below both our own pyproject floor
(>=1.84.0) and the floor openai-agents' litellm extra declares (>=1.83).
A user in a broken environment following it would land on a version the
package itself rules out. The hint now matches the declared floor.
* test: pin the bundle door's Agents-SDK model grammar; rename the cache-key test to extra_body
The config bundle hands its model string to the Agents SDK's own
MultiProvider grammar, which refuses unknown prefixes (probe on 0.20:
'anthropic/x' -> UserError: Unknown prefix). _normalize_retrieve_model's
litellm/ spelling is what keeps that door working — a link the existing
self-referential assert (config["model"] == client.retrieve_model)
could not catch. Pinned with a provider-slashed name.
Also renames the cache-key delivery test to its real channel,
extra_body — the extra_args name survived from the superseded delivery
attempt.
* refactor: name the retrieve_model helper for its reason — the Agents SDK's grammar
_normalize_retrieve_model said what it does, not why. The litellm/
spelling exists because the Agents SDK resolves raw model strings with
its own prefix grammar and refuses unknown prefixes — the name now
points at that constraint.
* test: the bundle-grammar test skips without openai-agents, like its file's siblings
Every agents-dependent test in this file importorskips; without the
guard this one errors where the others skip.
* feat: index_model + chat_model — two-knob model surface with full legacy fallback
The documented surface becomes two role knobs: index_model builds the
index, chat_model answers on the chat surfaces. model turns into the
set-both umbrella (its 0.2.8 indexing semantics are a strict subset, so
old configs run unchanged); summary_model and retrieve_model stay
accepted as legacy role names.
Resolution lives in ConfigLoader.load(), the one seam every consumer
already passes through (client, CLI standard/md paths, flash's
summary fallback, tree_optimize's default_model): new names win over
old, specific over general, model sets every role, and code constants
close each chain. The packaged yaml no longer ships model keys — key
presence is what separates a user's explicit choice from a built-in
default, and _validate_keys accepts the five model names explicitly.
Consequences: with no config at all, classic-mode structure extraction
now uses DEFAULT_INDEX_MODEL (gpt-5.6-luna) instead of the yaml's old
gpt-4o-2024-11-20 line (ratified; flash-default users see no change).
client.retrieve_model becomes a read-only alias for client.chat_model.
The resolution matrix test pins one row per released generation:
0.2.8 (model), 0.3.0.dev (model+retrieve_model), 0.2.10.dev (all three
legacy names), the new pair, umbrella-only, and mixed.
* feat: --index-model on the CLI; README flag docs follow
The CLI leads with --index-model; --model stays as its legacy synonym
(the CLI only indexes, so the umbrella and the index role coincide).
The flash branch's summary fallback gains the index position, and the
standard branch now forwards --summary-model, which it had silently
ignored — the flag's help always claimed it worked there. The md
branch's unfiltered model=None no longer clobbers the default: the
resolver treats None as unset.
* feat: default chat model becomes gpt-5.6-sol
Ray's pick for the out-of-box QA default; indexing stays on luna. sol
runs tools on the Responses lane — current litellm bridges chat()
there automatically; older litellm gets the guided 400 naming both
exits.
* chore: trim the model-keys comment to the constraint
The per-generation history lives in 7244ee4's message.
* feat: per-door reasoning passthrough — reasoning_effort / reasoning / thinking
Each chat door gains its own protocol's native thinking control,
forwarded verbatim with no invented vocabulary and no default of ours:
chat_completions(reasoning_effort=...), responses(reasoning={...}),
messages(thinking={...}). Unset sends nothing, so backend defaults
(sol: medium, adaptive) are untouched. chat() stays answer-only.
Delivery channels, each verified: the chat door rides
extra_args["reasoning_effort"] — LiteLLM's own top-level kwarg on
every supported openai-agents version, admitting non-enum values
("none"); newer openai-agents promotes it to the top-level argument
and pops the duplicate. Wire-captured on a mock backend
(/v1/chat/completions body carries it) and coexists with the Claude
cache marker in one dict. The responses door rides
ModelSettings.reasoning — coerced to the typed openai Reasoning object
and forwarded verbatim by the Responses model; the envelope echoes the
caller's dict. The messages door joins the existing anthropic
passthrough dict, asserted through the real tool runner.
LiteLLM semantics observed and accepted as-is: unknown models refuse
the param loudly with LiteLLM's own remedies, and gpt-5.4+ names with
an explicit effort route to /v1/responses even against a custom
api_base (its documented pre-existing arm). The sol-class 400 guidance
now names the third exit — an explicit effort routes on older litellm
releases too.
Cloud chat_completions rejects the new parameter like model/max_turns;
responses()/messages() are local-only already.
* feat: extra_body escape hatch on the three protocol doors
The industry-standard per-request extension channel (openai/anthropic
SDK trio): a dict merged verbatim into the backend request, last, so
caller keys win over SDK-set ones. Routing per door: OpenAI-compatible
destinations get a true body merge (ModelSettings.extra_body / the
anthropic SDK's native extra_body); LiteLLM-routed providers take the
keys as LiteLLM's own top-level kwargs instead, since LiteLLM plants
extra_body as literal fields other providers reject. Cloud mode rejects
it like the other local-only knobs. chat() stays answer-only.
* chore: litellm floor 1.84 -> 1.97.0
1.97.0 is where the unset-effort chatcmpl->responses bridge landed
(responses_api_bridge_check's on_constraint_enforcing_endpoint arm,
A/B-verified against 1.96.2), so sol-class models work through the
chat lane out of the box instead of 400ing until a manual upgrade.
Three spots move together: the pyproject floor, the requirements.txt
CI pin, and the install hint.
* feat: named top_p/max_tokens on the chat door, max_output_tokens on responses
extra_body could not carry these: openai-agents' LitellmModel passes
every ModelSettings sampling field as an explicit keyword and unpacks
extra_args into the same call, so the common knobs collided with a bare
TypeError on the LiteLLM lane (reproduced against a stub — Python call
semantics, callee-independent). Named params ride ModelSettings fields,
the one channel clean on every lane; responses() uses the protocol's
own name (openai_responses maps ModelSettings.max_tokens to
max_output_tokens on the wire) and the envelope now echoes the real
value instead of a constant None. Both caps bound each backend call in
the agent loop, not the whole run — documented. Cloud rejects them like
the other local-only knobs; the long tail (frequency_penalty etc.)
stays extra_body-blocked-loudly on that lane by choice.
* chore: the missing-key error names chat_model as the other exit
The default chat_model is what put keyless users on the OpenAI lane,
so the error now points at the knob that picks a different backend.
|
||
|
Ray
|
08a912c69c |
feat: openai-agents becomes a base dependency (#406)
feat: openai-agents becomes a base dependency — the chat engine ships with the SDK chat() is the SDK's front door, and its engine lived behind a vendor-named extra: pip install pageindex could index a document but failed on the first chat call, and chatting with Claude required installing '[openai]'. Measured before moving: the base tree already carries litellm (75 MB) + openai (13 MB), openai-agents adds ~15 MB (agents 8.1 + mcp 1.7 + griffe 1.4 + small pure-python deps), and current litellm's openai range (>=2.20,<3) intersects cleanly with openai-agents' (>=2.45,<3). The [openai] extra stays declared but empty, so existing pip install 'pageindex[openai]' commands keep resolving. Error messages and docstrings drop the extra; requirements.txt gains the dependency, so CI now runs the openai-agents test lane instead of skipping it. Extras now mean exactly one thing: a vendor's own SDK surface ([anthropic] for messages()/tool runner, [claude] for the Claude Agent SDK).v0.2.10.dev3 |
||
|
Ray
|
ec4851059f |
feat: chat() front door and Claude cache marking on LiteLLM routes (#405)
* feat: chat() — the answer-out front door over chat_completions * feat: cache-mark the managed prefix on anthropic-routed LiteLLM models * refactor: cache predicate asks litellm's own provider resolution * feat: extend cache marking to Claude on Bedrock and Vertex — both live-verified * fix: point anthropic-extra users at messages(); guard the two silent vendor chains * docs: the max-tokens table is a closed set — litellm's map prunes EOL'd entries, so it cannot replace it * chore: trim the max-tokens docstring to the contract * docs: state the local text-only history contract — cloud forwards tool turns, local rejects; extra fields dropv0.2.10.dev2 |
||
|
Ray
|
7c6e3c1c8f |
feat: Flash with full optimization becomes the default local indexing mode (#404)
* fix: break the phantom exception chain in _run_sync
Move asyncio.run(coro) out of the except RuntimeError block so real
errors no longer carry a bogus "no running event loop" context in
their traceback.
* feat: Flash with full optimization becomes the default local indexing mode
Every entrance now defaults to Flash with the full optimize pass
(deterministic merge, then LLM expand), replacing the standard LLM-built
tree as the default:
- submit_document(): mode=None now means "flash"; pass mode="standard"
for the LLM-built tree. _index_flash runs optimize="full" with the
expand model = summary_model, and fails fast with the missing key
name(s) via litellm.validate_environment before any work.
- page_index_flash(): optimize takes "full" (default) / "merge" / False;
True is accepted as "full" for compatibility, unknown values raise
instead of silently degrading to merge-only. optimize_expand stays
honored for legacy callers.
- CLI: --mode {flash,standard} replaces --flash (kept as a hidden
compatibility alias that forces flash). --optimize defaults to full in
flash mode with an `off` choice; explicitly passing it outside flash
still errors. Standard-only tuning flags (--toc-check-pages,
--max-*-per-node, --if-add-*) now error in flash mode instead of being
silently ignored, mirroring the existing flash-only flag errors. The
key pre-check runs only when an LLM will actually be called, so
--no-summary --optimize off|merge works keyless. Output drops the
_structure_flash suffix — always <name>_structure.json.
On the Disney earnings PDF the optimized default is also faster than
unoptimized flash (fewer nodes to summarize) and fixes hierarchy
mistakes; both modes emit identical schemas end to end.
Docs updated to match (mode flag, defaults, LLM usage honesty); tests
pin the new defaults: stored mode == "flash", optimize passthrough, and
the unknown-optimize rejection.
v0.2.10.dev1
|
||
|
Ray
|
ec342c4e28 |
fix: post-merge review fixes for v0.2.10
- Conformant responses() envelope: official output (model items only) + items (full transcript for round-trip), usage details aggregated across turns; verified against real OpenAI API - Python floor corrected to >=3.10 (litellm stable requires it) - Three stale anthropic>=0.84.0 hints updated to 0.108.0 - Image-stub behavior disclosed on tool-builder docstrings - Non-essential comments trimmed |
||
|
Ray
|
4e41acdc68 |
feat: agent tools and local chat for the PageIndex SDK (v0.2.10) (#396)
* feat: agent tools — the cloud MCP tool contract on the client Four new client methods make PageIndex documents available to agent frameworks, in both modes, with the mode decided solely by the client constructor: - agent_tools(): plain functions (browse_documents, get_document, get_document_structure, get_page_content) matching the PageIndex cloud MCP server's tools/list — same names, schemas, descriptions, and JSON response envelopes — so agent prompts port unchanged between the cloud MCP connection and these in-process tools. Tools never raise; errors come back in the same envelope. remove_document ships behind include_management=False. - as_openai_tools(): the same tools wrapped for the OpenAI Agents SDK. - as_claude_mcp(): one mcp_servers entry for the Claude Agent SDK — cloud clients get the remote MCP config (the framework connects to api.pageindex.ai/mcp and discovers the full cloud tool set), local clients get an in-process SDK MCP server. - agent_instructions(doc_id=None): orchestration guidance for the agent's system prompt; doc_id (same shape as chat_completions) appends the target documents. submit_document() gains wait=True: poll get_document status until completed, raise on failed or after 30 minutes — the manual polling loop every cloud caller writes today spins forever on a failed document. Neither framework becomes a dependency: imports happen at call time with actionable errors, and pageindex[openai] / pageindex[claude] extras are floor-only pins. tests/data/cloud_mcp_contract.json freezes the tool contract; a parity test guards against drift. 36 new tests (95 total), plus a live OpenAI Agents SDK run over a seeded local store verifying the structure-first navigation flow end to end. * fix: agent tools review — next_steps order, resolve caching, error semantics - Large-doc next_steps now says structure-first, consistent with tool descriptions and agent instructions - _remove_document fetches document list once instead of per-name - call_tool returns error envelope for unknown names instead of raising - _not_ready_error timed_out flag reflects actual wait outcome - openai_agents.py docstring corrected to match default (FunctionTools) - Removed unused ModelSettings import from demo * fix: agent tools review 2 — bridge thread safety, browse paging, metadata merge - McpBridge reads session/protocol headers under the lock (now RLock: _ensure_initialized posts while holding it). openai-agents runs sync tools on threads and executes parallel tool calls concurrently, so bridge functions genuinely race; a torn read sent a new session id with a stale protocol header. Measured: one session expiry under 8 threads cost 4 initializations before, minimal 2 after. - Session-expiry retry also resets the negotiated protocol version, so the re-handshake carries no stale MCP-Protocol-Version header. - browse_documents time sort pages list_documents natively instead of fetching the whole library to slice one window (relevance still needs the full list for scoring). - _await_completion: a status refetch that nulls out metadata no longer clobbers the listing's copy (setdefault was a no-op on existing None). - Structure tool reads the raw stored tree via a named LocalAPI raw_tree() seam instead of reaching into _api._store internals; drop the redundant deepcopy before _format_structure (store re-reads from disk, formatting builds fresh containers). - Shared pageindex/_version.py replaces _sdk_version duplicated in mcp_bridge and the Claude integration. Left as-is after source verification against the cloud MCP: first-page budget bypass, pageNum falsy-zero, and the page-gap fallback text are letter-for-letter cloud behavior — parity wins over local repair. * fix: agent tools review 3 — page-span cap, duplicate names, wait resilience, contract drift - _parse_page_spec bounds the requested span arithmetically (10k pages) before materializing it; pages="1-1000000000" previously expanded to a billion integers inside the caller's process. - Local submit_document uniquifies document names the way the cloud upload does (taken name -> _1.._99, then reject with the cloud's own message). Same-name duplicates broke name-addressed tools: resolution always picks the newest, so older duplicates were unreachable. - agent_instructions(doc_id=...) now fails loud when the pinned doc's name is shadowed by a newer same-name document (legacy stores predate the rename) — it previews resolution with the same _resolve_document the tools use, so the check cannot drift from actual behavior. - submit_document(wait=True) tolerates transient network errors, not just API errors; a dropped connection at minute 25 of a 30-minute wait no longer kills it. Third strike wraps into PageIndexAPIError per the documented contract. - The live contract-parity test compares full per-param schemas, not just names and descriptions. It immediately caught real drift the shallow check had been passing: the server now emits nullables as anyOf unions and stamps MAX_SAFE_INTEGER maxima on offset/part. Contract and snapshot updated to the served wire form; _annotation_for learned anyOf so bridge signatures stay Optional[str] instead of degrading to Any. Adjudicated, not changed: the allowed_tools wildcard example stays (docstring advice covers scoping; Ray's call), and raw-length response accounting stays (letter-for-letter cloud behavior, parity wins). * feat: surface the stored document name from submit_document Compute PR #558 makes /doc/ return {"doc_id", "name"} carrying the post-dedup-rename name. Mirror it end to end: local submit returns the stored name, the client warns when it differs from the uploaded file name (read via .get so older cloud servers stay compatible), the local name-exhaustion check runs before indexing instead of after the LLM spend, and the demo caches doc_id in a file instead of name-matching — a renamed document made the name lookup re-index on every run. * fix: add missing page_list kwarg in duplicate-name test mock * revert: keep README.md unchanged from main — SDK section deferred * feat: serve cloud agent instructions live from the MCP server The cloud MCP server publishes its agent instructions in the initialize result, adapted to each key's tool set. agent_instructions() previously returned the SDK's local-subset text in both modes — a silently forked copy that lacks the guidance for cloud-only tools (search_documents escalation, folders, images) and drifts as the server's prompt evolves. Cloud clients now serve the server's live instructions, captured from the initialize handshake on a per-client bridge shared with agent_tools() (one session, no extra request). An empty server response raises instead of silently substituting the subset text — same posture as the annotation-regression guard. The local constant stays as the honest subset for the in-process tools, with its provenance noted and a consistency test that every tool it names exists in the local registry. * fix: local relevance sort answers honestly instead of imitating sort="relevance" is cloud-side semantic ranking; the local substring imitation could satisfy the letter of the interface while silently missing semantically relevant documents. Per the honest-subset rule (same treatment as folders), local now returns the "not available here" envelope for sort="relevance" or a stray query, and the local instructions steer discovery through name/description matching plus full-library paging instead of prescribing a capability that does not exist here. The tool schema keeps the cloud contract verbatim, like folder_id: honesty lives in the runtime answer, not a forked contract. * docs: note the cloud+Claude instructions duplication trade-off in as_claude_mcp * fix: unsupported-capability envelopes say local-mode-yet, point to cloud "Not available here" read as a broken feature; the honest framing is that folders and semantic ranking exist on PageIndex cloud and are not in local mode yet. Both envelopes now say so and name the cloud client in next_steps, so agents relay an accurate story to the user. * fix: local tool descriptions pre-announce cloud-only capabilities The cloud-verbatim browse_documents description invites sort="relevance" and folder drilling, so a local agent's first semantic search attempt was a guaranteed dead end discovered only from the runtime error envelope. Local registration now appends a LOCAL MODE note to the description — the agent learns what is cloud-only before calling; the runtime envelope stays as the backstop for prompts that ignore descriptions. The cloud-facing contract stays byte-verbatim. * refactor: localized tool guidance replaces the appended LOCAL MODE note Appending a retraction to the cloud-verbatim description left the model parsing an instruction and its negation — and kept the cloud text recommending search_documents and get_folder_structure, tools that are not registered locally (get_page_content likewise pointed at get_document_image). Guidance now adapts to the local surface the way AGENT_INSTRUCTIONS already does: schema structure stays byte-identical to the contract (mechanically asserted by a strip-descriptions test), while local description strings teach only what works here and point to PageIndex cloud for the rest. A dead-reference test forbids local guidance from naming tools outside the local registry, so a contract refresh that reintroduces a cloud-only reference fails loudly. * feat: hide cloud-only parameters from the local tool surface folder_id, sort, query, and recursive were exposed locally with localized "cloud-only" descriptions, leaving the dead-end calls expressible and discovered at runtime. Schema constraints beat guidance: the local surface now serves the contract minus these parameters, so strict-schema frameworks make the calls inexpressible and a prompt that insists on sort="relevance" degrades to the bare call (the correct local behavior) instead of an error round-trip. The implementations still accept the hidden parameters and answer with the guided "works on PageIndex cloud" envelope — the backstop for direct call_tool callers and hosts without schema enforcement. wait_for_completion stays: seeded or torn stores can hold documents that are genuinely not completed. The structural guard now asserts the local schema equals the contract minus the documented hidden set, descriptions aside. * fix: incremental-review findings — bridge cache, guards, envelope drift Three independent review passes over the agent-instructions increment surfaced six fixes: - The per-client bridge moved off the instance into a weak-keyed, lock-guarded module cache: cloud clients stay picklable (threading.RLock no longer rides on the client) and concurrent first calls can no longer construct duplicate bridges/sessions. - Blank or non-string initialize.instructions now hit the same honest error as a missing one — a whitespace-only or structured value could previously become the system prompt (or crash the doc_id append with a raw TypeError). - The invalid-sort envelope no longer prescribes sort="relevance" — the one error text that still taught the cloud-only value it would then reject. - "Page through the rest of the library" is emitted only when has_more is true; a fully-listed library no longer instructs a pointless call. - The mandatory full-library paging step now says limit: 50 — 6 calls instead of 30 on a 300-document library. - Docstrings and comments rescoped to what is actually true: the never-raise contract covers invocations the signatures accept (unknown params fail at the Python boundary; call_tool answers them with the guided envelope), recursive is accepted as the identity rather than errored, lenient framework arg models drop hidden params pre-call, and the module header no longer claims full schema parity. The capability-phrase guard now covers every local docstring, not just browse_documents. * chore: keep the demo's doc_id cache file out of the repo * test: live envelope field-parity guard against cloud response drift The frozen contract guards tools/list, but the response envelopes the local tools emit were hand-built to mirror the cloud's and had no drift detector. A key-gated live test now asserts every field local emits exists in the live cloud response for the analogous call (top-level keys, next_steps, document entries, structure nodes, content entries). Guidance wording is deliberately localized and not compared. Verified green against the live server: local and cloud field structures currently match exactly. * feat: local chat — three protocol surfaces over the agent tools (v0.2.10) Local mode gains managed document QA: an agent over the #393 local tool set, reachable through three wire protocols, each 1:1 with the backend and with no translation layer. - chat_completions(): standard chat.completions semantics on any OpenAI-compatible backend (openai-agents engine). Final answer only, cross-turn aggregated usage, streaming as text pieces or chunk dicts (the existing cloud signature, now implemented locally; model and max_turns are local-only additions). - responses(): the agentic surface — OpenAI Responses format, the tool process is standard output items, streaming forwards native events (tool outputs emitted as response.output_item.done, the way the platform streams its own server-side tools). Round-tripping output into the next input keeps provider prompt-cache prefix continuity and the agent's memory — live-verified: the follow-up call answered from round-tripped tool output with zero new tool calls. - messages(): Anthropic-native via the SDK's own tool runner (new pageindex[anthropic] extra, floor 0.68.0 verified for tool_runner/beta_tool(input_schema)). tool_use/tool_result round-trip is the format's native behavior; the envelope is the final message with aggregated usage plus the full new-turn sequence; the managed system blocks carry cache_control breakpoints. Shared skeleton: thin chat header + the local AGENT_INSTRUCTIONS (caller system content is appended, not rejected), the doc_id targeting block as a leading context item (factored out of build_agent_instructions), read-only toolset, structural-only validation (no arbitrary caps — backend limits govern), sampling params passed through, per-run tracing disabled, enable_citations rejected as cloud-only. Design basis is industry-standard formats rather than the cloud chat endpoint; responses()/messages() raise on cloud clients until the cloud converges. Tests run the real engines against scripted backends (a Model fake for openai-agents, a mock HTTP transport under the real anthropic SDK) with real tool execution against a seeded store, including the round-trip prefix-extension assertions on both engines. * fix: local-chat review findings — truncation, serialization, streams Three independent review passes (bug scan, claims-vs-code, adversarial runtime probes) over the local-chat increment; every fix below was reproduced before being fixed. messages(): - A max_turns cut no longer duplicates the final assistant turn: the runner has already appended it when iterations exhaust, so the round-trip history carried a duplicate tool_use id and ended on an unanswered tool_use — a guaranteed 400 on continuation. The append now keys on stop_reason, and truncation reads natively as stop_reason: "tool_use" with a continuable history. - The envelope is JSON-serializable end to end: runner-stored turns carry pydantic content blocks; everything is dumped to plain dicts, excluding SDK-internal __api_exclude__ fields (parsed_output) that the API rejects on round-trip. - Bounded by default (max_iterations 10, like the OpenAI surfaces); usage aggregation now preserves the final turn's native fields and sums the token counters None-safely; empty caller system strings are skipped; non-dict message entries and bad doc_id types raise PageIndexAPIError; anthropic < 0.68 gets an actionable version error; the doc block no longer spends a cache_control breakpoint. chat_completions()/responses(): - MaxTurnsExceeded wraps into PageIndexAPIError on all four run paths. - responses(stream=True) is one logical response: per-turn backend lifecycle events are collapsed (a canonical consumer previously stopped at turn 1's response.completed and never saw the answer), sequence numbers are reassigned monotonically, and the synthesized tool-output event carries output_index/sequence_number. - The responses envelope carries the real request surface (instructions, the actual function tool definitions, tool_choice, parallel_tool_calls, error/incomplete_details). - RunConfig(group_id) pins a stable prompt_cache_key: openai-agents otherwise stamps each run with a fresh key, tagging round-tripped prefixes as different cache groups and defeating the feature the round-trip exists for. - Abandoning a stream now cancels the run: a watchdog task lets the cancellation land even while the pump awaits the backend, and the per-call AsyncOpenAI client is closed before its loop ends (fixes "Task exception was never retrieved" noise). The opening role chunk is emitted even for empty outputs; empty responses() input and enable_citations-before-extra ordering fixed. Docs rescoped to what is true: finish_reason/status reflect loop completion on the OpenAI surfaces (the engine does not surface per-turn backend reasons); chat streaming yields visible narration including pre-tool text; messages(stream=True) forwards the Anthropic SDK's native event objects (not wire-verbatim); the doc block is a leading conversation item on OpenAI surfaces and a system block on messages(). Tests: 25 in the file (11 new), with per-extra skip sections so a machine with only one framework still covers the other surface; without-frameworks matrix re-verified; live smoke re-run green with a clean exit. * feat: as_anthropic_tools — Anthropic tool-runner export, both modes Fills the last cell of the agent-connection matrix: users driving their own anthropic tool_runner loop get runnable tools directly. Cloud wraps the live MCP tool set with input schemas passing through verbatim (MCP inputSchema is the Messages API schema shape); local exposes the same set messages() runs internally. The beta_tool wrapping moves from local_chat into integrations/anthropic_sdk.py, parallel to openai_agents.py, and messages() now consumes the shared builder. agent_tools grows _bridge_invoker/_read_only_tools so the plain-function and beta_tool cloud paths share invocation containment and the read-only gate. * fix: as_anthropic_tools review findings — async flavor, schema isolation Adversarial + best-practice review of |
||
|
Ray
|
d375c00a5a |
feat: local mode for the PageIndex SDK (v0.2.9) (#389)
* feat: add local mode to the PageIndex SDK client
One PageIndexClient, two backends. With api_key: the 0.2.x cloud SDK,
request for request (with the reviewed fixes: bounded timeouts on JSON
endpoints, none on uploads, URL-encoded ids, 401 key hint, empty DELETE
body tolerated). Without api_key: the same methods run locally —
page_index builds the tree in submit_document (mode="flash" uses
PageIndex Flash), documents are stored as plain JSON per doc under
storage_path, submit_query is LLM tree search with retrieve_model, and
chat_completions answers over the retrieved nodes with OpenAI-style
responses and streaming.
Local responses mirror the cloud wire shapes verified against the server
source: tree nodes rename start_index to page_index and drop end_index,
a non-leaf summary becomes prefix_summary, and the metadata/list/delete/
retrieval envelopes match key for key. Cloud-only features (folders,
beta_headers, enable_citations) raise instead of pretending.
Replaces the demo-only workspace client (index/get_document_structure/
get_page_content had no real users) and its retrieve.py helpers.
page_index_main gains an optional logger param so the SDK can keep
./logs out of the caller's working directory; pymupdf import is now lazy
(only the optional PyMuPDF parser path needs it).
* chore: package pageindex 0.3.0.dev4 for PyPI
Poetry packaging for the combined SDK + local pipeline: every production
import is a declared dependency (openai and requests join requirements.txt
for the same reason), config.yaml and the flash data tables ship in the
wheel, the benchmark PNG does not. pymupdf drops to an optional note now
that its import is lazy. dev4 follows the already-published 0.3.0.dev1-3;
pip still resolves plain 'pip install pageindex' to 0.2.8 until a final
0.3.0 — install with --pre.
* docs: add SDK section to README; move the agentic demo onto the SDK
The demo keeps its flow and the post-cutoff demo paper, swapping the
removed workspace client for PageIndexClient local mode (list_documents
for the doc-id cache, get_tree/get_ocr behind the agent tools). The old
examples/workspace JSONs demoed the removed format and go with it.
* refactor: rebuild cloud_api on the 0.2.8 client text
Ray's rule for the cloud half: his 0.2.8 code is the base; a Kylin-lineage
change survives only when strictly better — invisible on healthy traffic
while fixing a real failure mode. Kept under that bar: request timeouts
(dead connections hung forever; uploads still pass none, exactly like
0.2.8), the upload handle closed via with (leaked on request errors),
URL-encoded path ids (a crafted id could reroute the URL), the
empty-DELETE-body guard, and stream hardening (choices guard,
response.close in finally). Reverted as not strictly better: the
lowercase summary param (the server accepts both spellings), the 401
message hint (visible text change; AUTH_HINT dropped from errors.py), and
the _request/requests.request reorganization — every method body,
docstring, and section comment is 0.2.8's text again.
diff -w against ../pageindex_sdk/pageindex/client.py now reads as that
surgical patch plus plumbing: CloudAPI reads BASE_URL/api_key through the
owning client, PageIndexAPIError comes from errors.py, and
is_retrieval_ready lives verbatim on PageIndexClient shared by both
modes. A mocked-requests harness driving 0.2.8 and this file through 19
identical calls shows the only remaining request-level difference is
timeout.
* refactor: drop the local retrieval endpoints — cloud-only, deprecated
Ray: the cloud already marks POST /retrieval/ and GET /retrieval/{id}/
deprecated in favor of chat completions, so local mode should not grow a
fresh implementation of a retiring surface. submit_query/get_retrieval
now raise in local mode with a pointer to chat_completions; cloud mode is
untouched (the endpoint still works there and 0.2.8 code keeps running).
The tree search that backed them stays as chat_completions' retrieval
engine; the retrievals/ storage goes away. This also closes the one real
cross-mode parity gap — the retrieved_nodes inner shape — by removing
its local half.
* feat: manifest.json — one-file document listings for the local store
Ray wanted a metafile that shows every document in one place instead of
per-directory reads. It is a cache, never a second source of truth:
writers update it best-effort after save/delete (atomic replace, no
locks), and list_metas trusts it only while its id set matches the docs/
directory names — documents are immutable, so matching names imply valid
content. Any mismatch (lost concurrent update, crash, corrupt or deleted
manifest) rebuilds it from the doc.json files, reading only the missing
entries. Incomplete dirs (no doc.json) stay invisible and are never
recorded, so a save that completes later is still picked up.
1000-doc listing: 37ms of per-dir reads -> 2.3ms warm (scandir names +
one manifest read); one-time rebuild 160ms.
* fix: align local doc_id prefix and createdAt format with the cloud
Local doc ids now carry the cloud's pi- namespace prefix (random token
stays uuid4 hex — nothing parses cuid internals, the prefix is the
contract; chat ids already mirrored chatcmpl-). createdAt now matches
the server byte for byte: the cloud emits the DB datetime's bare
isoformat — naive UTC, second precision — while we emitted microseconds
plus +00:00.
* feat: optional metadata tags on submit_document (both modes)
Ray's call after the alignment review: the metadata key in tree/OCR
envelopes and list entries should carry real data, and the only honest
way is exposing the field the server already accepts. Cloud mode
forwards it as the existing metadata form field; local mode validates it
early (a JSON-serializable dict, checked before any LLM spend), stores
it in doc.json, and returns it from the same three places. get_document
still omits it, mirroring the server, whose metadata-endpoint SQL never
selects that column. Scope stays deliberately narrow: set at submit and
read back — no metadata_filter, no update API.
* docs: state that createdAt is UTC and show how to localize it
The value is naive UTC in both modes (the cloud column is timestamp
DEFAULT CURRENT_TIMESTAMP on a UTC server, emitted via bare isoformat).
Wall-clock display is the consumer's layer: emitting local time under
the same format would silently change meaning per machine, and adding an
offset marker would break both the byte-format parity and string-order
sorting.
* fix: createdAt carries milliseconds, matching the cloud's datetime(3)
The earlier second-precision alignment was reasoned from the postgres
schema file, but production is MySQL (DATABASE_BACKEND defaults to
mysql) and its FilePageIndex.createdAt is datetime(3) DEFAULT
CURRENT_TIMESTAMP(3) — so the server isoformat()s a millisecond-
precision naive-UTC datetime, emitting .XXX000 fractions (bare seconds
only when the millisecond happens to be zero). Local now generates
through the same mechanism. The docs' 2024-01-15T10:30:00.000Z sample is
a JS-style string Python isoformat cannot produce — not evidence.
* fix: createdAt at millisecond precision, matching the timestamp(3) column
The cloud column is timestamp(3); its datetimes render through bare
isoformat() as six fractional digits ending in 000 (or no fraction when
the millisecond is exactly zero). Truncate to the millisecond and render
the same way, replacing the second-precision guess from
|
||
|
Ray
|
b723c9f0a7 |
ci: run the test suite on every push and pull request
Publishing was the only automation touching code: a tag builds and ships to PyPI without ever running a test, and pull requests get no checks at all. This runs pytest on a small matrix — Python 3.10 (the floor) and 3.13, each with and without the agent frameworks installed, so the lazy-import contract (the package must work with neither framework present) is enforced rather than assumed. |
||
|
Ray
|
d5c4e62c20 |
Exclude common words from roman numeral conversion (#387)
Update PageIndex Flash |
||
|
Ray
|
933df35b2f |
Add a Flash summary flag (#386)
* Update PageIndex Flash * Add a summary flag |
||
|
Ray
|
502b763876 | Update README (#385) |