@fffangchenghao/dsh-web-search

LIMITATIONS

Known limitations

- **Relevance is capped by upstream result pages** and is below a commercial search API. Key-free scraping gets throttled in return: measured, 360 served `访问异常页面` with a captcha and another domestic engine a `x5sec` punish page. Such pages are now classified as `SEARCH_BLOCKED` instead of masquerading as zero results, and 360 is a fallback; cooldowns and multiple engines still only cover for it, not all of it. - **Google is unreachable from this network.** A tunnelled `www.google.com/search` returns a JavaScript shell (94 KB, 1–3 links); headless Chrome is redirected to `google.com.hk` with a reCAPTCHA. `gbv=1`, a consent cookie and a mobile user agent all failed. Eight public SearXNG instances were unusable too (429/403/not JSON). - **It depends on upstream HTML structure.** A redesign breaks a parser; that engine records a reason and yields to the next one rather than silently returning nothing. - **Titles, links and snippets only** — no page text. That is `web_fetch`'s job. - **`web_fetch` cannot resume a fetched page from an offset**: the host's `WebFetchRequest` carries only `url`, so text beyond `fetchMaxLength` needs `read_page`, a higher limit or a narrower page. - **`read_page` keeps at most 2,000,000 characters per page**: anything beyond the cap is cut and flagged `truncated`; the plugin caches at most eight bodies, so a ninth page evicts the least recently used one. - **Extraction keeps only what looks like the article**: a page built entirely by JavaScript comes back nearly empty by default; it is rendered only when you installed Playwright and enabled `renderJs`. Content outside the chosen article root — a sidebar table, say — is still dropped with the navigation. - **Browser rendering has no measured real-page body**: the development environment for this project has no Playwright. What is verified is shell detection, the silent fallback when no renderer exists, and the render/timeout/fallback paths against an injected fake renderer (`render-js.test.ts`). Rendered text from a real site is still unmeasured. - **The benchmark is only 20 queries**: news recall is 0.367 and `zh-linux-612` has zero hits; a difference of 1–4 hits cannot separate versions. It shows direction, not statistical significance. - **`search_many` is capped at five queries and three in flight.** - **Two processes sharing one cache file write last-writer-wins**: neither can corrupt the file, but one process's newer snapshot can be replaced by another's older one. - **A time window can only filter results that carry a date**: dates are parsed from result pages, so an engine that prints none leaves that row without an age; when nothing dated falls inside the window the undated list comes back, with a `Note` saying the window is not proven for those rows. Bing's and 360's time parameters were measured to have no effect; only DuckDuckGo's `df=` is a real pushdown. - **Domain search is discovery, not a complete record**: GitHub search omits release tags, Wikipedia search omits revision timestamps, Hacker News search omits the top comment, and Stack Overflow search omits accepted-answer text. Follow up with `web_fetch` or `read_page` when the exact record matters. - **PyPI matches exact package names only**: it has no public search API, so the plugin uses the first word of the query; a typo yields an empty answer. - **Community platforms are best-effort.** Bilibili, V2EX, Reddit, YouTube and Juejin change or block their public endpoints; the tool falls back to a domain-scoped web search and filters the rows to the platform host. Zhihu has no usable keyless API in this environment, so it is fallback-only and can still return nothing. - **`cache_search` is in-process.** Research bodies live for at most seven days, 24 pages and 400,000 characters each, and disappear on restart; they are never written to the search-result cache. - **Official-site hints are bounded.** `research` adds a focused `site:` query only for a small set of issuers (EU, CAC, Chrome