Skip to content

fix(search): sanitize FTS5 query tokens#374

Open
Ricky-7-Yan wants to merge 1 commit into
TencentCloud:mainfrom
Ricky-7-Yan:fix-issue-160-fts5-query-tokens-v2
Open

fix(search): sanitize FTS5 query tokens#374
Ricky-7-Yan wants to merge 1 commit into
TencentCloud:mainfrom
Ricky-7-Yan:fix-issue-160-fts5-query-tokens-v2

Conversation

@Ricky-7-Yan

Copy link
Copy Markdown

Description | 描述

Fixes #160.

Sanitize FTS5 MATCH query construction so user input cannot alter query semantics through FTS5 operators or control syntax.

This PR keeps the change intentionally scoped:

  • Strip FTS5 operator words such as AND, OR, NOT, and NEAR/5 before tokenization
  • Strip FTS5 control syntax such as quotes, parentheses, wildcard, column markers, and unary minus
  • Keep normal keyword search behavior by preserving sanitized text tokens
  • Apply the same sanitization before both jieba tokenization and regex fallback tokenization
  • Add focused regression tests for injection-like input and empty syntax-only input

Related Issue | 关联 Issue

Fix #160

Change Type | 修改类型

  • Bug fix | Bug 修复
  • New feature | 新功能
  • Documentation update | 文档更新
  • Code optimization | 代码优化

Self-test Checklist | 自测清单

  • Verified locally | 本地验证通过
  • No existing features affected | 无影响现有功能

Verification | 验证

npx.cmd vitest run src/core/store/sqlite.test.ts
npx.cmd vitest run

Results:

  • src/core/store/sqlite.test.ts: 3 tests passed
  • Full test suite: 5 files / 70 tests passed

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program!
We have successfully received your submission. The program is currently in full swing, and we will complete the Code Review for you as soon as possible. Please keep an eye on the status notifications for this PR so you can follow up promptly once the review feedback is provided.
Thanks again for your contribution and open-source spirit! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(search): buildFtsQuery does not sanitize FTS5 operators — user input alters query semantics

2 participants