mward4/deepseek-harness
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 29 Wiki Activity

test(python): accept standalone live smoke acknowledgements

Browse Source 
The installed-wheel smoke checks real model tool use and externally verifies
the created file, a fresh host-only challenge, the copied receipt, and the
unchanged source. DeepSeek can return the requested acknowledgement before
a file-details paragraph; requiring it to be the last line makes correct
SDK execution fail according to the placement of explanatory prose.

Accept the exact acknowledgement on any standalone line, retaining completed
turns, model-requested tool calls, exact file bytes, and session-log checks.
Replay the observed response through both turns and keep negative coverage
for absent or embedded acknowledgements and incorrect files.
This commit is contained in:
Tianyi Cui committed 2026-09-23 23:43:04 +08:00
1 parent a523d8ec58
commit 8c32bcf97d
5 files changed
+23 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

python/development.md
+1 -1
View File
@@ -38,7 +38,7 @@ SDK scenarios compare committed expected output under `scripts/snapshots/python-
The `advanced` and `restart` comparisons match native delivery qualifiers against each input's own Session generation, including SDK events and notifications. Captured and other delivery generations retain their numeric values. Each side must use one generation across its Session roles, and fresh logs must identify the current writer. Comparison never rewrites committed recordings.
Trusted pull requests and master pushes also run `--scenario sdk-live --installed-wheel` on each selected native target. That scenario performs two tool-using turns against `https://api.deepseek.com`: it checks the created file immediately, replaces its content with a host-only random challenge, and requires the second turn to copy the changed content into a fresh receipt without modifying the source. Both turns must complete with the exact sentinel answer and a model-requested tool call; external byte comparisons check the files. Missing repository secrets fail instead of self-skipping. Fork and Dependabot pull requests run the complete keyless installed-wheel path but receive no key.
Trusted pull requests and master pushes also run `--scenario sdk-live --installed-wheel` on each selected native target. That scenario performs two tool-using turns against `https://api.deepseek.com`: it checks the created file immediately, replaces its content with a host-only random challenge, and requires the second turn to copy the changed content into a fresh receipt without modifying the source. Both turns must complete with a standalone sentinel acknowledgement and a model-requested tool call; the acknowledgement may precede or follow explanation, and external byte comparisons check the files. Missing repository secrets fail instead of self-skipping. Fork and Dependabot pull requests run the complete keyless installed-wheel path but receive no key.
An interactive smoke test needs `DEEPSEEK_API_KEY` in the environment or repository-root `.env`: