dsh-computer-control

LIMITATIONS

Known limitations

- **The recorder is not literally free while idle.** It is started by the first call of a burst and stopped by a watchdog once `recorderIdleStopMs` elapses with no further call. During that grace window the screen is still being captured; set `recorderIdleStopMs` to a small value to shrink the window at the cost of re-priming (each cold start costs about 1.5-2 s before the recorder's first frame). - **Frame time is the moment the file was written, not the moment the pixels were read.** `gdigrab` capture, JPEG encode, and DWM composition put a lag of roughly one frame interval between the screen and the frame that represents it. A frame's offset is therefore accurate to about one interval, not exact. - **A live pointer is composited into recorded frames.** The cursor may be drawn at a position that does not match the moment the rest of the frame was captured. - **Recording is Windows-only.** It depends on `gdigrab`; there is no equivalent path for macOS or Linux, and no plan for one in this package. - **The in-process fallback is much slower.** When the recorder cannot start, the helper falls back to GDI+ capture, which this machine measures at 2-3 fps at 1024 px wide because the downscale and JPEG encode dominate — an order of magnitude below the 9.9-10.7 fps the ffmpeg recorder holds. The envelope reports the measured gap either way, so a fallback call is visibly slower rather than silently wrong. - **Route capability decides what the envelope can suggest.** When the calling route rejects images, the envelope says so and the frames stay on disk; the route's own capability is still resolved, so the envelope distinguishes "this deployment turned images off" — where `read_image` can still display a saved frame — from "this route refuses images", where nothing on that route can. Recovering the evidence then requires a route that accepts images. - **Returned frames are a sampled subset.** A call captures far more than `maxFrames`, and the selection keeps two anchors — the last frame at or before injection and the final frame — then fills the remaining budget with a contiguous run starting at injection, and only then works backwards from the pre-action reference. Frames that changed most are considered before that padding, so a one-frame flash survives the cap; a rule that ranked purely by position would spend the whole budget on the first frames that twitched. The envelope reports two repeat counts — `repeatedFrames` by signature and `identicalFrames` by JPEG bytes — because repetition there means the sampler had nothing new to add, not that the screen was watched continuously. Only the byte count is reproducible from the images a caller received, and the two differ by a few frames whenever re-encoding shifts pixels below the signature threshold. - **A recorded frame's signature is computed by the helper.** The frame triage needs one signature per frame to rank by change, and nothing else can supply it: the recorder writes JPEG files, no pixels pass through the helper, and the Node side has no image decoder. Each new frame therefore costs one 32x32 thumbnail decode — a few milliseconds — on the call path. The grid is sampled point by point rather than averaged, because an averaged cell hides a pointer, a checkbox, or a highlighted menu row inside its own mean. - **The two capture paths reach the same threshold through different grids.** A recorded frame carries a 32x32 signature sampled point by point, so one cell is 0.000977 of the frame and the default `changeThreshold` admits three; the in-process fallback computes 16x16 *averaged* cells, where one cell is 0.0039 and the same threshold admits less than one. A single noisy averaged cell can therefore read as a change on the fallback path, and a feature small enough to live inside one recorded cell can be missed on the recorder path. Raising the zoom is the way to see such a feature; it does not change eith