Stereo Logic, Sheridan
Vision-language models for spreadsheet activity
Applied research on recognising business actions from screenshots and keyboard and mouse activity, using vision-language models and LLMs on self-hosted infrastructure.
- Role
- AI Research Assistant / ML Developer
- When
- Oct 2025 – Nov 2026
Context
Stereo Logic’s pipeline turns mouse and keyboard event streams into business actions such as text entry, field selection and navigation. I worked on extending it to spreadsheets, and on moving its inference from OpenAI’s API to self-hosted models.
What I worked on
- Computer-vision and vision-language model components for understanding spreadsheet screenshots.
- Prompts for spreadsheet interactions such as clicks, typing, drag-and-drop and paste.
- Tooling to evaluate and compare models against annotated data.
- Self-hosted inference with multimodal LLMs (Gemma 4 31B, Qwen3.8-27B, Qwen3-VL and Qwen3-Omni) on vLLM and an NVIDIA DGX.
Specific results and implementation details are confidential and aren’t shown here.