Collaborative Research Featuring LemoniLab Accepted to AIST 2026’s Data Analysis and Machine Learning Track
The collaborative paper “Autoresearch with Coding Agents: Generalizers and Metric-Maximizers on Quran Recitation Data” has been officially listed among the accepted papers at AIST 2026 under Data Analysis and Machine Learning — LNCS. The work brings together researchers affiliated with LemoniLab FZCO in Dubai, Innopolis University, and the Skolkovo Institute of Science and Technology.
AIST 2026—the 13th International Conference on Analysis of Images, Social Networks and Texts—will take place in Astana, Kazakhstan, on 16–18 October 2026. The conference states that its main proceedings are scheduled for publication in Springer Lecture Notes in Computer Science.
The research collaboration
The paper is authored by:
- Nursultan Askarbekuly — Innopolis University
- Mohamad Al Mdfaa — LemoniLab FZCO, Dubai, UAE
- Ahmed Helaly — Innopolis University
- Gonzalo Ferrer — Skolkovo Institute of Science and Technology
- Manuel Mazzara — Innopolis University
LemoniLab extends its appreciation to every researcher and participating institution, as well as the reviewers and collaborators whose feedback and exploratory runs strengthened the final work.
What does the paper investigate?
Modern coding agents can be given a dataset, a fixed evaluator, and an editable file, then left to iterate: modify the code, measure the result, keep an improvement, discard a regression, and repeat. This pattern is commonly described as autoresearch.
The paper asks a question with consequences far beyond coding:
When the score improves, has the agent improved the underlying solution—or learned how to optimize the measurement?
The researchers investigated this through a real production task: detecting which Quranic verses appear in noisy automatic speech-recognition transcripts, segmenting those transcripts by verse, and rejecting non-recitation input.
Why does this matter beyond the task?
The recitation task provides a precise, real-world substrate. The broader issue is how any autonomous agent behaves in a long optimization loop: which data it can inspect, what failure reports reveal, whether state can cross between runs, and whether final performance is measured on untouched data.
In the first study, the agents converged on a similar algorithmic core and then diverged in their optimization behavior. The researchers subsequently redesigned the evaluation environment by adding a held-out test set, removing expected answers from failure reports, isolating run state, and disclosing the split. Literal memorization disappeared, and the dramatic training-side score gap disappeared with it.
From research to production
The outcome did not remain inside a benchmark. On this task and metric, the strongest agent-built artifact scored 0.079 on held-out evaluation, compared with 0.760 for the incumbent hand-engineered pipeline—approximately an order-of-magnitude reduction in error under the same evaluation contract. The artifact was then deployed behind a feature flag, with automatic fallback to the legacy system and a CI regression gate.
This is an applied reference point, not an effort-matched human-versus-agent experiment. The incumbent predates the research metric, and no expert-human arm operated under the same one-hour budget. The supported conclusion is narrower: an unattended one-hour loop produced an artifact that was strong enough to replace an existing production component.
What this signals about LemoniLab
Reliable agentic systems require more than capable models. They require disciplined metrics, held-out evidence, isolated execution, visibility into state channels, and deployment controls that preserve reversibility.
For institutions moving from AI pilots to agentic production systems, the question is no longer only what an agent can do. It is how the institution can verify that the agent is improving the outcome it actually values.
Read the research and public artifacts
- Full paper on arXiv
- Official AIST 2026 accepted-papers list
- Public reproducibility repository
- De-identified dataset
If your organization is preparing AI agents for production, contact LemoniLab to discuss evaluation design, governance, and deployment readiness.
