Ehsaneddin Asgari

I am delighted to announce we are launching Project Euler on MathArena to track LLM performance on new kinds of challenging problems at the intersection of math and code. ➡️ Project Euler (https://projecteuler.net/) has been around for > 20 years with over 1.3 million human competitors that solved at least 1 problem. An original, uncontaminated problem is published every week on the site. ➡️ Solving each Project Euler problem requires two core skills: (i) math insight to arrive at an elegant and efficient method, and (ii) programming skill to implement the solution and compute the answer. ➡️ Our results show that these problems are significantly more challenging than those in competitions considered by LLMs so far, with o4-mini achieving a top score of only 31.25%. ➡️ We begin by evaluating o4-mini, grok-4, gemini-2.5-pro and claude-sonnet-4.0 (we will be adding more over time). Each LLM is allowed to use code execution as a tool and is asked to produce a final answer when ready. ➡️ For each model, we experiment with both our own scaffold that performs multi-turn code execution via function calling, and the model provider’s code interpreter (if provided via API), and select whatever works better for that model. ➡️ More on MathArena: matharena.ai

١٥٤

٧ تعليق

Mathematical modeling and Artificial Inteligence can improve delivery and efficacy of cancer therapeutics. New article of our Cancer Biophysics Laboratory , University of Cyprus in collaboration with Rakesh K. Jain's Lab at Harvard Medical School and Massachusetts General Hospital published in Nature Reviews Cancer! #artificialinteligence #AI #Cancer https://lnkd.in/d8Mdm2ug

١٧٥

١ تعليق واحد

This is the first fully AI-generated paper that has passed the same peer-review process that human scientists go through. The researchers at Sakana AI submitted three papers fully generated by AI (with no human modifications) to an ICLR workshop with the organizers' knowledge and cooperation. Reviewers knew some papers might be AI-generated but weren't told which ones, ensuring an unbiased double-blind review process. One of the three AI-generated papers received an average score of 6.33 from reviewers, placing it above the acceptance threshold and higher than many human-written papers. The accepted paper discussed a negative result related to neural network regularization, showing the AI system could identify and document unsuccessful approaches (an important aspect of scientific research). The papers were withdrawn after review as part of the pre-established protocol, as the scientific community has not yet established norms for publishing fully AI-generated research. What this research demonstrates is that AI systems can now conduct independent scientific inquiry that meets professional standards, which may significantly change how research is conducted and evaluated.

٢٩٧

٤١ تعليق

Advancing AI in Computational Oncology: Introducing GraphComm published in Nature Scientific Reports --> GraphComm, a graph-based deep learning framework, predicts cell–cell communication (CCC) from single-cell RNA-seq data by: ✔ Integrating ligand–receptor networks with pathway & protein complex info ✔ Capturing spatial context and intracellular signalling ✔ Identifying CCC changes under drug treatment (e.g., PC9 lung cancer cells) ✔ Supporting applications in tumour microenvironment and therapy response This approach bridges AI, systems biology, and oncology, enabling deeper insights into disease mechanisms and potential therapeutic targets. 📄 Read the full paper: https://lnkd.in/g524mkze 🔗 Code & reproducibility: https://lnkd.in/gB5gQXVK University Health Network Princess Margaret Cancer Centre Benjamin Haibe Kains Lab

١١٤

١ تعليق واحد

PhD student Siqi Wei and Master student Christo Sasi published an excellent review on the use of #knowledgegraphs and #neuralnetworks for #drugrepurposing. They also compared the properties of different graph neural network (GNNs) methods for the prediction of new drug repurposing candidates. Access the open access paper at: https://lnkd.in/e_ybMVVN. SIMPATHIC REMEDi4ALL REPO4EU

٧١

١ تعليق واحد

"Shall we give up writing scientific papers to AI? ACS Energy Letters Editorial by Filippo De Angelis* https://lnkd.in/g_PxGTxE ......One may be tempted to extend the use of AI tools to the write up of the entire paper, but most of the time (with the probable exception of uncritical reviewing of a given field) AI tools would likely generate stylistically good but scientifically trivial text. We now have to face reality and use our own creativity to discuss the main paper outcome and present them in a catchy way. As a matter of fact, one can probably recognize a scientific breakthrough even if its description is not stylistically outstanding. Most notably, we cannot expect AI tools to make connections beyond what they already know.

١٢٣

٤ تعليق

UPDATE: Link to the pre-print is now available at https://lnkd.in/ekYgGtN7 ***** Happy to share that our paper "We politely insist: Your LLM must learn the Persian art of Taarof" has been accepted at EMNLP 2025. This collaborative research with Ali Emami (Emory University), Nikta Gohari Sadr (Brock University), and other colleagues explores Persian pragmatics in LLMs. We introduce TaarofBench, a benchmark designed to evaluate large language models’ ability to recognize, interpret, and generate 'taarof', a form of indirect speech central to Persian politeness and social interaction. Our results show that even top-performing LLMs struggle with nuanced socio-pragmatic reasoning in Persian, revealing key gaps in current multilingual AI capabilities. Many thanks to my brilliant co-authors who led the way on this project!

١٬٠٩٦

٨٧ تعليق

Together with my outstanding collaborators Vittorio Del Tatto, Debarshi Banerjee, and Ali Hassanali we just posted on the archive a work where we address a quasi-philosophical question 😃 : can causality be defined and estimated in a molecular system sampling the canonical distribution by a time-reversible dynamics? The answer, according to us, is yes: genuine causal links do exists even "at equilibrium", and can be detected even without performing an external intervention on the system. WE WARMLY WELCOME COMMENTS AND SUGGESTIONS: THIS, FOR US, IS A HOT RESEARCH TOPIC, AND MANY QUESTIONS ARE STILL OPEN  https://lnkd.in/e9Hpt-BC

١٣١

Here is a new paper by Shahin Mahmoudi "Addressing Class Imbalance in Micro-CT Image Segmentation: A Modified U-Net Model with Pixel-Level Class Weighting" that introduces a novel approach for handling class imbalance in micro-CT image segmentation. Shahin is Omid Asghari, PhD, MAusIMM PhD student, and I provided some guidance for the work. Shahin developed a modified U-Net architecture enhanced with a pixel-level class weighting strategy that significantly improves segmentation accuracy, especially for minority classes. This advancement has implications not only for digital rock physics but also for broader geoscience applications, such as mineral mapping and geochemical anomaly detection. Enjoy! https://lnkd.in/gXe4X_vn

٥٨

٣ تعليق

Our new full technical research paper at the core A* conference ACM International Conference on the Foundations of Software Engineering (FSE) on hallucination detection in LLMs, with my amazing student Afif Al Mamun, Borui Yang, and my wonderful colleague Jie Zhang. Hallucination Detection in Large Language Models with Metamorphic Relations. Borui Yang, Afif Al Mamun, Jie Zhang, Gias Uddin. ACM International Conference on the Foundations of Software Engineering (FSE). Abstract: Large Language Models (LLMs) are prone to hallucinations, e.g., factually incorrect information, in their responses. These hallucinations present challenges for LLM-based applications that demand high factual accuracy. Existing hallucination detection methods primarily depend on external resources, which can suffer from issues such as low availability, incomplete coverage, privacy concerns, high latency, low reliability, and poor scalability. There are also methods depending on output probabilities, which are often inaccessible for closed-source LLMs like GPT models. This paper presents MetaQA, a self-contained hallucination detection approach that leverages metamorphic testing and prompt mutation. Unlike existing methods, MetaQA operates without any external resources and is compatible with both open-source and closed-source LLMs. MetaQA is based on the hypothesis that if an LLM’s response is a hallucination, the designed metamorphic relations will be violated. We compare MetaQA with the state-of-the-art zero-resource hallucination detection method, SelfCheckGPT, across multiple datasets, and on two open-source and two closed-source LLMs. Our results reveal that MetaQA outperforms SelfCheckGPT in terms of precision, recall, and f1 score. For the four LLMs we study, MetaQA outperforms SelfCheckGPT with a superiority margin ranging from 0.041 - 0.113 (for precision), 0.143 - 0.430 (for recall), and 0.154 - 0.368 (for F1-score). For instance, with Mistral-7B, MetaQA achieves an average F1-score of 0.435, compared to SelfCheckGPT’s F1-score of 0.205, representing an improvement rate of 112.2%. MetaQA also demonstrates superiority across all different categories of questions.

١١٣

٧ تعليق

🚨 New paper: DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors 🚨 Link to paper: https://lnkd.in/eBC6W7GJ Open benchmarks are foundational for evaluating large language models, but their accessibility leaves them vulnerable to misuse. In this paper, we introduce a new framework that leverages backdoor attacks—not as threats, but as tools—to detect when a model has memorized the test set. DyePack embeds stochastic backdoors into test data, allowing us to provably flag contaminated models with negligible false positive rates—all without access to model internals. It’s an approach inspired by how dye packs silently mark stolen currency. We demonstrate its effectiveness across multiple-choice and open-ended tasks, detecting all contaminated models with FPRs as low as 0.000017%. This is, we believe, a step toward restoring trust and rigor in LLM benchmarking. Joint work with Yize Cheng Wenxiao Wang Mazda Moayeri

٩٩

٢ تعليق

Artificial intelligence for development (AI4D): A contested notion. New paper by Sophie Toupin and Roda Siad published in the special issue on Datafied Development that Mohammad Amir Anwar, Nanna Thylstrup, Julian P. and I are editing for Big Data & Society https://lnkd.in/g36FNcHM

٥٠

٣ تعليق