Vineet J Nair, PhD

This year we submitted a few papers to ACL. These papers, in my opinion, differs significantly in quality: some papers are well above the acceptance bar, and some probably will get clear rejections, and one paper may be comparable with (or even better than) past best papers. Suprisingly, ALL papers got almost the same rating scores (boarderline). Lots of review comments are irrelavant to our work. AI detection tools suggest that some review comments are highly likely generated by AI models! I think this is a natural outcome when the review system ENFORCES that every submission should list at least one reviewer for ACL. Reviewers will NOT do a high-quality job when they are enforced to do something. Probably it's better to have a two-step review process: --Step 1: the AE desk rejects part of the papers that are obviously below the acceptance bar; --Step 2: The remaining papers go through a full review process handled by a smaller pool of experienced reviewers. ——— PS: Some peer researchers questioned: “why do you still submit a paper when you think it’s a clear rejection in your opinion?” Here is some additional information: — I’m very strict (or probably harsh) in paper quality control. A paper below my expectation is often still above the conference acceptance bar. What I care is NOT publication quantity, but the real contribution & impact to a research field. As a result, I often withhold a paper for a few rounds when it does not reach my standard; — When I say a paper is a clear rejection in “my own” opinion (which has a very high bar), its quality is normally at the boundary of acceptance. For such a paper, I normally withhold it for a while to further improve it, but sometimes I just let it get submitted because: (1) other co-authors think it’s a good paper and can get accepted, (2) student authors may get highly discouraged if I withhold the paper for a long time (especially when other groups may publish a similar idea), (3) student authors need review feedback to grow & improve their research even if the paper is finally rejected. This post is to share my concern regarding the extremely random review outcome in the AI community: even the papers obviously differ significantly in quality, they received almost the same average score. This concern is shared by some other PIs this year (see the comments below).

709

43 Comments

Stanford Center for Research on Foundational Model's Marin project has released the first fully open model in JAX. It’s an 'open lab' sharing the entire research process - including code, data, and logs, to enable reproducibility and further innovation. Check out the project: https://goo.gle/44AMeLY

41

2 Comments

Designing reward functions for RL agents is broken. The default approach remains manual engineering: domain experts iteratively craft reward signals through trial-and-error. This requires significant expertise, takes enormous human effort, and often fails when task complexity increases. But what if agents could discover their own optimal reward functions? This new research introduces a bilevel optimization framework that automatically discovers optimal reward functions for embodied RL agents through regret minimization. The optimal reward function can be defined as one that minimizes the gap between the learned policy and the true optimal policy. No expert demonstrations needed. No human feedback required. How it works: Two optimization levels run simultaneously. The lower level trains the RL agent to maximize rewards as usual. The upper level continuously updates the reward function itself, guided by a meta-gradient that minimizes policy regret. The reward function learns to assign high values to critical states like success or failure, while providing dense feedback throughout the state space. The framework works across both value-based agents (DQN) and policy-based agents (PPO, SAC, TD3) without task-specific tuning. In data center energy management, all RL agents using discovered rewards achieved energy reductions exceeding 60%, compared to 21-52% for baseline RL. In UAV trajectory tracking, the approach enabled PPO agents to successfully complete tasks where hand-designed rewards failed entirely. In sparse-reward OpenAI benchmarks, agents using discovered rewards outperformed baselines in both convergence speed and final performance. The discovered reward functions also reveal interpretable structure: they automatically identify critical states and encode latent relationships between states and rewards that match physics-based reward designs, despite having no explicit mathematical model.

92

6 Comments

Impact Scholars Spotlight: Biological Connectivity Patterns as a Blueprint for Efficient Neural Architectures in Reinforcement Learning 🎉 Congratulations to this Impact Scholars team: Zahra Moradimanesh, Ruman Ahmed Shaikh, Nima Zargarnezhad, Austin Meek, Paolo Muratore 👏 And to the team's mentor Ilya Kuzovkin This team participated in the #NeuroAI steam of Neuromatch 's Impact Scholars Program and developed an interesting project: 📘 Biological Connectivity Patterns as a Blueprint for Efficient Neural Architectures in Reinforcement Learning Check out their final project: 🎬 Presentation: https://lnkd.in/g4T_pSGZ 🤩 General public summary: https://lnkd.in/gzr4Kk6p 📃 Publication: https://lnkd.in/gQFeEDvn 🧠 About Neuromatch's Impact Scholars Program: The Impact Scholars Program supports scientists in expanding the research they began during Neuromatch Academies—including Computational Neuroscience, Deep Learning, NeuroAI, and Climate Science. Over six months, scholars gain hands-on experience, mentorship, and the chance to translate data into a real-world research project. Stay tuned as we share more projects from the 2025 cohort!

45

2 Comments

New Journal Paper! My excellent Ph.D. student Nicholas Nurre and I have been working on developing advanced regularization indirect methods for low-thrust trajectory optimization subject to duty-cycle constraints and comparing the results against direct optimization methods. This paper presents insightful results and comparisons. Title: Comparison of Indirect and Direct Methods for Operationally Constrained Low-Thrust Trajectories Abstract: We propose two novel #indirect and #direct #optimization methods for generating #minimumfuel #lowthrust trajectories for variable-specific-impulse variable-thrust (VIVT) solar #electric #propulsion (SEP) systems subject to solar power and #dutycycle #constraints. We consider both regularized and non-regularized formulations of the indirect methods. Furthermore, we use an #L2norm based regularization for smoothly #embedding #discontinuous events (e.g., throttle-control and power-input switches) to form a multiparameter family of smooth #Hamiltonian #boundaryvalue problems (BVPs). The solution to the regularized BVP is then used to initialize and solve an event-detection-based, non-regularized BVP. For the direct method formulation, a #forwardshooting scheme is proposed with the introduction of an important power constraint to avoid overestimation of the performance of the VIVT SEP system. Trajectories are solved with over 300 duty-cycle-driven #thrust #switches. https://lnkd.in/gY_Sa3aN #Aerospace #spacecraft #trajectory #indirect #direct #optimization #lowthrust #regularization #optimal #control #L2norm #variableimpulse #variablethrust #boundaryvalue #Pontryagin Auburn University Auburn University, Samuel Ginn College of Engineering

77

1 Comment

Research papers we love 🔬 📄 Targeted materials discovery using Bayesian algorithm execution. 👥 Sathya R. Chitturi, Akash Ramdas, Yue Wu, Brian Rohr, Stefano Ermon, Jennifer Dionne, Felipe H. da Jornada, Mike Dunne, Christopher Tassone, Willie Neiswanger, and Daniel Ratner. Rapid discovery and synthesis of future materials requires intelligent data acquisition strategies to navigate large design spaces. A popular strategy is Bayesian optimization, which aims to find candidates that maximize material properties; however, materials design often requires finding specific subsets of the design space which meet more complex or specialized goals. We present a framework that captures experimental goals through straightforward user-defined filtering algorithms. These algorithms are automatically translated into one of three intelligent, parameter-free, sequential data collection strategies (SwitchBAX, InfoBAX, and MeanBAX), bypassing the time-consuming and difficult process of task-specific acquisition function design. Our framework is tailored for typical discrete search spaces involving multiple measured physical properties and short time-horizon decision making. We demonstrate this approach on datasets for TiO2 nanoparticle synthesis and magnetic materials characterization, and show that our methods are significantly more efficient than state-of-the-art approaches. Overall, our framework provides a practical solution for navigating the complexities of materials design, and helps lay groundwork for the accelerated development of advanced materials. Read more: https://lnkd.in/dnmCd26a 🤔 What do you think about it? Share with us in the comments 👇 #materials #RnD #mat3ra #exabyteio #materialsscience #materialsdesign #materialsmodeling #science #technology

26

CDS founding director Yann LeCun and NYU Courant Institute of Mathematical Sciences PhD student Jiachen Zhu have challenged a fundamental assumption in deep learning: normalization layers aren't as essential as previously thought. Their paper introduces Dynamic Tanh (DyT), a surprisingly simple replacement for normalization layers in transformer models. DyT operates as an element-wise function that matches or exceeds the performance of traditional normalization across vision, language, speech, and genomic sequence models. The findings open new possibilities for more efficient neural network design while providing important insights into why normalization has worked so well. https://lnkd.in/dWjdRPj8 #DeepLearning #MachineLearning #AIResearch

57

1 Comment

It’s been almost a year since OpenAI claimed their reasoning models deliver PhD-level research. Here’s why a transformer can never be compared to a PhD student: 1. PhD students don’t (usually) just say what’s most likely to be said in a given context. A good dissertation aims for originality. Originality comes from common sense and subjective experiences that LLMs, by definition, do not have. 2. Research is not about complying with user needs and that’s what LLMs are trained for. It’s about discovering hidden truths based on prior research. 3. Good research prioritizes truth and factuality. Something LLMs don’t care about. 4. LLMs don’t create hypotheses or ask questions. They don’t challenge the status quo or disagree with the majority. At best, they summarize related work occasionally mixing up citations or hallucinating quotes. #AI #Tech

1,040

179 Comments

This paper is a banger, extremely elegant and impactful. They first show that the gradient of Maximum Likelihood (ML) includes a 1 / p(x) factor relative to the RL gradient, where p(x) is the probability that the model produces a correct output for input x. This implies more emphasis on harder samples. They then show that the gradient of the ML objective can be written as a sum over k from 1 to infinity of (1 / k) times the gradient of pass@k(x), where pass@k is the probability of obtaining at least one correct sample in k trials. They show that the RL gradient is simply the gradient of pass@1(x). Thus, the RL objective is a first-order approximation of the ML objective. They approximate the infinite sum with a finite one: for T = 1, this recovers RL; as T → ∞, it approaches ML. They then exploit the fact that the ML gradient can be expressed as an expectation under the success-conditioned distribution, leading to an extremely simple gradient estimator: (N / K) times the standard REINFORCE estimator, where N is the number of trajectories and K is the number of successful samples. They show that this estimator is unbiased for the truncated ML objective when the number of sampled trajectories satisfies N = T. They further reduce the variance of the estimator by subtracting a zero-mean control variate given by the unconditional average score, preserving unbiasedness. The key takeaway is that they introduce a straightforward method that trades compute for performance: as N increases, the objective approaches ML, which dominates RL. They also show this empirically, with performance improving as N increases. The method scales more favorably in the data-rich regime. In the data-scarce regime, it exhibits less overfitting and converges to higher average performance. Finally, while RL is notorious for distribution sharpening, they observe little to no degradation in diversity. They also show that MaxRL produces stronger gradients for harder prompts and leads to a larger fraction of prompts with at least one correct rollout.

686

9 Comments

Orbyfy Labs | A New Paper on Closing the Blow-Up Question in Navier–Stokes Link to our paper on ResearchGate: https://lnkd.in/ggqqsjWJ When Navier and Stokes wrote down their equations in the 19th century, they weren’t trying to describe turbulence in all its chaos. They were trying to express something much older: how motion redistributes energy and entropy through space. The Clay Millennium Navier–Stokes problem asks whether that redistribution can ever fail catastrophically; whether smooth, finite-energy motion can collapse into a singularity in finite time. Leray’s answer in the 1930s was pragmatic: even if smoothness breaks down, physically meaningful Leray–Hopf weak solutions should still exist and conserve energy in an averaged sense. What remained unclear for nearly a century is whether those weak solutions still allow uncontrolled accumulation of motion; whether vortex stretching can pile up faster than entropy can dissipate it. In this work, we show that it cannot. Rather than approaching the problem through sharper estimates, we treat Navier–Stokes as a geometric entropy evolution. Using Lyapunov monotonicity and Perelman-style entropy regulation, together with self-adjoint operator theory at the Leray–Hopf level, we show that curvature amplification is always integrable in time. Motion can intensify locally, but it cannot accumulate into a finite-time blow-up. This reframes turbulence itself. What has long been treated as randomness turns out to be structured: motion borrows energy locally, but entropy enforces repayment globally. The equations don’t merely describe flow, they regulate it. What makes this especially compelling is falsifiability. We tested this framework against strongly driven Weber packed-bed transport experiments and high-resolution turbulence simulations from the Johns Hopkins Turbulence Database. In both cases, despite sustained forcing and intense local amplification, curvature growth remained integrable, spectra stayed confined, and no runaway accumulation was observed, exactly as the theory predicts. Historically, this places Navier–Stokes closer to other great structural results in mathematics: where apparent instability is resolved not by suppression, but by identifying the invariant that prevents runaway behavior. As further research unfolds, this perspective may reshape how we think about motion, entropy, and stability across physics, from fluids to plasmas to energy transport. #NavierStokesRegularity #NavierStokes #FluidDynamics #LerayHopf #WeakSolutions #SobolevSpaces #SelfAdjointOperators #SpectralTheory #FunctionalAnalysis #GeometricAnalysis #EntropyMethods #LyapunovTheory #PerelmanEntropy #HighReynoldsFlows #DigitalTwins #PhysicsBasedModeling

26

10 Comments