AI Model Development

Every board is betting big on AI. Almost none are asking the question that actually protects them. I’ve been in boardrooms across industries, from finance to healthcare, and I keep seeing the same thing: Board members ask: - “What’s the AI budget?” - “What’s the timeline?” - “What’s the ROI?” But almost no one asks the most important question: “How do we even know this is AI?” Here’s the problem… Most boards are approving AI initiatives without a clear definition of what qualifies as AI because the lines are blurry. Vendors show up with polished demos and pitch tools labeled “AI-powered.” But without clarity, boards end up greenlighting: ✗ Rule-based systems dressed up as intelligence ✗ Traditional software relabeled with buzzwords ✗ Proof-of-concept demos, not scalable AI infrastructure ✗ “AI-washed” features that don’t actually learn or adapt Before the next AI contract crosses your desk, ask leadership: → Where exactly does machine learning happen in this system? → How does it improve over time with use? → What data powers it, and who owns that data? → How much human intervention is required for results? Because the companies truly win with AI? They’re not the ones with the flashiest tools. They’re the ones whose boards can differentiate real intelligence from noise. What’s your take - have you seen “AI” claims fall apart under scrutiny?

BREAKING! The FDA just released this draft guidance, titled Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, that aims to provide industry and FDA staff with a Total Product Life Cycle (TPLC) approach for developing, validating, and maintaining AI-enabled medical devices. The guidance is important even in its draft stage in providing more detailed, AI-specific instructions on what regulators expect in marketing submissions; and how developers can control AI bias. What’s new in it? 1) It requests clear explanations of how and why AI is used within the device. 2) It requires sponsors to provide adequate instructions, warnings, and limitations so that users understand the model’s outputs and scope (e.g., whether further tests or clinical judgment are needed). 3) Encourages sponsors to follow standard risk-management procedures; and stresses that misunderstanding or incorrect interpretation of the AI’s output is a major risk factor. 4) Recommends analyzing performance across subgroups to detect potential AI bias (e.g., different performance in underrepresented demographics). 5) Recommends robust testing (e.g., sensitivity, specificity, AUC, PPV/NPV) on datasets that match the intended clinical conditions. 6) Recognizes that AI performance may drift (e.g., as clinical practice changes), therefore sponsors are advised to maintain ongoing monitoring, identify performance deterioration, and enact timely mitigations. 7) Discusses AI-specific security threats (e.g., data poisoning, model inversion/stealing, adversarial inputs) and encourages sponsors to adopt threat modeling and testing (fuzz testing, penetration testing). 8) And proposed for public-facing FDA summaries (e.g., 510(k) Summaries, De Novo decision summaries) to foster user trust and better understanding of the model’s capabilities and limits.

NVIDIA just open-sourced a whole family of weather and climate ai models, and I think this is the moment serious #forecasting stops being something only national weather agencies can do. #Earth2 isn't one model, it's a stack. #Atlas does 15-day global forecasts and beats GenCast on benchmarks. #StormScope is the first AI to outperform physics-based systems on storm dynamics. #HealDA spins up initial atmospheric conditions in seconds on a GPU, the kind of step that used to take hours on a supercomputer. #CorrDiff downscales 500x faster while using 10,000x less energy. what gets me excited isn't any one of these though, it's that the whole pipeline is just sitting on #HuggingFace and #GitHub now. running weather ai used to mean physics models, now a small team in any country can fine-tune these and run them on their own hardware. bigger models are interesting, but this feels more important to me, taking something that lived behind a national lab's firewall and just handing it out. Link to Earth-2: https://lnkd.in/exReKuTV #climateAI #remotesensing #foundationmodel #weather #climate

Many engineers can build an AI agent. But designing an AI agent that is scalable, reliable, and truly autonomous? That’s a whole different challenge.  AI agents are more than just fancy chatbots—they are the backbone of automated workflows, intelligent decision-making, and next-gen AI systems. However, many projects fail because they overlook critical components of agent design.  So, what separates an experimental AI from a production-ready one?  This Cheat Sheet for Designing AI Agents breaks it down into 10 key pillars:  🔹 AI Failure Recovery & Debugging – Your AI will fail. The question is, can it recover? Implement self-healing mechanisms and stress testing to ensure resilience.  🔹 Scalability & Deployment – What works in a sandbox often breaks at scale. Using containerized workloads and serverless architectures ensures high availability.  🔹 Authentication & Access Control – AI agents need proper security layers. OAuth, MFA, and role-based access aren’t just best practices—they’re essential.  🔹 Data Ingestion & Processing – Real-time AI requires efficient ETL pipelines and vector storage for retrieval—structured and unstructured data must work together.  🔹 Knowledge & Context Management – AI must remember and reason across interactions. RAG (Retrieval-Augmented Generation) and structured knowledge graphs help with long-term memory.  🔹 Model Selection & Reasoning – Picking the right model isn't just about LLM size. Hybrid AI approaches (symbolic + LLM) can dramatically improve reasoning.  🔹 Action Execution & Automation – AI isn't useful if it just predicts—it must act. Multi-agent orchestration and real-world automation (Zapier, LangChain) are key.  🔹 Monitoring & Performance Optimization – AI drift and hallucinations are inevitable. Continuous tracking and retraining keeps your AI reliable.  🔹 Personalization & Adaptive Learning – AI must learn dynamically from user behavior. Reinforcement learning from human feedback (RHLF) improves responses over time.  🔹 Compliance & Ethical AI – AI must be explainable, auditable, and regulation-compliant (GDPR, HIPAA, CCPA). Otherwise, your AI can’t be trusted.  An AI agent isn’t just a model—it’s an ecosystem. Designing it well means balancing performance, reliability, security, and compliance.  The gap between an experimental AI and a production-ready AI is strategy and execution.  Which of these areas do you think is the hardest to get right?

AI is completely rewriting the rules of weather forecasting, and this video from NVIDIA is a perfect example of how fast things are moving. In just under 5 minutes, the video demonstrates Earth-2, a platform that allows you to run global weather forecasts in mere seconds using just a few lines of Python. You can seamlessly switch between data sources (like ERA5, GFS, IFS) and even swap out entire AI models (like FourCastNet, GraphCast, or Aurora) with a single line of code. But NVIDIA isn’t alone. We are witnessing an arms race among big tech to solve weather prediction: - Google DeepMind has GraphCast and NeuralGCM, which have already outperformed gold-standard physical models in many metrics. - Microsoft released Aurora, a foundation model trained on over a million hours of data, claiming to be 5000x faster than traditional numerical systems. - IBM & NASA recently open-sourced Prithvi, a "geospatial foundation model" designed not just for weather, but to be fine-tuned for specific climate applications. - Huawei has Pangu-Weather, which famously predicted the path of a typhoon more accurately than traditional methods. Why is this happening? - Compute: Traditional Numerical Weather Prediction (NWP) solves complex physics equations requiring massive supercomputers. AI models, once trained, infer results in seconds on a few GPUs. - Ensemble Forecasting: Because they are so cheap to run, we can generate thousands of scenarios (ensembles) instead of just a few. This is a game changer for predicting low probability extreme weather events. - Data Fusion: These models are proving incredibly good at learning patterns from historical data that pure physics equations might miss. For the geospatial practice, this is a big change. Weather is moving from a static dataset we download to a dynamic capability we run. You no longer need a supercomputer to generate high-resolution forecasts; you just need a GPU and a Python script. We may soon see fine-tuned weather models for specific geospatial use cases like hyper local wind for drones, precise precip for agriculture, or cloud cover for satellite tasking. The latency between data in and forecast out is shrinking to near zero, enabling true real time geospatial intelligence. Have you tried any of these models? What are your thoughts? 🌎 I'm Matt Forrest and I talk about modern GIS, earth observation, AI, and how geospatial is changing. 📬 Want more like this? Join 12k+ others learning from my daily newsletter → forrest.nyc

How AI is changing storm response in the U.S. — technically. Have you experienced it? Extreme weather response is no longer driven by single forecasts. It’s driven by ensembles + AI acceleration + real-time data fusion. Here’s what’s happening under the hood: AI-accelerated Numerical Weather Prediction (NWP) Deep learning models (graph neural nets, transformers) are trained on decades of reanalysis data to approximate full physics-based solvers. Result: • Inference in seconds instead of hours • Enables rapid ensemble generation (hundreds of scenarios, not dozens) This allows forecasters to update storm tracks and intensity continuously, not on fixed cycles. Multi-modal data fusion AI ingests: • Satellite imagery (GOES) • Doppler radar volumes • Ocean buoys & atmospheric soundings • Ground IoT sensors • Historical climatology Models correlate spatial-temporal patterns across modalities — something classical models struggle with at scale. Severe weather nowcasting Computer vision models detect: • Convective initiation • Tornadic signatures • Rapid intensification signals Lead times improve by 30–60 minutes for fast-forming events — which is operationally massive for emergency management. Probabilistic forecasting, not single answers ML-driven ensembles output probability distributions, not deterministic paths: • Flood depth likelihoods • Wind gust exceedance • Ice accumulation risk This feeds directly into risk-based decision systems. Infrastructure impact modeling Utilities combine AI weather outputs with: • Grid topology • Asset age & failure history • Load forecasts This enables pre-storm optimization: • Crew pre-positioning • Targeted grid isolation • Faster restoration paths Operational decision intelligence AI systems now bridge forecast → action: • When to evacuate • Where to stage responders • Which assets fail first This is no longer meteorology alone — it’s real-time systems engineering. Storms are getting more chaotic. Our response is getting more computational. AI doesn’t replace physics. It compresses it into time we can actually use. #AI #WeatherModeling #Nowcasting #ClimateTech #InfrastructureAI #DigitalTwins #ResilienceEngineering #HPC

$7,225 for one day of coding. And Cursor isn't even the worst example. Replit's margins went negative. Anthropic throttles its best users. I mapped pricing across 50 AI startups. Six distinct patterns emerged. The core tension: traditional SaaS has near-zero marginal cost per user. AI products pay for compute on every interaction. A casual Claude user costs pennies. A developer running Claude Code all day costs tens of thousands per month. Your best users are your most expensive users. That tension is breaking every pricing model in the market. Cursor charged a flat 500 requests/month. Worked fine until users leaned into multi-step agent workflows. They switched to credit pools. One developer burned 500 requests in a single day. The plan description changed from "Unlimited" to "Extended" twelve days after launch. Replit grew 15x in ten months ($16M to $252M ARR). But they were buying revenue with compute. When they launched a more autonomous agent, margins crashed to negative 14%. They had to invent "effort-based pricing" mid-flight. Anthropic played it differently. Their $17/$100/$200 tiers map to genuinely different user personas, not volume bands. A casual user and a Claude Code developer are different products with different willingness to pay. The lesson across all 50 companies: before you set any price, pull the cost distribution. What does your P10 user cost? P50? P90? If the ratio exceeds 10x, flat pricing will break. In AI products, it almost always exceeds 10x. Full guide with all 6 models, 4 case studies, and a decision tree: https://lnkd.in/gdKaQSMk

❗ Today, the European Commission not only launched a public consultation to gather input for its forthcoming guidelines on general-purpose AI models, but also unveiled its “preliminary approach” on how it interprets the GPAI rules of the AI Act. This approach is detailed in a dense 21-page document and marks the first time the EU executive has clarified its interpretation of some key provisions of the law and outlined its intended application. Perhaps the most notable aspect of the document is the establishment of a compute threshold—set at 10^22 floating point operations per second (FLOPs)—to determine whether a model falls under the AI Act. A similar threshold is used to decide if a modified model should be considered a new model, with all the accompanying legal implications. Remarkably, if a GPAI model initially falls short of the systemic threshold but meets it after modification, the entity responsible for the modification will be designated as a provider of a model with systemic risk. While this measure may be necessary to prevent circumvention of the systemic risk categorization, it might also discourage modifications to models that are just below 10^25 FLOPs. Another key concept the Commission aims to clarify in the guidelines is when a GPAI model is considered placed in the EU market, with the preliminary approach already including several examples. Moreover, two methodologies, one hardware-based and one architecture-based, are provided to calculate the computational resources. Additionally, the working document appears to encourage GPAI model providers to sign the upcoming code of practice. Signatories can expect that the Commission’s enforcement efforts will focus on their adherence to the code. In contrast, companies that choose not to sign will have to demonstrate compliance through other means, conduct a gap analysis, and be prepared to provide additional information upon request. Finally, the Commission outlined its enforcement approach for the first time, emphasizing a collaborative and proportionate strategy. It anticipates close, informal collaboration with model providers and a proactive stance from those supplying models with systemic risks. My full analysis for MLex.

In the past few months, we've worked with partners who've run into the same challenge with AI adoption. They rolled out policies or guidelines without bringing people into the conversation first—no workshop, no consensus building, just documents that needed signatures or implementation. Unsurprisingly, the result was frustrated staff expected to enforce or follow rules they had no part in creating, and leaders facing resistance instead of adoption. Both AI policies and guidelines are critical for responsible AI adoption, but they have to be built intentionally, with stakeholders driving consensus, or they most likely won't work. After working with hundreds of districts, we've created the resource below. Here are the best practices we recommend. Policies are your compliance layer and are designed to protect your district. We suggest adaptations to existing: ✔️ Acceptable use policies ✔️ Data privacy/FERPA protections ✔️ Academic integrity standards ✔️ Cyberbullying policies (to add deepfakes) Guidelines are your change management layer. They are the "why" that brings people along. We recommend including the following in your AI guidelines: 💡 Vision for GenAI adoption across your district 💡 GenAI misuse/academic integrity response protocols 💡 GenAI chatbot and EdTech tool vetting processes 💡 Digital wellbeing, data privacy, and student safety practices 💡 Implementation tips and instructional supports 💡 AI Literacy training opportunities and expectations What matters most is that both policies and guidelines should be built with stakeholders, not handed down to them. They should evolve with feedback, evidence of impact, and technical advancements. In all of our guideline and policy development work, we always start with AI literacy. It's important to build foundational understanding across stakeholders so that when policies and guidelines are developed, people can contribute meaningfully to the process and understand the "why" behind what they're being asked to implement. Intentional stakeholder engagement isn't a nice-to-have. It's what we've seen drive adoption. #AIforEducation #GenAI #ChangeManagement #AI

New research proves your AI model already knows the answer before it starts "thinking." A paper from Goodfire AI and Harvard tested two frontier reasoning models and found something security teams need to hear. On recall-based tasks, the kind that dominate enterprise agentic AI workflows, models committed to their final answer within the first tokens of reasoning. Then they generated hundreds of additional tokens that looked like careful deliberation. 80% of those tokens were unnecessary. Every one of them passed the CoT monitor. The researchers have a name for it: Reasoning Theater. Here's what kept me up. They applied Grice's maxims of cooperative communication to explain the gap. Turns out reasoning models follow the maxims that earn reward (stay relevant, be factual) and violate the ones that don't cost them anything (be concise, be clear). The model performs what reasoning is supposed to look like without any obligation to communicate its internal state. If you've sat through a compliance audit where someone recites a scripted answer that sounds thorough but reveals nothing about practice on the ground, you've seen the human version. Three separate research groups landed on the same finding. OpenAI showed models learn to hide intent when you train against CoT monitors. Anthropic found models disclose their reasoning shortcuts fewer than 20% of the time. A 40-author coalition called CoT monitorability "fragile." I wrote up the full breakdown: what the research found, why it matters for financial services agentic AI deployments, and a CARE framework response for your next governance meeting. The person at your AI governance table who hasn't read this paper is making decisions based on an assumption that no longer holds. Dropped the link below. What's your team using as a primary safety control for agentic AI right now? Curious whether CoT monitoring is the default in your org. 👉 Full blog at: https://lnkd.in/gkBeFA2T 👉 Follow and connect for more AI and cybersecurity insights with the occasional rant #AgenticAISecurity #AIGovernance #CISOInsights