dynamical.org

The dynamical.org scorecard has been crunching the ECMWF AIFS metrics -- we also added bias-corrected versions of the temperature metrics. With all the usual caveats of scientific imprecision due to different spatial resolutions, time steps, issue frequencies --- impressive.

🧊 #Icechunk 2 is out today 🧊 When we released 1.0 last July, we committed to format stability and declared it production-ready. Since then, teams across weather forecasting, climate science, neuroscience, and AI/ML pushed it in amazing ways (and some we didn't fully anticipate — repos with tens of thousands of commits, 100k+ arrays, and multi-terabyte distributed write pipelines running around the clock). That real-world usage shaped this release. Icechunk 2 delivers: 💪 Stronger consistency — a unified repo info file makes all repository state atomic and serializable, including branch ops, GC, and config changes 🚂 New data ops — move_node, shift_array, and reindex_array are now cheap metadata-only operations (no chunk rewrites) ⏹️ Rectilinear chunk grids — variable-sized chunks per dimension, enabling prepends, inserts, and irregular time intervals 🚀 Better performance — O(1) ancestry lookups, parallel flush, concurrent transaction log fetching 🪨 Hardened reliability — expanded retries, timeout controls, and fault injection tested against real network failures 💻 WASM support — Icechunk core now compiles to WASM, opening the door to browser and Node.js usage The migration path is smooth: Icechunk 2 fully reads and writes V1 repos, and upgrading is a metadata-only operation: no chunk data is copied or rewritten. Already running in the Earthmover Platform for weeks. So many thanks to the community for their comments, contributions, and collaboration, especially dynamical.org 💙 Get it today: pip install icechunk

"Homer" 😉 wrote the Odyssey nearly three millennia ago. Randall Cerveny read it in 1993 and thought: "these storm descriptions are suspiciously accurate." In the latest Weathering episode with Marta, we take a diversion from AI x Wx papers to dig into Cerveny's "Meteorological Assessment of Homer's Odyssey." Turns out, yes: the storm sequences in the Odyssey track remarkably well against what we know about wind shifts, wave behavior, the timing of squalls in the Mediterranean. Can a poem be an "observation"? Is dactylic hexameter an enduring data format? > Web: https://lnkd.in/e7aJrDKC > Spotify: https://lnkd.in/eKWJXnzc > Apple: https://lnkd.in/efD93WKS

GitHub CoPilot CLI with Claude Sonnet 4.6 spent 55 minutes working on a GRIB2 reading issues from NOAA: National Oceanic & Atmospheric Administration MRMS data. https://lnkd.in/etCrP48Y I am working with AI to develop some ML to test reinforcement learning for tornado tracks. One of the first steps is to grab the historical MRMS data for the parameters I am interested in and get formatted into zarr dataset to make it #aiml ready. The first cut of this was slow and I wasn't seeing progress so I had a previous code snippet that could run in marimo pretty quick for a viewer I had. I gave it that code snippet and told it that it was too slow. 55 minutes later AI had taken the process from taking roughly 108 minutes per tornado to 2.3 minutes! No wonder I wasn't seeing the progress I wanted to see. It optimized the previous implementation to not cache the mrms locally before building the #zarr. It figured out that cfgrib was downloading the entire CONUS first and then applying the bounding box filtering because it didn't support that and swapped out for another package that could handle it. Then it put it into parallelized streams of downloading and processing in memory straight to zarr with no intermediate cache. During that process it identified that there was no thread safety and that needed to be resolved. The point is that the CLI did a great job of testing theories about ways to fix and improve this code to figure out my speed issue with only a little input of a snippet that worked better for me. It also can do a great job of summarizing what it did as well for documentation sake. FYI: Yes I am turning this process into articles later and sharing the code results. GRIB2 files are not painful to translate even if they still do suck to access. Thank you dynamical.org for working to fix this! #github #copilot #cli #ai #llm