Image Processing in Science

New in Nature Methods: NimbusImage — cloud-native image analysis for modern biology!! Image analysis has always been a bottleneck in biology. Not because we lack algorithms—but because deploying them, scaling them, and interacting with results still requires substantial technical expertise. A new Nature Methods (2025) paper introduces NimbusImage, a cloud-based platform designed to close this gap. https://lnkd.in/ePz_hJgy What stands out is not just another tool, but a shift in how image analysis is done: • Web-based, cloud-native analysis of very large microscopy images • Human-in-the-loop workflows that combine automation with expert correction • Built-in deep learning (Cellpose, SAM, RNA FISH spot detection) without scripting • Asynchronous, scalable compute via containerized workers • Full exportability and API access for reproducibility and downstream analysis Crucially, NimbusImage separates annotation from measurement, allowing users to fix model errors before quantification—something most pipelines still struggle with. This feels like an important step toward making advanced image analysis: ✔️ scalable ✔️ reproducible ✔️ accessible to non-programmers ✔️ still rigorous enough for power users As imaging datasets continue to grow in size and complexity, platforms like this may become as essential as ImageJ once was—just built for the cloud and the deep-learning era. #Bioinformatics #ImageAnalysis #ComputationalBiology #Microscopy #MachineLearning #AIinBiology #NatureMethods

Imagine we could map every cell in the human body, revealing its precise location and molecular identity. This tantalizing possibility is at the heart of our latest perspective piece published in Nature Methods, where we explore a groundbreaking approach to understanding biological systems at unprecedented depth and scale: Deep 3D Histology. In this perspective article, we discuss three key pillars of this emerging field: Advanced Tissue Clearing and Imaging: -Cutting-edge tissue clearing techniques for intact specimen visualization -High-resolution light-sheet microscopy pushing the boundaries of 3D imaging -Applications ranging from mouse embryos to entire human organs Spatial Omics Technologies: -Integration of single-cell omics data with 3D spatial context -Creation of comprehensive molecular atlases of entire organisms -Bridging the gap between molecular profiles and tissue architecture Artificial Intelligence in Image Analysis: -Deep learning revolutionizing 3D histology data processing -Automation of tasks from image enhancement to cell segmentation -Unveiling information invisible to the human eye through "virtual staining" The potential impact of combining these technologies is staggering. By accelerating our understanding of diseases and drug discovery, we could compress centuries of insights into just a few years of research. Challenges remain, including improving resolution, increasing imaging speed for large samples, and developing user-friendly AI tools. But as we overcome these hurdles, Deep 3D Histology could become a routine tool in both research and clinical settings. The future of biomedical research is three-dimensional, molecularly detailed, and AI-enhanced. This new era of 3D omics has the potential to revolutionize medicine and our understanding of life itself. You can read the full perspective and join the discussion on this exciting frontier of science: https://rdcu.be/dNBe8 More technical details are here as tweetorial: https://lnkd.in/d48cTXDE #AI #DeepLearning #Clearing #3D #Imaging #Omics #Deep3DHistology

Tried something interesting with automated annotations on histology images. Using a simple text prompt, Grounding DINO was able to detect glomeruli in a kidney H&E image and generate bounding boxes. I then passed those boxes to SAM 2, which converted them into clean, pixel-level segmentation masks. So essentially: text prompt → object detection → precise segmentation. What stood out to me is how well this worked on scientific imagery, not just natural images. Annotation is often a bottleneck in biomedical workflows, and this kind of pipeline could significantly speed things up. Would be interesting to see how robust this is across different stains, tissues, and imaging conditions. If there’s interest, I can put together a short video walkthrough. #microscopy #computervision #deeplearning #imageanalysis #digitalpathology

🔍 ����𝐨𝐦𝐩𝐮𝐭𝐞𝐫 𝐕𝐢𝐬𝐢𝐨𝐧 𝐈𝐬 𝐍𝐨𝐭 𝐉𝐮𝐬𝐭 𝐀𝐛𝐨𝐮𝐭 𝐒𝐞𝐞𝐢𝐧𝐠 𝐏𝐢𝐱𝐞𝐥𝐬 — 𝐈𝐭 𝐈𝐬 𝐀𝐛𝐨𝐮𝐭 𝐔𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 𝐒𝐢𝐠𝐧𝐚𝐥𝐬 Most people look at an image and see objects, colors, edges, textures, and brightness. But in 𝐂𝐨𝐦𝐩𝐮𝐭𝐞𝐫 𝐕𝐢𝐬𝐢𝐨𝐧, an image is more than what appears on the screen. An image is a 𝐬𝐢𝐠𝐧𝐚𝐥 → and every signal carries hidden information. This is why 𝐅𝐢𝐥𝐭𝐞𝐫𝐢𝐧𝐠 is one of the most important concepts I teach, especially 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐃𝐨𝐦𝐚𝐢𝐧 𝐅𝐢𝐥𝐭𝐞𝐫𝐢𝐧𝐠 and 𝐅𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐲 𝐃𝐨𝐦𝐚𝐢𝐧 𝐅𝐢𝐥𝐭𝐞𝐫𝐢𝐧𝐠. 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐃𝐨𝐦𝐚𝐢𝐧 → works directly on pixel values. It helps smooth noise, sharpen edges, enhance contrast, and improve local image details. 𝐅𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐲 𝐃𝐨𝐦𝐚𝐢𝐧 → works after transforming the image, usually through Fourier Transform. Here, smooth regions appear as 𝐥𝐨𝐰 𝐟𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐢𝐞𝐬, while edges, fine details, sharp changes, and some types of noise appear as 𝐡𝐢𝐠𝐡 𝐟𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐢𝐞𝐬. Blur → suppression of high-frequency components. Edge → strong high-frequency signal. Noise → sometimes a visible pattern in the frequency domain. Important filters: 𝐋𝐨𝐰-𝐏𝐚𝐬𝐬 𝐅𝐢𝐥𝐭𝐞𝐫 → smooths images and reduces noise 𝐇𝐢𝐠𝐡-𝐏𝐚𝐬𝐬 𝐅𝐢𝐥𝐭𝐞𝐫 → enhances edges and fine details 𝐁𝐚𝐧𝐝-𝐏𝐚𝐬𝐬 𝐅𝐢𝐥𝐭𝐞𝐫 → preserves selected frequency ranges 𝐍𝐨𝐭𝐜𝐡 𝐅𝐢𝐥𝐭𝐞𝐫 → removes periodic noise such as stripes or sensor interference In research papers, filtering is often used as a 𝐩𝐫𝐞𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐬𝐭𝐞𝐩 before segmentation, feature extraction, classification, or deep learning model training. 𝐌𝐞𝐝𝐢𝐜𝐚𝐥 𝐢𝐦𝐚𝐠𝐢𝐧𝐠 → enhances tumors, vessels, lesions, and tissue boundaries. 𝐒𝐚𝐭𝐞𝐥𝐥𝐢𝐭𝐞 𝐢𝐦𝐚𝐠𝐢𝐧𝐠 → removes artifacts and improves land-cover analysis. 𝐈𝐧𝐝𝐮𝐬𝐭𝐫𝐢𝐚𝐥 𝐢𝐧𝐬𝐩𝐞𝐜𝐭𝐢𝐨𝐧 → highlights cracks, defects, and surface irregularities. A good research paper should clearly explain: What filter was used → why it was selected → what feature it targeted → how it improved results. A strong Computer Vision student should not only apply filters. They must understand 𝐰𝐡𝐚𝐭 𝐚 𝐟𝐢𝐥𝐭𝐞𝐫 𝐫𝐞𝐦𝐨𝐯𝐞𝐬, 𝐰𝐡𝐚𝐭 𝐢𝐭 𝐩𝐫𝐞𝐬𝐞𝐫𝐯𝐞𝐬, and 𝐡𝐨𝐰 𝐢𝐭 𝐜𝐡𝐚𝐧𝐠𝐞𝐬 𝐭𝐡𝐞 𝐢𝐦𝐚𝐠𝐞 𝐬𝐢𝐠𝐧𝐚𝐥. 🎯 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧: For periodic stripe noise, which filter is most suitable → Low-Pass, High-Pass, or Notch Filter? #ComputerVision #ImageProcessing #FrequencyDomain #Filtering #SignalProcessing #AI #DeepLearning #MedicalImaging #RemoteSensing

📢 A decade of "Big-deep-smart data in imaging for guiding materials design" Almost ten years ago as one of the first step in my position as the director of the Institute for Functional Imaging of Materials (IFIM) at ORNL, I and my colelagues Bobby Sumpter and Rick Archibald charted a roadmap on how quantitative imagign tools can be used to advance materials science and condensed matter physics. After all, astronomers learn a lot from just observing the universe (without any experiments). Surely we can harness the multitude of imaging data to understand matter on the atomic scale? And we definitely need machine learning to do so. At that time, we posed specific questions (quoting): - Can we establish the parameters of quantum theory (for example, Hubbard U or parameters of the chosen pseudopotential) given experimental observations of the atomic structure and functionality for given materials? - Can the parameters of Ginzburg–Landau free energy be improved given the observations of the ferroelectric domain structure and topology in a solid? - Can the size-dependent thermodynamic properties of nanoparticles be determined from in situ observations of shape evolution during the growth process? - Can elementary reaction mechanisms and reaction rates be obtained from the observation of the dynamics of step-edge motion during deposition or dissolutions of solids in liquids, or direct observation of molecular motion on surfaces? We also posed that strucutre of the solids cannot be described bottom up based on symemtry theory and will require statistical descriptions, that microscopic observations give rise to vastly expanded region of the chemcial space of the same system, the need for establishing defect genome libraries, and so on. Looking back 10 years, it is very surprising to which extent many of this predictions have been realized - both by us and our colleagues across the ML and physics worlds. Some of these predictins just start to get implemented - Figure 4 predicts "Here the active feedback between the expert and the experimental instrument (i) provides the inputs for training of the automatic expert systems (ii) that subsequently can perform automatic data interpretation (iii). Such a system can also explore information contained in the extant analyses via correlative analysis of citation networks and semantic text content with subsequent knowledge integration. Smart-data systems can provide a gateway ..... Ideally, we envision an image acquired by a microscope in real time being interpreted in terms of relevant functionalities, and the smart-data system offering suggestions based on exploration history of the potential impact of observed features on materials functionalities, possible origin, and so on." We just started to delve in these areas with the human in the loop AI and LLM co-scientists - looks like a right track! https://lnkd.in/e6QSwrgM

Recent advances in imaging and computation have enabled analysis of large three-dimensional (3D) biological datasets, revealing spatial composition, morphology, cellular interactions and rare events. However, the accuracy of these analyses is limited by image quality, which can be compromised by missing data, tissue damage or low resolution due to mechanical, temporal or financial constraints. Here, we introduce InterpolAI, a method for interpolation of synthetic images between pairs of authentic images in a stack of images, by leveraging frame interpolation for large image motion, an optical flow-based artificial intelligence (AI) model. InterpolAI outperforms both linear interpolation and state-of-the-art optical flow-based method XVFI, preserving microanatomical features and cell counts, and image contrast, variance and luminance. InterpolAI repairs tissue damages and reduces stitching artifacts. We validated InterpolAI across multiple imaging modalities, species, staining techniques and pixel resolutions. This work demonstrates the potential of AI in improving the resolution, throughput and quality of image datasets to enable improved 3D imaging. Full paper title: InterpolAI: deep learning-based optical flow interpolation and restoration of biomedical images for improved 3D tissue mapping Interesting paper in Nature Portfolio introducing InterpolAI, an optical flow-based deep learning method that adapts frame interpolation techniques from video processing to biological imaging for improved 3D tissue reconstruction. Paper and research by @Saurabh Joshi, André Forjaz, @Kyu Sang Han Denis Wirtz and larger team at The Johns Hopkins University School of Medicine

Why are there so many scientific imaging tools for biology? Each has a distinct function and reveals interesting research questions and methods. Unlike molecular assays, imaging datasets provide clear visual structure. From developing zebrafish embryos to neural activity in two-photon microscopy, the raw outputs are growing in complexity and size. We’ll dive into some open source imaging tools and what they actually do: Napari, ImageJ, Cellpose, CellProfiler, Suite2p, highlighting the original motivation for each tool and how they fit into common lab workflows. We’ll then construct some real-world scientific use cases, from segmenting microglia in brain slices to high-throughput compound screening in cancer models. Finally, we’ll explore some data infrastructure and software principles that lead to better analysis for scientists.

➡️ X-ray diffraction (XRD) and electron microscopy (EM) are among the most important techniques in materials science and many other fields. Yet, there has historically been no direct physics-based method to go from an XRD pattern or STEM image to the actual atomic structure. ➡️ Last week, Prof. Sergei Kalinin group demonstrated the use of our MicroscopyGPT model on a real STEM image, helping to determine the underlying structure. Here’s an example of obtaining atomic structure from a powder diffraction pattern using DiffractGPT. ➡️ You can now literally go to atomgpt.org, upload your 2θ-intensity data, and get candidate atomic structures or try it on the Google Colab notebook. ➡️ Compared to other AI approaches in materials science, this seems to be one of the most promising areas of research, with exciting potential for large-scale commercial applications. ➡️Google Colab: https://lnkd.in/eeHhTRYS ➡️YouTube Demo: https://lnkd.in/ee2fHxKp #MaterialsScience #XRD #STEM #AI #MachineLearning #AtomisticModeling #MicroscopyGPT #DiffractGPT

🔬 Why CellProfiler is the Go-To Tool for Bioinformaticians Working with Images 🔬 When it comes to analyzing cell images, CellProfiler is a game-changing, open-source tool widely used by bioinformaticians and researchers alike. Developed at the Broad Institute, it simplifies complex image analysis tasks and enables high-throughput cell imaging for everything from drug screening to gene expression studies. What is CellProfiler? CellProfiler is a powerful, open-source software designed specifically for biological image analysis. It allows researchers to automate the extraction of quantitative data from thousands of images, making it invaluable for high-throughput screening and cellular analysis. 🔹 Top Benefits of Using CellProfiler Easy-to-Use Interface 🖥️ CellProfiler's drag-and-drop interface lets users build pipelines for cell segmentation, measurement, and tracking without heavy coding, making it accessible to researchers at all levels. Automation of Image Analysis ⚙️ CellProfiler enables batch processing of images, meaning you can extract data from hundreds (or thousands!) of images at once. This is especially useful in high-content screening, where manually analyzing each image is impossible. Quantitative Data Extraction 📊 With CellProfiler, you can extract detailed quantitative data about cell size, shape, intensity, and texture. This data is essential for identifying patterns and understanding cellular behavior, especially in complex, large-scale studies. Integration with Machine Learning 🤖 CellProfiler can integrate with machine learning tools like CellProfiler Analyst to classify cells and discover patterns that might not be visible with traditional analysis. This opens up powerful possibilities for predictive modeling in drug discovery and biomarker identification. Customizable Pipelines 🔄 Each CellProfiler pipeline can be customized and saved, making it easy to standardize workflows and ensure reproducibility across projects and experiments. Where CellProfiler Shines CellProfiler is perfect for image-based bioinformatics workflows like: High-throughput screening: Automating analysis for drug discovery or genetic studies. Cell morphology studies: Measuring cellular changes in response to treatments. Single-cell analysis: Extracting single-cell data for downstream applications like RNA-seq or proteomics. For bioinformaticians and researchers working with microscopy images, CellProfiler is an essential tool that saves time, boosts reproducibility, and provides deep insights from cellular images. #Bioinformatics #CellProfiler #ImageAnalysis #MachineLearning #HighThroughput #DataScience #DrugDiscovery #Microscopy

After 18 years of infertility, a couple turned to a Columbia University team who deployed something called S.T.A.R. - Sperm Tracking And Recovery: it’s AI scanning 8 million images in under 60 minutes, locating sperm cells technicians missed in two full days. Let that sink in: the same AI approach used for discovering stars is now spotting sperm. We’re talking about replacing laborious human scrutiny with image‑processing horsepower and robotics. In one case, STAR found 44 sperm in under an hour where manual methods found none. The end result? A healthy pregnancy and the first documented case of AI‑confirmed conception via this technique . 🧠 Why this matters to us in enterprise and tech: Tech/biology convergence: The fusion of astrophysics‑grade image analysis with micro‑robotics showcases how cross‑disciplinary algorithms can revolutionize established industries: fertility science today, your sector tomorrow. Operational leverage: STAR achieved in 60 minutes what took human teams two days. Multiply that by hundreds of tasks and imagine the scale. Ethics meets engineering: Deploying AI for reproductive medicine invites us to re-examine trust, regulation, and accountability: not just for boardrooms but for society at large. Competitive foresight: If your product involves edge‑case detection (fraud, anomalies, rare events), take note; STAR’s architecture is a template. My take (tongue half in cheek): If you thought AI’s “obvious” use‑cases were limited to chatbots and predictive marketing, meet STAR. It proves the real frontier is where domain experts declare “we’ve tried everything”. That’s your cue- find that frontier in your industry and ask: what could AI find in “haystacks” we’ve written off? Final thought: Entrepreneurs, view this as a road map. Identify the domain where humans have “tried it already,” then deploy cross‑disciplinary AI to reveal what we’ve overlooked. What other “haystacks” have you written off too soon? ↓ ↓ ↓ 👉 Stay ahead—Follow me on LinkedIn and subscribe to the newsletter: www.michaelhousman.com #AIInnovation #CrossDisciplinaryTech #OperationalExcellence #EthicalAI #ThoughtLeadership