Monocular infrared imaging for high-throughput time-series phenotyping and instance-level growth modeling of mushrooms

Understanding mushroom growth dynamics is essential for precision cultivation and automated harvesting. However, monitoring mushroom phenotypes in real farms remains challenging due to extremely low-light environments, dense growth patterns, and the lack of large-scale annotated datasets for 3D geometry reconstruction. This project presents a monocular infrared-based pipeline for mushroom growth modeling using segmentation and geometry estimation. First, mushroom instances are segmented from infrared images using a YOLO-assisted SAM2 framework to obtain precise object boundaries under dense canopy conditions. Then, an optimized MoGe2 model is applied to estimate dense depth maps from monocular infrared observations. By combining segmentation masks with geometry estimation, the system reconstructs key 3D phenotypic attributes of each mushroom, including height, projected area, and geometric structure. The framework further analyzes time-series observations to model mushroom growth dynamics and extract biologically meaningful growth curves. Experimental results demonstrate that the proposed approach enables reliable phenotyping and geometry estimation for mushrooms under challenging dark cultivation environments, providing a scalable solution for automated mushroom monitoring.

1. Infrared image acquisition: Time-lapse infrared images are captured from a fixed top-view camera in a mushroom cultivation environment.
2. Mushroom instance segmentation: YOLO detection provides coarse localization, while SAM2 refines segmentation masks to extract accurate mushroom boundaries.
3. Monocular depth estimation: An optimized MoGe2 model predicts dense depth maps from infrared images to recover geometric structure.
4. 3D phenotype extraction: Segmentation masks and depth maps are fused to compute mushroom geometric attributes such as height and projected area.
5. Growth curve modeling: Time-series observations are analyzed to construct mushroom growth curves and quantify developmental patterns.

Figure 1: Time-series visualization of mushroom growth, instance segmentation, and depth estimation under infrared imaging. From top to bottom: raw infrared observations, segmentation results, and estimated depth maps across different growth stages.

Figure 2: Comparison of mushroom imaging under different lighting conditions, including normal RGB illumination, dark-environment RGB, infrared imaging with 20% IR intensity, and infrared imaging with 100% IR intensity.

Figure 3: Time-series growth analysis of a single mushroom, showing the evolution of projected area, cap surface area, and height across developmental stages.

Figure 4: Visualization of 3D reconstruction results for a mushroom cultivation scene across different growth stages. Colored mushroom surfaces indicate individual segmented instances reconstructed from monocular infrared geometry estimation.