Publications

2026

1. ECCV 2026

   

   DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection

   Gautam Rajendrakumar Gare^*, Neehar Peri^*, Matvei Popov, Shruti Jain, John Michael Galeotti, and Deva Ramanan

   In European Conference on Computer Vision (ECCV), 2026

   Abs arXiv Bib PDF Code Website

   Multi-Modal LLMs (MLLMs) demonstrate strong visual grounding capabilities on popular object detection benchmarks like OdinW-13 and RefCOCO. However, state-of-the-art models still struggle to generalize to out-of-distribution classes, tasks and imaging modalities not typically found in their pre-training. While in-context prompting is a common strategy to improve performance across diverse tasks, we find that it often yields lower detection accuracy than prompting with class names alone. This suggests that current MLLMs cannot yet effectively leverage few-shot visual examples and rich textual descriptions for object detection. Since frontier MLLMs are typically only accessible via APIs, and state-of-the-art open-weights models are prohibitively expensive to fine-tune on consumer-grade hardware, we instead explore black-box prompt optimization for few-shot object detection. To this end, we propose Detection Prompt Optimization (DetPO), a gradient-free test-time optimization approach that refines text-only prompts by maximizing detection accuracy on few-shot visual training examples while calibrating prediction confidence. Our proposed approach yields consistent improvements across generalist MLLMs on Roboflow20-VL and LVIS, outperforming prior black-box approaches by up to 9.7 mAP. Our code and optimized prompts are available at https://github.com/ggare-cmu/DetPO.

   @inproceedings{gare2026detpo,
     title = {DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection},
     author = {Gare, Gautam Rajendrakumar and Peri, Neehar and Popov, Matvei and Jain, Shruti and Galeotti, John Michael and Ramanan, Deva},
     booktitle = {European Conference on Computer Vision (ECCV)},
     year = {2026},
   }

2. arXiv

   

   Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection

   Gautam Rajendrakumar Gare, John Michael Galeotti, Michael C. Mozer, Deva Ramanan, and Nan Rosemary Ke

   arXiv preprint arXiv:2606.03251, 2026

   Abs arXiv Bib PDF

   In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic was an intervention by the coronavirus on the sub-population infected with COVID. We ask, \emphdo natural experiments occur in existing real-world datasets? If yes, how should we treat them? To detect natural experiments in data, we use causal discovery to recover the underlying causal graph and perform feature selection based on causal links. If downstream performance improves by treating the data as interventional rather than observational, we argue that this suggests the dataset contains natural experiments. We first validate this hypothesis by simulating datasets with and without natural experiments using synthetic graphs. We then perform a systematic empirical evaluation on a large suite of real-world datasets. Our results indicate that \emphreal-world datasets do contain natural experiments and we can take advantage of those natural experiments to improve model performance using causal inference. Our work represents the initial foray into this area, offering a preliminary exploration within a limited scope.

   @article{gare2026naturalexp,
     title = {Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection},
     author = {Gare, Gautam Rajendrakumar and Galeotti, John Michael and Mozer, Michael C. and Ramanan, Deva and Ke, Nan Rosemary},
     journal = {arXiv preprint arXiv:2606.03251},
     year = {2026},
   }

3. ECCV-W 2026

   

   Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models

   Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin, Deepak Pathak, John Michael Galeotti, and Deva Ramanan

   In PFATCV Workshop at ECCV, 2026

   Oral Abs arXiv Bib PDF Website

   Oral presentation at PFATCV Workshop, ECCV 2026

   A popular route to interpretable zero-shot classification asks a large language model (LLM) to describe each class name and prompts CLIP with the resulting descriptors. We show that these descriptors carry little visual evidence of their own: removing the class name from the prompt collapses ImageNet accuracy from 59.5% to 15.5%. The diagnosis is that the descriptors are conditioned on the label rather than on the images, so they describe the concept in general and mislead exactly when the data shifts; an LLM insists that strawberries are red, but every strawberry in ImageNet-Sketch is a colorless line drawing. We therefore select attributes from the target image collection instead: we score a large attribute pool against the images in CLIP’s joint embedding space and keep the top-scoring attributes per class. Selected this way, class-name-free attribute prompts reach 23.8% on ImageNet (against 15.5% for LLM descriptors), the gain holds on four shifted ImageNet variants, and reselecting from the LLM’s own pool isolates the selection mechanism as the cause. With one image per class, the selected attributes outperform the prompt-tuning method CoOp by 3 points while fitting in under a minute instead of 14 hours, with no learned soft prompt to obscure the decision. Because the attribute set is chosen by the data, it doubles as a readable summary of a dataset, which we use to describe distribution shift in words.

   @inproceedings{gare2026attributes,
     title = {Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models},
     author = {Gare, Gautam Rajendrakumar and Shi, Jia and Lin, Zhiqiu and Pathak, Deepak and Galeotti, John Michael and Ramanan, Deva},
     booktitle = {PFATCV Workshop at ECCV},
     year = {2026},
   }

4. ECCV-W 2026

   

   Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models

   Rakshanda Hassan Abhinandan, John Michael Galeotti, Deva Ramanan, and Gautam Rajendrakumar Gare

   In eXCV Workshop at ECCV, 2026

   Abs arXiv Bib PDF Website

   Where should the question go in a vision-language model (VLM) prompt: before the image or after it? Intuition says before: knowing what is asked should tell the model where to look. Yet across visual question answering benchmarks, question-first prompting consistently underperforms the image-first ordering recommended for frontier VLMs, a phenomenon we term the \emphquestion-first paradox. We trace the paradox to a conflict between two stages of VLM computation. Logit-lens and attention probes show the intuition is half right: a question placed before the image genuinely steers perception, moving image patch representations toward question-relevant concepts. The failure lies downstream. Stranded behind hundreds of image tokens, the question is barely attended by the answer token, which instead commits to image-driven (often wrong) answers; a causal attention knockout confirms that the answer reads the question only when the question follows the image. Better perception is squandered by worse access. The diagnosis yields a training-free fix: \emphquestion echoing, restating the question on both sides of the image so that one copy steers perception while the other is read out at answer time. The same division of labor appears in a fifty-year-old finding on human “adjunct questions”, where repeating a question before and after a passage aids comprehension more than either position alone. \emphEchoing the image as well brings further gains, restoring the whole-image view a causal decoder otherwise loses. The paradox holds across five open VLMs, costing up to 17.5 group-accuracy points. Echoed prompts close it and surpass the best single-pass ordering on NaturalBench, POPE, Winoground, and open-ended VQAv2, by up to 19 Winoground group-accuracy points, with no training, fine-tuning, or architecture change. The paradox exposes a tension in multimodal prompting between steering what a model \emphsees and preserving access to what it was \emphasked; echoing resolves it with prompt design alone.

   @inproceedings{abhinandan2026prompt,
     title = {Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models},
     author = {Abhinandan, Rakshanda Hassan and Galeotti, John Michael and Ramanan, Deva and Gare, Gautam Rajendrakumar},
     booktitle = {eXCV Workshop at ECCV},
     year = {2026},
   }

5. ACL 2026

   

   Activation Reward Models for Few-Shot Model Alignment

   Tianning Chai, Chancharik Mitra, Brandon Huang, Gautam Rajendrakumar Gare, Zhiqiu Lin, Assaf Arbelle, and 5 more authors

   In Findings of the Association for Computational Linguistics (ACL), 2026

   Abs Bib HTML PDF

   Aligning Large Language Models (LLMs) and Large Multimodal Models (LMMs) to human preferences is crucial for improving their real-world behavior. A common approach is to use reward models that enable reinforcement-learning post-training. However, traditional reward modeling requires finetuning on large preference datasets, limiting adaptability to new preferences. We introduce Activation Reward Models (Activation RMs), the first mechanistic interpretability approach that steers LLM activations to align with few-shot preference data without finetuning. Our method combines activation denoising and output token likelihood scoring, achieving state-of-the-art performance on standard reward modeling benchmarks, surpassing zero-shot, few-shot, and voting-based baselines. We further demonstrate that Activation RMs mitigate reward hacking behaviors and remain robust to noisy exemplars and spurious reward signals. To evaluate this, we propose PreferenceHack, a novel few-shot benchmark testing reward models on reward hacking in a paired preference format, where Activation RMs achieve state-of-the-art performance, surpassing GPT-4o.

   @inproceedings{chai2026activation,
     title = {Activation Reward Models for Few-Shot Model Alignment},
     author = {Chai, Tianning and Mitra, Chancharik and Huang, Brandon and Gare, Gautam Rajendrakumar and Lin, Zhiqiu and Arbelle, Assaf and Karlinsky, Leonid and Feris, Rogerio and Darrell, Trevor and Ramanan, Deva and Herzig, Roei},
     booktitle = {Findings of the Association for Computational Linguistics (ACL)},
     year = {2026},
   }

6. ISBI 2026

   

   Label Uncertainty for Ultrasound Segmentation

   Malini Shivaram, Gautam Rajendrakumar Gare, Laura Hutchins, Jacob Duplantis, Thomas Deiss, Thales Nogueira Gomes, and 9 more authors

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2026

   Oral Abs arXiv Bib PDF

   Oral presentation at ISBI 2026

   In medical imaging, inter-observer variability among radiologists often introduces label uncertainty, particularly in modalities where visual interpretation is subjective. Lung ultrasound (LUS) is a prime example-it frequently presents a mixture of highly ambiguous regions and clearly discernible structures, making consistent annotation challenging even for experienced clinicians. In this work, we introduce a novel approach to both labeling and training AI models using expert-supplied, per-pixel confidence values. Rather than treating annotations as absolute ground truth, we design a data annotation protocol that captures the confidence that radiologists have in each labeled region, modeling the inherent aleatoric uncertainty present in real-world clinical data. We demonstrate that incorporating these confidence values during training leads to improved segmentation performance. More importantly, we show that this enhanced segmentation quality translates into better performance on downstream clinically-critical tasks-specifically, estimating S/F oxygenation ratio values, classifying S/F ratio change, and predicting 30-day patient readmission. While we empirically evaluate many methods for exposing the uncertainty to the learning model, we find that a simple approach that trains a model on binarized labels obtained with a (60%) confidence threshold works well. Importantly, high thresholds work far better than a naive approach of a 50% threshold, indicating that training on very confident pixels is far more effective. Our study systematically investigates the impact of training with varying confidence thresholds, comparing not only segmentation metrics but also downstream clinical outcomes. These results suggest that label confidence is a valuable signal that, when properly leveraged, can significantly enhance the reliability and clinical utility of AI in medical imaging.

   @inproceedings{shivaram2026label,
     title = {Label Uncertainty for Ultrasound Segmentation},
     author = {Shivaram, Malini and Gare, Gautam Rajendrakumar and Hutchins, Laura and Duplantis, Jacob and Deiss, Thomas and Gomes, Thales Nogueira and Tran, Thong and Patel, Keyur H. and Kirby, Krystal and Fox, Thomas H. and Krishnan, Amita and Ramanan, Deva and DeBoisblanc, Bennett and Rodriguez, Ricardo and Galeotti, John},
     booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
     year = {2026},
   }

7. ISBI 2026

   

   LEARNER: Learning Granular Labels from Coarse Labels using Contrastive Learning

   Jana Armouti, Nikhil Madaan, Rohan Panda, Tom Fox, Laura Hutchins, Amita Krishnan, and 5 more authors

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2026

   Abs arXiv Bib PDF

   Predicting whether a treatment leads to meaningful improvement is a central challenge in personalized medicine, particularly when disease progression manifests as subtle visual changes over time. While data-driven deep learning (DL) offers a promising route to automate such predictions, acquiring large-scale longitudinal data for each individual patient remains impractical. To address this limitation, we explore whether inter-patient variability can serve as a proxy for learning intra-patient progression. We propose LEARNER, a contrastive pretraining framework that leverages coarsely labeled inter-patient data to learn fine-grained, patient-specific representations. Using lung ultrasound (LUS) and brain MRI datasets, we demonstrate that contrastive objectives trained on coarse inter-patient differences enable models to capture subtle intra-patient changes associated with treatment response. Across both modalities, our approach improves downstream classification accuracy and F1-score compared to standard MSE pretraining, highlighting the potential of inter-patient contrastive learning for individualized outcome prediction.

   @inproceedings{gare2026learner,
     title = {LEARNER: Learning Granular Labels from Coarse Labels using Contrastive Learning},
     author = {Armouti, Jana and Madaan, Nikhil and Panda, Rohan and Fox, Tom and Hutchins, Laura and Krishnan, Amita and Rodriguez, Ricardo and DeBoisblanc, Bennett and Ramanan, Deva and Galeotti, John and Gare, Gautam Rajendrakumar},
     booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
     year = {2026},
   }

8. ISBI 2026

   

   Human-Aligned Learning of Interpretable Biomarkers for Lung Ultrasound Diagnosis

   Gautam Rajendrakumar Gare, Tom Fox, Peter Lowery, Kevin Zamora, Hai V. Tran, Laura Hutchins, and 6 more authors

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2026

   Abs arXiv Bib PDF

   Contemporary artificial neural networks (ANN) are trained end-to-end, jointly learning both features and classifiers for the task of interest. Though enormously effective, this paradigm imposes significant costs in assembling annotated task-specific datasets and training large-scale networks. We propose to decouple feature learning from downstream lung ultrasound tasks by introducing an auxiliary pre-task of visual biomarker classification. We demonstrate that one can learn an informative, concise, and interpretable feature space from ultrasound videos by training models for predicting biomarker labels. Notably, biomarker feature extractors can be trained from data annotated with weak video-scale supervision. These features can be used by a variety of downstream Expert models targeted for diverse clinical tasks (Diagnosis, lung severity, S/F ratio). Crucially, task-specific expert models are comparable in accuracy to end-to-end models directly trained for such target tasks, while being significantly lower cost to train.

   @inproceedings{gare2026biomarkers,
     title = {Human-Aligned Learning of Interpretable Biomarkers for Lung Ultrasound Diagnosis},
     author = {Gare, Gautam Rajendrakumar and Fox, Tom and Lowery, Peter and Zamora, Kevin and Tran, Hai V. and Hutchins, Laura and Montgomery, David and Krishnan, Amita and Ramanan, Deva and Rodriguez, Ricardo Luis and deBoisblanc, Bennett P. and Galeotti, John Michael},
     booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
     year = {2026},
   }

9. arXiv

   

   Prognostic Value of Lung Ultrasound Biomarkers for Readmission Risk in Congestive Heart Failure: A Pilot Data-Driven Analysis

   Jana Armouti, Laura Hutchins, Jacob Duplantis, Thomas Deiss, Thales Nogueira Gomes, Keyur H. Patel, and 9 more authors

   arXiv preprint arXiv:2605.18878, 2026

   Abs arXiv Bib PDF

   Hospital readmission within 30 days of discharge is a leading driver of morbidity, mortality, and avoidable healthcare expenditure in congestive heart failure (CHF). Current clinical risk stratification tools rely primarily on non-imaging data and exhibit limited predictive performance. Point-of-care lung ultrasound (LUS) offers a sensitive, noninvasive window into the pulmonary congestion that characterizes CHF decompensation, yet its prognostic utility for readmission prediction remains largely unexplored. We present a pilot feasibility study, the first systematic machine learning study using B-mode LUS acquired during hospitalization to predict 30-day CHF readmission. Quantitative spatiotemporal embeddings are extracted from a pretrained Temporal Shift Module (TSM) ResNet-18 encoder, and interpretable biomarker features are separately evaluated. Through structured ablations over lung view, temporal representation, multi-view fusion, and cross-lung augmentation, we identify the key imaging factors driving readmission risk. Our findings reveal that (1) dependent lower-lung regions (Left-3, Right-3) carry the strongest prognostic signal, consistent with their greater susceptibility to hydrostatic congestion; (2) temporal difference features between sequential examinations substantially outperform single-timepoint representations, highlighting the importance of capturing disease trajectory; and (3) multi-view feature concatenation yields the best overall performance, with our top MLP model achieving an F1 score of 0.80 (95% CI: 0.62-0.96). Biomarker analysis further reveals that pleural-line abnormalities, including breaks and indentations, are as informative as the canonical A-line and B-line markers. These results support POCUS-derived biomarkers as practical, interpretable tools for noninvasive CHF risk stratification.

   @article{armouti2026prognostic,
     title = {Prognostic Value of Lung Ultrasound Biomarkers for Readmission Risk in Congestive Heart Failure: A Pilot Data-Driven Analysis},
     author = {Armouti, Jana and Hutchins, Laura and Duplantis, Jacob and Deiss, Thomas and Gomes, Thales Nogueira and Patel, Keyur H. and Walvekar, Seema and Guillory, Shane and Fox, Thomas H. and Krishnan, Amita and Rodriguez, Ricardo and DeBoisblanc, Bennett and Ramanan, Deva and Galeotti, John and Gare, Gautam Rajendrakumar},
     journal = {arXiv preprint arXiv:2605.18878},
     year = {2026},
   }

2025

1. ISBI 2025

   

   Weakly Supervised Contrastive Learning for Better Severity Scoring of Lung Ultrasound

   Gautam Rajendrakumar Gare, Hai V. Tran, Bennett P. deBoisblanc, Ricardo Luis Rodriguez, and John Michael Galeotti

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2025

   Abs arXiv Bib PDF

   With the onset of the COVID-19 pandemic, ultrasound has emerged as an effective tool for bedside monitoring of patients. Due to this, a large amount of lung ultrasound scans have been made available which can be used for AI based diagnosis and analysis. Several AI-based patient severity scoring models have been proposed that rely on scoring the appearance of the ultrasound scans. AI models are trained using ultrasound-appearance severity scores that are manually labeled based on standardized visual features. We address the challenge of labeling every ultrasound frame in the video clips. Our contrastive learning method treats the video clip severity labels as noisy weak severity labels for individual frames, thus requiring only video-level labels. We show that it performs better than the conventional cross-entropy loss based training. We combine frame severity predictions to come up with video severity predictions and show that the frame based model achieves comparable performance to a video based TSM model, on a large dataset combining public and private sources.

   @inproceedings{gare2025weakly,
     title = {Weakly Supervised Contrastive Learning for Better Severity Scoring of Lung Ultrasound},
     author = {Gare, Gautam Rajendrakumar and Tran, Hai V. and deBoisblanc, Bennett P. and Rodriguez, Ricardo Luis and Galeotti, John Michael},
     booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
     year = {2025},
   }

2024

1. ICML 2024

   

   LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies

   Jia Shi, Gautam Rajendrakumar Gare, Jinjin Tian, Siqi Chai, Zhiqiu Lin, Arun Vasudevan, and 3 more authors

   In International Conference on Machine Learning (ICML), 2024

   Oral Abs arXiv Bib PDF Code Website

   ICML Oral

   We tackle the challenge of predicting models’ Out-of-Distribution (OOD) performance using in-distribution (ID) measurements without requiring OOD data. Existing evaluations with "Effective Robustness", which use ID accuracy as an indicator of OOD accuracy, encounter limitations when models are trained with diverse supervision and distributions, such as class labels (Vision Models, VMs, on ImageNet) and textual descriptions (Visual-Language Models, VLMs, on LAION). VLMs often generalize better to OOD data than VMs despite having similar or lower ID performance. To improve the prediction of models’ OOD performance from ID measurements, we introduce the Lowest Common Ancestor (LCA)-on-the-Line framework. This approach revisits the established concept of LCA distance, which measures the hierarchical distance between labels and predictions within a predefined class hierarchy, such as WordNet. We assess 75 models using ImageNet as the ID dataset and five significantly shifted OOD variants, uncovering a strong linear correlation between ID LCA distance and OOD top-1 accuracy. Our method provides a compelling alternative for understanding why VLMs tend to generalize better. Additionally, we propose a technique to construct a taxonomic hierarchy on any dataset using K-means clustering, demonstrating that LCA distance is robust to the constructed taxonomic hierarchy. Moreover, we demonstrate that aligning model predictions with class taxonomies, through soft labels or prompt engineering, can enhance model generalization.

   @inproceedings{shi2024lca,
     title = {LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies},
     author = {Shi, Jia and Gare, Gautam Rajendrakumar and Tian, Jinjin and Chai, Siqi and Lin, Zhiqiu and Vasudevan, Arun and Feng, Di and Ferroni, Francesco and Kong, Shu},
     booktitle = {International Conference on Machine Learning (ICML)},
     year = {2024},
   }

2023

1. BIAS 2023

   

   Improving Model’s Interpretability and Reliability using Biomarkers

   Gautam Rajendrakumar Gare, Tom Fox, Beam Chansangavej, Amita Krishnan, Ricardo Luis Rodriguez, Bennett P. deBoisblanc, and 2 more authors

   In Bern Interpretable AI Symposium (BIAS), 2023

   Abs arXiv Bib PDF

   Accurate and interpretable diagnostic models are crucial in the safety-critical field of medicine. We investigate the interpretability of our proposed biomarker-based lung ultrasound diagnostic pipeline to enhance clinicians’ diagnostic capabilities. The objective of this study is to assess whether explanations from a decision tree classifier, utilizing biomarkers, can improve users’ ability to identify inaccurate model predictions compared to conventional saliency maps. Our findings demonstrate that decision tree explanations, based on clinically established biomarkers, can assist clinicians in detecting false positives, thus improving the reliability of diagnostic models in medicine.

   @inproceedings{gare2023bias,
     title = {Improving Model's Interpretability and Reliability using Biomarkers},
     author = {Gare, Gautam Rajendrakumar and Fox, Tom and Chansangavej, Beam and Krishnan, Amita and Rodriguez, Ricardo Luis and deBoisblanc, Bennett P. and Ramanan, Deva and Galeotti, John Michael},
     booktitle = {Bern Interpretable AI Symposium (BIAS)},
     year = {2023},
   }

2022

1. MIA 2022

   

   W-Net: Dense and diagnostic semantic segmentation of subcutaneous and breast tissue in ultrasound images by incorporating ultrasound RF waveform data

   Gautam Rajendrakumar Gare, Jiayuan Li, Rohan Joshi, Rishikesh Magar, Mrunal Prashant Vaze, Michael Yousefpour, and 2 more authors

   Medical Image Analysis, 2022

   Abs DOI Bib PDF

   We study the use of raw ultrasound waveforms, often referred to as the "Radio Frequency" (RF) data, for the semantic segmentation of ultrasound scans to carry out dense and diagnostic labeling. We present W-Net, a novel Convolution Neural Network (CNN) framework that employs the raw ultrasound waveforms in addition to the grey ultrasound image to semantically segment and label tissues for anatomical, pathological, or other diagnostic purposes. To the best of our knowledge, this is also the first deep-learning or CNN approach for segmentation that analyzes ultrasound raw RF data along with the grey image. We chose subcutaneous tissue (SubQ) segmentation as our initial clinical goal for dense segmentation since it has diverse intermixed tissues, is challenging to segment, and is an underrepresented research area. SubQ potential applications include plastic surgery, adipose stem-cell harvesting, lymphatic monitoring, and possibly detection/treatment of certain types of tumors. Unlike prior work, we seek to label every pixel in the image, without the use of a background class. A custom dataset consisting of hand-labeled images by an expert clinician and trainees are used for the experimentation, currently labeled into the following categories: skin, fat, fat fascia/stroma, muscle, and muscle fascia. We compared W-Net and attention variant of W-Net (AW-Net) with U-Net and Attention U-Net (AU-Net). Our novel W-Net’s RF-Waveform encoding architecture outperformed regular U-Net and AU-Net, achieving the best mIoU accuracy (averaged across all tissue classes). We study the impact of RF data on dense labeling of the SubQ region, which is followed by the analyses of the generalization capability of the networks to patients and analysis on the SubQ tissue classes, determining that fascia tissues, especially muscle fascia in particular, are the most difficult anatomic class to recognize for both humans and AI algorithms. We present diagnostic semantic segmentation, which is semantic segmentation carried out for the purposes of direct diagnostic pixel labeling, and apply it to breast tumor detection task on a publicly available dataset to segment pixels into malignant tumor, benign tumor, and background tissue class. Using the segmented image we diagnose the patient by classifying the breast lesion as either benign or malignant. We demonstrate the diagnostic capability of RF data with the use of W-Net, which achieves the best segmentation scores across all classes.

   @article{gare2022wnet,
     title = {W-Net: Dense and diagnostic semantic segmentation of subcutaneous and breast tissue in ultrasound images by incorporating ultrasound RF waveform data},
     author = {Gare, Gautam Rajendrakumar and Li, Jiayuan and Joshi, Rohan and Magar, Rishikesh and Vaze, Mrunal Prashant and Yousefpour, Michael and Rodriguez, Ricardo Luis and Galeotti, John Michael},
     journal = {Medical Image Analysis},
     volume = {76},
     pages = {102326},
     year = {2022},
     doi = {10.1016/j.media.2021.102326},
   }

2021

1. MICCAI-W 2021

   

   The Role of Pleura and Adipose in Lung Ultrasound AI

   Gautam Rajendrakumar Gare^*, Wanwen Chen^*, Alex Ling Yu Hung, Edward Chen, Hai V. Tran, Tom Fox, and 5 more authors

   In LL-COVID19 Workshop at MICCAI, 2021

   Abs DOI Bib PDF

   In this paper, we study the significance of the pleura and adipose tissue in lung ultrasound AI analysis. We highlight their more prominent appearance when using high-frequency linear (HFL) instead of curvilinear ultrasound probes, showing HFL reveals better pleura detail. We compare the diagnostic utility of the pleura and adipose tissue using an HFL ultrasound probe. Masking the adipose tissue during training and inference (while retaining the pleural line and Merlin’s space artifacts such as A-lines and B-lines) improved the AI model’s diagnostic accuracy.

   @inproceedings{gare2021pleura,
     title = {The Role of Pleura and Adipose in Lung Ultrasound AI},
     author = {Gare, Gautam Rajendrakumar and Chen, Wanwen and Hung, Alex Ling Yu and Chen, Edward and Tran, Hai V. and Fox, Tom and Lowery, Peter and Zamora, Kevin and deBoisblanc, Bennett P. and Rodriguez, Ricardo Luis and Galeotti, John Michael},
     booktitle = {LL-COVID19 Workshop at MICCAI},
     pages = {141--149},
     year = {2021},
     doi = {10.1007/978-3-030-90874-4_14},
   }

2. ISBI 2021

   

   Dense Pixel-Labeling for Reverse-Transfer and Diagnostic Learning on Lung Ultrasound for COVID-19 and Pneumonia Detection

   Gautam Rajendrakumar Gare, Andrew Schoenling, Vipin Philip, Hai V. Tran, Bennett P. deBoisblanc, Ricardo Luis Rodriguez, and 1 more author

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2021

   Abs DOI Bib PDF

   We propose using a pre-trained segmentation model to perform diagnostic classification in order to achieve better generalization and interpretability, terming the technique reverse-transfer learning. We present an architecture to convert segmentation models to classification models. We compare and contrast dense vs sparse segmentation labeling and study its impact on diagnostic classification. We compare the performance of U-Net trained with dense and sparse labels to segment A-lines, B-lines, and Pleural lines on a custom dataset of lung ultrasound scans from 4 patients. Our experiments show that dense labels help reduce false positive detection. We study the classification capability of the dense and sparse trained U-Net and contrast it with a non-pretrained U-Net, to detect and differentiate COVID-19 and Pneumonia on a large ultrasound dataset of about 40k curvilinear and linear probe images. Our segmentation-based models perform better classification when using pretrained segmentation weights, with the dense-label pretrained U-Net performing the best.

   @inproceedings{gare2021dense,
     title = {Dense Pixel-Labeling for Reverse-Transfer and Diagnostic Learning on Lung Ultrasound for COVID-19 and Pneumonia Detection},
     author = {Gare, Gautam Rajendrakumar and Schoenling, Andrew and Philip, Vipin and Tran, Hai V. and deBoisblanc, Bennett P. and Rodriguez, Ricardo Luis and Galeotti, John Michael},
     booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
     pages = {1406--1410},
     year = {2021},
     doi = {10.1109/ISBI48211.2021.9433826},
   }

3. arXiv

   

   Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions

   Gautam Rajendrakumar Gare and John Michael Galeotti

   2021

   Abs arXiv Bib PDF

   Class labels used for machine learning are relatable to each other, with certain class labels being more similar to each other than others (e.g. images of cats and dogs are more similar to each other than those of cats and cars). Such similarity among classes is often the cause of poor model performance due to the models confusing between them. Current labeling techniques fail to explicitly capture such similarity information. In this paper, we instead exploit the similarity between classes by capturing the similarity information with our novel confidence labels. Confidence labels are probabilistic labels denoting the likelihood of similarity, or confusability, between the classes. Often even after models are trained to differentiate between classes in the feature space, the similar classes’ latent space still remains clustered. We view this type of clustering as valuable information and exploit it with our novel projective loss functions. Our projective loss functions are designed to work with confidence labels with an ability to relax the loss penalty for errors that confuse similar classes. We use our approach to train neural networks with noisy labels, as we believe noisy labels are partly a result of confusability arising from class similarity. We show improved performance compared to the use of standard loss functions. We conduct a detailed analysis using the CIFAR-10 dataset and show our proposed methods’ applicability to larger datasets, such as ImageNet and Food-101N.

   @misc{gare2021confidence,
     title = {Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions},
     author = {Gare, Gautam Rajendrakumar and Galeotti, John Michael},
     year = {2021},
   }

2017

1. Mechatronics

   

   An iteratively optimized resolution to hyper redundancy for dissimilarly doped compliant IPMC actuators

   Ritwik Chattaraj, Siladitya Khan, Aritra Dasgupta, Gautam Gare, Debabrata Chatterjee, and Subhasis Bhaumik

   Mechatronics, 2017

   DOI Bib PDF

   @article{chattaraj2017ipmc,
     title = {An iteratively optimized resolution to hyper redundancy for dissimilarly doped compliant IPMC actuators},
     author = {Chattaraj, Ritwik and Khan, Siladitya and Dasgupta, Aritra and Gare, Gautam and Chatterjee, Debabrata and Bhaumik, Subhasis},
     journal = {Mechatronics},
     volume = {46},
     pages = {154--167},
     year = {2017},
     doi = {10.1016/j.mechatronics.2017.08.004},
   }

2016

1. IJEECSE

   

   Custom 8 Bit Microprocessor Designing and Implementation on FPGA Board

   Gautam Rajendrakumar Gare, Kenneth Peter, and Amaresh L R

   In International Journal of Electrical Electronics and Computer Science Engineering, 2016

   Bib PDF

   @inproceedings{gare2016microprocessor,
     title = {Custom 8 Bit Microprocessor Designing and Implementation on FPGA Board},
     author = {Gare, Gautam Rajendrakumar and Peter, Kenneth and R, Amaresh L},
     booktitle = {International Journal of Electrical Electronics and Computer Science Engineering},
     year = {2016},
   }

2. IJMTER

   

   Autonomous Vehicle Technology: A Brief Overview of the Technology and Current Trends in Autonomous Systems

   Gautam Rajendrakumar Gare

   In International Journal of Modern Trends in Engineering and Research, 2016

   Bib PDF

   @inproceedings{gare2016autonomous,
     title = {Autonomous Vehicle Technology: A Brief Overview of the Technology and Current Trends in Autonomous Systems},
     author = {Gare, Gautam Rajendrakumar},
     booktitle = {International Journal of Modern Trends in Engineering and Research},
     year = {2016},
   }