I am pursuing my Masters in Robotics with research at Carnegie Mellon University. I am advised by Dr. Howie Choset and Dr. John Galeotti
and am majorly interested in machine learning and computer vision applications for robotics. My current research is on implicit neural representations for medical ultrasound and their applications to segmentation, 3D reconstruction and modeling deformation in soft tissue.
Before joining CMU, I completed my undergrad in CS from Vellore Institute of Technology where I focused on core ML and NLP projects. I have experience with building recommendation systems, ensemble classifiers, feature-based medical image diagnosis systems, and named-entity recognition models. I briefly interned at Fidelity Investments where I worked on intelligent document layout analysis for automated information retrieval from semi-structured legal documents.
As a part of the Biorobotics Lab, I have worked on projects in Medical Robotics and Robotic Search and Rescue. These projects, as well as my graduate coursework have helped me focus on 3D perception, 3D representations, geometric vision, and physics-inspired learning.
Most Robotic Ultrasound Systems (RUSs) equipped with ultrasound-interpreting algorithms rely on building 3D reconstructions of the entire scanned region or specific anatomies. These 3D reconstructions are typically created via methods that compound or stack 2D tomographic ultrasound images using known poses of the ultrasound transducer with the latter requiring 2D or 3D segmentation. While fast, this class of methods has many shortcomings. It requires interpolation-based gap-filling or extensive compounding and still yields volumes that generate implausible novel views. Additionally, storing these volumes can be memory-intensive.
These challenges can be overcome with neural implicit learning which provides interpolation in unobserved gaps through a smooth learned function as well as a lighter representation for the volume in terms of memory. In this thesis, a neural implicit representation (NIR) based on the physics of ultrasound image formation is presented.
...
With this NIR, a physically-grounded version of tissue reflectivity function (TRF) is learned by regression using observed intensities in ultrasound images. Additionally, this NIR also learns a spatially-varying point spread function (PSF) of the ultrasound imaging system to improve the photorealism of rendered images. The TRF learned through this method can handle contrasting observations from different viewing-directions due to a differentiable rendering function that incorporates the angle of incidence between ultrasound rays and the tissue interfaces in the scanned volume. It is a stable representation of the tissue volume that when combined with the viewing-direction, can produce true-to-orientation ultrasound images.
Given that many diagnostic and surgical applications, robotic or otherwise, require anatomy-specific 3D reconstructions, it is not sufficient to learn entire ultrasound volumes without discerning the required anatomies. To circumvent the use of traditional 3D segmentation methods that are computationally-heavy, I demonstrate that the obtained TRF can be used to learn a neural implicit shape representation for anatomies that are largely homogeneous. This is formulated as a weakly-supervised binary voxel occupancy function that is learned in parallel with the NIR. All these contributions are substantiated on simulated, phantom-acquired and live subject-acquired ultrasound images capturing blood vessels. Finally, an application for the anatomy-specific reconstruction is discussed in the context of physical simulations for deformation modeling of soft tissue.
Selected Projects
Adding Viewing Direction-dependence to Ultra-NeRF Ananya Bal,
Magdalena Wysocki,
Mohammad Farid Azampour,
John Galeotti,
Howie Choset
Slides
Incorporating viewing direction-dependence to Ultra-NeRF for rendering true-to-orientation reflectance in ultrasound images. A normal surface field is learned to supplement a modified rendering function.
Implicit Shape Representation for 3D Anatomy Reconstruction from Ultrasound Images Ananya Bal,
Magdalena Wysocki,
Mohammad Farid Azampour,
John Galeotti,
Howie Choset
Slides
Weakly supervised volumetric implicit shape representation method. Applied to 3D anatomy reconstruction from multi-view 2D ultrasound images trhough Ultra-NeRF.
Ultra-NeRF++: Learning Imaging System Properties in addition to Scene Parameters Ananya Bal,
Magdalena Wysocki,
Mohammad Farid Azampour,
John Galeotti,
Howie Choset
Slides
Enhanced Ultra-NeRF by approximating spatially varying Point Spread Functions for the ultrasound imaging system in addition to the tissue reflectivity function of the scene. Network architecture also improved for rednering high-frequency artifacts.
Making RAFT training unsupervised and applying it to predict vessel deformation under forces in ultrasound images.
We use this further to generate ultrasound images at multiple force values and improve segmentation by 12%
A 3D deformation simulation framework where we reduce the sim2real gap by optimizing for
material properties through maximizing IoU of the vessel area
from the simulation and the real-world ultrasound images.
A 3D reconstruction optimization-based method to identify a high curvature region for autonomous ultrasound scanning. A novel, comprehensive 3D reconstruction evaluation score is proposed.
Our pipeline learns deformation using 3D point clouds of the deforming object, material properties, force and its point of application and predicts a deformed version of the object.
As we go from images to point clouds, our method uses 2D RGB images to learn 3D deformations.
Feel free to steal this website's source code. Do not scrape the HTML from this page itself, as it includes analytics tags that you do not want on your own website — use the github code instead. Also, consider using Leonid Keselman's Jekyll fork of this page.