Research Statement Morteza MardaniStanford University

OverviewThe rapidly emerging fields of Machine Learning (ML) and Artificial Intelligence (AI) are disrupting manytraditional business and industries, and promise to ultimately reorganize many aspects of daily life. Suchreorganization would be especially beneficial in Medicine, where life-and-death decisisions might be significantlyimproved using data and algorithms.

The overarching theme of my research in recent years has been transformation of medical imaging usingML and AI. Medical imaging is a huge industry, producing 100’s of million images yearly and several tens ofbillion dollars in revenue. My research has delivered improvements in medical imaging that I am convincedover time will be very consequential for patient benefit and economic impact. These improved results provokenew challenges. (C1) when ML is used for image reconstruction and diagnosis, how can you ensure that theresult is accurate? It is not enough that the images look believable, they must reflect reality. Next, if ML isto be used in medical imaging the physicians need to be confident in the results. (C2) How do you explainthe results to the physician, who is ultimately responsible for the diagnosis? Finally, pathology is relativelyrare, and is quite variable. (C3) How do you train systems to be robust for such data-scarce scenarios?

In the coming phase of my research career, I plan to continue the delivery of further improvements tomedical imaging using ML/AI and also to attack the attendant challenges. My background in algorithms,analysis, and applications of ML and statistical signal processing (SP) tools for data science and AI hasprovided me with a unique set of tools to address C1-C3. To facilitate ML from big data my PhD researchcontributed to mining important information from a sheer volume of raw data that is typically incomplete(misses), anomalous (outliers), multiway (tensor), and streaming in nature. To mine information from suchdata I leveraged data parsimony and put forth a framework for scalable inference from streaming matrix/tensordata. Being optimal, it also nicely integrates with large-scale ML tasks.

I was fortunate to be situated in recent years at Stanford, epicenter of various important recent developmentsin medical imaging, and to see first hand the coming future technology, both my own and that of my colleagues.I expect to be fully occupied in coming years with realizing the potential for ML/AI in medical imaging.I pioneered the application of generative adversarial networks (GANs) for robust MRI translationcreating diagnostic MRI from fast scans. The impact of this method is prominent in diagnosis of pediatricpatients, and attracted a lot of attention in the MRI community. I also contributed a principled neural networkdesign that offers an interpretable framework with high generalization power for data-scarce situations.

My research contributions are well recognized in the field by receiving the 2017 IEEE Signal ProcessingSociety Young Author Best Paper Award, and 2012 IEEE SPAWC Best Paper Award. In addition,for various other ML projects I have mentored 1 postdoc, 4 PhD, 5 undergraduate, and 2 high-schoolstudents. I have also been involved in writing grant proposals for NSF and NIH that were successfully funded.Moreover, my extensive collaboration with faculties in Electrical Engineering, Computer Science, Statistics,and Radiology has empowered me to overcome the “language barrier.” All in all, my long-term vision is tocontribute with my expertise and interdisciplinary experience to solve AI problems inspired by Medicinethat positively impact humans health.

Mining Big Data for Machine Learning via SparsityTh primary challenge of our era is to use the sudden wealth of data and the recent arrival of overwhelmingcomputer power in every field of science and technology. I have adopted this as my own mission as well andhave steadily accumulated the skills and knowledge needed to contribute productively in this era. In comingparagraphs I summarize some of the forces and trends that are shaping this era, and which also have servedas my formative influences. The sheer volume of data makes it often impossible to run analytics using centralprocessors and storage units. Network data are also often decentralized, and collecting the data might be

Figure 1. Representative coronal Abdominal images for a representative test patient with 5-fold undersampling. The firstcolumn (1st) shows the full view gold standard image, and the rest of columns from left to right indicate the enlargedareas of liver (top row) and kidney (bottom row) for (2nd) the gold-standard, and reconstruction under (3rd) GANCSwith 1% GAN loss, (4th) GANCS with 25% GAN loss, and (5th) CS Wavelet.

infeasible due to communication costs or privacy concerns. Disparate origins of data also makes the datasetsoften incomplete, and thus a sizable portion of entries are missing. Moreover, large-scale data are prone tocontain corrupted measurements, communication errors, and even exhibit anomalies. Also, as many sourcescontinuously generate data in real time, analytics must often be performed online without an opportunity torevisit past data. Last but not least, due to variety, data is typically multiway. To facilitate ML, my doctorateresearch contributed to cope with these challenges by leveraging the inherent low-dimensionality of datacaptured via sparsity and low rank regularization.Unveiling Anomalies via Data Parsimony. Spotting anomalous patterns is an important task to understande.g., the source of Cyberattacks in IP backbone networks, and to facilitate ML by discarding the outliersfrom datasets. We developed a framework that decomposes data arrays into a low rank background andcompressed sparse outlier components by leveraging the sporadic nature of anomalies. Theoretical guaranteesare established to evaluate the quality of decomposition, and our experiments with Internet traffic traces fromthe US backbone network accurately diagnose the Cyberattacks.Scalable Algorithms for Rank Regularization. Despite the success of rank surrogates in capturing dataparsimony, they scale very poorly with data due to their nonseparable structure, which hinders decentralized andstreaming analytics. To mitigate this computational challenge, we put forth scalable solvers for nuclear-normregularization that leverage a bilinear (nonconvex) variant of nuclear-norm to decompose across differentdimensions. Despite nonconvexity we proved that even with the budget of distributed computation andsequential acquisition one can hope to achieve the optimal solution offered by an oracle batch method thathas access to the entire data at once with unlimited computational power. This method enabled robustprincipal component analysis (PCA) for large-scale and incomplete datasets. The outcomes of this projectresulted publications such as [1] that received the IEEE Signal Processing Society Young Author BestPaper Award at 2017, and [4] that received SPAWC Best Paper Award at 2012.Dimensionality Reduction of Incomplete Multiway Data Arrays. Extracting latent structure from high-dimensional arrays is at premium for downstream ML tasks. Data arrays are however multiway and incomplete.To unravel the low-dimensional structure and represent it with a small latent vector we put forth a subspacelearning framework that benefits from a canonical decomposition of tensors to devise a tractable surrogate forthe tensor rank. The resulting latent code can be used for clustering network communities, or, as a byproductit can impute missing MRI measurements to image the dynamic evolution of motion and contrast in MRI.

Robust Deep Learning for Medical ImagingMedical imaging is a domain that can tremendously benefit from AI for enhanced and automated clinicaldecision making. The existing algorithms are not however cognizant of the challenges in the medical imagingdomain such as: 1) label scarcity, 2) risk of hallucinations, and 3) need for interpretability. Inspired by iterativeinference algorithms, my vision is to design robust and interpretable neural network architectures for medicalimage recognition and recovery tasks based on GANs and RNNs that are computationally scalable. Myresearch at Stanford contributed in the following main directions:GANs for Robust MR Image Translation. MRI is a major imaging modality in clinics due to its superb softtissue contrast. It however suffers from aliasing artifacts when it is highly undersampled for real-time imaging.Conventional compressed sensing (CS) MRI is not however cognizant of the image diagnostic quality, andsubstantially trades off accuracy for speed in real-time imaging. To cope with these challenges we put forth anovel image translation framework (so termed GANCS) that permeates benefits from GANs to modeling amanifold of MR images from historical patient records [2]. We examined extensive evaluations on a large MRIdataset of pediatric patients, and the ratings received from expert radiologists corroborate that our GANCSretrieves images with improved diagnostic quality and finer details relative to the conventional CS and deeplearning schemes. It also offers around much faster inference than existing CS-MRI schemes. Fig. 1 showsthat GANCS reveals fine liver vessels that are not seen in the conventional CS-Wavelet. It is worth notingthat this is the first attempt to use GANs for MR image translation.Neural Proximal Learning via Recurrent Neural Networks. Recovering high-resolution images fromlimited sensory data typically leads to a serious ill-posed inverse problem, demanding inversion algorithms thateffectively capture the prior information. Learning a good inverse mapping from training data faces severechallenges, including:(i) scarcity of training data; (ii) need for plausible reconstructions that are physicallyfeasible; (iii) need for fast reconstruction, especially in real-time applications. We develop a successful systemsolving all these challenges using as basic architecture the recurrent application of proximal gradient descent(PGD) algorithm. We learn a proximal map that works well with real images based on residual networks(ResNets). We conducted extensive experiments for translating abdominal MRI of pediatric patients fromhighly undersampled scans and superresolving natural face images. Our key findings include: 1. a recurrentResNet with a single residual block unrolled from an iterative algorithm yields an effective proximal whichaccurately reveals MR image details. 2. Our architecture significantly outperforms conventional non-recurrentdeep ResNets by 2dB SNR, and it is trained much faster [3].Robust Variational Networks for Pathology Recognition. Despite the excellent performance of DL inobject detection from natural images, their performance is quite limited in identifying the abnormalities frommedical images. This is mainly due to the subtle nature of pathologies that involve only a small group ofpixels; such features rarely happen compared with the normal cases. To solve this problem we put forth aframework using variational networks that first finds a soft probabilistic mask around the pathology, andthen refines it in a recurrent fashion to correctly spot the pathology pixels, and accordingly diagnoses theabnormality. Our observations on the Chest X-ray diagnosis exceed state-of-the-art algorithms driven fromnatural images and are comparable with the radiologists ratings.

Future Research AgendaIn my future research, I plan to continue building upon the theoretical and algorithmic foundation developedduring my doctoral study, and the multidisciplinary experience gained during my postdoctoral training toexpand towards new directions at the intersection of AI and Medicine. While recent advances in DL hasgreatly facilitated perception tasks associated with computer vision and language processing, Medicine havecertain features that asks for special algorithmic treatment for reliable and interpretable decision making. Inthis direction, I am passionate to work on the following directions.Unsupervised Geometry Encoding. Embedding images with complex structures into a low-dimensional andinterpretable latent code representation greatly facilitates subsequent decision making tasks. For instance,the popular face recognition system by Google Photos embeds faces into equivariant latent codes that arecomparable using Euclidean distance metrics. The code elements are not however interpretable individually.

Among many reasons to disentangle the code for different geometrical attributes, an interpretable codesignificantly empowers generative models to create images with specific combination of objects. An importantapplication pertains to spotting pathologies in medical images, where the abnormality involves a tiny fractionof pixels that are subtly differentiated from the normal background. The overly heterogeneous nature ofpathologies however demands a lot of training information that is hard to obtain for medical images. Thishinders learning abnormality distribution from images using deep generative models, and thus identificationof abnormalities. To cope with this challenge, my idea is to develop a geometric encoding technique thatdecomposes the image into normal and abnormal code parts, that signifies the pathologies and their degree ofseverity. This can significantly speed up and enhance the quality of diseases diagnosis in clinical practice.Data-Scarce Supervised Learning with Physical Constraints. Collecting labels for training deep neuralnetworks is very expensive in data-scarce applications such as medical imaging. For instance, high resolutionMRI scan of dynamic organs is constrained by high motion and contrast variations in pediatric patients. Thus,the neural networks lack generalization to a large and diverse population of patients. Also, certain diseasesare rare and under-represented, which biases the label prediction towards the healthy population. To improvegeneralization my idea is to leverage the structural constraints dictated by the underlying physics. AI generallylearns a mapping of input data to the closest plausible prediction using a certain metric. In the case ofmultiple constraints one can envision it as projection onto the intersection of manifolds including the labelmanifold and other ones capturing the physical constraints. The physical constraints in general may not beeasy to explicitly model, and one may have only input-output observations from them. My vision is to designa network architecture that projects onto the intersection manifold. The hope is that taking the structuresinto account one can choose shallower networks for better capturing the effective low dimensionality of thedata than the classical deep feed-forward neural networks. A fundamental question to address then pertainsto end-to-end training of multiple concurrent networks.Robustifying Neural Networks against Adversarial Effects. Motivated by the the ill-posed MRI recon-struction problem, one theoretical risk with DL-based image translation methods is the introduction of realisticartifacts, or so-termed hallucinations which can prove costly in a domain as sensitive as medical imaging.The hallucination is due to memorization risk of neural networks, or, the low scan quality. It can mask outpathologies, or, introduce new abnormalities thus risking the diagnosis. Hence, it is critical to examine theextent of these hallucinations and, in so doing, analyze the robustness of DL techniques in medical imagetranslation tasks. In this context, my vision is to leverage my expertise in statistical learning to first developa hallucination risk map from training data that (blindly) evaluates the confidence of the reconstructedpixels. Second, I aim to devise effective regularizers for DL translation techniques that control the risk ofhallucinations.

Research Mentoring and SupervisionMentoring is an important step towards becoming an independent researcher. I have enjoyed mentoringpostdocs, graduate, undergraduate, and high school students and helping them develop critical thinking aswell as improve their research skills. As a senior PhD student I mentored two junior PhD students at theUniversity of Minnesota for projects on large-scale classification and categorical dimensionality reductionfrom incomplete big data. In addition, at Stanford I mentored 1 postdoc, 2 PhDs, 3 undergraduates, and2 high school students on various projects about deep learning for medical imaging such as segmentation,diagnosis, and reconstruction. My responsibility was to define research projects, formulate the problems, andguide students with solving the problem, writing and revising papers, and helping to prepare for professionalpresentations and talks. This was a rewarding (and ongoing) experience which has led to a few journal andconference papers. I am pleased that my summer undergraduate student Jordan Harrod received theBest Amgen Scholar Research Presentation Award at Stanford, Summer 2016.

Funding Experience and PlansExternal funding is important to conduct research and support graduate students. During my research atStanford I have been actively involved and trained for writing grants. I served as PI for DoD Prostate Cancer

Research Program (FY2017), and drafted aims for various NSF and NIH proposals including large programgrants such as P41. I believe my extensive technical collaboration with faculty and colleagues from ElectricalEngineering, Computer Science, Statistics, and Radiology is valuable towards overcoming a potential “languagebarrier.” In terms of funding programs, the importance of AI for healthcare is apparent. Giant companiessuch as Google, Facebook, and Microsoft have already entered this business. As Sundar Pichai, Google CEO,said "So tomorrow, if AI can shape healthcare, it has to work through the regulations of healthcare ... Infact, I see that as one of the biggest areas where the benefits will play out for the next 10-20 years." Tofacilitate the adoption of AI in Medicine, FDA early this year voiced its qualified support, and promised torefine its regulatory approach for the new era, with initiatives such an FDA-supported technology incubatorfor AI innovation and a new ML partnership with Harvard.

In addition, the U.S. government has also initiated funding opportunities to support AI in healthcare. Inparticular, the NSTC recommends strategically directed R&D funds “accelerate advances in AI that woulddirectly support improved value in medical imaging." It also recommends making available federated datasets"to provide a wealth of past outcomes on which to base predictions and prescriptions.” NSF and NIHhave several programs to support AI research. In particular, CISE’s Division of Information and IntelligentSystems supports research and education projects that develop new knowledge in three core programs: (i)Cyber-Human Systems, (ii) Information Integration and Informatics, and (iii) Robust Intelligence (RI). Thebudget per proposal ranges from 0.5-3M for periods of 3-5 years. The RI “encompasses the broad spectrumof foundational computational research needed to understand and enable intelligent systems in complex,realistic contexts. RI may be characterized by flexibility, resourcefulness, creativity, real-time responsivenessand long-term reflection; use of a variety of representation or reasoning approaches; ability to learn and adaptperformance at human levels and beyond; and awareness of and competence in complex environments andsocial contexts.”

NIH also extensively promotes AI research by hosting workshops on harnessing AI and ML to advancebiomedical research, or, AI and medical imaging workshop (with NIBIB) to discuss state-of-the-art AIapplications for medical imaging. The recent program on NLM Research Grants in Biomedical Informaticsand Data Science (R01 Clinical Trial Optional) (PAR-18-896) also funds “well-defined research problems thatpropose a rigorous research design, based on preliminary studies, which will result in innovations that advancewhat is known in the field of informatics and have the capacity to improve human health.”

During my 3-4 years stay in Silicon Valley, I was privileged to closely work with academic researchers atStanford and UC Berkeley, and interact/consult with incorporations working to bring AI to Medicine suchas Arterys, Enlitic, HeartVista, Subtle, Facebook FAIR, and Google Brain. I believe such an experience isvaluable to understand the practical concerns, and to make industry relations for research collaboration andsponsorship. As a junior faculty I will apply for the NSF CAREER Award in July 2019, and NIH R01 Grant inDecember 2019, and will actively seek external funding opportunities through suitable channels.

References[1] Mardani M, Mateos G, Giannakis GB. Subspace learning and imputation for streaming big data matrices and tensors.IEEE Transactions on Signal Processing. 2015 May 15;63(10):2663-77.

[2] Mardani M, Gong E, Cheng JY, Vasanawala SS, Zaharchuk G, Xing L, Pauly JM. Deep Generative Adversarial NeuralNetworks for Compressive Sensing (GANCS) MRI. IEEE Transactions on Medical Imaging, July 2018.

[3] Mardani M, Sun Q, Vasawanala S, Papyan V, Monajemi H, Pauly J, Donoho D. Neural Proximal Gradient Descentfor Compressive Imaging. In Proc. Neural Information Processing Systems (NIPS), Montreal, Canada, December 2018.

[4] Mardani M, Mateos G, Giannakis GB. Distributed nuclear norm minimization for matrix completion. In Proc. IEEE13th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Turkey, June 2012.