RESEARCH ARTICLE SUMMARY◥

ORGANIC CHEMISTRY

Mapping the dark space of chemicalreactions with extended nanomolesynthesis and MALDI-TOF MSShishi Lin, Sergei Dikler, William D. Blincoe, Ronald D. Ferguson,Robert P. Sheridan, Zhengwei Peng, Donald V. Conway, Kerstin Zawatzky,Heather Wang, Tim Cernak, Ian W. Davies, Daniel A. DiRocco, Huaming Sheng*,Christopher J. Welch*, Spencer D. Dreher*

INTRODUCTION:The inventionofnewchem-ical reactions provides new bond constructionstrategies for improved access to diverse regionsof structural space. However, a pervasive, long-standing bias toward reporting successfulresults means that the shortcomings of evenmature reaction methods remain poorly de-fined, making practical syntheses of structur-ally diverse targets far from certain. Distincttools and experimental approaches are re-quired to expose and record the problematicstructural elements that limit different syntheticmethods. The experimental space required tosystematically survey reaction failure is vast,and existing ultrahigh-throughput (uHT) re-action screening approaches are inadequate forexploring the diversity of conditions pertainingin modern synthetic methods. Additionally,analytical approaches must continuously im-

prove to meet the throughput demands of thisexpansive reaction screening.

RATIONALE: We report a nanomole-scalescreening protocol that can be used to executeheterogeneous reactions with heating andagitation, use of volatile solvents, and capacityfor photoredox chemistry. These advances inminiaturized chemistry screening were com-bined with the use of matrix-assisted laserdesorption/ionization–time-of-flightmass spec-trometry (MALDI-TOF MS), enabling analysisof 1536 reactions in ~10 min. Together, theseadvances create a platform that can enable sys-tematic reaction evaluation and data capture tosurvey the dark space of chemical reactions.

RESULTS: Using the Buchwald-Hartwig C–Ncoupling reaction to exemplify this process, an

uHT Glorius fragment additive poisons diag-nostic approach was first applied to demon-strate thatMALDI-MS could provide adequatedata quality to monitor the formation of asingle product under a wide variety of differentsynthetic conditions. Four catalytic methods—Ir/Ni andRu/Nidual-metal photoredox catalysis,as well as heterogeneous and high-temperatureCu and Pd catalysis—with extended nanomolechemistry requirements were evaluated for thesynthesis of a single product in the presence of383 structurally diverse simple and complexpotential poisons. Using a normalizing internal

standard thatwas closelyrelated to the product andoptimized operating pa-rameters,MALDI-MSpro-vided good correlationwith existing ultra per-formance liquid chroma-

tography (UPLC)–MS approaches (coefficientof determination R2 up to 0.85), allowing cor-rect binning of “hits” and “misses” (defined as>50% product signal knockdown) up to 95%of the time. Next, the more challenging goalof exploring diverse whole-molecule C–N cou-plingswas explored. In this case, it was not prac-tical to have either product standards or closelyrelated internal standards to enable analyticalquantitation. A “simplest-partner test”was em-ployed, in which 192 aryl bromides and 192secondary amines were each coupled with aMS-active “simplest partner,” guaranteeing asomewhat normalized MS response for all pro-ducts. The formation of 384 different productsusing the four aforementioned synthetic meth-ods was monitored by MALDI-MS, with pass-fail binning of results correlating well withUPLC-MS in the identification of commonstructural elements (such as functional groupcounts,H-bonddonors and acceptors, andpolarsurface area) that lead to reaction failure.

CONCLUSION: In the near future, each prob-lematic structural element that is identifiedthrough systematic dark-space exploration canbe promoted for in-depth examination to pre-cisely define the specific parameters that deter-mine reaction outcome at the atomic andquantum molecular level. Predictive machinelearning models will use this focused data toenable synthetic practitioners to select themostappropriate reactions for use in a particularsynthetic setting. In addition, functionality thatpersistently fails across synthetic methods cansharply define important challenges for the in-vention of improved chemical reactions.▪

RESEARCH

Lin et al., Science 361, 569 (2018) 10 August 2018 1 of 1

The list of author affiliations is available in the full article online.*Corresponding author. Email: huaming.sheng@merck.com (H.S.); christopher@welchinnovation.com (C.J.W.);spencer_dreher@merck.com (S.D.D.)Cite this article as: S. Lin et al., Science 361, eaar6236(2018). DOI: 10.1126/science.aar6236

Extended nanomole chemistry and MALDI-TOF MS for systematic reaction profiling.Nanomole-scale chemistry tools that can execute awide varietyof synthetic protocols are combinedwith rapid MALDI-TOF MS analysis to enable broad reaction profiling to map the dark space ofchemical reactivity. DMSO, dimethyl sulfoxide; DABCO, 1,4-diazabicyclo[2.2.2]octane.

ON OUR WEBSITE◥

Read the full articleat http://dx.doi.org/10.1126/science.aar6236..................................................

on June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

RESEARCH ARTICLE◥

ORGANIC CHEMISTRY

Mapping the dark space of chemicalreactions with extended nanomolesynthesis and MALDI-TOF MSShishi Lin1, Sergei Dikler2, William D. Blincoe3, Ronald D. Ferguson1,Robert P. Sheridan4, Zhengwei Peng4, Donald V. Conway5, Kerstin Zawatzky3,Heather Wang3, Tim Cernak6*, Ian W. Davies7†, Daniel A. DiRocco7, Huaming Sheng3‡,Christopher J. Welch3‡§||, Spencer D. Dreher1‡

Understanding the practical limitations of chemical reactions is critically important forefficiently planning the synthesis of compounds in pharmaceutical, agrochemical, andspecialty chemical research and development. However, literature reports of the scope ofnew reactions are often cursory and biased toward successful results, severely limitingthe ability to predict reaction outcomes for untested substrates. We herein illustratestrategies for carrying out large-scale surveys of chemical reactivity by using a material-sparing nanomole-scale automated synthesis platform with greatly expanded syntheticscope combined with ultrahigh-throughput matrix-assisted laser desorption/ionization–time-of-flight mass spectrometry (MALDI-TOF MS).

The field of synthetic organic chemistry hasspawned a vast array of creative reactionsthat can be logically combined to preparenearly any molecule. However, efficient se-lection of the precise reaction sequence that

leads to a particular product remains a challenge,as poorly understood substrate-specific inter-actions often necessitate laborious screening ofcombinations of catalysts, reagents, and condi-tions. High-throughput experimentation (HTE)chemistry facilitates these investigations by in-creasing the pace of problem-solving (1–7), andan emerging strategy usesmachine learning (ML)data generated with these screening tools to createpredictive models for specific problems (8, 9).Data mining surveys of existing published orproprietary databases containing informationon hundreds of millions of reactions have beensomewhat encouraging (10–14), but a pervasive,long-standing bias toward reporting successful

results limits the utility of this information formodel building. Additionally, these data have notbeen collected under controlled experimentalconditions, and most of the substrates are notrepresentative of the complexity that arises inapplied synthetic problems. Hence, no existinglarge, structured datasets meet all of the require-ments for effective reaction modeling. Con-sequently, intentional surveys are required tosystematically map reactivity patterns, with therapid identification of inaccessible regions, thedark space of chemical reactivity, being partic-ularly important for focusing subsequent MLinvestigations.Chemical reactivity space is vast, even for a

single synthetic transformation, with permuta-tions of possible substrate structures, catalysts,and reaction conditions soon proliferating intounmanageable experimental complexity. Large-scale reactivity surveys have thus far been lim-ited by the throughput of reaction screening andanalysis technologies. Additionally, the scarcityand high cost of relevant complex substrates re-quires that screening be carried out on as smalla scale as possible. Existing miniaturized HTEchemistry tools have worked only under a narrowrange of conditions and have thus far not beenapplicable to the vast majority of high-qualitysynthetic protocols. In this work, we describe howengineering advances in automated, miniaturizedreaction experimentation and the use of fastmatrix-assisted laser desorption/ionization massspectrometry (MALDI-MS) for routine reactionanalysis affords an opportunity to rapidly surveythe chemical reactivity landscape. We demon-strate how these advances can be used to sys-tematically reveal diverse sources of reaction

failure within large substrate sets, informationthat is largely missing from current chemistrydata sources, representing a notable advancetoward broadly effective reaction profiling.

Extending nanomole synthesis forsystematic reaction profiling

Our interest in reaction profiling has grown froman internal effort termed the C–N Coupling Ini-tiative, which aims to systematically build toolsto completely understand and leverage a givensynthetic transformation. A recent examinationof our electronic notebooks (ELNs) shows C–Ncoupling to be a hit-or-miss affair (~10,000 ex-amples, ~35% failure rate) that, despite its po-tential to render druglike molecules, may beunderused because of its higher demand for op-timization than, for example, the Suzuki reaction(~50,000 example, 18% failure). Attempts to usethis ELN dataset for creating predictive modelshave been disappointing, as the data are frac-tured, with many different classes of amines thatwould be expected to have different reactivitypatterns (primary, secondary, NH-heterocyclic,amides, sulfonamides) while also containing deepchannels of project-driven substrate similarityin the aryl halide coupling partners. Additionally,these data are collected under a wide array ofsynthetic protocols, with specific conditions sel-dom being repeated. Given this lack of a suit-able dataset, a real and urgent goal of appliedresearch is to increase the understanding of C–Ncoupling reactions by building an experimentalprocess suitable for fast reactionmapping. Clearly,this systematic reaction profiling capability willalso be of general value for surveying other re-action types where available knowledge is scarce.Failure in metal-catalyzed reactions can re-

sult from a variety of causes, including inhibitionof catalysts due to substrate binding, competitiveside reactions, local steric and electronic inter-actions, and substrate decomposition caused byharsh reaction conditions. In addition, small per-turbations in molecular structure may result inchanges in reactivity, especially for complex sub-strates. We perceived that mapping reactionfailure arising from this diverse patchwork ofmechanisms would require a large collectionof structurally diverse analogs as well as atremendous number of experiments. Emergingnanomole-scale tools have the material-sparingminiaturized scale and ultrahigh-throughput(uHT) required for broad profiling; however,existing experimental strategies involve sim-plified engineering and are inadequate for ex-amining most high-quality synthetic protocols.The first-generation nanomole synthesis platformthat we developed (1) was limited to plastic-compatible, homogenous, ambient temperaturereactions in low-volatility polar, aprotic solvents,encompassing only a small subset of organictransformations. The recent work by Perera et al.(2) extends nanomole synthesis capabilities in aflow chemistry format, allowing heating and theuse of diverse solvents; however, this formatalso requires that reactions remain homogeneous,necessitating the use of very high dilution and

RESEARCH

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 1 of 7

1Chemistry Capabilities Accelerating Therapeutics, Merck & Co.,Inc., Kenilworth, NJ 07033, USA. 2Bruker Daltonics, Inc.,Billerica, MA 01821, USA. 3Analytical Research andDevelopment, Merck & Co., Inc., Rahway, NJ 07065, USA.4Modeling and Informatics, Merck & Co., Inc., Kenilworth, NJ07033, USA. 5Discovery Sample Management, Merck & Co.,Inc., Kenilworth, NJ 07033, USA. 6Discovery Chemistry, Merck &Co., Inc., Boston, MA 02115 USA. 7Process Research andDevelopment, Merck & Co., Inc., Rahway, NJ 07065, USA.*Present address: Department of Medicinal Chemistry, Departmentof Chemistry, Program in Chemical Biology, University of Michigan,Ann Arbor, MI 48109, USA. †Present address: Princeton CatalysisInitiative, Princeton University, Princeton, NJ 08544, USA.‡Corresponding author. Email: huaming.sheng@merck.com (H.S.);christopher@welchinnovation.com (C.J.W.);spencer_dreher@merck.com (S.D.D.) §Present address:Indiana Consortium for the Analytical Sciences, Indianapolis,IN 46202, USA. ||Present address: Welch Innovation, LLC,Cranbury NJ 08512, USA.

on June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

limiting study to extremely fast kinetics thatcan keep up with serial high-performance liquidchromatography (HPLC) analysis. Rather thanaccepting such constraints, we systematicallyengineered and validated new plate-based nano-mole synthesis tools with a general ability tocarry out a wide variety of typical synthetic re-actions. We identified effective chemically com-patible glass microplate reactors, performed fast384-tip dosing of reagent solutions in volatilesolvents, and designed aluminum sealing blocksthat can retain volatile solvents on heating. Inaddition, we used LabRam resonant acousticmixing to both agitate reactions and createmilkyslurries of solid inorganic bases that can beadded to reactions by parallel liquid handling.Finally, we created a nanomole-scale photo-chemistry tool to enable reaction evaluation inthe rapidly growing field of photoredox catalysis.We describe the use of these tools in reaction

profiling experiments below, but they are ex-pected to benefit extensions to nanoscale chem-ical and biological evaluation as well (15).

MALDI-TOF MS reaction analysis

Owing to the sheer number of experimentsnecessary, surveying chemical reactivity can belimited by the rate of analytical measurement.Despite considerable recent progress, HPLC-MSassays typically used for such screening are con-strained by the fastest speed with which samplescan bemechanically taken up and injected (~10 sper sample) (16). Spectroscopic techniques canbe faster but often lack the resolving powerneeded to study a diverse collection ofmolecules.High-throughput mass spectrometry providesexcellent mass-to-charge ratio separation of in-dividual products (17–20), and MALDI-MS hasbeen intensively optimized in recent years forrapid and robust molecule-specific MS imaging

of biological tissue slices. The new generation ofMALDI–time-of-flight (TOF) instruments equippedwith a 10-kHz scanning beam laser, substantiallyfasterX,Y stage, and faster plate loading-unloadingcycle substantially improves sample throughput tohundreds of thousands of measurements per day.We were therefore intrigued by the possibilityof applying this fast-analysis approach (severalsamples per second) to high-throughput reactionscreening (HTS). AlthoughMALDI is commonlyused for high-throughput analysis, applicationsare typically limited to either quantitative bio-chemical assays that feature a single substrate orproduct within each well (21–28) or chemistrydiscovery using anMS-active tag or specialMALDIplate to ensure MS detection of all labeled pro-ducts (29–31). Neither of these approaches en-abled the rapid, label-free analysis of a diversecollection of products that we required.Direct MS analysis of small molecules can

be vulnerable to interference from reaction com-ponents that would typically be removed bychromatography. To analyze samples byMALDI-TOF MS, we found a suitable ionization matrix,and instrument settings afforded excellent per-formance for diverse pharmaceutically relevantproducts, with the addition of an ionizing in-ternal standard helping to normalize the effectsof different reaction conditions on product ion-ization and MALDI spot formation. Screeningand optimization of MALDI ionization matriceswere carried out using a library of druglike in-former compounds (32), leading to the identi-ficationofa-cyano-4-hydroxycinnamic acid (CHCA)as a generally preferredmatrix [4mg/ml in 50%H2O/acetonitrile (ACN) with 0.1% trifluoroaceticacid (TFA)] affording an optimal signal-to-noiseratio across a wide range of protonated modelcompounds. A general protocol includingMALDIplate preparation, automated data acquisition,and data processing was established for MALDIanalysis, in which 175 nl of quenched reactionmixture in dimethyl sulfoxide (DMSO)with addedinternal standard was directly spotted on an HTSMALDI target plate in 1536 format using apositive-displacement liquid-handling robot with16 channels. In our experiments, the spotting timefor a single plate was ~30 min, but MALDI platepreparation time could be shortened by using aliquid-handling robotwith a 1536-channel pipett-ing head. The resulting DMSO on the targetplate could be evaporated either under vacuumat 0.01 bar (2 min) or at atmospheric pressure(1 hour). CHCAmatrix solution (150 nl) was thenapplied onto the dried reaction spots through thesame liquid-dispensing procedure. The readoutspeed for each 1536 MALDI plate ranged from8 to 11 min, depending on the number of lasershots per spectrum. All automated MALDI runswere set up, triggered, and processed using ded-icated uHTS software for MALDI-TOF instru-ments (Bruker MALDI PharmaPulse 2.0). In thewhole-molecule analysis experiment, a customprocessing script was used to simultaneouslyextract nearly 400 masses, peak intensities, andpeak areas for reaction products and the internalstandard into a single spreadsheet, in addition to

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 2 of 7

Fig. 1. uHT MALDI-MS approach to screening potential reaction poisons. (A) 1536 reactions wererun with four leading C–N coupling methods to evaluate the impact of 383 single and polyfunctionalfragments on one simple coupling reaction, requiring 11 min of MALDI acquisition time. Ph, phenyl;DiF, difluoro; RuPhos, 2-dicyclohexylphosphino-2′,6′-diisopropoxybiphenyl; ppy, 2-phenyl pyridinyl;dtbbpy, di-tert-butyl di-pyridyl; DABCO, 1,4-diazabicyclo[2.2.2]octane; bpy, bipyridinyl. (B) Raw MALDIproduct response is not well correlated with UPLC-MS or UPLC-UV measures, but normalizationusing an internal standard reveals very good correlation. (C) Ranked, normalized MALDI productresponse data with corresponding UPLC-MS (EIC) data below for comparison.

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

standard MALDI PharmaPulse processing. Asignal-to-noise-ratio of 5/1 was used in the soft-ware to extract the final MALDI-MS data.

uHT fragment poisons analysis

To test our MALDI analytical approach for gen-eral reaction profiling, we began with an ana-lytically simplified screening protocol, developedby Glorius and co-workers, for identifying re-action poisons—i.e., functional groups and re-agents that interfere with a given reaction (Fig. 1)(33, 34). Because the same reaction is investigatedin each experimental well, a single productstandard can be used throughout, greatly sim-plifying analysis. We realized that the throughputof the MALDI technique could enable screeningof much larger sets of steric and electronic func-tional variants and higher-order polyfunctionalmolecular fragments to precisely identify specificdeleterious bindingmotifs and cooperative inter-actions. At the same time, we could evaluate theability of the MALDI system to provide accurateproduct quantification in the presence of dif-ferent reaction conditions and a wide variety ofadditives.We selected four different C–N coupling con-

ditions (Fig. 1A)—Ir/Ni and Ru/Ni photoredox(35) as well as Pd and Cu conditions (32)—thatwere previously demonstrated to perform wellusing our chemistry informer libraries approach.Eachmethod was evaluated using the new nano-mole experimentation platforms in the presenceof 383 polar molecular fragments (with onecontrol reaction containing no additive) rangingfrom simple compounds containing a single func-tional group (such as amides, esters, and N-heterocycles) to more complex polyfunctional

compounds. The resulting 1536 fragment-additivereactions were analyzed in a single MALDI ex-periment in 11 min, with the settings describedabove and using a deca-deuterated product analog,added to each well after reaction completion,as an internal standard (Fig. 1A). The reactionswere also analyzed using a conventional 2-minultra performance liquid chromatography (UPLC)–MS method. The raw MALDI product responseshows only marginal correlation with the UPLC-MS orUPLC–ultraviolet absorption (UV) data forthe same reactions [extracted ion count (EIC),coefficient of determinationR2 = 0.24; UV 210 nm,R2 = 0.16]; however, normalization of theMALDIsignal with the closely related internal standardsignal affords notably good correlation (EIC,R2 =0.85), showing that chemistry-specific effectson MALDI signal intensity can be mitigated bythe use of internal standards (Fig. 1B). The norm-alizedMALDI product response metric, rankedfrom high to low for all four methods, is shown inFig. 1C, with the correspondingUPLC-MSEICdatashown below for comparison. This analytical ap-proach is similar to the application of MALDI-MSfor HTS of enzyme inhibitors, but this work dem-onstrates that MALDI, in conjunction with anappropriate product standard, can be used forroutine reaction screening in the presence of adiversity of interfering elements.To simplify MALDI-based assessments in

fragment-additive screening, we binned the nor-malized MALDI product response for each re-action into poisons (>50% signal knockdown)and nonpoisons (<50% signal knockdown) foreach fragment and synthetic method (Fig. 2).Across all four methods, the MALDI assignmentmatched the UPLC-MS EIC signal ~95% of the

time and the UPLC-UV 210-nm data ~88% of thetime (we found that UPLC-UV 210-nm data oftenshow peaks that interfere with the product peak).The fragment screeningdata showed the expectedoutcome that each of the synthetic methods issusceptible to poisoning by particular single func-tional groups (see figs. S19 and S20 for completelists). We were also able to observe that whenthese single-functional poisons are incorporatedas parts of polyfunctional fragments, reactionpoisoning almost always occurs (>90% of thetime). Notably, polyfunctional fragments com-posed of functional groups that are not them-selves poisons often (~50% of the time) act aspoisons as well. The structural details of thispoisoning are very specific. For example, weobserved cases in which the combination of N-heterocycle and ketone strongly poisons a givencatalytic system, whereas other similar structuralcombinations do not (Fig. 2). This work providessome indication of how screening large permu-tations of multifunctional structures can identifyprecise motifs that lead to reaction failure. Wewere gratified to find that with a large numberof experiments, the effects of false negativesand positives on identifying trends are, to somedegree, offset by the internal validation thatstems from using multiple fragments that arestructurally related. The fragment-based analysisherein reveals Cu to be substantially more re-silient to single and poly-functional poisoningthan the other methods evaluated, which shouldsupport its wider use in complex, densely func-tionalized substrates. The Ir/Ni and Ru/Ni photo-redox methods were very similar to each otherin their inhibition profiles and also generally per-formed very similarly to Pd. Most importantly,

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 3 of 7

Fig. 2. Fragmentadditives poisoningstudy. Comparative ratesof poisoning determinedby MALDI analysisfor single functionalfragments andpolyfunctional fragmentsfor the four testedmethods are displayed.This study reveals that Cuis well-suited to managingdiverse functionality.Polyfunctional fragmentsthat are composed oflinked, single functionalfragments that are notpoisons themselves oftenlead to differentialpoisoning.

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

the specific problematic functional arrays thatpoison eachmethod can be roughlymapped, andsubsequent detailed nanomole chemistry ex-ploration with structural variants can providefocused data for effective ML predictive model-ing [as in (8)].

uHT simplest-partnerwhole-molecule analysis

Wenext investigated themuchmore challenginguse of MALDI-MS to characterize the effectspresent in large, whole-molecule substrate sets(Fig. 3). Whereas the fragments approach isuseful for rapid identification of problematicfunctionality, classifying local steric and elec-tronic as well as bulk molecular effects such assolubility can be accomplished only by usingwhole-molecule substrates. Previous work inour labs explored the effects of pharma-relevantwhole-molecule informer compounds (32) onthe performance of different synthetic methods.Though useful to begin to systematically evaluatedifferent synthetic methods, the 18 compoundsin these test sets do not provide enough struc-tural diversity to begin to assign general causesof reaction failure. In addition, the impact ofamines in complex reactivity space was not pre-viously taken into account. Hence, we envisioneda much larger virtual array of 192 N-heterocycle–containing aryl bromides crossed with 192 cyclicsecondary amines (Fig. 3A), both of increasingmolecular complexity, representing 36,864 dis-tinct potential products. This array, if evaluatedwith the four aforementioned C–N coupling pro-tocols, would result in 147,456 experiments. Al-though experimental assessment of all substratesand protocols is conceivable, we explored a sys-tematic “simplest-partner” approach to study theisolated structural effects of individual buildingblocks in this space. In this protocol, each bro-mide is subjected to reaction with the simplestamine in the set and each amine with the sim-plest bromide. Applying the four different syn-thetic methods to this 384-substrate test setaffords a total of 1536 experiments, or ~1% ofthe experimental space. To ensure that everyreaction in the set would have a strongMALDI-MS response, the simplest coupling partners werechosen to incorporate a basic nitrogen atom,thereby ensuring MS detectability in the positive-ion mode. Although measuring products innegative-ion mode was not the focus of thiswork, analysis of small molecules by MALDI-TOF in negative-ion mode can be successfullyachieved when appropriate sample prepara-tion methods are used (36–38). Clearly, thisgeneral strategy would be poorly suited for thesynthesis of hydrocarbons or other species thathave poor MS ionization properties.Traditional exploration of complex substrate

space using HPLC analysis requires the prepara-tion of product standards to obtain responsefactors for quantification and to confirm productidentity for every new molecule of interest, anuntenable proposition for our envisioned ex-periment. The prospect of adding an internalstandard that is tailored to each product for

MALDI normalization is equally daunting. Wereasoned that in our search for causes of re-action failure, a focus on common structuralreactivity trends rather than absolute productyields might afford valuable insights without thetraditional need for individual product standards.Also, we hoped that the use of a single internalstandard in all wells, although clearly not control-ling for structure-based variations in MALDIsignal intensity, might still be useful for normal-izing the effects of MALDI spotting variability,thereby vastly simplifying experimental execution.Hence, we endeavored to use rapid MALDI re-sponses for 384 different products coupled withparameterized and clustered molecular descrip-tors (such as functional group counts, sterichindrance, and bulk properties such as H-bonddonors and acceptors and polar surface area) toidentify fundamental incompatibilities between

given sets of reaction conditions and the diversestructural properties within an enormous com-pound set. Once identified, such reactivity trendscan be confirmed by conventional analyticalmethods. The MALDI signal for all 1536 reac-tions was acquired using a single internalstandard for normalization across all reactions,aswell as a conventional 2-minUPLC-MS analysisfor comparison. Two-dimensional scatter plotsfor all 1536 reactions (Fig. 3B) reveal that the nor-malized product MALDI responses are poorlycorrelated compared with the UPLC-MS (EIC,R2 = 0.33) and UPLC-UV data [total wavelengthchromatogram (TWC), R2 = 0.30; we found anaveraged wavelength to be more useful in thisdiverse compounds set than a single wavelength,such as UV 210 nm]. However, we found thatwe could use the MALDI data to create binarythresholds for reaction success or failure that

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 4 of 7

Fig. 3. Whole-molecule simplest-partner evaluation with MALDI analysis. (A) Bromides andamines (192 of each) are each crossed with a simple, mass-active coupling partner under the fourpreviously described (Fig. 1A) catalytic methods. (B) Using a single internal standard, the normalizedMALDI data show poor correlation with UPLC-MS and UPLC-UV metrics.

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

permitted us to identify general structural en-tities that correlate with reaction failure within alarge, whole-molecule set. To this end, we setan arbitrary pass-fail threshold of 20% of theaverageMALDI value across the entire set.Whenthe binned data for each synthetic method areclustered as a percentage of failed reactions fordifferent structural parameters that have at least10 examples within a set (Fig. 4A), the data from

the normalized MALDI experiment show essen-tially the same trends revealed in the UPLC-MSdata (both EIC and TWC responses), suggestingthat MALDI can be used to stack-rank the struc-tural entities that most often cause reaction fail-ure for each set of conditions (see figs. S26 to S33for complete lists). Again, any of these identifiedproblematic structural parameters can be pro-moted for higher-order systematic ML studies.

Although lists of potential poisons for specificsynthetic methods are useful, understandingthe comparative reactivity of different syntheticmethods for problematic functionality is arguablyeven more important. Figure 4B shows how theclustered binary pass-fail data for all four syn-thetic methods can be combined into a singlegraph that reveals structural entities that aregenerally problematic for bromides and amines

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 5 of 7

Fig. 4. Whole-molecule reactivity trends from binary thresholdinganalysis. (A) Failure percentage for different clustered parameters determinedby pass-fail binary binning using thresholded MALDI data correlates very wellwith similarly binned UPLC-MS EIC and UPLC-UV TWC data. Each point on thegraph represents the failure rate of different specific aggregated structuralparameters, such as NH heterocycles or H-bond donors, for each syntheticmethod.Circles and crosses denote whether the trend is for amines (circles) orbromides (crosses).The symbol color of each circle or cross indicates whichsynthetic method was used for the data point. (B) The average failure rate forclustered parameters for all four methods reveals the functionality that is

problematic across methods.The color of each data point reveals whichmethod has the lowest failure rate. (C and D) The most problematicparameters for aryl bromides (blue shaded area from Fig. 4B) and amines(red shaded area in Fig.4B) are listed in descendingorderof failure rates acrossmethods, along with the number of examples in the test set. In nearly all cases,Cu is the preferred method. However, several very specific structural typesin amines are found to be problematic for Cu and show greater reactivity withPd. MR, membered ring; TPSA, total polar surface area; NHR, nitrogen withan alkyl group and a hydrogen; MW, molecular weight; clogP, calculated logP(logarithm of the partition coefficient between n-octanol and water); Bn, benzyl.

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

across all methods. The data can serve as a valu-ablemap for academic research focused on creat-ing new methods to overcome these commonproblems. This graph also reveals which methodaffords the lowest failure rate for each parameter.Figure 4B clearly shows the synthetic advantageof the Cu system in nearly all parameters used todescribe aryl bromides (themost problematic arelisted in Fig. 4C), which again substantiates itsvalue in complex synthesis. At the same time, fora number of parameters within the amine struc-tural space (Fig. 4B, red), Pd is the best catalyst.Diving deeper (Fig. 4D), we can identify veryspecific structural features within amines thatare problematic for Cu and for which Pd canprovide a synthetic advantage. Particularly no-table is the reproducibly higher performance ofPd versusCu for 3-substituted-4-N-carbonyl piper-azines and 3,4-N-heterocycle–appended piper-azines, revealing subtle conformational effectsseveral bonds removed from the reactive aminethat are not considered in the current understand-ing of C–Ncoupling reactivity. These systemswereexplored at a larger reaction scale (10 mmol, 40×scale-up) with additional structural examples toconfirm the trends (see figs. S37 to S39 for fur-ther details on Cu versus Pd amine trends).The fragment-additive and whole-molecule

studies both indicate that Cu has a substantialadvantage over the other methods examined inthis study, with respect to single and poly-functional group tolerance and scope. Theseobservations hold with aryl bromides, but Cualso appears to have specific liabilities in theamine structural space. This study demonstratesthat uHT fragment and whole-molecule ap-proaches can provide complementary reactivityinformation, and both approaches will likelybe inextricably linked in the future of predictivechemistry. Given the observed importance ofprecisely defined structures evident in this work,as well as a shortage of diverse test substrates,chemists will likely alternate between the twoapproaches to refine their maps of holistic re-activity effects. The most important messagefrom this work is that, rather than accuratelydetermining reaction yields, broad uHT map-ping of substrate space with no isolated productstandards can reveal specific molecular inter-actions using adequately discriminating binaryanalytical approaches that simply separate suc-cesses from failures. The reactivity trends iden-tified by MALDI in these experiments wereconfirmed with conventional UV/MS scoring,which suggests that MALDI alone can be used touncover results previously accessible only viaslower analytical methods. Looking forward, wehope that the convenience, robustness, and speedof MALDI will facilitate creation of large sets ofstructural andmass response data to enable quan-titative yield prediction to further increase thevalue of the MALDI-MS approach.

Full factorial whole-molecule space

The remaining 99% of the full factorial whole-molecule crossover substrate coupling spacemaycontain additional information on higher-order

bimolecular interactions that can also influencereaction performance. The simplest-partner ap-proach can be used to triage the experimentsrequired to map the enormity of the remainingspace. We reasoned that if either compound ina complex pair performs poorly in the simplest-partner test, then structural poisoning will belikely in themore complex combination. Likewise,when both partners perform well with simplepartners, the combination is likely to work welltogether. We investigated 288 experiments inthis crossover space and found that we couldassign >50% of the space into “hits” or “misses”(with 90% accuracy, see supplementary materialsfor details). The remaining space, with substrateshaving middling MALDI responses, could not beeffectively resolved. This pruning approach allowsus to chart complex bimolecular space usingthe minimal coverage provided by the simplest-partner experiments, thereby eliminating >70,000experiments that do not need to be performed.

Toward predictive informatics

The miniaturized uHT reaction engineering andMALDI-TOF MS analytical advancements de-scribed in this work enable generation of large,structured datasets that pinpoint problematicstructural elements within complex substratespace. Identified problems can then be promotedfor more detailed mapping with the use of uHTexperimentation coupled with atomic and quan-tum molecular physical descriptors to enablestructure-based ML. In this work, we ran morethan 3000 experiments studying the effects ofjust four synthesis conditions on a relativelynarrow area of chemistry space (the couplingof cyclic secondary amines and N-heterocycle–containing aryl bromides). The real power in thisapproach will harness the pronounced analyticalspeed of MALDI analysis for the iterative evalua-tion of large substrate arrays against diversecatalysts, bases, solvents, reagent stoichiometries,and temperatures, enabling big data chemistryinformatics in the search for general solutions toproblematic areas in organic synthesis.A persistent focus on revealing structural lim-

itations in chemistry is a win-win scenario foracademic researchers and synthetic practition-ers alike. Dark space that remains inaccessibleacross syntheticmethods becomes fodder for newexperimental research of known value, and spe-cific knowledge of limitations will lead to predic-tivity and understanding that will increase thespeed and success of the design-make-test cycle,helping to remove synthetic problem-solvingfrom the critical path. Even with optimally ef-ficient tools and strategies in place, mapping theentire landscape of useful chemical reactivity iscurrently beyond the reach of any single organi-zation. Given the promising enabling value ofsuch amassive survey, a precompetitive public-private partnership to address this gap mighteven be an achievable and worthwhile goal.

Materials and methods summary

The specific extended nanomole-scale chemistryprotocols used for the four describedC–Ncoupling

synthetic methods (Ir/Ni and Ru/Ni photoredox,Cu, and Pd), as well as the MALDI-MS and com-parative UPLC-MS analysis protocols, are de-scribed in detail in the supplementary materials.A brief summary is provided below.

Description of extended nanomole-scalechemistry platform

Nanomole-scale chemistry reactions, to thispoint, have been run using plastic plates withlow-volatility, plastic-compatible solvents (polaraprotics, DMSO, and N-methyl-2-pyrrolidone)and homogeneous components (bases and cat-alysts) at room temperature. In thiswork, reactionengineering advancements have enabled a muchwider scope of potential reaction conditions. Theuse of glass 1536-well plates and rapid 384-tippipetting enables the use of volatile, non–plastic-compatible solvents (such as dioxane, used inthis study) and substrates (piperidine). Develop-ment of a resonant acoustic LabRam grindingprotocol to produce long-lasting slurries thatcan be dosed with liquid-handling robotics hasenabled the use of heterogeneous inorganic bases(Cs2CO3 and K3PO4). An aluminum reactor blockwas designed that can provide a high-quality sealto prevent solvent loss, and a heating mechanismwas devised that can be equipped to heat thealuminum block inside the LabRam, which pro-vides a robust mechanism for parallel reactionagitation. Finally, a nanomole-scale photochemistrytool was created using a modified aluminumreactor with an acrylic plastic bottom that allowsuniform light penetration.

Description of MALDI-TOF analysis

All reactionmixtures were quenched and dilutedto standard UPLC-MS concentration. Thesequenched reactions were then spotted using aMosquito HTS liquid-handling robot on BrukerHTS MALDI targets with barcodes (1.0-mmthickness) in 1536 format [HTS MALDI plate1.0 mm, BC, part 1833280]. The targets weremounted on Bruker HTS MALDI adapters (part1847571). Reactionmixture (175 nl) was depositedand allowed to dry, followed by depositing 150 nlof a 4-mg/ml solution ofa-cyano-4-hydroxycinnamicacid in 0.1% TFA, 50% ACN/H2O. These targetswere analyzed on a Bruker Rapiflex MALDI-TOF/TOF system.

Description of comparativeUPLC-MS analysis

The quenched reactions were also monitoredusing a Waters Acquity UPLC I-Class system(Waters Corp.) equipped with a binary pump,flow-through needle sampler, column manager,photodiode array detector, SQ detector 2 withelectrospray ionization source in the positivemode, andMassLynx software. Separations wereperformed on a Waters CORTECS UPLC C18+column (dimensions: 30 mm by 2.1 mm; particlesize: 1.6 mm).

REFERENCES AND NOTES

1. A. Buitrago Santanilla et al., Nanomole-scale high-throughputchemistry for the synthesis of complex molecules.

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 6 of 7

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

Science 347, 49–53 (2015). doi: 10.1126/science.1259203;pmid: 25554781

2. D. Perera et al., A platform for automated nanomole-scalereaction screening and micromole-scale synthesis in flow.Science 359, 429–434 (2018). doi: 10.1126/science.aap9112;pmid: 29371464

3. S. W. Krska, D. A. DiRocco, S. D. Dreher, M. Shevlin, TheEvolution of Chemical High-Throughput Experimentation ToAddress Challenging Problems in Pharmaceutical Synthesis.Acc. Chem. Res. 50, 2976–2985 (2017). doi: 10.1021/acs.accounts.7b00428; pmid: 29172435

4. V. I. Martin, J. R. Goodell, O. J. Ingham, J. A. Porco Jr.,A. B. Beeler, Multidimensional reaction screening forphotochemical transformations as a tool for discovering newchemotypes. J. Org. Chem. 79, 3838–3846 (2014).doi: 10.1021/jo500190b; pmid: 24697145

5. K. Troshin, J. F. Hartwig, Snap deconvolution: An informaticsapproach to high-throughput discovery of catalyticreactions. Science 357, 175–181 (2017). doi: 10.1126/science.aan1568; pmid: 28706066

6. A. McNally, C. K. Prier, D. W. MacMillan, Discovery of ana-amino C-H arylation reaction using the strategy ofaccelerated serendipity. Science 334, 1114–1117 (2011).doi: 10.1126/science.1213920; pmid: 22116882

7. M. N. Hopkinson, A. Gómez-Suárez, M. Teders, B. Sahoo,F. Glorius, Accelerated Discovery in Photocatalysisusing a Mechanism-Based Screening Method. Angew.Chem. Int. Ed. 55, 4361–4366 (2016). doi: 10.1002/anie.201600995; pmid: 27000485

8. D. T. Ahneman, J. G. Estrada, S. Lin, S. D. Dreher, A. G. Doyle,Predicting reaction performance in C-N cross-couplingusing machine learning. Science 360, 186–190 (2018).doi: 10.1126/science.aar5169; pmid: 29449509

9. A. Milo, A. J. Neel, F. D. Toste, M. S. Sigman, A data-intensiveapproach to mechanistic elucidation applied to chiral anioncatalysis. Science 347, 737–743 (2015). doi: 10.1126/science.1261043; pmid: 25678656

10. C. W. Coley, R. Barzilay, T. S. Jaakkola, W. H. Green,K. F. Jensen, Prediction of Organic Reaction Outcomes UsingMachine Learning. ACS Cent. Sci. 3, 434–443 (2017).doi: 10.1021/acscentsci.7b00064; pmid: 28573205

11. S. Szymkuć et al., Computer-Assisted Synthetic Planning: TheEnd of the Beginning. Angew. Chem. Int. Ed. 55, 5904–5937(2016). doi: 10.1002/anie.201506101; pmid: 27062365

12. A. Lavecchia, Machine-learning approaches in drugdiscovery: Methods and applications. Drug Discov. Today20, 318–331 (2015). doi: 10.1016/j.drudis.2014.10.012;pmid: 25448759

13. M. H. Todd, Computer-aided organic synthesis.Chem. Soc. Rev. 34, 247–266 (2005). doi: 10.1039/b104620a;pmid: 15726161

14. R. Sheridan et al., Toward structure-based predictive tools forthe selection of chiral stationary phases for thechromatographic separation of enantiomers. J. Chromatogr. A1467, 206–213 (2016). doi: 10.1016/j.chroma.2016.05.066;pmid: 27318509

15. N. J. Gesmundo et al., Nanoscale synthesis and affinityranking. Nature 557, 228–232 (2018). doi: 10.1038/s41586-018-0056-8; pmid: 29686415

16. C. J. Welch, Are We Approaching a Speed Limit for theChromatographic Separation of Enantiomers? ACS Cent. Sci. 3,

823–829 (2017). doi: 10.1021/acscentsci.7b00250;pmid: 28852695

17. M. Wleklinski et al., High throughput reaction screeningusing desorption electrospray ionization mass spectrometry.Chem. Sci. 9, 1647–1653 (2018). doi: 10.1039/C7SC04606E;pmid: 29675211

18. S. Sun, T. R. Slaney, R. T. Kennedy, Label free screening ofenzyme inhibitors at femtomole scale using segmented flowelectrospray ionization mass spectrometry. Anal. Chem. 84,5794–5800 (2012). doi: 10.1021/ac3011389; pmid: 22656268

19. K. J. Boggio et al., Recent advances in single-cell MALDImass spectrometry imaging and potential clinical impact.Expert Rev. Proteomics 8, 591–604 (2011). doi: 10.1586/epr.11.53; pmid: 21999830

20. X. W. Diefenbach et al., Enabling Biocatalysis by High-Throughput Protein Engineering Using Droplet MicrofluidicsCoupled to Mass Spectrometry. ACS Omega 3, 1498–1508(2018). doi: 10.1021/acsomega.7b01973

21. M. S. Ritorto et al., Screening of DUB activity and specificityby MALDI-TOF mass spectrometry. Nat. Commun. 5, 4763(2014). doi: 10.1038/ncomms5763; pmid: 25159004

22. C. Haslam et al., The Evolution of MALDI-TOF MassSpectrometry toward Ultra-High-Throughput Screening:1536-Well Format and Beyond. J. Biomol. Screen. 21, 176–186(2016). doi: 10.1177/1087057115608605; pmid: 26428484

23. M. Winter et al., Establishing MALDI-TOF as Versatile DrugDiscovery Readout to Dissect the PTP1B Enzymatic Reaction.SLAS Discov. 23, 561–573 (2018). doi: 10.1177/2472555218759267; pmid: 29466676

24. K. Beeman et al., Integration of an In Situ MALDI-Based High-Throughput Screening Process: A Case Study with ReceptorTyrosine Kinase c-MET. SLAS Discov. 22, 1203–1210 (2017).doi: 10.1177/2472555217727701; pmid: 28820955

25. R. E. Heap et al., Identifying Inhibitors of Inflammation: A NovelHigh-Throughput MALDI-TOF Screening Assay for Salt-Inducible Kinases (SIKs). SLAS Discov. 22, 1193–1202 (2017).doi: 10.1177/2472555217717473; pmid: 28692323

26. T. Si et al., Profiling of Microbial Colonies for High-ThroughputEngineering of Multistep Enzymatic Reactions via OpticallyGuided Matrix-Assisted Laser Desorption/Ionization MassSpectrometry. J. Am. Chem. Soc. 139, 12466–12473 (2017).doi: 10.1021/jacs.7b04641; pmid: 28792758

27. M. W. Duncan, H. Roder, S. W. Hunsucker, Quantitativematrix-assisted laser desorption/ionization massspectrometry. Brief. Funct. Genomics 7, 355–370 (2008).doi: 10.1093/bfgp/eln041; pmid: 19106161

28. P. Wang, R. W. Giese, Recommendations for quantitativeanalysis of small molecules by matrix-assisted laser desorptionionization mass spectrometry. J. Chromatogr. A 1486, 35–41(2017). doi: 10.1016/j.chroma.2017.01.040; pmid: 28118972

29. J. R. Cabrera-Pardo, D. I. Chai, S. Liu, M. Mrksich, S. A. Kozmin,Label-assisted mass spectrometry for the acceleration ofreaction discovery and optimization. Nat. Chem. 5, 423–427(2013). doi: 10.1038/nchem.1612; pmid: 23609094

30. T. J. Montavon, J. Li, J. R. Cabrera-Pardo, M. Mrksich,S. A. Kozmin, Three-component reaction discovery enabled bymass spectrometry of self-assembled monolayers. Nat. Chem.4, 45–51 (2012). doi: 10.1038/nchem.1212; pmid: 22169871

31. A. B. Diagne, S. Li, G. A. Perkowski, M. Mrksich, R. J. Thomson,SAMDI Mass Spectrometry-Enabled High-ThroughputOptimization of a Traceless Petasis Reaction. ACS Comb. Sci.

17, 658–662 (2015). doi: 10.1021/acscombsci.5b00131;pmid: 26521847

32. P. S. Kutchukian et al., Chemistry informer libraries:A chemoinformatics enabled approach to evaluate andadvance synthetic methods. Chem. Sci. 7, 2604–2613 (2016).doi: 10.1039/C5SC04751J; pmid: 28660032

33. K. D. Collins, F. Glorius, A robustness screen for the rapidassessment of chemical reactions. Nat. Chem. 5, 597–601(2013). doi: 10.1038/nchem.1669; pmid: 23787750

34. J. Richardson, J. C. Ruble, E. A. Love, S. Berritt, A Methodfor Identifying and Developing Functional Group Tolerant CatalyticReactions: Application to the Buchwald-Hartwig Amination.J. Org. Chem. 82, 3741–3750 (2017). doi: 10.1021/acs.joc.7b00201; pmid: 28245358

35. E. B. Corcoran et al., Aryl amination using ligand-free Ni(II)salts and photoredox catalysis. Science 353, 279–283 (2016).doi: 10.1126/science.aag0209; pmid: 27338703

36. A. R. Korte, Y. J. Lee, MALDI-MS analysis and imaging of smallmolecule metabolites with 1,5-diaminonaphthalene (DAN).J. Mass Spectrom. 49, 737–741 (2014). doi: 10.1002/jms.3400;pmid: 25044901

37. L. J. Soltzberg, P. Patel, Small molecule matrix-assistedlaser desorption/ionization time-of-flight mass spectrometryusing a polymer matrix. Rapid Commun. Mass Spectrom.18, 1455–1458 (2004). doi: 10.1002/rcm.1505; pmid: 15216505

38. R. Shi et al., Hydroxyl-Group-Dominated Graphite DotsReshape Laser Desorption/Ionization Mass Spectrometry forSmall Biomolecular Analysis and Imaging. ACS Nano 11,9500–9513 (2017). doi: 10.1021/acsnano.7b05328;pmid: 28850220

ACKNOWLEDGMENTS

We thank A. Donofrio (Merck & Co., Inc., Kenilworth, NJ, USA)for helpful discussions. Funding: S.L. and K.Z. are grateful to theMRL Postdoctoral Research Fellows program for financial support.Additional financial support was provided by Merck & Co., Inc.,Kenilworth, NJ, USA. Author contributions: S.L., R.P.S., Z.P.,I.W.D., D.A.D., and S.D.D. designed the chemistry experiments. S.L.,R.D.F., H.S., and S.D.D. performed the chemistry experiments.S.L., R.D.F, D.V.C., T.C., I.W.D., D.A.D., and S.D.D. developed theextended nanochemistry tools. S.L., S.D., W.D.B., H.W., H.S., andC.J.W. developed the MALDI-MS analytical platform. S.L., K.Z., andS.D.D. analyzed comparative UPLC-MS data. S.L., R.P.S., Z.P.,and S.D.D. analyzed the reactivity trends. Z.P. conducted ELN datamining. S.L., S.D., T.C., I.W.D., H.S., C.J.W., and S.D.D. wrote themanuscript. Competing interests: None declared. Data andmaterials availability: All data described in this work are includedin the Excel file associated with the supplementary materials.

SUPPLEMENTARY MATERIALS

www.sciencemag.org/content/361/6402/eaar6236/suppl/DC1Materials and MethodsFigures S1 to S55Tables S1 to S8ReferencesData S1 to S5

1 December 2017; accepted 15 May 2018Published online 24 May 201810.1126/science.aar6236

Lin et al., Science 361, eaar6236 (2018) 10 August 2018 7 of 7

RESEARCH | RESEARCH ARTICLEon June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from

MALDI-TOF MSMapping the dark space of chemical reactions with extended nanomole synthesis and

Spencer D. DreherKerstin Zawatzky, Heather Wang, Tim Cernak, Ian W. Davies, Daniel A. DiRocco, Huaming Sheng, Christopher J. Welch and Shishi Lin, Sergei Dikler, William D. Blincoe, Ronald D. Ferguson, Robert P. Sheridan, Zhengwei Peng, Donald V. Conway,

originally published online May 24, 2018DOI: 10.1126/science.aar6236 (6402), eaar6236.361Science 

, this issue p. eaar6236Sciencespectrometry of nanomole-scale reaction mixtures without any need for intervening chromatography.

N coupling across a wide range of complex substrates. The product detection scheme relies on mass−conditions for C developed a protocol for rapidly screening different catalytic et al.often happens in pharmaceutical research. Lin

remains a major challenge to fine-tune reported conditions when the reactants become more structurally complex, as Chemists engaged in reaction discovery tend to report outcomes involving a few, relatively simple reactants. It

A rapid screen for complex reactants

ARTICLE TOOLS http://science.sciencemag.org/content/361/6402/eaar6236

MATERIALSSUPPLEMENTARY http://science.sciencemag.org/content/suppl/2018/05/23/science.aar6236.DC1

REFERENCES

http://science.sciencemag.org/content/361/6402/eaar6236#BIBLThis article cites 38 articles, 7 of which you can access for free

PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions

Terms of ServiceUse of this article is subject to the

is a registered trademark of AAAS.ScienceScience, 1200 New York Avenue NW, Washington, DC 20005. The title (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement ofScience

Copyright © 2018, American Association for the Advancement of Science

on June 29, 2020

http://science.sciencemag.org/

Dow

nloaded from