Automatically Extracting Action Graphs from Materials ...strubell/doc/nips17ws-action-graphs.pdf ·...

Automatically Extracting Action Graphs From Materials Science Synthesis Procedures

December 8, 2017 NIPS Workshop on ML for Molecules and Materials

Sheshera Mysore Edward Kim Emma Strubell Ao Liu Haw-Shiuan Chang Srikrishna Kompella Kevin Huang Andrew McCallum Elsa Olivetti

Automatically Extracting Action Graphs from Materials Science Synthesis Procedures

Sheshera Mysore1, Edward Kim2, Emma Strubell1, Ao Liu1, Haw-Shiuan Chang1, Srikrishna Kompella1, Kevin Huang2, Andrew McCallum1, Elsa Olivetti2

1College of Information and Computer Sciences, University of Massachusetts Amherst2Department of Materials Science and Engineering, Massachusetts Institute of Technology

Summary

• Want to analyze, predict inorganic materials synthesisprocedures at large scale.

• Unlike organic synthesis, inorganic procedures exist only asnatural language narratives in scientific journal articles.

• We automatically extract structured representations ofsynthesis procedures from materials science article text usinga pipeline of supervised and unsupervised NLP methods.

Overall extraction pipeline architecture

… the suspension was transferred to an

autoclave and sealed…

PDF to textEntity extraction

Dependency parsing

Event extraction(suspension, transferred, autoclave) (suspension, sealed, autoclave) (suspension, maintained, autoclave)

Edge induction

Entity extraction

Let x = [x1, . . . , xT ] be a sentence of input text and y =[y1, . . . , yT ] be per-token output tags. We predict the most likelyy, given a conditional model P (y | x). We experiment with twofactorizations of P (y | x). First:

P (y|x) =TY

t=1P (yt|F (x)), (1)

tags are conditionally independent given features F (x), encodedby a deep neural network (DCNN, Bi-LSTM). Second:

P (y|x) = 1Zx

TY

t=1Ât(yt|F (x))Âp(yt, yt≠1), (2)

a linear-chain CRF that couples all of y together, enforcing con-straints between labels during prediction w/ local factor Ât, pair-wise factor fÂp, partition function Zx (La�erty et al., 2001). F (x)is parameterized by a neural network (Bi-LSTM-CRF) or a log-linear model over sparse binary features (CRF-ling, CRF-hand,CRF-both) indicating e.g. lexicon membership, part-of-speech.

Example action graph

Typical synthesis procedure textIn a typical procedure for the synthesisof —-MnO2 nanowires, 2.5 mL of 50wt.% Mn(NO3)2 solution was dilutedto 25.0 mL, and ozone was fed intothe bottom of the solution for 30 minunder vigorous stirring. With the in-draught of ozone, black solid appearedgradually and the clear solution turnedinto black slurry finally. Then the sus-pension was transferred into an auto-clave of 48.0 mL, sealed and main-tained at 200 ¶C for 8 h. After this,the autoclave was cooled to room tem-perature naturally. The resulting solidproducts were washed with water, anddried at 120 ¶C for 8 h.

Event extraction

We extract events from sentences by applying heuristic rules on the syntacticdependency parse of the sentence. The most important of these splits everyphrase whose head links to the root with a conj relation. All other tokens areassociated with the root and constitute the main phrase. Each split phrase andthe main phrase is considered an event.Example dependency parse: Post-processed parse

representing 3 events:

Edge induction

We consider two methods for edge induction: Our baseline(Seq) simply attaches events in the order they occur in thetext; Our generative probabilistic model (Prob) is based onthat of Kiddon et al. (2015), which uses hard EM to learnan attachment model given strong priors based on parse treestructure and typical attachment in procedural text.

Experimental results

Entity extraction:

Model Precision Recall F1CRF-ling 76.98 67.41 71.88CRF-hand 75.59 69.32 72.48CRF-both 74.97 72.12 73.52DCNN 77.85 77.16 77.50Bi-LSTM 74.25 77.83 76.00Bi-LSTM-CRF 74.64 80.74 77.57

Action graph extraction:

Setting 1: Ignore edges between unaligned events.Model Aligned Unaligned Precision Recall F1

End-to-end evaluationSeq 39.85% 30.95% 73.04 94.38 82.35

Prob 39.85% 30.95% 68.38 89.89 77.67Perfect node segmentationSeq 63.80% 0% 99.29 99.29 99.29

Prob 63.80% 0% 95.36 95.36 95.36

Setting 2: Penalize edges between unaligned events.Model Aligned Unaligned Precision Recall F1



Prob 63.80% 0% 95.36 88.70 91.91

2

Goal: automatically extract materials synthesis procedures from text.

2


• Want to accelerate materials science via large-scale analysis, prediction of inorganic synthesis routes

2


b-MnO2 nanowires: A novel ozonation catalyst for water treatment

Yuming Dong a, Hongxiao Yang a,c, Kun He b, Shaoqing Song a, Aimin Zhang a,*a School of Chemistry and Chemical Engineering, Key Laboratory of Mesoscopic Chemistry, Nanjing University-Jinchuan Group Ltd. Joint Laboratory of Metal Chemistry,Nanjing University,22 Hankou road, Nanjing, Jiangsu Province 210093, PR Chinab Department of Earth Science, Nanjing University, Nanjing 210093, PR Chinac Analysis and Testing Center, Nanjing Normal University, Nanjing 210097, PR China

1. Introduction

For its great significance, the degradation of organicpollutants in waste water is one of the focuses in watertreatment. Kinds of advanced oxidation technologies, such asphoto-catalysis, wet-oxidation and catalytic ozonation havebeen developed. In these oxidation processes, many nanoma-terials were proposed as heterogeneous catalysts for their welldispersal and high efficiency. For example, TiO2 nanomaterialswere developed as photo-catalyst [1–5]; other nanomaterialssuch as ZnO, ZnS, SrTiO3–Fe2O3 and Co3O4 were used in catalyticozonation process or photocatalytic methods [6–10]. However,most nanomaterials used currently in water treatment arenanoparticles, which are difficult to be separated from water.This becomes the main limitation for the application ofnanomaterials in this field [11–15]. Therefore, it is verydesirable to develop novel catalysts with good separability aswell as remarkable catalysis for water treatment. One-dimen-sional (1D) nanostructures such as nanowire and nanofibers areeasily sedimentated in water due to their large 1D size and highaspect ratios. Although there are only a few reports about theapplication of 1D nanostructure in water treatment [16–18], the

usage of 1D nanostructure may supply a kind of novel andpractical catalysts.

Recently, the synthesis of b-MnO2 1D nanostructures hasattracted much attention for their novel potential properties. Itis found that b-MnO2 is one of the metal oxides which are easierto be present as nanowires [19–23], and b-MnO2 has potentialutility as catalyst, ion-sieves and electrode materials. There aremany reports on the usage of b-MnO2 nanostructure as catalyst[17,18,24–27]. For example, it was found that b-MnO2

nanomaterials had catalytic performance on H2O2 decomposi-tion [17,18,26,27]. It is worth to note that, b-MnO2 1Dnanostructures (nanorods) revealed good catalysis activity onthe degradation of dye in water in the presence of H2O2 [17,18].Herein, using Mn(NO3)2 and ozone as raw materials, b-MnO2

nanowires were obtained through a facile hydrothermal route.In this method, as an oxidant which cannot introduce anyimpurities, ozone could be generated instantly and convenientlyfrom air or oxygen, avoiding dangerous factors from thepreservation of strong oxidants.

Being a strong oxidation process, catalytic ozonation is efficientand practical for the degradation of organic pollutants [28–30]. Inthis paper, as-prepared b-MnO2 nanowires were applied as acatalyst for the degradation of phenol by ozone. b-MnO2

nanowires revealed remarkable catalysis for the degradation ofphenol and the removal of chemical oxygen demand (COD), whichdenotes a promising prospect in water treatment.

Applied Catalysis B: Environmental 85 (2009) 155–161

A R T I C L E I N F O

Article history:Received 15 January 2008Received in revised form 7 July 2008Accepted 8 July 2008Available online 15 July 2008

Keywords:b-MnO2 nanowiresCatalytic ozonationPhenolOne-dimensional nanostructuresSeparability

A B S T R A C T

Using Mn(NO3)2 and ozone as raw materials, b-MnO2 nanowires with diameters of about 6–12 nm,lengths of 2–5 mm and surface area of 73.54 m2 g!1 were synthesized by a simple hydrothermal process.The influences of synthesis conditions such as hydrothermal temperature, reaction time and ozone wereinvestigated, and the growth process of b-MnO2 nanowires was discussed. The catalytic properties of b-MnO2 nanowires for the degradation of phenol were evaluated. b-MnO2 nanowires revealed goodseparability and remarkable catalysis for the degradation of phenol.

! 2008 Elsevier B.V. All rights reserved.

* Corresponding author. Tel.: +86 25 83686235; fax: +86 25 83317761.E-mail address: [email protected] (A. Zhang).

Contents lists available at ScienceDirect

Applied Catalysis B: Environmental

journa l homepage: www.e lsev ier .com/ locate /apcatb

0926-3373/$ – see front matter ! 2008 Elsevier B.V. All rights reserved.doi:10.1016/j.apcatb.2008.07.007


• Unlike organic synthesis, no tabulation of synthesis routes — have to read papers!

2




1. Introduction












A B S T R A C T








2. Experimental

2.1. Materials and preparation

All chemicals were of analytical grade and were used asreceived without further purification. The water used throughoutwas distilled water. Ozone was generated in a laboratory ozonegenerator from pure oxygen, and the flow rate of ozone was0.30 mg min!1.

In a typical procedure for the synthesis of b-MnO2 nanowires,2.5 mL of 50 wt.% Mn(NO3)2 solution was diluted to 25.0 mL, andozone was fed into the bottom of the solution for 30 min undervigorous stirring. With the indraught of ozone, black solidappeared gradually and the clear solution turned into blackslurry finally. Then the suspension was transferred into anautoclave of 48.0 mL, sealed and maintained at 200 8C for 8 h.After this, the autoclave was cooled to room temperaturenaturally. The resulting solid products were washed with water,and dried at 120 8C for 8 h. The obtained products were collectedfor the following characterization.

For comparison of catalysis activity, we also synthesized b-MnO2 nanowires and dandelion-like b-MnO2 microstructuresaccording to literatures [20,31]. The b-MnO2 bulk phase materialand TiO2 nanoparticles (P25) used were from commercial sources.

2.2. Characterization

The compositions of the products were identified by powder X-ray diffraction (XRD) on an X’TRA X-ray diffractometer (ARL,Switzerland) using Cu Ka radiation (l = 1.5418 A). The morphol-ogies and sizes of the products were examined by transmissionelectron microscopy (TEM) with a JEM-200CX (JEOL, Japan)transmission electron microscope at an accelerating voltage of200 kV. The high-resolution transmission electron microscopy(HRTEM) images were obtained by a 2010 high-resolutiontransmission electron microscope (JEOL, Japan). The nitrogenadsorption and desorption isotherms at 77 K were measured usingan ASAP2020 surface area and porosity analyzer (Micromeritics,USA). The Brunauer–Emmett–Teller (BET) specific surface area wascalculated using isotherms data and the pore size distribution(PSD) curves were calculated from the analysis of the desorptionbranch of the isotherm using the Barrett–Joyner–Halenda (BJH)algorithm. The sedimentation rates of nanomaterial in water weredetermined through a 50-mL measuring cylinder and stopwatch,with 0.05 g nanomaterials in 50 mL of water. The X-ray photo-electron spectroscopy (XPS) was obtained by a 250 ElectronSpectrometer for Chemical Analysis (Thermo, ESCALAB, USA).

2.3. Study of catalytic properties

The catalytic degradation experiments of phenol in thepresence of ozone were performed in a semi-continuous flowmode at 20 8C. Ozone was generated in a laboratory ozonegenerator using pure oxygen as gas source. The flow rate of oxygenwas 15 mL min!1, and the flow rate of ozone was 0.30 mg min!1.The simulative waste water was phenol solution (the initialconcentration of phenol was 100 mg L!1), and the initial pH was6.35 without any adjustment. In a typical catalytic degradationprocedure, 0.16 g catalyst and 160 mL of simulative waste waterwere mixed in a flask under stirring and thermostatic control. Thenozone was fed into the bottom of the flask with continuous stirring.After certain intervals, samples (3.0 mL) were taken from thereactor, and the clear solutions after separation were used foranalytical determination. The control experiments of singleozonation (without catalyst) were also carried out under the same

conditions. The concentrations of phenol were determined by a LC-10AD high performance liquid chromatography (HPLC, Shimadzu)with a reversed-phase 4.6 mm " 250 mm C18 waters column(Cosmosil, Japan) at room temperature. Elution was carried out bypumping methanol and water (5:5 v/v) at a flow rate of1.0 mL min!1. An attached SPD-10A UV–vis detector (Shimadzu)at 210 nm wavelength was used. The values of chemical oxygendemand (COD) were obtained through oxidation with K2Cr2O7

under acidic conditions and titrate analysis with (NH4)2Fe(SO4)2

aqueous solution according to national criterion of P.R. China [32].The relative concentrations of ozone in water were determinedwith a UV–vis spectrophotometer (752, Shanghai, China) byrecording the absorption at 258 nm in a 1-cm quartz cell [33,34].The total concentration of manganese ions was determined with J-A1100 Inductively Coupled Plasma (ICP, Jarrell-Ash, USA).

3. Results and discussion

3.1. Characterization of b-MnO2 nanowires

According to the conditions described in Section 2.1, ozone wasfed into the bottom of the 25 mL of 5 wt.% Mn(NO3)2 solution for30 min, then the suspension was transferred into an autoclave of48.0 mL, sealed and maintained at 200 8C for 8 h. The obtainedproducts were collected for the following characterization.

The XRD pattern of as-obtained products is shown in Fig. 1. Allthe observed diffraction peaks can be assigned to a pure tetragonalphase (JCPDS24-0735). The results denoted that the productsobtained by this method were pure b-MnO2. The TEM image of as-obtained b-MnO2 nanowires is shown in Fig. 2a. It can be seen that,the products were uniform nanowires with diameters of about 6–12 nm and lengths of 2–5 mm. It can be said that the nanowireshave a high aspect ratio and a narrow distribution of diameters.The HRTEM of a single nanowire and its corresponding fast-Fourier-transform (FFT) pattern are shown in Fig. 2b and c. Theinterlayer distance was calculated as about 0.320 nm, whichcorresponds to the 1 1 0 plane of tetragonal b-MnO2. The HRTEMimage and its FFT pattern further confirmed the good crystal-lization of nanowires and indicated that nanowires extended alongthe 1 1 0 direction.

The BET specific surface area of b-MnO2 nanowires wasdetermined as 73.54 m2 g!1. The sedimentation rate of b-MnO2

nanowires in water was calculated as 8.0 " 10!2 cm s!1. In order togive a visual understanding to the separability of b-MnO2

nanowires, we also determined the sedimentation rate ofcommercial TiO2 nanoparticles (P25), which is a well known

Fig. 1. XRD pattern of b-MnO2 nanowires. The products were obtained through firstozonation of 5 wt.% Mn(NO3)2 solution and subsequent hydrothermal reaction at200 8C for 8 h.

Y. Dong et al. / Applied Catalysis B: Environmental 85 (2009) 155–161156



2




1. Introduction












A B S T R A C T








2. Experimental

2.1. Materials and preparation

All chemicals were of analytical grade and were used asreceived without further purification. The water used throughoutwas distilled water. Ozone was generated in a laboratory ozonegenerator from pure oxygen, and the flow rate of ozone was0.30 mg min!1.

In a typical procedure for the synthesis of b-MnO2 nanowires,2.5 mL of 50 wt.% Mn(NO3)2 solution was diluted to 25.0 mL, andozone was fed into the bottom of the solution for 30 min undervigorous stirring. With the indraught of ozone, black solidappeared gradually and the clear solution turned into blackslurry finally. Then the suspension was transferred into anautoclave of 48.0 mL, sealed and maintained at 200 8C for 8 h.After this, the autoclave was cooled to room temperaturenaturally. The resulting solid products were washed with water,and dried at 120 8C for 8 h. The obtained products were collectedfor the following characterization.

For comparison of catalysis activity, we also synthesized b-MnO2 nanowires and dandelion-like b-MnO2 microstructuresaccording to literatures [20,31]. The b-MnO2 bulk phase materialand TiO2 nanoparticles (P25) used were from commercial sources.

2.2. Characterization

The compositions of the products were identified by powder X-ray diffraction (XRD) on an X’TRA X-ray diffractometer (ARL,Switzerland) using Cu Ka radiation (l = 1.5418 A). The morphol-ogies and sizes of the products were examined by transmissionelectron microscopy (TEM) with a JEM-200CX (JEOL, Japan)transmission electron microscope at an accelerating voltage of200 kV. The high-resolution transmission electron microscopy(HRTEM) images were obtained by a 2010 high-resolutiontransmission electron microscope (JEOL, Japan). The nitrogenadsorption and desorption isotherms at 77 K were measured usingan ASAP2020 surface area and porosity analyzer (Micromeritics,USA). The Brunauer–Emmett–Teller (BET) specific surface area wascalculated using isotherms data and the pore size distribution(PSD) curves were calculated from the analysis of the desorptionbranch of the isotherm using the Barrett–Joyner–Halenda (BJH)algorithm. The sedimentation rates of nanomaterial in water weredetermined through a 50-mL measuring cylinder and stopwatch,with 0.05 g nanomaterials in 50 mL of water. The X-ray photo-electron spectroscopy (XPS) was obtained by a 250 ElectronSpectrometer for Chemical Analysis (Thermo, ESCALAB, USA).

2.3. Study of catalytic properties

The catalytic degradation experiments of phenol in thepresence of ozone were performed in a semi-continuous flowmode at 20 8C. Ozone was generated in a laboratory ozonegenerator using pure oxygen as gas source. The flow rate of oxygenwas 15 mL min!1, and the flow rate of ozone was 0.30 mg min!1.The simulative waste water was phenol solution (the initialconcentration of phenol was 100 mg L!1), and the initial pH was6.35 without any adjustment. In a typical catalytic degradationprocedure, 0.16 g catalyst and 160 mL of simulative waste waterwere mixed in a flask under stirring and thermostatic control. Thenozone was fed into the bottom of the flask with continuous stirring.After certain intervals, samples (3.0 mL) were taken from thereactor, and the clear solutions after separation were used foranalytical determination. The control experiments of singleozonation (without catalyst) were also carried out under the same

conditions. The concentrations of phenol were determined by a LC-10AD high performance liquid chromatography (HPLC, Shimadzu)with a reversed-phase 4.6 mm " 250 mm C18 waters column(Cosmosil, Japan) at room temperature. Elution was carried out bypumping methanol and water (5:5 v/v) at a flow rate of1.0 mL min!1. An attached SPD-10A UV–vis detector (Shimadzu)at 210 nm wavelength was used. The values of chemical oxygendemand (COD) were obtained through oxidation with K2Cr2O7

under acidic conditions and titrate analysis with (NH4)2Fe(SO4)2

aqueous solution according to national criterion of P.R. China [32].The relative concentrations of ozone in water were determinedwith a UV–vis spectrophotometer (752, Shanghai, China) byrecording the absorption at 258 nm in a 1-cm quartz cell [33,34].The total concentration of manganese ions was determined with J-A1100 Inductively Coupled Plasma (ICP, Jarrell-Ash, USA).

3. Results and discussion

3.1. Characterization of b-MnO2 nanowires

According to the conditions described in Section 2.1, ozone wasfed into the bottom of the 25 mL of 5 wt.% Mn(NO3)2 solution for30 min, then the suspension was transferred into an autoclave of48.0 mL, sealed and maintained at 200 8C for 8 h. The obtainedproducts were collected for the following characterization.

The XRD pattern of as-obtained products is shown in Fig. 1. Allthe observed diffraction peaks can be assigned to a pure tetragonalphase (JCPDS24-0735). The results denoted that the productsobtained by this method were pure b-MnO2. The TEM image of as-obtained b-MnO2 nanowires is shown in Fig. 2a. It can be seen that,the products were uniform nanowires with diameters of about 6–12 nm and lengths of 2–5 mm. It can be said that the nanowireshave a high aspect ratio and a narrow distribution of diameters.The HRTEM of a single nanowire and its corresponding fast-Fourier-transform (FFT) pattern are shown in Fig. 2b and c. Theinterlayer distance was calculated as about 0.320 nm, whichcorresponds to the 1 1 0 plane of tetragonal b-MnO2. The HRTEMimage and its FFT pattern further confirmed the good crystal-lization of nanowires and indicated that nanowires extended alongthe 1 1 0 direction.

The BET specific surface area of b-MnO2 nanowires wasdetermined as 73.54 m2 g!1. The sedimentation rate of b-MnO2

nanowires in water was calculated as 8.0 " 10!2 cm s!1. In order togive a visual understanding to the separability of b-MnO2

nanowires, we also determined the sedimentation rate ofcommercial TiO2 nanoparticles (P25), which is a well known

Fig. 1. XRD pattern of b-MnO2 nanowires. The products were obtained through firstozonation of 5 wt.% Mn(NO3)2 solution and subsequent hydrothermal reaction at200 8C for 8 h.

Y. Dong et al. / Applied Catalysis B: Environmental 85 (2009) 155–161156






Summary








Dependency parsing


Edge induction

Entity extraction


P (y|x) =TY

t=1P (yt|F (x)), (1)


P (y|x) = 1Zx

TY





Event extraction



Edge induction



Entity extraction:






Prob 63.80% 0% 95.36 95.36 95.36




Prob 63.80% 0% 95.36 88.70 91.91

Overall pipeline architectureb-MnO2 nanowires: A novel ozonation catalyst for water treatment


1. Introduction












A B S T R A C T










PDF to text



1. Introduction












A B S T R A C T











Dependency parsing



1. Introduction












A B S T R A C T











Dependency parsing

Event extraction

(suspension, transferred, autoclave) (suspension, sealed, autoclave) (suspension, maintained, autoclave)



1. Introduction












A B S T R A C T











Dependency parsing

Event extraction


Edge induction



1. Introduction












A B S T R A C T








Entity extraction



Entity extraction

… the suspension was transferred to an autoclave and sealed…MATERIAL OPERATION OPERATIONAPPARATUS



Entity extraction


susp

ensio

n NNwas VBD

trans

ferredVBN to

PRP

suspension was transferred to an…

an DET

…



Entity extraction


susp

ensio

n NNwas VBD

trans

ferredVBN to

PRP

suspension was transferred to an…

an DET

…

MAT OPERO O O

Event extraction


Event extraction• Rules over dependency parse tree + entities

Edge induction


Edge induction

Baseline sequential model:


Edge induction

Baseline sequential model:• Attach each event to the previous event in text.


Edge induction


Unsupervised probabilistic model:

• Attach each event to the previous event in text.


Edge induction


Unsupervised probabilistic model:• Define prior over connections: P(C)



Edge induction



- Example: given dependency label, probability of incoming connections?



Edge induction



- Example: given dependency label, probability of incoming connections?• Model probability of entities, text given connections: P(S|C)



Edge induction




- Example: ozone more likely to be added while stirring than while heating



Edge induction




- Example: ozone more likely to be added while stirring than while heating - Example: suspension more likely an intermediate than a raw ingredient



Edge induction





• Maximize P(S,C) = P(S|C)P(C) w/ hard EM



Edge induction


Unsupervised probabilistic model: [Kiddon et al. 2015, food recipes]• Define prior over connections: P(C)



• Maximize P(S,C) = P(S|C)P(C) w/ hard EM





Summary








Dependency parsing


Edge induction

Entity extraction


P (y|x) =TY

t=1P (yt|F (x)), (1)


P (y|x) = 1Zx

TY





Event extraction



Edge induction



Entity extraction:






Prob 63.80% 0% 95.36 95.36 95.36




Prob 63.80% 0% 95.36 88.70 91.91

Visit our poster for details and results!



1. Introduction












A B S T R A C T








Thank you!

Automatically Extracting Action Graphs from Materials ...strubell/doc/nips17ws-action-graphs.pdf ·...

Documents

Transcript of Automatically Extracting Action Graphs from Materials ...strubell/doc/nips17ws-action-graphs.pdf ·...