AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.
-
Upload
toni-lattin -
Category
Documents
-
view
231 -
download
1
Transcript of AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.
![Page 1: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/1.jpg)
AgMIP SSA MeetingAccra, Ghana
12 September, 2012
Importing and translating crop model data
![Page 2: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/2.jpg)
AgMIP IT ProductsWebsites
Public site www.agmip.org (under renovation) Research site research.agmip.org AgMIP toolshed tools.agmip.org (under construction)
Tools Climate tools Crop model data translation tools Web and desktop interfaces for modeling tools Open source repositories github.org/agmip
Databases Crop Experiment / Sentinel site (operational) Crop model outputs (operational) Crop scenarios (conceptual) Gridded data (conceptual)
![Page 3: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/3.jpg)
AgMIP harmonized databases
Model data stored in a consistent format
Filters allow import of data from various model-ready files and other data sources
Translators allow export of data to various model-ready formats
![Page 4: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/4.jpg)
Web-based data searchhttp://test.agmip.org/ace-ui2/#mappage
![Page 5: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/5.jpg)
Guide for Regional Integrated Assessments HandbookItem 3: Assemble Existing Data from
Experiments and Calibrate Crop ModelsItem 5: Assemble Data and Calibrate Crop
Models for Regional Yield Simulations and Assessments
Item 10: Archive Data and Analyses Results for Integrated Assessments
![Page 6: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/6.jpg)
Types of dataData rich field experiments for calibration
of cropping system parameterssufficiently complete to parameterize crop
modelsSurvey data – evaluation of baseline vs
future conditionsmay be incompleteassumptions based on local knowledge of
agricultural practices
![Page 7: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/7.jpg)
Steps to data preparation and translation
1. Collect raw data2. Using spreadsheet templates, transform the
data to harmonized units and variables – weather, soils, and field / management / metadata
3. Save data in csv format files4. QuadUI – desktop application for data
translation to DSSAT and APSIM formats 5. View assumptions and errors (repeat steps 2-
5 as needed)6. Run models7. Collect output data in ACMO.csv format for
input to economic models
![Page 8: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/8.jpg)
Step 1 – Collect raw dataMachakos data as an example (Survey.xls)
![Page 9: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/9.jpg)
Step 2 – Prepare crop model data3 ways to input data
DSSAT format (use Xbuild and other DSSAT tools to prepare crop model files). These can be translated directly into AgMIP harmonized format.
Use spreadsheet templates to fill in data with proper ICASA variable names and units.multiple templates will allow different kinds of data
User Interface for data entry – under development
![Page 10: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/10.jpg)
Step 2 – Survey data template
![Page 11: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/11.jpg)
Step 2 – Survey data template (continued)
Excel spreadsheet for calculations, conversions, date formatsOne line per site for metadata, field, and
management dataseparate csv file for soil dataseparate csv file for weather dataseparate csv file for regional assumed
management data (initial soil conditions, fertilizer rates, planting conditions)
See sample template (Machakos_import_2.5.xlsx)
![Page 12: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/12.jpg)
ICASA Master Variable ListHarmonized database uses ICASA Master
Variable List to interpret data definitions and units.
See ICASA_Master_Variables_10Sep2012.xlsx or online (always up-to-date) at https://docs.google.com/spreadsheet/pub?key=0AiUhvGN3ZVgXdGJud3lDNThFSnlGWGlkTjgxWXRwbWc&output=htmlliving document – go to web version for the
latestadding variables to spreadsheet templates:
use code_display column from ICASA list
![Page 13: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/13.jpg)
Step 3 - Save csv format filesSeparate csv file for each data type:
soilweathermanagement / field / metadata (initial
conditions, observed data)regional assumptions
Beware of date formats when you re-open a csv file with Excel!
Zip all csv files together for import with QuadUI
![Page 14: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/14.jpg)
Step 4 - QuaDUIQuick and Dirty User InterfaceTemporary desktop application for
conversion of spreadsheet templates or DSSAT-format files to AgMIP harmonized format, APSIM or DSSAT formats.
Near future: web-based application to upload and archive data, search and query, download in multiple formats.
![Page 15: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/15.jpg)
Step 4 – QuaDUI (continued)Open by double-clicking the
quadui.bat file and selecting the zip file and location for output files.
![Page 16: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/16.jpg)
Step 4 – QuaDUI (continued)Select format(s) for conversion• DSSAT• APSIM• JSON
Currently working on additional translators for:• STICS• Aquacrop• InfoCrop• EPIC• ORYZA
• CropSyst• Salus• WOFOST• RZWQM• Need to add SARRAH
translator!• Hit “Convert” button
![Page 17: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/17.jpg)
Step 5 – QuaDUI troubleshootingLook at crop model files for notes on
assumptions made by the translatorsLook at QuadUI log file (where the QuadUI
jar file is located) for notes on invalid variables
Application requires that headers are spelled exactly as in code_display of ICASA Master Variable List (not case-sensitive)
Variables must appear in correct file (e.g., no weather data in soils file)
![Page 18: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/18.jpg)
Step 6 – Run ModelsDSSAT – bat file provided which will run the
experiments created.APSIM – use APSIM UI
![Page 19: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/19.jpg)
Step 7 - Crop model outputsMachakos_ACMO.csv file – format for
harmonization of output from crop models.
![Page 20: AgMIP SSA Meeting Accra, Ghana 12 September, 2012 Importing and translating crop model data.](https://reader036.fdocuments.net/reader036/viewer/2022062307/551c5971550346b1458b5172/html5/thumbnails/20.jpg)
Spreadsheet template for sentinel site dataUFGA8201_MZX_trt1.xlsx
Legend:
Symbol Location Description! Column 1 Comment
#Column 1, headers
Summary data variable names
%Column 1, headers
Series data variable names (time series or soil layer data)
Unique id (1, 2, A, B, …)
First field of data
Relational data requires unique id to show relationships between summary and series sections of data
*First field of data
Complete data (one line per experiment)