FME Server Tutorial - FME Data Conversion - Convert GIS, CAD
FME Server Workspace Patterns - Continued
-
Upload
safe-software -
Category
Technology
-
view
173 -
download
2
Transcript of FME Server Workspace Patterns - Continued
![Page 1: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/1.jpg)
FME Server Workspace Patterns - ContinuedReed WhittingtonBurns & McDonnell
![Page 2: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/2.jpg)
About Us
FME Certified Pro ServicesBurns & McDonnell, Dallas, TX
Reed Whittington
Burns & McDonnell is a full-service engineering, architecture, construction, environmental and consulting solutions firm.
Our multidiscipline staff of more than 5,000 employee-owners includes engineers, architects, construction experts, planners, estimators, economists, technicians and scientists, representing virtually all design disciplines.
![Page 3: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/3.jpg)
Business Technology Services Burns and McDonnell Technology Consulting Services is a virtual team of technical experts around the globe offering a wide array of services and deliverables including:• Web Sites/Information Management Portals/Doc Management solutions• 2d/Web Maps/3D/4D Globe Applications• Field Data Collection/Inspections/Real-Time Monitoring• CAD/GIS/Database Integration• Automated Report Generation and Distribution• Project Metrics/Analytics/KPI Dashboards
Integrated, field tested, project proven solutions
Many of these services are bundled under OneTouchPM which is a hybrid cloud based, web enabled, managed service allowing project managers to engage their customers and share information, summary progressing with aggregated executive views from all disciplines.
![Page 4: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/4.jpg)
What is a pattern?
In software engineering, a software design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. It is a description or template for how to solve a problem that can be used in many different situations. Not a specific implementation; design patterns are formalized best practices that the programmer can use to solve common problems when designing an application or system.https://en.wikipedia.org/wiki/Software_design_pattern
![Page 5: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/5.jpg)
4 FME Server Workspace Patterns• Workflow Submitter Iterator (or Work Status)
• Strategy for Asynchronous processing of large datasets• Runner / Controller Workspace (or Tool Chain)
• General approach using FMEServerJobSubmitter for workflow control
• Dataset Tool Switch (or Multi-Pass)• Strategy for Synchronous Sequential processing• Operates on same dataset with different modes of operation
• Flatten, Explode, Export (or Import/Merge/Publish)• 3 stages of processing typically used for master data management
![Page 6: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/6.jpg)
Submitter Iterator with Status• Uses a selection strategy to select mutually exclusive
parts of a source data set so that the parts can be operated on independently and run on different fme engines.
• Used to process files or db operations so that sql select statements can be used.
• Needs a better name• Work Status, Delegator, …
• This was the subject of a previous presentation.• http://cdn.safe.com/events/FME2011WorldTour/Austin_Reed_Whittington.pdf
![Page 7: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/7.jpg)
Example using Path reader to Iterate and track status.can also use url as path or list of SQL queries
Submitter Iterator with StatusWait=No
![Page 8: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/8.jpg)
Runner / Controller Workspace• General strategy for Job Control and Tool Chain• Lets you run the controller on desktop• FME 2015 improves FMEServerJobsubmitter:
• In Engine Sequential Run with Wait=Yes• Output datasets to temp files or as specified
Excellent Article by DeWet@Safe:https://knowledge.safe.com/articles/1439/fme-server-workflow-management-1.html
![Page 9: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/9.jpg)
Dataset Tool Switch (aka multi-pass)• The “Multi-pass" pattern presents a common interface
to iterate over datasets using successive refinement. • Meant to be run multiple times with different RUN_MODEs• RUN_MODE Choice defines the Switch behavior• TestFilter on RUN_MODE controls Feature RoutingUseful for
processing directory/tree of files and/or database exports or publishing.
• Interface includes:• SourceDir, StagingDir, Run Modes, DB connection, Publish
location
![Page 10: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/10.jpg)
Use the Path reader to Iterate and Route based on Run Mode
![Page 11: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/11.jpg)
Flatten, Explode, Export• 3 separate workspaces that act together but run asynchronous• Each may use “multi-pass”• Flatten
• Denormalization into one big table• Pack attributes into dictionaries• Think NoSQL and Operational Data Store
• Explode• Unpacks big table• Data model mapping and normalization happens here• May use templates for standard data models• Might also use Dynamic Schema
![Page 12: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/12.jpg)
Flatten, Explode, Export (cont)• Export• Database to files (.kmz, .csv, .json, .gdb, …)• Heavy use of Fanout (ds and ft)• May use Tool Switch RUN_MODE to define
export type or staging/publish processing steps
![Page 13: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/13.jpg)
More possibilities with FME 2016…What do you use?
Tool Chain Workspace• FeatureReader
– Selective reading/caching– Error Checking
• FeatureWriter– Write then read back in
same .fmw– In workspace tool chain– Removes the need for temp
output files of the FMEServerJobSubmitter
Before FME 2016• Required chaining
FMEJobSubmitters together
• Next in chain reads output of Previous
![Page 14: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/14.jpg)
Main point: FME Server Patterns help you develop and organize your data management tools.
![Page 15: FME Server Workspace Patterns - Continued](https://reader035.fdocuments.net/reader035/viewer/2022062821/588896761a28ab3e658b7cff/html5/thumbnails/15.jpg)
Thank you!Reed WhittingtonFME Certified Pro ServicesBurns & McDonnellDallas, [email protected]