Data Integration Workflows Power Utility Operations

By November 16, 2020 November 23rd, 2020 No Comments

An organization works with  many systems that each hold their own data, often in different data formats. When data comes from multiple sources, data integration is an integral part of a team’s processes.  Often the data is siloed in systems, existing independent of one another within a business. Bringing that data together means it can be used in meaningful ways and to help inform business decisions. Integration and automation also give your team time to focus on tasks that are beyond the everyday functions of the business, validating the quality of the data and delivering an up-to-date single source of truth, saving you time and resources.

Applying FME to Utilities

Utility companies often have large amounts of data from multiple sources that need to be converted, transformed and validated. FME allows these companies to accomplish a variety of automation tasks across multiple workstreams, some of these include:

  • Design and Planning
  • Company Operations
  • Asset Management
  • Service Outages and Leaks
  • Distribution Networks
  • Cross Team Communication and Collaboration
  • Information Sharing Between the Public, Shareholders and Others

There are a range of ways FME can be used to integrate data across multiple applications, let’s take a look at some common uses across the utility industry.

(1) Data Migration

Utility companies often need to migrate incoming data or data from a legacy system, converting it and integrating it with new data, as a repeatable task.

  • Task: Data migration between systems where integration from GIS to CAD or CAD to GIS needs to be carried out on a regular basis, as a repeatable migration workflow and which can be easily automated using FME
  • Solution: Building a migration workflow that connects to your systems
  • Result: Data is written out to where it needs to be


(2) Data Consolidation and Quality Control

Utility companies regularly collect and analyze data from different sources whether it be from out in the field or from multiple departments within the organisation to meet stakeholder requirements. Running an automated self-serve validation portal process allows your organisation to attain accurate, complete and compliant data

  • Task: Data validation and quality control
  • Problem: Data needs to be good quality and of value to an organisation
  • Solution: Implement data portal self-serve for data validation purposes. Workspaces can use FME’s validation or data quality transformers – which includes the geometry validator or attribute validator. Giving the ability to then generate reports, PDFs, Spreadsheets or HTML pages that can be automatically delivered via email
  • Result: Data is now correct, complete and compliant to industry or organisation standards and can be used throughout the business

(3) Synchronising Data and Integration of Data

Decision makers across businesses require a complete picture of all available information to reach conclusions and make recommendations. This can be achieved through the integration of multiple systems and the delivery of automated workflows. Data integration is carried out between systems and applications from different parts of a business, external and internal sources or geospatial and business software. The output of these systems including; GIS, CAD, Bentley and Microsoft are brought together to a central repository (or kept separate and synchronized) while needing to be kept up to date and coordinated.

  • Task: Synchronize data across different systems and formats e.g. GIS, CAD, Bentley, Microsoft
  • Problem: Difficult to make decisions if you don’t have all the data available to you
  • Solution: Use FME to build data and application integration workflows
  • Result: Synchronized and up to date data from across systems that is used to make informed decisions

A workflow that builds integration between data and applications can run as an automation on FME server to synchronize the data automatically.

(4) Information Delivery and Distribution

Utility companies often have self-service portals for stakeholders to access data.  This provides high quality data in the format required by the general public, customers, design and contractors or other clients and is used to inform the public about scheduled or unscheduled outages or in times when emergency management is needed.

  • Task: Data needs to be accessible and distributed to shareholders and the public
  • Problem: Critical information demands to be delivered and distributed in a range of formats suitable to the end user
  • Solution: Data integration workflow to collect data into appropriate formats and hosted on an FME Cloud portal for sharing
  • Result: User friendly data formats that are accessible at the self-service portal

Generate Report Workflow: Water Utility Company

The FME platform is used to generate reports for technicians, where information is captured about network issues. The workspace pulls together detail about the leaks and locations of the water mains. The extent and severity are also captured while information around the nearest located valve is included so they can be turned off.

A simple workflow like this can run as an automation on FME server. The automation would be set up to listen for events or triggers that initiate the workspace. This can be done in a host of ways i.e. scheduled to run at a particular time, activate when a file is added to a  Dropbox account, via a Webhook or a directory and sending the output to an email.

(5) Beyond Basic Data Types

FME allows you to easily experiment with integrating data and delivering the output to different formats/functions or run analytics.  Some of the exciting data sources that could be leveraged include:

  • Drones/UAV
  • Augmented Reality
  • Satellite Imagery
  • Game Engines
  • Cognitive Services and Machine Learning
  • Virtual Reality and Interactive Web Visualisations
  • Sensor Data

Many businesses have similar workflows that are often repeated.  FME gives you the power to automate simple tasks and perform more complex data integration modelling that improve operational efficiency and enable data-based decision making.

There are FME licensing and subscription options to suit businesses and budgets of all sizes.  Chat to Locus to get started.

This article is a synopsis of the Safe Software Webinar; How Utilities Can Improve Operations Using Data Integration Workflows (recorded June 2020), reviewed and repurposed by Pip Norris, FME Data Solutions, Locus

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.