Category

Blog

Our FME 2020.0 Top Picks!

The latest version of FME is here, with some significant improvements to the 2020 release giving you greater performance, flexibility and accessibility to the power of FME.

We asked our Locus FME Certified technical team to take a closer look at some of the 2020.0 enhancements.

FME Server

Access to FME Server has never been easier, with new options for authentication and the ability to run jobs from a mobile device. This gives you the power to check on or run jobs remotely, monitor logs, or even build a data capture service via mobile. Download the FME Data Express mobile application now to get started.

FME Server Applications can now use authentication, giving you greater flexibility with security whilst still opening up the power of FME Server to a wider audience. This can be controlled on the application level, letting you choose which apps are shared with different individuals or even roles or groups.

A key enhancement to FME Server apps is also the ability for users to input geometry with a map picker as a published parameter. This enhances the user experience of running server apps, and opens your workspaces to an even wider audience by allowing non-technical staff to select areas of interest, straight out of the box.

 

FME Desktop

As always, FME Desktop has undergone significant performance enhancements. When upgrading your version of FME Desktop, always check to look for upgradeable transformers which will improve the efficiency of your workspace. The Statistics Calculator is roughly 100x faster than it’s last version, so it’s definitely worth upgrading sooner rather than later.

Two of the most used formats; Shapefile and Excel reader/writers are now lightning fast. These popular formats use new underlying data structures, so your workflows that connect to these formats will run more efficiently than ever.  After upgrading, when you open your workspaces that use these formats, make sure you update the reader/writers to the newer version.

You also now have the ability to apply custom connection line styles in your FME Workspaces to enhance the user experience when building up your workflows.

Like any release of FME there has been a series of new formats added for both Reading and Writing. Connect to Big Data systems, spatial data and 3D models, XML and JSON formats, and more.

New readers, writers, connectors, and transformers in FME 2020.0 include:

  • Azure and Google cognitive services for natural language processing and computer vision
  • CityJSON (Tech Preview)
  • Entwine Point Tile (EPT)
  • Google BigQuery
  • Mapbox Vector Tiles (MVT)
  • Microsoft DirectDraw Surface (DDS)
  • NIfTI (for MRI data)
  • OGC S-121 GML
  • PROJ transformers (leverage more coordinate systems by connecting to the PROJ reprojection library)
  • Revit (including floor plans and easy wall simplification)
  • Snowflake
  • UK MasterMap variants: Water, Highways, Sites, Topography
  • XSD-driven XML

Article Author: Kieran O’Donnell, FME Data Solutions, Locus

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

Integrating FME with ArcGIS Online

ArcGIS Online is a great tool for hosting your data online, and building web maps or apps which you can use to share data with your users. Using FME, you can make sure you’re getting the most from your ArcGIS Online subscription by building some really great workflows to keep your data up to date, pull data down into your organisation, or even administer your ArcGIS Online account. 

ArcGIS Online Feature Service Writer 

The ArcGIS Online Feature Writer is one of many ways you can use FME to integrate with ArcGIS Online. Using this writer, you can create, update or remove data which is in your ArcGIS Online organization. For many organisations publishing datasets to ArcGIS Online creates a bit of a data management headache, as Hosted Feature Services are duplicates of data which is maintained internally.  

Using FME however, you can create a simple process to read from your internally maintained dataset, and write directly to the feature service. By running this process on a regular basis, the data hosted in ArcGIS Online will stay up to date and you can have confidence in all your data without a burdensome process to maintain. 

ArcGIS Online Feature Service Reader 

Using the ArcGIS Online Reader, you can also access any Hosted Feature Service you have published. By adding this into an FME Workspace, you can read the data like any other format. This is particularly useful if you have projects utilising ESRI’s Collector application, and FME even allows you to download the attachments stored against each feature. 

Just like using the Feature Service Writer, you can keep two datasets in-sync without needing to constantly process the data yourself. In this case, the master dataset is held in the ArcGIS Online Feature Service, and an internal dataset is the replica which you use to display in your in-house GIS systems. 

ArcGIS REST API 

Another way to integrate FME with ArcGIS Online is using the REST API. This is slightly more advanced, so users who are unfamiliar with accessing APIs in FME should check out our FME and the art of the API blog article. We’ve also put together a sample workspace you can use to get started. Some instructions are included in the workspace to set this up to work with your own ArcGIS Online account. 

The ArcGIS REST API has a wealth of functions you can use to perform geospatial, mapping and administrative functions using your ArcGIS Enterprise organisation or ArcGIS Server. Using FME, you can make HTTP Calls to the ArcGIS REST API to perform these functions in an automated fashion. 

In the sample workspace, the process takes you through a fairly simple workflow to perform a query on your ArcGIS Online organisational account. Like most APIs you need to authenticate with ArcGIS Online first to fetch a token, in order to validate you have permissions to perform this function. 

Next, the workspace performs another request to the REST API, querying the portal with the user parameter you have set. This could be for content owned by a specific user, containing a certain tag, or a combination of criteria – the potential is endless. After making the request, the JSON is unpacked and split into individual records. In this case, the output is written to a spreadsheet of items which do or don’t contain tags, but again you can perform any function you wish. Querying the organisation is a great way to keep on top of data or items hosted in ArcGSI Online, in order to ensure duplicates aren’t kept and metadata standards are adhered to over time. 

The functions aren’t just limited to querying the portal though. You can use FME to publish services, create new items, or migrate content from one user account to another. If you’re willing to spend a little time developing these processes, you can get some great functionality which will save you plenty of time and effort in administering your organisational account, and ensure it’s always kept up to date. More information on the ArcGIS REST API. 

Article Author: Kieran O’Donnell, FME Data Solutions, Locus

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

FME and the art of the API

Far from the typical transfer of shapefiles and spreadsheets, a new era of web resources and APIs is coming. As GIS professionals, this has typically been a skillset not needed within our domain – but with the increase in web based technologies and a need to constantly pull resources from external sources, APIs are definitely something worth exploring.

What is an API

An API can vary greatly, depending on what it’s for. But put simply, it’s a set of functions and procedures, allowing the creation of applications that access the features or data of an operating system, application, or other service. These days pretty much any data service you’re accessing will have an API in order for you to query the system for data that you need.

What format can I access?

The format of an API’s data depends largely on the system you’re accessing, but JSON is likely to be the most common format you’ll come across. For this reason, we’ll explore a few API examples which return their data in JSON. The key challenge, is in turning a large chunk of text into tabular data, separated into distinct columns and records. Once you’re comfortable with this, using FME it’s really simple to write that data into a typical GIS format like a shapefile, or a File Geodatabase.

Example: Auckland Transport Developer Portal

The Auckland Transport Developer Portal is a great place to start exploring the use of APIs. In this example, we’re going to look at the Realtime Transit Feed (GTFS) which gives users the potential to query the locations of Auckland Transport’s vehicles in realtime.

 Most developer portals have the option to try the API and return sample data in order for the user to verify it’s what they’re looking for. This is a great place to start, as users can make a sample request with correct syntax, and check out what data is returned really easily.

Here you can see with a very simple request, and using an API key unique to the user, you can return a lot of information – including spatial attributes. This response is formatted as JSON, which is returned as a single chunk of text.

Using FME to access an API

Within FME, you can replicate exactly the same request made above via the developer portal. Using the HTTP Caller transformer, you can make web requests to return data like above, and then use a series to unpack the data into the correct columns and rows.

This chain of transformers takes the nested JSON and flattens the attributes into legitimate attributes. Next it explodes the list into individual features, which you can see by the feature counts changing dramatically. At this step, one feature enters which is exploded into the 1150 features that the JSON text actually makes up. 

Lastly, we use a Vertex Creator and Coordinate System Setter to create geometry using the attributes that the API returns, and sets the coordinate system in order to display it correctly on a map. The result, is that the JSON we originally started with has been split into individual records which can be displayed on a map and queried, like any other dataset. From here, it’s simple to output the data to any format that FME supports. 

APIs are a great way to keep some of your datasets up to date, as you can build processes which are repeatable, run on a regular schedule, and require no manual processes due to the automatic transfer of data over the web. For the next dataset or data service you come across, why not try and master the art of the API! 

Article Author: Kieran O’Donnell, FME Data Solutions, Locus

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

Fully automate your Data Integration with Web Feature Services

Web Feature Services (WFS) are a great way of accessing GIS resources over the web without the need to download static files. The WFS Standard provides an interface for allowing requests for geographical features across the web, using platform-independent calls. As opposed to Web Map Services (WMS), Web Feature Services gives you access to the actual data which drive maps, and not just display it.

FME and the WFS Reader
FME has a WFS reader enabling a connection to Web Feature Services. You can input the URL, use authentication (if required), and configure the reader to use the WFS for whatever way you need it.

Performance
One of the main issues you can face when using a WFS is accessing large datasets. Try some of these tips to limit the features you pull from the server, to ensure a stable connection that you can rely on.

Geographic Filter
If you’re only trying to access features within a certain geographic extent, you can apply a bounding box on the reader, to only return features within your extent. This works particularly well if you’re a local government with a constant area of interest, or pulling features from an international service but are only interested in data for New Zealand.

Attribute Filter Expressions
Alternatively if you’re interested in a specific set of data which you can define using an attribute, you can enter this in the WFS parameters. This is particularly useful if you’re looking for data within a certain date range.

Multipart Transfer
Sometimes it’s not possible to  filter data using an attribute, or if you’re trying to access a table there’s no geographic extent to limit your features by.  When accessing very large datasets, this can cause issues with connection timeouts and lead to an unstable process. Using the WFS Reader’s constraints, you can make multiple requests to the WFS in order to pull smaller and more manageable chunks of data which make up a complete dataset. By setting the Count field to a smaller number, rather than asking for the entire dataset in a single request, FME batches up the requests based on your count size.

Using these constraints, a request is sent to the server for only features 1-5000. After this request is made, another will be made for features 5001-10,000 – and so on until the entire dataset is received. By using this method, smaller requests to the service are much more reliable, resulting in a more stable process with greatly reduced risk of timeout errors.

Did you know?… The LINZ Data Service has support for WFS on it’s datasets. This can save you plenty of time by taking away the need to download datasets manually, and allows for fully automated updates using FME as your ETL tool.

On the LINZ data service, check out the Services tab on any of the datasets you need to download on a regular basis. Using some of the tips and tricks above, you can set up FME workspaces to run on a regular schedule, pulling only the data that you and your organization need, when you need it.  Read more about how to use the LINZ WFS.

Read more on the FME Web Feature Service reader

Article Author:
Kieran O’Donnell
FME Data Solutions
Locus

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

Ready to upgrade to FME 2019.1

With the release of 2019.1 on both FME Desktop and Server some impressive new enhancements deliver visual and performance based gains.

What’s new in 2019?

FME Server Automations: Released in 2019.0, Automations are a revolutionary new tool in FME Server where you can use a graphical builder to create event based workflows. You can build workflows by defining triggers and actions, such as receiving an email, watching a directory or using a schedule. Automations are revolutionary because you can build workflows that leverage all the power of FME Server, in an easy to use drag and drop interface.

 

Visual Preview: This feature allows you to preview your data from within a dockable pane inside FME Workbench, without the need to open a data inspector window. Our users find this particularly helpful when working with tricky logic and minor tweaks  to their workspace, and definitely makes building and debugging your workspaces more efficient.

Performance Enhancements: With all FME releases there are some enhancements to performance, but this version in particular brought some really great improvements. Shapefiles have had a redevelopment to become incredibly fast and efficient, which is great news for the most popularly used format. The Dissolver transformer has also been seriously enhanced, running as much as 75x faster than before.

New Formats: As always the number of formats supported in FME has increased, with this year the number reaching more than 450. Of note is a new native Revit reader, where you can read your Revit files directly without needing to use RVZ intermediate files. Some users will also be interested that there is now a 12d Reader released in beta, with expected support in FME 2020.

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

Locus Achieves Safe Software Platinum Status

December 2018.  Locus, market leaders in the provision of certified FME training and consultancy has been awarded Platinum Value Added Reseller (VAR) status by Safe Software Inc. the developers of FME.

Founded in 2008 by John Arnerich, Locus is a team of business practitioners and FME technical experts engaged in the sale, support and servicing of FME products including FME Desktop, FME Server and FME Cloud.

With its breadth of FME knowledge, technical understanding and expertise in New Zealand and Australia, Locus has assisted over 200 organisations transform the way they manage their data. The powerful data integration capabilities, flexible and scaleable licencing editions and models make FME a natural choice for many organisations, from the small operator to enterprise level organisation.

“We are pleased to award Locus with Platinum Partner, a highly regarded position within the FME partner community,” says Sonia Hobbs, VP, Business Development and Strategy, Safe Software. “Their business acumen, technical expertise, and customer-centric approach make Locus an exemplary partner and we look forward to seeing their success and growth in the future.”

“We couldn’t be more thrilled to be awarded Platinum status in Safe Software’s partner program, commented John Arnerich, Group Director, Locus. Our relationship with Safe spans more than 10 years and this accolade further deepens our commitment to Safe and the FME suite of products. Our customers rely on the extensive capabilities of this flexible software in solving a variety of both every day and often more complex data interoperability challenges. We believe in the power of FME in improving organisational productivity and ultimately saving you time, and money,” added John.

Visit Locus at www.locus.co.nz and www.locusglobal.com.au to learn more about the FME product suite, FME certified training courses and software support.

More about Locus …Locus is a team of business practitioners and FME technical experts ready to help you maximise the value of your business through better data management. As a Safe Software Platinum Partner and Value Added Reseller with offices in New Zealand and Australia, we hold the unique position of being engaged solely in the sale, support and servicing of FME products; this is our competitive advantage.

More about FME…Built by Safe Software and used by over 20,000 organisations worldwide, FME is a data translation and transformation tool for solving problems of data interoperability, without the need for coding. With support for 400+ formats and applications, FME is the integration platform that converts, transforms and automates data whenever and however it’s needed.

Registration opens for the FME World Tour 2018 hosted by Locus Limited

Locus Limited invites you to register for the FME World Tour 2018 and join us in Wellington, Christchurch or Auckland in May 2018.

Learn about the newest features in FME 2018, meet fellow users, update your FME knowledge with time-saving tips/practices, receive a sneak peek of upcoming features and grab yourself some take-home freebies!

This event is perfect for you if you are:

  • An FME user who would like to be inspired by new FME ideas, discover best practices, and meet fellow FME users
  • Work in an organisation that would like to understand more about FME’s capabilities and how it may be able to assist you in managing your data challenges

Secure your place on the FME World Tour 2018 now:

Monday, 14 May 2018 – Wellington
Wednesday, 16 May 2018 – Christchurch
Friday, 18 May 2018 – Auckland

Cost: $95 + GST per person

REGISTER NOW

Interested in presenting at FME World Tour 2018?
Perhaps you’ve used FME to deliver a challenging or innovative project or have some unique tips and tricks you would like to share.    Presenting at an FME World Tour event is an opportunity to get recognition for your success.  We have presenter places available and would like to hear from you.  To be eligible to present please submit an abstract of your presentation via the Safe Software site or contact Locus to discuss further hello@locus.co.nz

More about Locus: Locus Limited is a Safe Software Gold Value Added Reseller. Since 2004, our team of FME Certified Professionals have been enabling businesses globally to unlock the power of FME to manage their enterprise level data.

What is FME?  The Feature Manipulation Engine (FME) is a platform that streamlines the translation of spatial data between geometric and digital formats.   FME enables you to move data between 350+ formats and applications while preserving data quality throughout the conversion process.  With hundreds of tools (transformers) at your disposal you have complete control over the structure and composition of your data.  Manual data processing tasks are a thing of the past with automated workflows scheduled to run at any time interval.

New to FME or looking to develop your skills?

Registrations for our Introduction to FME Desktop training course are now open.

Unlock the power of FME to manage your data and join Locus Limited at the Grand Mercure, 8 Customs Street East, Auckland on 9th & 10th of October 2017.

Conducted by one of our Safe Software FME Certified Trainers, this two-day course will take place in a classroom environment and include hands-on, problem-solving exercises.

Introduction to FME Desktop is perfect for you if you are:

  • A new user of FME and are looking for some formal instruction
  • You’re a modest user without formal training and would benefit from a greater understanding of the fundamentals
  • You’re less technical more strategic and want to improve business processes in your organisation
  • You would like to leverage the latest version of FME to improve your knowledge and capability

Training will include:

– Data translation basics
– Data transformation
– Best Practice
– Readers and Writers
– Practical Transformer Use

Cost: $1,750 + GST per person ($1,500 + GST per person for two or more people attending from the same organisation)

To register for the Introduction to FME Desktop training course or for more information, telephone us on 0800 LOCUS NZ or  email training@locus.co.nz

More about Locus: Locus Limited is a Safe Software Gold Value Added Reseller. Since 2004, our team of FME Certified Professionals have been enabling businesses globally to unlock the power of FME to manage their enterprise level data.

What is FME?  The Feature Manipulation Engine (FME) is a platform that streamlines the translation of spatial data between geometric and digital formats.   FME enables you to move data between 350+ formats and applications while preserving data quality throughout the conversion process.  With hundreds of tools (transformers) at your disposal you have complete control over the structure and composition of your data.  Manual data processing tasks are a thing of the past with automated workflows scheduled to run at any time interval.

It’s Arrived! Safe Software Release FME 2017.1

The much anticipated Safe Software FME 2017.1 release arrived on 1 August 2017 and is packed with features and fixes!   Here at Locus, we’ve welcomed the opportunity to take this release for a test drive and to follow area few highlights we wanted to bring to your attention.

Maximise your automation

Direct jobs to one or more specific engines with a new FME Server interface Queues feature.   Lengthy jobs can be pushed to one engine and quick jobs to another thereby maximising your automation capabilities and ensuring no one job holds another up.

A new transformer; DateTimeConverter

The DateTimeConverter replaces the DateFormatter and this new transformer has some far superior functionality including greater precision (fractional seconds), the ability to parse more complex input strings (wildcards and advanced flags), and the ability to convert between time zones.

You’ll be able to enjoy working faster with presets, partial matches and a Quick Reference feature for flags. It also has a rejected port for invalid values and the ability to repair certain errors.

Remote sensing just got bigger

In FME 2017 Safe introduced support for cloud hosted satellite imagery but managing this data can be a challenging task.    FME2017.1 will help you access more services – Landsat 8, Landsat 8 on AWS and Terrain Tiles on AWS – as well as more effectively handle these datasets.

Usability gains for FME Desktop

A faster Inspector transformer’s connection to the Data Inspector will make authoring more efficient.  You can now select, copy and paste multiple user parameters at once.  An updated reader/writer right click option provides 2 key enhancements: it allows you to update existing readers and writers from older versions of FME to access the capabilities of new versions, and it also allows you to adjust all settings, including those which could affect the feature type schemas.  All of this without having to delete and re-add the affected reader or writer.

To experience all of these new features and more, download a free trial of FME now or talk to Locus about how we and FME can help you manage your data.

Download a free trial of FME – https://www.locus.co.nz/try-fme-for-free/

hello@locus.co.nz or 0800 562 8769

About Safe Software and FME
Safe Software is the maker of FME and the pioneering global leader in spatial data transformation technology. FME empowers users to confidently transform spatial data so it can be used and shared – where, when and how it’s needed. Available in seven languages, FME’s unmatched transformation capabilities and support for 350+ spatial and non-spatial formats enables you to quickly overcome data challenges so you can focus on your objectives. Discover why leading vendors and thousands of organizations worldwide rely on FME to quickly overcome barriers to using and sharing data. Visit
www.safe.com.

FME is a registered trademark of Safe Software Inc. All other product names may be trademarks or registered trademarks of their respective owners.

Gary Nicholson of Locus Limited achieves FME® Server Professional certification from Safe Software

Locus Limited, Partner of Safe Software and gold value added reseller is delighted to confirm that Gary Nicholson of Locus has achieved Safe Software’s Certified FME Server Professional designation, the first person to do so in New Zealand.

The FME Server Professional accreditation formally recognises individual expertise and proven ability in the delivery of knowledgeable FME support, training and consulting services.  Locus Limited is proud to acknowledge Gary’s FME Server Professional designation, an additional capability to his FME Professional accreditation.

Commenting on Gary’s achievement, John Arnerich, CEO, Locus Limited added “We’re thrilled with Gary’s accreditation, it extends his existing experience and enhances Locus’s FME-based offerings.   Committed to delivering a best in class experience to all our customers this accreditation demonstrates that Gary and Locus conform to Safe Software’s high standards for its resellers”.

John was also quick to add that “Gary’s advanced level of FME Server expertise is unparalleled in New Zealand. For our customers and organisations considering FME, that level of knowledge translates into superior technical support, and training”.

Gary’s multiple accreditations sit alongside Locus’s colleague Kim Ollivier’s FME Certified Professional and FME Certified Trainer certifications.

Locus Limited is a Safe Software Gold Value Added Reseller. Since 2004, our team of FME Certified Professionals have been enabling businesses globally to unlock the power of FME to manage their enterprise level data. Our solutions, support and bespoke training ensures that our customers maximise the benefits from their FME investment today, and on an ongoing basis.   For more information about FME and how Locus can help you, visit locus.co.nz or contact us on hello@locus.conz

 

About Safe Software and FME
Safe Software is the maker of FME and the pioneering global leader in spatial data transformation technology. FME empowers users to confidently transform spatial data so it can be used and shared – where, when and how it’s needed. Available in seven languages, FME’s unmatched transformation capabilities and support for 350+ spatial and non-spatial formats enables you to quickly overcome data challenges so you can focus on your objectives. Discover why leading vendors and thousands of organizations worldwide rely on FME to quickly overcome barriers to using and sharing data. Visit
www.safe.com.

FME is a registered trademark of Safe Software Inc. All other product names may be trademarks or registered trademarks of their respective owners.