Voyager's Indexing Pipeline

Owned by Heather Geyer
Last updated: Aug 09, 2022
1 min read

Voyager's Indexing Pipeline provides functions to transform and manipulate the properties (metadata) of data records as it adds them to the Index.

Metadata can include common attributes like name, path, size or creation date as well as more specific information such as spatial reference and extent of spatial files or readable text content.

Pipeline functions generally fall into three categories - Metadata Extraction, Document Transformation and Geotagging.

  • Metadata Extraction allows administrators to select and manipulate specific entries from a data set's associated XML metadata. The XML source can be filename.shp's filename.xml file or an XML blob stored in the XMP/IPTC section of an image file's EXIF header. For more information, see Metadata Extraction.

  • Document Transformers allow administrators to add, remove or update data properties directly in the index. For more information, see Document Transformers.

  • Geotagging allows administrators to associate geographic location (and other fields) with non-spatial data using spatial records in a look-up table (gazetteer).
    For more information, see Geotagging.

Creating a Custom Pipeline Step Using Python

You can also create pipeline steps using Python, customized for your environment and data. See Creating a Custom Pipeline Step Using Python for more information and examples.