Customizing Connectors and Extractors with Plugins
The Voyager platform is designed with extensibility in mind, enabling the integration of custom plugins developed by third parties. This flexible system allows for the support of new file formats through the implementation of customizable connectors and extractors. Below, we detail how developers can extend the capabilities of our platform to meet specific data integration and processing needs.
Connector Customization:
Connectors serve as the bridge between our platform and various data sources. They can be customized or entirely developed by third parties to enhance or introduce new data retrieval functionalities. Our architecture supports a wide range of programming languages, offering developers the flexibility to use the language with which they are most comfortable or that is most suited to the task at hand.
A key feature of our customizable connectors is their integration into the user interface, providing an intuitive experience for configuration and real-time status monitoring. This direct integration simplifies the process of connecting to and iterating over content, making it as seamless as possible for the end user.
The development time for a new connector can vary, but in many cases, a basic version can be implemented within a single day, depending on the developer's familiarity with our system and the APIs of the data sources they are integrating with.
Extractor Customization:
Extractors are responsible for processing the retrieved data, converting it into a structured format that our platform can utilize. Similar to connectors, extractors can also be developed by third parties using a variety of techniques. The most straightforward approach involves creating an executable that can be invoked via command line, passing arguments necessary for processing.
When an extractor is called, it receives information such as the path to the data item, arguments for where to write output metadata (including standard output), and any special instructions related to the file types being processed, such as generating thumbnails.
This loose coupling via command-line executables allows for easy integration and flexibility in how data is processed, accommodating a wide range of data types and extraction requirements.
Summary:
By supporting the customization of connectors and extractors through third-party plugins, our platform offers a highly flexible and adaptable solution for data integration and processing. Whether you are looking to extend the platform's native capabilities or integrate with new data sources, our system provides the foundation necessary for developers to implement tailored solutions efficiently.
For more information see: