How to Run the Connector Externally
To run the Ingestion via the UI you’ll need to use the OpenMetadata Ingestion Container, which comes shipped with custom Airflow plugins to handle the workflow deployment. If, instead, you want to manage your workflows externally on your preferred orchestrator, you can check the following docs to run the Ingestion Framework anywhere.External Schedulers
Get more information about running the Ingestion Framework Externally
Requirements
Python Requirements
To run the Exasol ingestion, you will need to install:Database Privilege Requirements
For the required database privileges, see the Requirements section of the Exasol connector documentation.Metadata Ingestion
The Exasol service connection is defined by the Exasol connection JSON schema. To create and run a Metadata Ingestion workflow, define a YAML configuration that connects to Exasol, processes the required entities, and sends the metadata to OpenMetadata. The workflow configuration follows the metadata ingestion workflow JSON schema.1. Define the YAML Config
2. Run with the CLI
First, we will need to save the YAML file. Afterward, and with all requirements installed, we can run:Lineage
After running a Metadata Ingestion workflow, we can run Lineage workflow. While theserviceName will be the same to that was used in Metadata Ingestion, so the ingestion bot can get the serviceConnection details from the server.
1. Define the YAML Config
This is a sample config for Lineage:- You can learn more about how to configure and run the Lineage Workflow to extract Lineage data from here
2. Run with the CLI
After saving the YAML config, we will run the command the same way we did for the metadata ingestion:Query Usage
The Query Usage workflow will be using thequery-parser processor.
After running a Metadata Ingestion workflow, we can run Query Usage workflow.
While the serviceName will be the same to that was used in Metadata Ingestion, so the ingestion bot can get the serviceConnection details from the server.