Creating a New Data Pipeline in Forge AI - Revision history

Chris.Hansen at 10:04, 10 August 2024

2024-08-10T10:04:50Z

← Older revision		Revision as of 10:04, 10 August 2024
Line 2:		Line 2:

	=== 1. Access the Data Pipelines Section ===		=== 1. Access the Data Pipelines Section ===
	Navigate to the ~~Data Pipelines~~ section in the Forge AI platform ~~under Configuration~~. You will find the option to create a new data pipeline in the upper right-hand corner of the page.		Navigate to the Configurations section in the Forge AI platform. You will find the option to create a new data pipeline in the upper right-hand corner of the page when viewing the Data Pipelines tab.

	=== 2. Initiate a New Data Pipeline ===		=== 2. Initiate a New Data Pipeline ===
Line 13:		Line 13:

	* '''Action:''' Select from the drop-down menu.		* '''Action:''' Select from the drop-down menu.
	* '''Options:''' The list will ~~include various data sources~~ available within your ~~Forge AI~~ environment~~, such as databases, APIs, and IoT devices~~.		* '''Options:''' The list will includes the Data Sets available within your FojiSoft environment.

	==== '''Destination''' ====		==== '''Destination''' ====

	* '''Action:''' Select from the drop-down menu.		* '''Action:''' Select from the drop-down menu.
	* '''Options:''' Choose the target ~~location~~ where the data will be sent. ~~This could~~ be a ~~database, data warehouse, or another storage solution supported by Forge AI~~.		* '''Options:''' Choose the target relational data table where the data will be sent. A new table will automatically be created to match the configuration if you do not explicitly select a table.

	==== '''Mode''' ====		==== '''Mode''' ====

Chris.Hansen at 09:55, 10 August 2024

2024-08-10T09:55:43Z

← Older revision		Revision as of 09:55, 10 August 2024
Line 63:		Line 63:

	By following these steps, you can effectively set up and manage data pipelines within the Forge AI platform, ensuring smooth data integration and processing across your systems.		By following these steps, you can effectively set up and manage data pipelines within the Forge AI platform, ensuring smooth data integration and processing across your systems.
	[[Category:~~Monitor~~]]		[[Category:ForgeAI]]

Mitchell.Hirsche: Creating a new data pipeline for your AI models

2024-08-07T21:08:16Z

Creating a new data pipeline for your AI models

New page

=== Step-by-Step Guide ===

=== 1. Access the Data Pipelines Section ===
Navigate to the Data Pipelines section in the Forge AI platform under Configuration. You will find the option to create a new data pipeline in the upper right-hand corner of the page.

=== 2. Initiate a New Data Pipeline ===
Click on the "New Data Pipeline" button to start the creation process.

=== 3. Fill Out the Pipeline Parameters ===
You will be prompted to fill out several parameters for the new data pipeline. Ensure you complete each section as follows:

==== '''Source''' ====

* '''Action:''' Select from the drop-down menu.
* '''Options:''' The list will include various data sources available within your Forge AI environment, such as databases, APIs, and IoT devices.

==== '''Destination''' ====

* '''Action:''' Select from the drop-down menu.
* '''Options:''' Choose the target location where the data will be sent. This could be a database, data warehouse, or another storage solution supported by Forge AI.

==== '''Mode''' ====

* '''Options:'''
** '''Replace data in destination table:''' This option will overwrite the existing data in the destination table.
** '''Append data to table:''' This option will add new data to the existing data in the destination table.

==== '''Execution Interval''' ====

* '''Action:''' Specify the frequency at which the pipeline should run.
* '''Options:'''
** '''Second'''
** '''Minute'''
** '''Hour'''
** '''Day'''
** '''Week'''
* '''Example:''' Enter a number and select the interval (e.g., 5 minutes).

=== 4. Set Up the Date Range Step ===
Define the date range for the data to be processed within the pipeline.

==== '''From''' ====

* '''Mode:'''
** '''Absolute:''' Select a specific start date and time.
** '''After Now:''' Define an offset relative to the current time (e.g., 5 minutes after now).
** '''Before Now:''' Define an offset relative to the current time (e.g., 5 minutes before now).
* '''Offset:''' Enter the number and select the unit (seconds, minutes, hours, days, weeks, years).

==== '''To''' ====

* '''Mode:'''
** '''Absolute:''' Select a specific end date and time.
** '''After Now:''' Define an offset relative to the current time (e.g., 5 minutes after now).
** '''Before Now:''' Define an offset relative to the current time (e.g., 5 minutes before now).
* '''Offset:''' Enter the number and select the unit (seconds, minutes, hours, days, weeks, years).

=== 5. Finalize and Save the Pipeline ===
After filling out all the required parameters, review your configurations. If everything is correct, click on the "Save" button to create your new data pipeline.

=== 6. Monitor and Manage the Pipeline ===
Once the pipeline is created, it will appear in your list of data pipelines. You can monitor its execution, check logs for any issues, and make adjustments as needed.

By following these steps, you can effectively set up and manage data pipelines within the Forge AI platform, ensuring smooth data integration and processing across your systems.
[[Category:Monitor]]