SFTP Integrations

SFTP Integration Overview

This document outlines the standard workflow, directory structure, supported file types, and hosting options for SFTP integrations with Graphite.

The integration follows a cyclical process involving data export from Graphite, processing by your system, and data import back into Graphite.

Data Export: Graphite periodically scans for entity changes based on a predefined schedule. For each modified entity, Graphite generates a data file and places it in the /export directory on the SFTP server.
File Retrieval and Processing: Your integration system should monitor the /export directory. Upon detecting new files, your system retrieves and processes them according to your integration logic.
Data Import: After processing, your integration system prepares update or acknowledgement files based on the outcomes. These files are then uploaded to the /import directory. Each supported file type has a specific format; refer to Supported File Types for details.
Graphite Processing and Error Handling: Graphite monitors the /import directory.
- Success: For each valid file, Graphite attempts to apply the specified updates (patch) or record the acknowledgements (acknowledge).
- Failure: If any errors occur while processing a file for a particular entity, Graphite will not apply any updates for that entity from that file. The file encountering the error is moved to the /errors directory for investigation.
Archiving: Successfully processed files from the /import directory are moved by Graphite to the /processed directory. These files are retained for a specified period, serving as an audit log of successfully imported data.

All specified directories reside within the root directory of the SFTP user's home path.

📘
Important
All paths are absolute and must start with /.

/export: Contains data files exported by Graphite detailing entity changes, ready for retrieval and processing by your system.
/import: Used by your system to upload files (containing updates or acknowledgements) intended for processing by Graphite.
/errors: Contains files that Graphite failed to process due to errors. These files should be reviewed to diagnose integration issues. See Common Errors for troubleshooting tips.
/processed: Archive of files successfully processed by Graphite from the /import directory. Files are retained here for a defined duration as an audit log.

Graphite's SFTP integration supports the following file formats:

Integrations can utilize either a Graphite-hosted or a customer-hosted SFTP server.

Management: Graphite manages all server maintenance and configuration.
Authentication: Graphite utilizes SSH key-based authentication.
- Customer Provides: You will need to provide your public SSH key(s) and the corresponding source IP address(es) for whitelisting (required for both production and non-production environments).
- Graphite Provides: Graphite will supply the necessary SFTP host, port, and username details.

Management: Your organization is responsible for server setup, maintenance, and configuration.
Authentication: Graphite requires SSH key-based authentication to connect. Password authentication is not supported.
- Graphite Provides: Graphite will supply its public SSH key(s) (for staging and production) and the Graphite IP addresses that require whitelisting on your server's firewall.
- Customer Provides: You will need to provide Graphite with the SFTP host, port, and username required for connection.