JSON SFTP Integration

JSON File Format for Importing Data to Graphite

When your integration needs to send data back to Graphite (e.g., updating fields with external identifiers or acknowledging processed data), you can upload a JSON file to the /import directory on the SFTP server.

Graphite processes these JSON files to perform two main actions: patching data onto the counterparty entity profile and acknowledging the processing status of data previously sent by Graphite.

Overall JSON Structure

Each JSON file uploaded to /import should be a single JSON object with the following top-level keys:

{  
  "connectionId": "...",  
  "upTo": "...",
  "patch": {  
    ...  
  },  
  "acknowledge": [  
    ...  
  ]  
}

connectionId (String, Required): This is the unique identifier for the connection (relationship) between your entity (the interface owner) and the counterparty entity whose data is being affected. Graphite uses this ID to identify the correct connection and the associated counterparty entity for patching and acknowledgements.
patch (Object, Optional): Contains the data fields you want to update or add to the counterparty entity's profile within the context of this connection.
acknowledge (Array, Optional): An array of acknowledgement objects, used to inform Graphite about the processing status (success or failure) of data previously received from Graphite, often up to a specific point in time.
upTo (String - ISO 8601 Date/Time, Optional but Recommended): Specifies the timestamp up to which data has been successfully processed or attempted. This typically corresponds to a timestamp associated with data previously exported by Graphite. Using upTo helps Graphite understand the synchronization point.

`patch` Object Details

The patch object contains key-value pairs representing the fields to be updated on the counterparty entity profile associated with the connectionId.

Updating Top-Level Fields: To update regular fields (not part of a group/repeater), simply include them as key-value pairs within the patch object. The value should match the expected data type (e.g., string, number, boolean).

"patch": {  
  "externalVendorId": "VENDOR-12345",  
  "paymentTerms": "Net 30"  
}

Updating Group (One-to-Many) Fields: Groups represent repeatable sets of fields (like multiple contacts or locations). To update specific instances within a group, the value associated with the group's key must be an array of objects. Each object in the array represents one instance within the group and must include an _id key to identify the specific instance being updated. Other keys within the object represent the fields to update for that specific instance.

"patch": {  
  "contacts": [  
    {  
      "_id": "65aabcd1234567890abcdef0", // Instance ID of the contact to update  
      "email": "[email protected]",  
      "phone": "555-123-4568"  
    },  
    {  
      "_id": "65aabcd1234567890abcdef1", // Instance ID of another contact  
      "isActive": false  
    }  
  ]  
}

Note: The specific _id values for group instances are provided in the data exported _from Graphite._

`acknowledge` Array Details

The acknowledge array allows your integration to report the processing status of data back to Graphite. This is crucial for tracking data synchronization and handling errors. Each element in the array is an object representing a specific acknowledgement action.

"acknowledge": [  
  {  
    "errors": [  
      "Invalid payment terms code 'N40'"  
    ],
  },  
  {  
    "errors": [  
      "Missing required field 'department'"  
    ],
    "groupKey": "contacts",  
    "instanceId": "65aabcd1234567890abcdef5"  
  }  
]

Each acknowledgement object can contain the following keys:

errors (Array of Strings, Optional): If your system encountered errors while processing data related to this acknowledgement, provide an array of error messages here.
- If errors is null, omitted, or an empty array, the acknowledgement is considered successful.
- If errors contains one or more strings, the acknowledgement is considered unsuccessful, and the errors are logged in Graphite.
groupKey (String, Optional): If the acknowledgement (especially an error) pertains to a specific instance within a group, provide the technical key (field name) of the group here. Required if instanceId is provided.
instanceId (String, Optional): If the acknowledgement pertains to a specific instance within a group, provide the _id of that instance here. Required if groupKey is provided.

Acknowledgement Scenarios:

Successful Acknowledgement (Global): Omit errors or set to []. Omit groupKey and instanceId
Unsuccessful Acknowledgement (Global): Provide one or more messages in the errors array. Omit groupKey and instanceId.
Successful Acknowledgement (Specific Group Instance): Omit errors or set to []. Provide the relevant groupKey and the specific instanceId (_id) of the group item that was processed correctly.
Unsuccessful Acknowledgement (Specific Group Instance): Provide error messages in errors. Provide the relevant groupKey and the specific instanceId (_id) of the group item that failed processing.

Processing Logic and Error Handling

File Processing: Graphite processes each JSON file found in /import individually.
Patching: If a patch object is present and valid, Graphite attempts to apply the updates to the counterparty entity identified by connectionId.
Acknowledging: If an acknowledge array is present, Graphite processes each acknowledgement object, updating the sync status and logging any reported errors.
File Movement:
- If Graphite successfully parses the JSON and processes both the patch (if present) and acknowledge (if present) sections without internal errors (like invalid connectionId), the file is moved to the /processed directory.
- If Graphite encounters an error parsing the JSON file itself, or if critical lookups fail (e.g., the connectionId doesn't correspond to a valid connection), the entire file is moved to the /errors directory, and no patching or acknowledging occurs for that file. Note that errors reported within the acknowledge array do not cause the file to go to /errors; they are logged as part of the acknowledgement process itself.

Example JSON File

{
  "connectionId": "conn_abc123def456", 
  "upTo": "2025-03-20T14:00:00Z",
  "patch": {  
    "externalVendorId": "EXT-SYS-9876",  
    "contacts": [  
      {  
        "_id": "65feabc1234567890defabc0",  
        "email": "[email protected]"  
      }  
    ]  
  },  
  "acknowledge": [  
    {  
      "errors": [  
        "Invalid address format for primary location"  
      ],  
      "groupKey": "Locations_List",  
      "instanceId": "65feabc1234567890defabc5"  
    }
  ]  
}

This example:

Targets the connection conn_abc123def456.
Patches the externalVendorId field.
Patches the email field for the specific contact instance with _id 65feabc1234567890defabc0.
Reports a processing error for the location instance 65feabc1234567890defabc5, also related to the 2025-03-20T14:00:00Z data point.