Connectors & EkDB

hub

ConnectCore Overview

ConnectCore is the foundation for all external system connectors in the Clarity platform. It provides the base interfaces, configuration patterns, and discovery mechanisms that every connector builds upon.

Core Interfaces

integration_instructions

IConnectorClient

Marker interface for all connector API clients. Exposes a ConnectorId property that uniquely identifies the connector instance. Enables mock support across all connectors for testing.

settings_input_component

IConnectorSettings

Base interface for all connector configuration classes. Each connector provides its own implementation with vendor-specific properties (credentials, endpoints, timeouts).

Configuration Classes

settings

ConnectSettings

Global connector settings shared across all connectors, including common timeouts and retry policies.

vpn_lock

ProxySettings

HTTP proxy configuration for connectors that need to route traffic through a corporate proxy or gateway.

Connector Discovery

Connectors register themselves with the platform through the GetConnectorSetupsPipeline hook. This pipeline is invoked at startup and whenever the admin UI needs to display available connectors and their configuration status.

// Each connector adds a hook to announce itself
[HookFor(typeof(GetConnectorSetupsPipeline))]
public class NetSuiteConnectorSetupHook : IPipelineHook
{
    public Task ExecuteAsync(PipelineContext context)
    {
        context.GetResult<List<ConnectorSetup>>().Add(new ConnectorSetup
        {
            ConnectorId = "netsuite",
            DisplayName = "NetSuite",
            Vendor = "Oracle",
            IsConfigured = _settings.IsValid()
        });
        return Task.CompletedTask;
    }
}

key

External Key Database (EkDB)

EkDB maps external system IDs to internal Phoenix entities, providing a unified identity layer that spans multiple ERP systems, payment providers, and third-party services. It stores metadata alongside each mapping and supports caching for high-throughput lookups.

Core Interfaces & Implementations

integration_instructions

IExtKeyDB

The primary interface for all external key database operations: resolve, store, update, and delete external ID mappings.

database

PGExtKeyDB

PostgreSQL implementation of IExtKeyDB. Stores mappings in a dedicated table with JSONB metadata columns for flexible, schema-less extension data.

Key Concepts

EntityKey

Defines an external ID for an entity. Composed of a key definition type and the actual external value. Represents a single external system's identifier for a Phoenix entity.

EntRef (Entity Reference)

Contains the full set of external ID mappings for a Phoenix entity. An EntRef can hold keys from multiple external systems simultaneously.

ResolveOp

Controls resolution behavior: Read for lookup-only operations, ReadWrite when the mapping should be created if it does not exist.

Metadata Storage

Each mapping can carry arbitrary JSON metadata, job ID tracking for sync auditing, and timestamps. Caching reduces repeated database lookups during batch operations.

Key Definition Types

Type	Description	Example
`StringEntKeyDef`	Simple string-based external key	`"CUST-12345"`
`CompositeEntKeyDef`	Multi-part key composed of several fields	`Company+Branch+Id`
`ParsableEntKeyDef`	Key that can be parsed from a formatted string	`int, Guid, etc.`
`ColonSepStringEntKeyDef`	Colon-separated composite key as a single string	`"NS:12345:INV"`

EkDB Architecture

inventory_2

Phoenix Entity

Internal ID (Guid)

Customer

Invoice

Contact

table_chart

EntityKey Table

EkDB Mapping Layer

PK: PhoenixId

FK: ExternalId

Metadata (JSON)

arrow_back Resolve

Map arrow_forward

cloud

External System

ERP / Service ID

NS:CUST-5001

NS:INV-8832

SAP:BP-0042

folder_open

Connector Structure

Every connector follows a standardized directory layout that separates backend logic from frontend UI. This consistency makes it straightforward to navigate any connector once you understand the pattern.

Directory Layout

Phoenix.Connector.XYZ/
  Backend/
    Clients/           — API client classes (HTTP/REST communication)
    Controllers/       — API endpoints for admin configuration
    DataModel/         — External system data models and DTOs
    Pipelines/         — Integration pipelines (Search, Get, Create, Update)
    Hooks/             — Setup and configuration hooks
    XyzPlugin.cs       — Plugin registration (IPlugin implementation)
    XyzSettings.cs     — Connector-specific configuration
  Frontend/
    plugin.json        — Frontend plugin metadata
    components/        — Setup UI components
    routes/            — Admin routes for connector management

Naming Conventions

Property	Purpose	Example
`Name`	Plugin class display name	`"NetSuite"`
`WorkflowVendor`	Vendor identifier for pipeline routing	`"Oracle"`
`WorkflowProduct`	Product identifier for pipeline routing	`"NetSuite"`

add_circle

Creating a Connector

Follow these steps to create a new connector plugin. Each step includes the interface or base class to implement and a code example.

1

Create Plugin Class

Implement IPlugin with Name, WorkflowVendor, and WorkflowProduct properties.

public class XyzPlugin : IPlugin
{
    public string Name => "XYZ ERP";
    public string WorkflowVendor => "XyzCorp";
    public string WorkflowProduct => "XyzErp";

    public void Configure(IServiceCollection services)
    {
        services.AddSingleton<XyzSettings>();
        services.AddScoped<IXyzClient, XyzClient>();
    }
}

2

Create Settings Class

Implement IConnectorSettings with connector-specific configuration properties.

public class XyzSettings : IConnectorSettings
{
    public string BaseUrl { get; set; }
    public string ApiKey { get; set; }
    public int TimeoutMs { get; set; } = 30000;
    public int MaxRetries { get; set; } = 3;

    public bool IsValid() =>
        !string.IsNullOrEmpty(BaseUrl) &&
        !string.IsNullOrEmpty(ApiKey);
}

3

Create API Client

Build an HTTP client class using HttpClient or RestSharp with authentication and retry logic.

public class XyzClient : IConnectorClient
{
    public string ConnectorId => "xyz";
    private readonly RestClient _client;

    public XyzClient(XyzSettings settings)
    {
        _client = new RestClient(new RestClientOptions(settings.BaseUrl)
        {
            Authenticator = new HttpBasicAuthenticator(settings.ApiKey, ""),
            MaxTimeout = settings.TimeoutMs
        });
    }
}

4

Create Controller

Add admin endpoints for connector configuration and health checks.

[ApiController]
[Route("api/connectors/xyz")]
public class XyzController : ControllerBase
{
    [HttpGet("status")]
    public IActionResult GetStatus() => Ok(new { Connected = true });

    [HttpPost("test-connection")]
    public async Task<IActionResult> TestConnection() { ... }
}

5

Create Pipelines

Implement domain-specific pipelines (Search, Get, Create, Update) with [WorkflowNode] attributes for automatic discovery.

[WorkflowNode(
    Vendor = "XyzCorp",
    Product = "XyzErp",
    Operation = "SearchCustomers")]
public class SearchXyzCustomersPipeline : SerialPipeline
{
    public override async Task ExecuteAsync(PipelineContext context)
    {
        var query = context.GetInput<SearchQuery>();
        var results = await _client.SearchCustomersAsync(query);
        context.SetResult(results);
    }
}

6

Add Discovery Hook

Register a GetConnectorSetupsPipeline hook so the platform discovers your connector.

7

Create Frontend Setup Components

Build the admin UI components in Frontend/components/ and routes in Frontend/routes/ for connector configuration and status display.

business

NetSuite Reference Implementation

Authentication

verified OAuth 2.0 (Strategic)

Required for all new integrations starting with NetSuite 2027.1. Uses client credentials flow with automatic token refresh on expiration.

token Token-Based Authentication (TBA)

Legacy authentication method. Still supported for existing integrations. Uses consumer key/secret with token key/secret for request signing.

Both authentication methods support automatic token refresh on expiration. The connector determines the active method from configuration and initializes the appropriate authenticator at client construction time.

Configuration (NetSuiteSettings)

public class NetSuiteSettings : IConnectorSettings
{
    // Account
    public string AccountId { get; set; }
    public string Environment { get; set; }       // "production" | "sandbox"
    public string BaseUrl { get; set; }

    // OAuth 2.0
    public string OAuthClientId { get; set; }
    public string OAuthClientSecret { get; set; }
    public string OAuthTokenUrl { get; set; }

    // Token-Based Auth (TBA)
    public string ConsumerKey { get; set; }
    public string ConsumerSecret { get; set; }
    public string TokenId { get; set; }
    public string TokenSecret { get; set; }

    // Behavior
    public int TimeoutMs { get; set; } = 30000;
    public int DefaultPageSize { get; set; } = 100;
    public int MaxRetries { get; set; } = 3;
}

API Client

The NetSuite client is built on RestSharp with lazy initialization and thread-safe access. It configures JSON serialization to handle NetSuite's specific date formats and null handling requirements.

RestSharp-based Lazy initialization Thread-safe Configurable JSON serialization

Pipelines (15+ by Domain)

Customers

Search QueryByDate Get Create Update

Contacts

Search Create Update

Invoices

Search QueryByDate Get Update

Payments

CreateCustomerPayment

Utility

QueryNetSuitePipeline Custom SuiteQL queries for advanced data access

Pipeline Pattern

[WorkflowNode(
    Vendor = "Oracle",
    Product = "NetSuite",
    Operation = "SearchCustomers")]
public class SearchNetSuiteCustomersPipeline : SerialPipeline
{
    private readonly INetSuiteClient _client;
    private readonly IExtKeyDB _ekdb;

    public override async Task ExecuteAsync(PipelineContext context)
    {
        var query = context.GetInput<CustomerSearchQuery>();

        // Use SuiteQL for search (bulk read strategy)
        var suiteQlResults = await _client.QueryAsync<NetSuiteCustomer>(
            $"SELECT * FROM customer WHERE companyname LIKE '%{query.Term}%'"
        );

        // Resolve external keys through EkDB
        var mapped = await _ekdb.ResolveAsync(suiteQlResults, ResolveOp.Read);
        context.SetResult(mapped);
    }
}

REST vs SuiteQL Strategy

api REST API

Used for Create, Update, and Delete operations where record-level access is needed.

Create Update Delete

database SuiteQL

Used for Search, Filter, and Bulk Read operations where SQL-like querying is more efficient.

Search Filter Bulk Read

NetSuite Pipeline Flow

devices

Client Request

conversion_path

Pipeline

code

NetSuiteClient

lock

OAuth / TBA

cloud

REST / SuiteQL

output

Response

update

Connector Versioning

package_2

Unified Deployment

All components move together per deployment. Every connector ships in the same Docker image with the same version tag. There is no per-customer or per-connector version selection in code.

tune

Configuration-Driven Behavior

Customer-specific behavior is achieved through configuration and data, not through different DLL versions. Feature flags, settings, and EkDB metadata drive behavioral differences between customers.

tag

Version Identity

The deployment version is the Docker image tag, typically a commit hash or latest. This tag is the single source of truth for what code is running in any environment.

check_circle

Current State

Unified deployment with config-driven behavior adaptation.
JsonExtensionData for handling custom fields without breaking deserialization.
Separate connector hashes per build.
Sandbox-to-production promotion path.

warning

Breaking Changes

When an ERP API introduces breaking changes, a new connector version is created.
Migration tooling assists with data transition.
Both versions can coexist during transition period.

rocket_launch

Roadmap

Planned

Low-code connector editor (Maestro).
Marketplace for community connectors.
AI-assisted field mapping for new ERP integrations.

account_tree

Dependency Handling

swap_horiz

ERP Version Differences

Version differences between ERP releases are handled inside each connector's client and DTOs. The connector layer absorbs API changes so that the rest of the platform sees a stable interface.

data_object

JsonExtensionData

DTOs use the [JsonExtensionData] attribute to capture custom and extra fields from external API responses without breaking deserialization. This ensures forward-compatibility when the external API adds new fields.

inventory

Isolated NuGet Packages

Each connector owns its own NuGet packages. There is no shared ERP SDK across connectors. This isolation prevents version conflicts and allows each connector to upgrade its dependencies independently.

// Example: Capturing extra fields from NetSuite API responses
public class NetSuiteCustomerDto
{
    public string Id { get; set; }
    public string CompanyName { get; set; }
    public string Email { get; set; }

    // Captures any fields not mapped to properties above
    [JsonExtensionData]
    public Dictionary<string, JsonElement> ExtensionData { get; set; }
}

info

Layered Dependency Model

Clarity uses a layered dependency model rather than strict plugin isolation. Core plugins (ConnectCore, Payments, Invoicing, Sales) are allowed to reference each other within domain boundaries — for example, the eCommerce plugin orchestrates products, sales, and payments as a composite plugin. However, dependency direction is strictly enforced: plugins depend on Core, never the reverse. Connectors depend on ConnectCore but not on each other. Client-specific customizations interact with plugins exclusively through the pipeline hook system, never by direct code reference. This approach provides practical modularity while acknowledging the reality that a payments platform requires coordination between invoicing, sales, and payment processing.