> ## Documentation Index
> Fetch the complete documentation index at: https://hexelstudio.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Connectors

> Sync data from Amazon S3 into knowledge stores with scheduled ingestion.

Use a connector when you have data in S3 (or another external system) that changes frequently and you want agents to always search the current content — without uploading files by hand every time something updates. For instance, if your engineering team pushes updated runbooks to a bucket nightly, a connector ensures your on-call agent always retrieves the latest version.

Connectors keep knowledge stores in sync with external systems. You configure a data source, run syncs on demand or on a schedule, and the platform ingests content automatically.

## Available sources

| Source    | Status    | Use for                              |
| --------- | --------- | ------------------------------------ |
| Amazon S3 | Available | Object storage buckets of documents. |

<Note>
  Additional connector sources are on the roadmap. Only Amazon S3 is available today. Check the [Console](https://console.hexelstudio.com) and [Changelog](/docs/changelog) for updates.
</Note>

## Key concepts

| Term            | Meaning                                                                |
| --------------- | ---------------------------------------------------------------------- |
| **Data source** | A configured connection to an external system.                         |
| **Sync**        | A single ingestion run that reads, parses, chunks, and embeds content. |
| **Schedule**    | How often a source syncs automatically.                                |

## Data source states

| State     | Meaning                             |
| --------- | ----------------------------------- |
| `active`  | Connected and ready to sync.        |
| `paused`  | Syncs suspended; resume to restart. |
| `error`   | Connection or configuration issue.  |
| `deleted` | Soft-deleted; no longer accessible. |

## Sync run states

| State       | Meaning                                   |
| ----------- | ----------------------------------------- |
| `pending`   | Queued, not yet started.                  |
| `running`   | Ingesting content.                        |
| `completed` | Finished successfully.                    |
| `failed`    | Encountered an error; check sync details. |

## How it works

<img src="https://mintcdn.com/hexelstudio-2127951d/utVkRjxsT1DGYlO5/assets/diagrams/datasource-sync.png?fit=max&auto=format&n=utVkRjxsT1DGYlO5&q=85&s=5b9d38c7e85bd3d638fd4922363ef1af" alt="Datasource sync flow" width="1536" height="1024" data-path="assets/diagrams/datasource-sync.png" />

## Create a data source

```bash theme={"dark"}
curl -X POST https://api.hexelstudio.com/data/v1/datasources \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "docs-bucket",
    "source": "s3",
    "config": {
      "bucket": "my-company-docs",
      "region": "us-east-1",
      "max_object_size_mb": 50
    },
    "knowledge_store_id": "YOUR_STORE_ID"
  }'
```

### Request parameters

<ParamField body="name" type="string" required>
  Display name for the data source.
</ParamField>

<ParamField body="source" type="string" required>
  Connector type. Currently only `s3`.
</ParamField>

<ParamField body="config" type="object" required>
  Source-specific configuration.
</ParamField>

<ParamField body="config.bucket" type="string" required>
  S3 bucket name.
</ParamField>

<ParamField body="config.region" type="string" required>
  AWS region of the bucket (e.g., `us-east-1`).
</ParamField>

<ParamField body="config.max_object_size_mb" type="integer" default="50">
  Maximum object size to ingest, in megabytes. Range: 1–500.
</ParamField>

<ParamField body="knowledge_store_id" type="string" required>
  Target knowledge store for ingested content.
</ParamField>

### Response

<ResponseField name="id" type="string">
  Unique data source identifier.
</ResponseField>

<ResponseField name="state" type="string">
  Current state: `active`, `paused`, `error`, or `deleted`.
</ResponseField>

<ResponseField name="created_at" type="string">
  ISO 8601 creation timestamp.
</ResponseField>

## Trigger a sync

```bash theme={"dark"}
curl -X POST https://api.hexelstudio.com/data/v1/datasources/YOUR_SOURCE_ID/sync \
  -H "Authorization: Bearer $TOKEN"
```

## Pause and resume

```bash theme={"dark"}
# Pause
curl -X POST https://api.hexelstudio.com/data/v1/datasources/YOUR_SOURCE_ID/pause \
  -H "Authorization: Bearer $TOKEN"

# Resume
curl -X POST https://api.hexelstudio.com/data/v1/datasources/YOUR_SOURCE_ID/resume \
  -H "Authorization: Bearer $TOKEN"
```

## List data sources

```bash theme={"dark"}
curl "https://api.hexelstudio.com/data/v1/datasources?page=1&page_size=20" \
  -H "Authorization: Bearer $TOKEN"
```

## Get sync history

```bash theme={"dark"}
curl "https://api.hexelstudio.com/data/v1/datasources/YOUR_SOURCE_ID/syncs" \
  -H "Authorization: Bearer $TOKEN"
```

## Errors

| `code`                   | HTTP | When                                                                       |
| ------------------------ | ---- | -------------------------------------------------------------------------- |
| `VALIDATION`             | 400  | Missing `bucket`/`region`, or `max_object_size_mb` out of range (1–500).   |
| `AUTH.PERMISSION_DENIED` | 403  | Missing `data-platform:connector:read` or `data-platform:connector:write`. |
| `RESOURCE.NOT_FOUND`     | 404  | The data source doesn't exist.                                             |

## Security

| Concern          | Detail                                                                                            |
| ---------------- | ------------------------------------------------------------------------------------------------- |
| Read permission  | `data-platform:connector:read` — list, get, and view sync history.                                |
| Write permission | `data-platform:connector:write` — create, sync, pause, resume, update, and delete.                |
| S3 access        | Grant the platform read access to the S3 bucket you connect. Hexel does not write to your bucket. |
| Scoping          | Data sources are scoped to your organization, workspace, and environment.                         |
| Storage          | Ingested content is stored on platform-managed infrastructure, not back in your S3 bucket.        |

## Troubleshooting

<AccordionGroup>
  <Accordion title="A sync fails or returns no documents">
    Confirm the bucket and region are correct and that the platform has read access to the bucket. Check the source's sync history for the failing run; a `failed` sync records the cause.
  </Accordion>

  <Accordion title="Content is stale">
    A data source only refreshes when a sync runs. Schedule recurring syncs, or trigger one manually after the source changes.
  </Accordion>

  <Accordion title="Large objects are skipped">
    Objects above `max_object_size_mb` (default 50, max 500) are not ingested. Raise the limit on the data source if you need larger files.
  </Accordion>
</AccordionGroup>

## Rate limits

10,000 requests per organization per hour. Pagination: `page` >= 1, `page_size` 1–100 (default 20).

<AccordionGroup>
  <Accordion title="Common errors">
    | Code  | Meaning                                                    |
    | ----- | ---------------------------------------------------------- |
    | `404` | Data source not found or deleted.                          |
    | `409` | Sync already running for this source.                      |
    | `422` | Invalid config (e.g., max\_object\_size\_mb out of range). |
    | `429` | Rate limit exceeded.                                       |
  </Accordion>

  <Accordion title="IAM permissions for S3">
    The platform needs `s3:GetObject` and `s3:ListBucket` on the target bucket. Configure a cross-account IAM role or bucket policy granting these permissions to Hexel's ingestion service. See [Authentication](/docs/getting-started/authentication) for the trust policy.
  </Accordion>
</AccordionGroup>

## Related pages

<CardGroup cols={2}>
  <Card title="Syncs" icon="arrows-rotate" href="/docs/data-platform/syncs">
    Monitor sync runs across your environment.
  </Card>

  <Card title="Knowledge Stores" icon="book" href="/docs/data-platform/knowledge-stores">
    Where synced content lands.
  </Card>

  <Card title="Documents & Uploads" icon="file-arrow-up" href="/docs/data-platform/documents">
    Add one-off files directly.
  </Card>

  <Card title="Data Platform Overview" icon="database" href="/docs/data-platform/overview">
    The full data model.
  </Card>
</CardGroup>

## Next steps

Continue to [Documents & Uploads](/docs/data-platform/documents).