Integrations as data, not code

The fourth connector pull request in a row landed on my desk, and I caught myself reading the same code again. Not similar code. The same code, retyped. A function that refreshes an OAuth token after a 401. A loop that follows a next cursor until the API stops handing one back. A switch statement mapping the vendor’s error codes onto ours. A backoff helper with jitter. The part that actually made this connector different from the last one was maybe fifteen lines buried in the middle. Everything around it was boilerplate we’d written three times already and would write dozens more times before we were done.

I work on Shopware Nexus, Shopware’s integration platform. It went generally available in May 2026, and the short version is that a merchant connects their shop to the other systems they run their business on (an ERP, a PIM, a warehouse, an email provider) and builds workflows that keep data flowing between them. Somewhere underneath all of that, something has to actually talk to each of those third-party APIs. For a while, that something was a Go package per integration. One folder, one connector, hand-written.

That model works great for the first five connectors and starts to hurt around the fifteenth.

Why a module per integration stops scaling

The problem isn’t any single connector. It’s that each one re-implements the same four or five concerns, slightly differently, in code only its author fully understands.

Auth is the obvious one. Some APIs want OAuth client credentials, some want a bearer token you mint yourself, some want an API key in a header, one memorable one wanted the key in a query parameter (please don’t). Every connector grew its own little auth block. Pagination is the same story: cursor here, page number there, Link header somewhere else, and each was hand-rolled. Retries, error mapping, rate limiting: all copied, all subtly drifting.

Then there’s review. When a connector is code, reviewing a new one means reading a few hundred lines of Go to answer questions that are actually simple: what does it call, how does it authenticate, is it safe to retry. Those are data questions, but we’d buried them inside behavior, so every new integration became a real code review, and real code reviews take real senior time.

And deploys. A connector living in the binary means shipping a new binary to change it, which couples every integration’s release cadence to every other one’s. A one-line fix to how one connector paginates rode along with whatever else happened to be on main that day.

Past twenty connectors, the pattern was clear: we were maintaining twenty implementations of the same handful of ideas.

The connector as data

The reframe was to stop asking “what code does this integration need” and start asking “what facts about this API would a generic runner need to talk to it.” The answer turned out to be small and mostly declarative. Endpoints. What kind of auth. The shape of pagination. How fields map onto our internal model. The rate limits to respect. That’s a descriptor, and a descriptor is data.

Here’s a sketch of one for an invented “orders” API. Nothing here is a real internal schema, it’s just to show the shape:

name: acme-orders
version: 3
base_url: https://api.example.com/v2

auth:
  kind: oauth2_client_credentials
  token_url: https://api.example.com/oauth/token
  # no client_id or client_secret here.
  # the executor resolves them from the secrets store at run time.
  credentials_ref: secret://acme/orders

rate_limits:
  requests_per_second: 5

operations:
  list_orders:
    method: GET
    path: /orders
    idempotent: true                 # safe to retry
    pagination:
      style: cursor
      cursor_param: page_token
      cursor_path: $.next_page_token
      items_path: $.orders
  create_order:
    method: POST
    path: /orders
    idempotent: false                # at-most-once
    idempotency_key_header: Idempotency-Key

field_mappings:
  - from: $.id
    to: external_id
  - from: $.customer.email
    to: customer_email
  - from: $.total_cents
    to: amount
    transform: cents_to_decimal

The thing I want to call out is what is not in there. No client secret, no token. The descriptor holds a reference (secret://acme/orders) and nothing else; credential values live in a secrets store, and the executor resolves them at run time. That’s the property that makes descriptors safe to move around. A descriptor can show up in a log, a diff, a support ticket, or an AI chat transcript, and none of that leaks anything. The worst you learn from an exposed descriptor is which APIs a merchant talks to, never how to authenticate as them. I treat “a leaked descriptor leaks nothing” as a hard invariant, because the moment you allow one convenient exception you’ve lost the property entirely.

One executor, and why the hints matter

On the code side, all of this collapses into a single generic executor. The connector contract is three methods:

// Illustrative sketch. Names invented for this post,
// same idea as the descriptor above: keep behavior generic,
// push everything integration-specific into data.
type Integration interface {
    // Describe returns the descriptor: endpoints, auth kind,
    // pagination, field mappings, rate limits. Data, not behavior.
    Describe() Descriptor

    // Retryability tells the engine how a given operation
    // may be retried.
    Retryability(op string) RetryPolicy

    // Call runs one operation against the live API.
    Call(ctx context.Context, op string, in Input) (Output, error)
}

In the module-per-integration world, every connector was its own implementation of that interface. In the data-driven world, one generic type implements it, and everything it needs comes from the descriptor it was handed.

Retryability is the method that earns its keep, and the reason is that we run workflows on Temporal, a durable-execution engine that replays workflow history to recover state. Replay means an activity can run more than once, and the engine has to know whether that’s fine. So the retry policy is declared per operation. A read is idempotent: if a list_orders call gets retried during a replay, no harm done, you just fetch the same page again. A write is not: retrying create_order blindly would create duplicate orders, the kind of bug that turns into an awkward customer call. Writes are executed at-most-once, guarded by an idempotency key so that even if the request does go out twice, the third party dedupes it. Getting the read-versus-write classification right per operation, in data, is what lets one executor be safe under a replaying engine instead of subtly wrong.

Calling hosts you don’t control

There’s a part of this that took me a while to be comfortable with. The executor makes outbound HTTP calls to hosts that come from a descriptor a merchant supplied. That is a server on our infrastructure connecting to an address someone outside our infrastructure chose. If you’ve read anything about SSRF, you already see it.

Nothing stops a base_url from pointing at http://169.254.169.254/, the cloud metadata endpoint, or at an internal service on a private range only our network can reach. A misconfigured or malicious descriptor turns our own executor into a confused deputy that fetches things on the attacker’s behalf.

So before the executor dials anything, it resolves the hostname itself and checks the resolved addresses against a deny-list: loopback, link-local, and the private ranges. If it resolves into any of those, the request never leaves. The subtle bit is DNS rebinding. A hostname can pass the check on the first lookup and then resolve to 127.0.0.1 on the second, so if you resolve to validate and then let the HTTP client resolve again to connect, you’ve validated nothing. The fix is to resolve once, validate that specific IP, and dial that exact IP, so there’s no gap between the check and the connection for an attacker to slip a private address through. None of this is exotic, but it stays invisible until someone points a descriptor somewhere it shouldn’t go, and I’d rather it be boring than a headline.

Letting an AI draft it, and a human keep it honest

Descriptors are data, and data can be generated. This is where the AI-assisted side comes in. Point a model at an API’s OpenAPI spec or its docs, and it’s genuinely good at proposing a first draft: it reads the endpoints, guesses the auth kind, spots the pagination shape, sketches the field mappings. What used to be a couple of weeks of a human writing a connector from scratch turns into a draft in minutes.

But a draft is not a connector, and the design decision I’d defend hardest is that the model’s output does not persist on its own. The merchant sees the proposed descriptor in a chat interface, reviews it, edits what’s wrong (the model guesses the wrong rate limit more often than you’d like, and sometimes invents an endpoint that reads plausibly but isn’t real), and only after they accept it does anything get written. The human review step isn’t a UX nicety bolted on at the end. It’s the thing that makes generation safe. An AI drafting config that a person confirms is a very different risk profile from an AI writing config straight into a system that will then go call live APIs with real credentials. The first one I’ll ship. The second one I won’t.

Versioning, so nothing changes under your feet

The last piece is boring and I love it for that. Notice the version: 3 at the top of the descriptor. A deployed workflow pins the descriptor version it was built against. Editing a descriptor produces a new version; it does not retroactively change the integrations already running on the old one. So when someone tweaks a field mapping on Tuesday, the workflows that have been quietly syncing orders since last month keep behaving exactly as they did, until someone deliberately moves them to the new version. Configuration that silently changes running systems is how you get an incident that nobody can explain, because “we didn’t deploy anything” is technically true. Pinning the version makes the change explicit and the rollback trivial.

What I’d do differently

A few things I’d redo, honestly.

I versioned descriptor instances early and the descriptor schema late, and that was backwards. Adding a new optional field to the schema was painless. But the first time I needed to change how pagination was represented, every existing descriptor was written against the old shape and I had no migration path for them. I ended up doing it by hand, which does not scale and is exactly what this effort was supposed to kill. If I did it again, I’d treat the descriptor schema like a database schema from day one: every shape change is a migration, with an up path, written before I need it.

I also tested the executor against well-behaved sandbox APIs for far too long. Real third-party APIs misbehave in ways sandboxes don’t. They return 200 OK with an error in the body. They hand you a next cursor that points at the same page forever, which is an infinite loop waiting to happen. They rate-limit you without a Retry-After header, so your backoff is guessing. We eventually stood up a small collection of deliberately hostile fake APIs to test against, and every one of them found a real bug. I’d build that adversarial harness first next time, before the happy path, not after production taught me which lies to expect.

And the honest tension: putting transforms like cents_to_decimal in the descriptor was convenient right up until it became a small programming language. The gap between “declarative config” and “we accidentally built an interpreter, complete with its own bugs and no debugger” is thinner than it looks. I still think field mapping belongs in the descriptor. I’m less sure every transform does, and I’d draw that line more deliberately, earlier, than I did.

The through-line for all of it is the same. When an integration is code, every question about it is a code question, answered by a person reading the code. When an integration is data, those same questions become things you can read off the page and diff, and the one piece of behavior you have to trust, the executor, is the one piece you actually get to test to death. That trade has been worth it every time.