|
1 | 1 | # Conduit Connectors |
2 | 2 |
|
3 | | -Connectors are an integral part of Conduit. Conduit ships with a couple of |
4 | | -[connectors that are directly built into](#built-in-connectors) the service, |
5 | | -but it can also be expanded with additional |
6 | | -[standalone connectors](#standalone-connectors). |
7 | | - |
8 | | -One of the main differences between Conduit connectors and those that you might find from other services is |
9 | | -that all Conduit connectors are Change Data Capture-first (CDC). |
10 | | - |
11 | | -CDC allows your pipeline to only get the changes that have happened over time |
12 | | -instead of pulling down an entire upstream data store and then tracking diffs |
13 | | -between some period of time. |
14 | | -This is critical for building real-time, event-driven pipelines and applications. |
15 | | -But, we'll note where connectors do or do not have CDC capabilities. |
| 3 | +> [!IMPORTANT] |
| 4 | +> The content of this page has been moved to [Conduit's documentation website](https://conduit.io/docs/core-concepts#connector). |
16 | 5 |
|
17 | 6 | ## Roadmap & Feedback |
18 | 7 |
|
19 | | -If you need support for a particular data store that doesn't exist on the connector list, check out the list of |
20 | | -requested [source connectors](https://github.com/ConduitIO/conduit/issues?q=is%3Aissue+label%3Aconnector%3Asource+is%3Aopen) |
21 | | -and the list of requested [destination connectors](https://github.com/ConduitIO/conduit/issues?q=is%3Aissue+label%3Aconnector%3Adestination+is%3Aopen). |
22 | | -Give the issue a `+1` if you really need that connector. The upvote will help the team understand demand for any |
23 | | -particular connector. If you find that an issue hasn't been created for your data store, please create a new issue in |
24 | | -the Conduit repo. |
| 8 | +> [!IMPORTANT] |
| 9 | +> The content of this page has been moved to [Conduit's documentation website](https://conduit.io/docs/future/roadmap). |
25 | 10 |
|
26 | 11 | ## More about Connectors |
27 | 12 |
|
28 | 13 | ### Built-in connectors |
29 | 14 |
|
30 | | -To help developers bootstrap pipelines much more quickly, |
31 | | -Conduit ships with several built-in connectors by default. |
32 | | -This includes the Postgres, File, Random Data Generator, Kafka and Amazon S3 connectors. |
33 | | - |
34 | | -If you are creating and distributing customized, pre-built Conduit binaries yourself, |
35 | | -you may want to modify the list of built-in connectors, too. |
36 | | -By modifying the list of built-in connectors, |
37 | | -your compiled binary will include a set of pre-installed connectors specific to your end users' needs, |
38 | | -and it can be run by others, as is, without them having to follow any additional installation instructions. |
39 | | - |
40 | | -If you want to create your own Conduit binary with a different set of built-in connectors, |
41 | | -you can build-in an [existing connector](#the-list). |
42 | | -Alternatively, you can also create your own |
43 | | -[using the Conduit connector template or the Conduit Connector SDK](https://conduit.io/docs/developing/connectors). |
44 | | - |
45 | | -Once you have chosen a connector to be built-in, you can: |
46 | | - |
47 | | -- Download the new package and its dependencies: `go get "github.com/foo/conduit-connector-new"` |
48 | | -- Import the Go module defining the connector |
49 | | -into the [builtin registry](https://github.com/ConduitIO/conduit/blob/main/pkg/plugin/connector/builtin/registry.go) |
50 | | -and add a new key to `DefaultDispenserFactories`: |
51 | | - |
52 | | -```diff |
53 | | -package builtin |
54 | | - |
55 | | -import ( |
56 | | - // ... |
57 | | - file "github.com/conduitio/conduit-connector-file" |
58 | | - // ... |
59 | | -+ myNewConnector "github.com/foo/conduit-connector-new" |
60 | | -) |
61 | | - |
62 | | -var ( |
63 | | - // DefaultDispenserFactories contains default dispenser factories for |
64 | | - // built-in plugins. The key of the map is the import path of the module |
65 | | - // containing the connector implementation. |
66 | | - DefaultDispenserFactories = map[string]DispenserFactory{ |
67 | | - "github.com/conduitio/conduit-connector-file": sdkDispenserFactory(file.Connector), |
68 | | - // ... |
69 | | -+ "github.com/foo/conduit-connector-new": sdkDispenserFactory(myNewConnector.Connector), |
70 | | - } |
71 | | -) |
72 | | -``` |
73 | | - |
74 | | -- Run `make` |
75 | | -- You now have a binary with your own, custom set of built-in connectors! 🎉 |
| 15 | +> [!IMPORTANT] |
| 16 | +> The content of this page has been moved to Conduit's documentation website |
| 17 | +> [here](https://conduit.io/docs/core-concepts#built-in-connector), and [here](/docs/using/connectors/additional-built-in-plugins). |
76 | 18 |
|
77 | 19 | ### Standalone connectors |
78 | 20 |
|
79 | | -In addition to built-in connectors, Conduit can be used together with standalone connectors |
80 | | -to enable even more data streaming use cases. |
81 | | -The Conduit team and other community developers create and maintain standalone connectors. |
82 | | - |
83 | | -Learn more about how you can install [standalone connectors to Conduit here](https://conduit.io/docs/using/connectors/installing). |
| 21 | +> [!IMPORTANT] |
| 22 | +> The content of this page has been moved to [Conduit's documentation website](https://conduit.io/docs/core-concepts#standalone-connector). |
84 | 23 |
|
85 | 24 | ### Source & Destination |
86 | 25 |
|
87 | | -Source means the connector has the ability to get data from an upstream data store. Destination means the connector can |
88 | | -to write to a downstream data store. |
| 26 | +> [!IMPORTANT] |
| 27 | +> The content of this page has been moved to [Conduit's documentation website](https://conduit.io/docs/core-concepts#source-and-destination). |
89 | 28 |
|
90 | 29 | ### The List |
91 | 30 |
|
|
0 commit comments