1
- # RapidPro Indexer
1
+ # Indexer
2
2
3
3
[ ![ Build Status] ( https://github.com/nyaruka/rp-indexer/workflows/CI/badge.svg )] ( https://github.com/nyaruka/rp-indexer/actions?query=workflow%3ACI )
4
4
[ ![ codecov] ( https://codecov.io/gh/nyaruka/rp-indexer/branch/main/graph/badge.svg )] ( https://codecov.io/gh/nyaruka/rp-indexer )
5
5
[ ![ Go Report Card] ( https://goreportcard.com/badge/github.com/nyaruka/rp-indexer )] ( https://goreportcard.com/report/github.com/nyaruka/rp-indexer )
6
6
7
- Simple service for indexing RapidPro contacts into ElasticSearch .
7
+ Service for indexing RapidPro/TextIt contacts into Elasticsearch .
8
8
9
- # Deploying
9
+ ## Deploying
10
10
11
- As Indexer is a Go application, it compiles to a binary and that binary along with the config file is all
11
+ As it is a Go application, it compiles to a binary and that binary along with the config file is all
12
12
you need to run it on your server. You can find bundles for each platform in the
13
- [ releases directory] ( https://github.com/nyaruka/rp-indexer/releases ) . You should only run a single indexer
13
+ [ releases directory] ( https://github.com/nyaruka/rp-indexer/releases ) . You should only run a single
14
14
instance for a deployment.
15
15
16
- Indexer can run in two modes:
16
+ It can run in two modes:
17
17
18
18
1 ) the default mode, which simply queries the ElasticSearch database, finds the most recently
19
- modified contact, then on a schedule queries the ` contacts_contact ` table on the RapidPro
19
+ modified contact, then on a schedule queries the ` contacts_contact ` table in the
20
20
database for contacts to add or delete. You should run this as a long running service which
21
- constantly keeps ElasticSearch in sync with your RapidPro contacts.
21
+ constantly keeps ElasticSearch in sync with your contacts.
22
22
23
23
2 ) a rebuild mode, started with ` --rebuild ` . This builds a brand new index from nothing, querying
24
24
all contacts on RapidPro. Once complete, this switches out the alias for the contact index
25
25
with the newly build index. This can be run on a cron (in parallel with the mode above) to rebuild
26
26
your index occasionally to get rid of bloat.
27
27
28
- # Configuration
28
+ ## Configuration
29
29
30
- Indexer uses a tiered configuration system, each option takes precendence over the ones above it:
30
+ The service uses a tiered configuration system, each option takes precendence over the ones above it:
31
31
32
32
1 . The configuration file
33
33
2 . Environment variables starting with ` INDEXER_ `
34
34
3 . Command line parameters
35
35
36
- We recommend running Indexer with no changes to the configuration and no parameters, using only
36
+ We recommend running it with no changes to the configuration and no parameters, using only
37
37
environment variables to configure it. You can use ` % rp-indexer --help ` to see a list of the
38
38
environment variables and parameters and for more details on each option.
39
39
40
- ## RapidPro Configuration
40
+ ### RapidPro
41
41
42
42
For use with RapidPro, you will want to configure these settings:
43
43
@@ -48,9 +48,42 @@ Recommended settings for error reporting:
48
48
49
49
* ` INDEXER_SENTRY_DSN ` : The DSN to use when logging errors to Sentry
50
50
51
- # Development
51
+ ### Reference
52
52
53
- Once you've checked out the code, you can build Indexer with:
53
+ These are the configuration options that can be provided as parameters or environment variables. If using environment
54
+ varibles, convert to uppercase, replace dashes with underscores and prefix the name with ` INDEXER_ ` , e.g. ` -log-level `
55
+ becomes ` INDEXER_LOG_LEVEL ` .
56
+
57
+ ```
58
+ -cleanup
59
+ whether to remove old indexes after a rebuild
60
+ -db string
61
+ the connection string for our database (default "postgres://localhost/rapidpro?sslmode=disable")
62
+ -debug-conf
63
+ print where config values are coming from
64
+ -elastic-url string
65
+ the url for our elastic search instance (default "http://localhost:9200")
66
+ -help
67
+ print usage information
68
+ -index string
69
+ the alias for our contact index (default "contacts")
70
+ -librato-username
71
+ the Librato username for metrics reporting
72
+ -librato-token
73
+ the Librato token for metrics reporting
74
+ -log-level string
75
+ the log level, one of error, warn, info, debug (default "info")
76
+ -poll int
77
+ the number of seconds to wait between checking for updated contacts (default 5)
78
+ -rebuild
79
+ whether to rebuild the index, swapping it when complete, then exiting (default false)
80
+ -sentry-dsn string
81
+ the sentry configuration to log errors to, if any
82
+ ```
83
+
84
+ ## Development
85
+
86
+ Once you've checked out the code, you can build the service with:
54
87
55
88
```
56
89
go build github.com/nyaruka/rp-indexer/cmd/rp-indexer
@@ -69,41 +102,3 @@ To run all of the tests:
69
102
```
70
103
go test ./... -p=1
71
104
```
72
-
73
- # Usage
74
-
75
- ```
76
- Indexes RapidPro contacts to ElasticSearch
77
-
78
- Usage of indexer:
79
- -cleanup
80
- whether to remove old indexes after a rebuild
81
- -db string
82
- the connection string for our database (default "postgres://localhost/rapidpro?sslmode=disable")
83
- -debug-conf
84
- print where config values are coming from
85
- -elastic-url string
86
- the url for our elastic search instance (default "http://localhost:9200")
87
- -help
88
- print usage information
89
- -index string
90
- the alias for our contact index (default "contacts")
91
- -log-level string
92
- the log level, one of error, warn, info, debug (default "info")
93
- -poll int
94
- the number of seconds to wait between checking for updated contacts (default 5)
95
- -rebuild
96
- whether to rebuild the index, swapping it when complete, then exiting (default false)
97
- -sentry-dsn string
98
- the sentry configuration to log errors to, if any
99
-
100
- Environment variables:
101
- INDEXER_CLEANUP - bool
102
- INDEXER_DB - string
103
- INDEXER_ELASTIC_URL - string
104
- INDEXER_INDEX - string
105
- INDEXER_LOG_LEVEL - string
106
- INDEXER_POLL - int
107
- INDEXER_REBUILD - bool
108
- INDEXER_SENTRY_DSN - string
109
- ```
0 commit comments