Skip to content

Commit

Permalink
manual: Indexer systemd extra params (#2256)
Browse files Browse the repository at this point in the history
---------

Co-authored-by: djklim87 <[email protected]>
Co-authored-by: Sergey Nikolaev <[email protected]>
  • Loading branch information
3 people authored Jun 25, 2024
1 parent e3cda52 commit a38a476
Showing 1 changed file with 22 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ In the typical scenario, indexer does the following:
* Writes the table files
* (Optional) Informs the search server about the new table which triggers table rotation

## Indexer tool
## Indexer tool
The `indexer` tool is used to create plain tables in Manticore Search. It has a general syntax of:

```shell
Expand Down Expand Up @@ -51,7 +51,9 @@ The exit codes for indexer are as follows:
* 1: there was a problem while indexing (and if `--rotate` was specified, it was skipped) or an operation emitted a warning
* 2: indexing went OK, but the `--rotate` attempt failed
Also, you can run `indexer` via a systemctl unit file:
### Indexer systemd service
You can also start `indexer` using the following systemctl unit file:
```shell
systemctl start --no-block manticore-indexer
Expand All @@ -62,7 +64,22 @@ Or, in case you want to build a specific table:
```shell
systemctl start --no-block manticore-indexer@specific-table-name
```
Find more information about scheduling `indexer` via systemd below.
Use the `systemctl set-environment INDEXER_CONFIG` command to run the Indexer with a custom configuration, which replaces the default settings.
The `systemctl set-environment INDEXER_ARGS` command lets you add custom startup options for the Indexer. For a complete list of command-line options, see [here](../../Data_creation_and_modification/Adding_data_from_external_storages/Plain_tables_creation.md#Indexer-command-line-arguments).
For instance, to start the Indexer in quiet mode, run:
```bash
systemctl set-environment INDEXER_ARGS='--quiet'
systemctl restart manticore-indexer
```
To revert the changes, run:
```bash
systemctl set-environment INDEXER_ARGS=''
systemctl restart manticore-indexer
```
### Indexer command line arguments
* `--config <file>` (`-c <file>` for short) tells `indexer` to use the given file as its configuration. Normally, it will look for `manticore.conf` in the installation directory (e.g. `/etc/manticoresearch/manticore.conf`), followed by the current directory you are in when calling `indexer` from the shell. This is most useful in shared environments where the binary files are installed in a global folder, e.g. `/usr/bin/`, but you want to provide users with the ability to make their own custom Manticore set-ups, or if you want to run multiple instances on a single server. In cases like those you could allow them to create their own `manticore.conf` files and pass them to `indexer` with this option. For example:
Expand Down Expand Up @@ -223,7 +240,7 @@ mem_limit = 256M
Plain table building RAM usage limit. Optional, default is 128 MB. Enforced memory usage limit that the `indexer` will not go above. Can be specified in bytes, or kilobytes (using K postfix), or megabytes (using M postfix); see the example. This limit will be automatically raised if set to an extremely low value causing I/O buffers to be less than 8 KB; the exact lower bound for that depends on the built data size. If the buffers are less than 256 KB, a warning will be produced.
The maximum possible limit is 2047M. Too low values can hurt plain table building speed, but 256M to 1024M should be enough for most, if not all datasets. Setting this value too high can cause SQL server timeouts. During the document collection phase, there will be periods when the memory buffer is partially sorted and no communication with the database is performed; and the database server can timeout. You can resolve that either by raising timeouts on the SQL server side or by lowering `mem_limit`.
The maximum possible limit is 2047M. Too low values can hurt plain table building speed, but 256M to 1024M should be enough for most, if not all datasets. Setting this value too high can cause SQL server timeouts. During the document collection phase, there will be periods when the memory buffer is partially sorted and no communication with the database is performed; and the database server can timeout. You can resolve that either by raising timeouts on the SQL server side or by lowering `mem_limit`.
#### on_file_field_error
Expand All @@ -245,7 +262,7 @@ Note that with `on_file_field_error = skip_document` documents will only be igno
```ini
write_buffer = 4M
```
```
Write buffer size, bytes. Optional, default is 1MB. Write buffers are used to write both temporary and final table files when indexing. Larger buffers reduce the number of required disk writes. Memory for the buffers is allocated in addition to [mem_limit](../../Data_creation_and_modification/Adding_data_from_external_storages/Plain_tables_creation.md#mem_limit). Note that several (currently up to 4) buffers for different files will be allocated, proportionally increasing the RAM usage.
Expand Down

0 comments on commit a38a476

Please sign in to comment.