Skip to content

Configuration

Jump to bottom
nathanmarz edited this page Jan 29, 2012 · 1 revision

Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on a topology by topology basis, whereas other configurations can be modified per topology.

Every configuration has a default value defined in defaults.yaml in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using StormSubmitter. However, the topology-specific configuration can only override configs prefixed with "TOPOLOGY".

Storm 0.7.0 and onwards lets you override configuration on a per-bolt/per-spout basis. The only configurations that can be overriden this way are:

  1. "topology.debug"
  2. "topology.max.spout.pending"
  3. "topology.max.task.parallelism"
  4. "topology.kryo.register": This works a little bit differently than the other ones, since the serializations will be available to all components in the topology. More details on Serialization.

The Java API lets you specify component specific configurations in two ways:

  1. Internally: Override getComponentConfiguration in any spout or bolt and return the component-specific configuration map.
  2. Externally: setSpout and setBolt in TopologyBuilder return an object with methods addConfiguration and addConfigurations that can be used to override the configurations for the component.

The preference order for configuration values is defaults.yaml < storm.yaml < topology specific configuration < internal component specific configuration < external component specific configuration.

Resources:

Clone this wiki locally