[Speculative decoding] Adding configuration object for speculative decoding

[cadedaniel]

This PR is a subset of PR 6/9: Integrate speculative decoding with LLMEngine. in the speculative decoding open sourcing plan. It introduces a SpeculativeConfig and plumbs it through to the executors. The new flags are as follows:

parser.add_argument(
    '--speculative-model',
    type=str,
    default=None,
    help=
    'The name of the draft model to be used in speculative decoding.')

parser.add_argument(
    '--num-speculative-tokens',
    type=int,
    default=None,
    help='The number of speculative tokens to sample from '
    'the draft model in speculative decoding')

In the future we can extend these flags to support non-draft-model speculative decoding.

Testing

• We assert that the GPUExecutor raises AssertionError when spec decode is enabled. This verifies the config works.

Misc.

• This wraps the various engine configs class in an EngineConfig. This removes the need to do parallel_config = engine_configs[2] and device_config = engine_configs[4].

[cadedaniel]

ready for review @LiuXiaoxuanPKU

[LiuXiaoxuanPKU]

LGTM!