[REQUEST] Add pipeline section in `config.json` for pp

Is your feature request related to a problem? Please describe. The pipeline section of config.json controls only part of pipeline parallelism. The engine reads activation_checkpoint_interval, pipe_partitioned, grad_partitioned and use_reentrant from it. The number of stages, the partition method and the schedule are set in Python, through the PipelineModule constructor and the choice of module class (see here).

The pipeline section already defines stages, partition and seed_layers in get_pipeline_config (See here), but nothing reads them.

Tensor parallelism and expert parallelism are both driven from the config, through tensor_parallel.autotp_size and expert_parallel.autoep_size.

Describe the solution you'd like Make the existing pipeline section the place to configure pipeline parallelism.

{
  "pipeline": {
    "stages": 4,
    "partition": "parameters",
    "schedule": "dualpipev",
    "activation_checkpoint_interval": 0
  }
}

• stages: the number of pipeline stages.
• partition: the partition method.
• schedule: 1f1b or dualpipev. deepspeed.initialize() selects the engine from this key. Today it selects by module class.

Existing code keeps working. Values passed to the PipelineModule constructor stay valid when the config does not set them.

Additional context

Open questions for discussion.

1. PipelineModule partitions its layers in __init__ (see here), before deepspeed.initialize() reads the config. Should partitioning move to initialize(), or should the module be re-partitioned there?
2. Which value wins when the constructor and the config disagree?

I am happy to work on this.

cc: @delock