Running all combinations of variants in promptflow
Is your feature request related to a problem? Please describe. According to https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-tune-prompts-using-variants?view=azureml-api-2#run-all-variants-with-a-single-row-of-data-and-check-outputs, the more node with variants, the more combinations you will miss in running
Each time you can only select one LLM node with variants to run while other LLM nodes will use the default variant.
This is a must-have feature for anyone familiar with grid-search in machine learning. At the beginning, I thought it was a bug in my code but according to the docs it is expected.
Describe the solution you'd like Instead of only being able to change one node variants at a time, support running flows with the combination of all variants, including the non-default ones.
Describe alternatives you've considered Use only one node with variants, but this obviously deprecate the idea of variants.
Additional context Relates to https://github.com/microsoft/promptflow/issues/944, which is labelled with long-term without arguments.
Source: microsoft/promptflow