[FEATURE]: Add privacy-aware business context for AI Assistant experiment design

Pre-flight checklist

• I have searched existing issues and this isn't a duplicate.
• I'm reporting against PyCaret 4.0 (the OOP Experiment(...).fit(df) API), not 3.x.
• I've confirmed the feature I'm asking about is not on the 4.0 kill list (docs/revamp/KILL_LIST.md).

Issue description

First of all, thank you for the impressive PyCaret 4.0 revamp. I really like the direction of PyCaret as an open-source, self-hosted ML platform with an Engine, Control Plane, Web UI, deployments, monitoring, and LLM-assisted experiment design.

I would like to propose a small AI Assistant enhancement: add an optional, privacy-aware business context field for experiment design.

The current AI Assistant can reason from the dataset profile and user goal, but many real-world ML decisions also depend on business context that cannot be inferred from data alone, such as:

• target definition
• false-positive vs false-negative cost
• preferred business metric
• interpretability requirements
• leakage-prone columns
• deployment constraints, such as batch vs online scoring
• monitoring priorities by business segment or KPI

Why this matters

For example, in a churn prediction project, the best experiment design may depend on whether false negatives are more expensive than false positives, whether the model must be explainable to customer success teams, whether some columns are unavailable at inference time, and whether the first deployment is daily batch scoring instead of a real-time endpoint.

Providing this context could help the AI Assistant make better advisory suggestions for:

• task type
• target confirmation
• primary metric
• validation strategy
• preprocessing choices
• model shortlist
• leakage warnings
• thresholding considerations
• deployment risk review
• monitoring recommendations

Privacy and trust model

Business context can be sensitive, so this should be explicit and opt-in.

Suggested principles:

• The field should be optional.
• Users should explicitly choose whether to include business context in AI Assistant requests.
• The UI should clearly explain that this context may be sent to the configured LLM provider when enabled.
• The feature should fit PyCaret’s self-hosted / bring-your-own-provider model.
• It should work well with trusted private providers, local providers, or enterprise-controlled LLM endpoints.
• Business context should not be silently sent to any LLM.
• The LLM should remain advisory only and should not execute actions directly.

Proposed scope

Affected modules:

• services/api: persist optional business context and include it in selected LLM consultations
• apps/web: add UI input and privacy copy in the experiment creation / AI Assistant flow
• docs/revamp: document behavior, intended usage, and privacy expectations
• packages/engine: no engine changes expected for the first version

Suggested MVP:

1. Add an optional business_context field at the project or experiment level.
2. Add a textarea in the experiment creation / AI Assistant flow.
3. Add an explicit toggle such as: Include this business context in AI Assistant suggestions.
4. Include the context in the experiment_design LLM consultation only when enabled.
5. Keep the existing advisory response pattern: suggested config, reasoning summary, and risk flags.

Out of scope

To keep the first version small and reviewable, this proposal does not include:

• document upload
• external knowledge-base ingestion
• vector search
• autonomous model execution
• autonomous deployment
• new data-source integrations
• changes to the deterministic engine contract

Example

A user could provide the following business context:

We are predicting subscription churn within the next 30 days.
False negatives are more expensive than false positives.
The model must be explainable to the customer success team.
Columns created after cancellation should be treated as leakage.
The first deployment will be daily batch scoring, not a real-time endpoint.
Recall should be favored for model selection.

The AI Assistant could then suggest recall or F-beta as the primary metric, flag likely leakage columns, prefer interpretable model candidates, recommend threshold tuning, and suggest batch deployment considerations.

Contribution

I would be happy to contribute an initial PR for this feature if the maintainers agree with the scope.

A first PR could focus only on:

• adding the metadata field
• wiring it into the experiment_design AI Assistant flow
• adding UI copy and tests
• updating docs

Minimal reproducible example

None

Expected behavior

• Existing AI Assistant flows continue to work without business context.
• Business context is included in LLM requests only when the user explicitly enables it.
• The UI clearly communicates the privacy implication.
• The backend persists the optional field at the agreed project or experiment level.
• experiment_design uses the context when enabled.
• Tests cover persistence, prompt construction, and frontend behavior.
• Documentation explains the feature and privacy model.

Actual behavior / traceback

None

Environment

None