StructuredDataRegressor in AutoKeras 3.0.0 produces implausible predictions for tabular regression target while other versions remain numerically plausible

I am using StructuredDataRegressor for a tabular regression task as part of a reproducibility study comparing AutoML framework versions. I observed a version-specific issue with AutoKeras 3.0.0 where one target produces numerically implausible predictions that are far outside the target scale.

This does not appear to be only a case of poor predictive performance. Other AutoKeras versions tested in the same experimental setup also do not always perform strongly, but they produce predictions in a realistic numerical range. In contrast, AutoKeras 3.0.0 produces predictions that are orders of magnitude outside the observed target values for one of the targets.

Environment

Python: 3.10

autokeras==3.0.0
keras==3.12.1
keras-tuner==1.4.8
tensorflow==2.19.0
tensorboard==2.19.0
tensorboard-data-server==0.7.2

The experiment was run inside a Docker environment.

Task and dataset

The task is tabular regression on a recycled concrete aggregate dataset with two possible target variables.

Target CS:
- 638 available instances
- 12 features
- AutoKeras 3.0.0 shows poor and variable, but still numerically plausible, performance

Target FS:
- 139 available instances
- 12 features
- AutoKeras 3.0.0 produces implausible predictions far outside the target scale

For the FS target, the true values in the test split are approximately in the range of normal flexural strength values, for example around 2 to 9. However, AutoKeras 3.0.0 produces predictions such as 48, 108, 146, 206, and 334.

Minimal code

from autokeras import StructuredDataRegressor

automl = StructuredDataRegressor(
    max_trials=50,
    column_names=list(feature_names),
    loss="mean_absolute_error",
    seed=seed,
)

automl.fit(
    x=X_train,
    y=y_train,
    epochs=100,
    verbose=False,
)

y_pred = automl.predict(X_test).ravel()

Example output

For one FS run with seed 1001, the true test values were:

y_test = [
    5.2, 3.4, 6.0, 5.0, 7.2, 4.9, 5.0, 4.4, 3.1, 4.6,
    6.5, 4.7, 5.7, 5.2, 4.4, 5.1, 6.5, 8.9, 5.5, 5.8,
    2.0, 4.0, 5.5, 4.4, 6.4, 5.0, 5.0, 5.2, 5.1, 7.0,
    5.2, 6.9, 3.6, 6.8, 5.2, 5.7, 2.4, 4.9, 5.1, 1.9,
    5.8, 2.1
]

The corresponding predictions were:

y_pred = [
    108.0866, 48.9043, 133.1933, 120.1106, 146.9046,
    91.6808, 87.6355, 55.5626, 66.3953, 76.9426,
    206.6751, 82.3499, 101.7845, 93.9274, 73.9414,
    84.0815, 111.2073, 67.2150, 72.4622, 89.9336,
    55.3766, 60.7624, 105.8543, 74.5728, 125.2033,
    92.7562, 131.0857, 60.6819, 99.7832, 136.1026,
    97.6079, 136.8613, 77.4261, 334.2567, 113.5422,
    110.6380, 68.0948, 64.8043, 96.9645, 54.6361,
    125.5471, 50.5962
]

The resulting metrics for this run were:

R2   = -5522.6881
MAE  = 94.7583
MSE  = 11242.5527
RMSE = 106.0309

Across 30 independent runs for the FS target, the mean R² was approximately -5858, indicating that this was not an isolated single-run outlier.

Expected behavior

The model may perform poorly on this dataset, especially given the smaller sample size for the FS target. However, predictions should remain in a numerically plausible range for the target variable. Other tested AutoKeras versions produced weaker or variable performance, but still generated realistic prediction values.

Actual behavior

With AutoKeras 3.0.0 and the environment above, StructuredDataRegressor produces predictions that are far outside the target scale for the FS regression task. This leads to extremely negative R² values and RMSE values that are not comparable to other AutoKeras versions or other AutoML tools in the same experiment.

Additional observation

When overwrite=True was used, the behavior was less severe. However, in my reproducibility experiment I kept the setup fixed across framework generations and did not change this parameter, because the goal was to compare framework versions under the same experimental script. Other AutoKeras versions used with the same setup produced numerically plausible predictions.

The attached plot shows the R² distribution across tested AutoKeras generations. Generation 1 corresponds to AutoKeras 3.0.0 and shows extreme negative R² values for the FS target, while later generations remain within a realistic performance range. Generation 2 is empty on purpose because there was no suitable AutoKeras version available in this timeframe.

Question

Is this behavior expected for StructuredDataRegressor in AutoKeras 3.0.0 with TensorFlow 2.19.0 / Keras 3.12.1, or could this indicate a bug or unstable interaction in this specific version stack for small tabular regression datasets?