S3FailureTest test no longer functional
Author: romain-intelCreated Nov 22, 2021Updated Sep 4, 2026
The S3FailureTest relies on the presence of a string TEST_S3_RETRY to determine whether or not a retry is actually happening. That string is printed in the aws_retry function. Unfortunately, with the new datastore, the aws_retry function is not used to push artifacts out (as it was prior to the new datastore) since the new datastore relies on the S3 client which doesn't use aws_retry everywhere.
We should do one or more of the following:
- make the use of
aws_retrymore pervasive in the S3 client - add this string elsewhere as well so that it is triggered properly.
Currently, the test may pass but it is because the string happens to be present in the mflog code but not in the main datastore code.
Source: Netflix/metaflow