[BUG] MKS Robin E3 V1.1: I2C EEPROM failures with mks_robin_e3 STM32duino environment; Maple passes identical stress test
Did you test the latest bugfix-2.1.x code?
Yes, and the problem still exists.
Bug Description
I am seeing reproducible external I2C EEPROM failures on a Longer LK5 Pro using the stock MKS Robin E3 V1.1 motherboard (STM32F103RC).
The board uses the V1.1-specific Marlin board definition and its onboard 4 KB I2C EEPROM.
The problem occurs when current Marlin is built using the supported:
mks_robin_e3environment, which uses the STM32duino / STM32 HAL stack.
Normal EEPROM operations such as M500 can intermittently fail with:
Error writing to EEPROM!I investigated this with GDB/OpenOCD and then created a small EEPROM stress test which writes and immediately verifies bytes in the unused upper EEPROM region 0x0F00-0x0FFF (trying to save write cycles in that memory regions that matter for non-volatile parameters..).
The important results are:
Same current Marlin source
Same Configuration.h / Configuration_adv.h
Same MKS Robin E3 V1.1 board
Same physical MCU
Same physical EEPROM
Same EEPROM addresses
Same test data
Same test implementation
FAIL: mks_robin_e3 (STM32duino / STM32 HAL)
PASS: mks_robin_e3_maple (Maple / libmaple) The Maple environment required only adding env:mks_robin_e3_maple to the MKS_ROBIN_E3_V1_1 environment whitelist in Marlin/src/pins/pins.h. No V1.1 board pin definitions were replaced.
A 50-pass Maple test completed 12,800 write + verify operations over the 256-byte scratch region without a mismatch. The STM32duino build fails the identical test.
During a failing STM32duino run I instrumented the STM32duino Wire implementation immediately around the HAL transaction-start calls.
GDB captured the following sequence during M500:
i2c_master_write()
-> TwoWire::endTransmission()
-> eeprom_read_byte()
-> PersistentStore::write_data()
-> MarlinSettings::save()
-> M500The initial HAL transaction call returned:
HAL_ERRORThe captured state was:
HAL I2C state = 0x20
HAL I2C ErrorCode = 0x20For the STM32F1 HAL used by this environment, 0x20 is HAL_I2C_ERROR_TIMEOUT.
After continuing, the corresponding receive operation also immediately failed. The final diagnostic values were:
failure count = 2
operation = 2 (RX; first failure was TX)
HAL status = 1 (HAL_ERROR)
HAL state = 0x20
HAL error = 0x20
BUSY snapshot = 0The BUSY snapshot was taken after HAL had already returned from its timeout path, so BUSY=0 at the diagnostic breakpoint does not mean that the preceding BUSY wait did not time out.
There is also a second issue in the STM32duino version used by this Marlin environment.
Its i2c_master_write() / i2c_master_read() implementations initialize:
i2c_status_e ret = I2C_OK;and only process errors inside:
if (HAL_I2C_Master_Transmit_IT(...) == HAL_OK) {
...
}or:
if (HAL_I2C_Master_Receive_IT(...) == HAL_OK) {
...
}There is no corresponding else path for an immediate HAL_ERROR / HAL_BUSY return. Therefore ret remains I2C_OK even though HAL failed to start the transaction.
This behavior matches stm32duino/Arduino_Core_STM32#1774, fixed by stm32duino/Arduino_Core_STM32#1775.
Before instrumenting HAL directly, I had already observed a failed EEPROM operation where:
write endTransmission() -> 0 (success)
address endTransmission() -> 0 (success)
requestFrom(..., 1) -> 1 bytewhile the byte returned was incorrect:
expected = 0x3F
actual = 0x80This explains how Marlin can receive apparently successful Wire return values while the underlying HAL transaction has actually failed.
For comparison, the older Maple/libmaple Wire stack used by known-working LK5 Pro firmware does not reproduce the EEPROM failure. Its Wire implementation propagates the result from its I2C transfer and its F1-specific error handling also resets/re-enables the I2C peripheral following protocol errors.
Further disclaimer: I am not claiming that the underlying STM32F1 BUSY/timeout problem still exists in current STM32duino.
The practical workaround on this hardware is current Marlin using the existing mks_robin_e3_maple environment.
Bug Timeline
Newly discovered while updating this printer from an older Marlin / Maple-based firmware to current Marlin. The printer previously ran a DreadedZombie LK5 Pro Marlin build using the mks_robin_e3_maple environment reliably for approximately 1-2 years. After moving to current Marlin and using the normally supported mks_robin_e3 environment for BOARD_MKS_ROBIN_E3_V1_1, intermittent EEPROM write/verification failures were observed. Testing the old Maple-based firmware on the same physical board and EEPROM did not reproduce the issue. Current Marlin was then built using the Maple environment, after locally allowing mks_robin_e3_maple for BOARD_MKS_ROBIN_E3_V1_1. The identical EEPROM stress workload also passed there. So I do not know when the underlying STM32duino/HAL behavior was originally introduced. This appears to be an existing issue exposed by this board/environment combination rather than a newly introduced hardware problem.
Expected behavior
EEPROM I2C transactions should either:
- complete successfully and return the data that was actually transferred, or
- propagate an I2C / HAL error to the caller.
M500 should reliably save settings to the onboard EEPROM.
Running repeated write/read/verify operations against valid EEPROM addresses should not intermittently return stale or incorrect data.
If HAL fails to start an I2C transaction, the Wire layer should not report the transaction as successful.
Actual behavior
Under the mks_robin_e3 STM32duino environment, EEPROM accesses intermittently fail.
At the Marlin/Wire level, the transaction may still appear successful:
endTransmission() -> 0
requestFrom(..., 1) -> 1while the returned byte is incorrect.
Direct GDB instrumentation showed that the underlying STM32F1 HAL transaction-start function had actually returned HAL_ERROR with HAL_I2C_ERROR_TIMEOUT.
The STM32duino Wire implementation used by this Marlin environment does not propagate this immediate HAL start failure and instead returns I2C_OK.
This can result in Marlin consuming stale/invalid data and eventually reporting:
Error writing to EEPROM!The same Marlin-level stress test passes using Maple/libmaple on the same board.
Steps to Reproduce
Hardware:
MKS Robin E3 V1.1, mainly the components MCU (STM32F103RC) and the onboard EEPROM.Build current Marlin for:
MOTHERBOARD BOARD_MKS_ROBIN_E3_V1_1using:
pio run -e mks_robin_e3Flash and boot the firmware.
Normal M500 operations can reproduce the EEPROM verification error intermittently.
For a more deterministic test, add the attached EEPROM stress-test patch. It uses only the final 256 bytes of the 4 KB EEPROM:
0x0F00 - 0x0FFFso the normal Marlin settings region is not deliberately stressed.
From GDB call:
eeprom_stress_test(50)The test iterates all 256 scratch addresses for 50 passes:
50 * 256 = 12,800 writes 12,800 immediate verification readsIt stops at
eeprom_stress_failure_breakpoint()on the first mismatch.With
mks_robin_e3, the test reproduces the failure.Build the exact same Marlin source with the existing Maple environment. The only required source change is allowing the Maple environment for the V1.1 board in pins.h:
env:mks_robin_e3 -> env:mks_robin_e3 env:mks_robin_e3_mapleBuild:
pio run -e mks_robin_e3_mapleRun the exact same:
eeprom_stress_test(50)
Result:
mks_robin_e3 -> FAIL
mks_robin_e3_maple -> PASS (12,800 verified operations)The GDB HAL instrumentation used to identify the failure is also included as a separate diagnostic patch / log and is not necessary merely to reproduce the EEPROM mismatch.
Version of Marlin Firmware
bugfix-2.1.x Commit used during investigation: 2c03a437cdaa7c0be45b34e60aec0c7dab8739e8
Printer model
Longer LK5 Pro
Electronics
Stock
LCD/Controller
LCD N/A. Board: Robin MKS E3 V1.1
Other add-ons
N/A
Bed Leveling
ABL Bilinear mesh
Your Slicer
None
Host Software
OctoPrint
Don't forget to include
- A ZIP file containing your
Configuration.handConfiguration_adv.h.
Additional information & file uploads
Additional investigation notes:
The PlatformIO STM32duino framework used by the failing environment during testing was:
framework-arduinoststm32 @ 4.10900.200819The issue is not limited to Marlin's high-level PersistentStore logic.
A comparison against an older known-working Maple-based LK5 Pro Marlin build showed that the high-level EEPROM write/verify sequence is substantially the same. The important difference is the lower I2C stack:
mks_robin_e3:
STM32duino Wire
-> STM32 HAL I2C
mks_robin_e3_maple:
Maple Wire
-> libmaple hardware I2C drivermks-robin-e3-v1.1-i2c-eeprom-debug.zip
The libmaple Wire implementation propagates the transfer result from process(), and its protocol-error path disables and re-enables the I2C peripheral.
The old STM32duino behavior observed here closely matches stm32duino/Arduino_Core_STM32#1774, fixed by stm32duino/Arduino_Core_STM32#1775.
I believe there are therefore two related questions:
Why does the STM32F1 HAL stack used by this Marlin environment intermittently reach the pre-transaction I2C timeout on this board?
Since the STM32duino false-success behavior was already fixed upstream, should the framework used by this Marlin environment be updated or otherwise patched?
Separately, the existing mks_robin_e3_maple environment has been tested successfully on the MKS Robin E3 V1.1 hardware. Marlin currently permits Maple for MKS_ROBIN_E3 and MKS_ROBIN_E3D_V1_1 but not MKS_ROBIN_E3_V1_1.
The one-line V1.1 whitelist change is:
- env:mks_robin_e3
+ env:mks_robin_e3 env:mks_robin_e3_mapleEDIT: This change also needs an update to timers.h for the STM32F1 HAL..
- #if MB(BTT_SKR_MINI_E3_V1_0, BTT_SKR_E3_DIP, BTT_SKR_MINI_E3_V1_2, MKS_ROBIN_LITE, MKS_ROBIN_E3D, MKS_ROBIN_E3, VOXELAB_AQUILA)
+ #if MB(BTT_SKR_MINI_E3_V1_0, BTT_SKR_E3_DIP, BTT_SKR_MINI_E3_V1_2, MKS_ROBIN_LITE, MKS_ROBIN_E3D, MKS_ROBIN_E3, MKS_ROBIN_E3_V1_1, VOXELAB_AQUILA)I plan to submit that whitelist change separately so it remains independent of the root-cause investigation.
Source: MarlinFirmware/Marlin