I2CTargetMemory does seem like a bit of an unusual name, but "I2C target" is officially what an "I2C slave" is called now. And in the future we may have a machine.I2CTarget class which implements a generic target, so machine.I2CTargetMemory seems like the only option for naming.

I guess it could be shortened to machine.I2CMemory...

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.44%. Comparing base (628d53d) to head (aa92591).
Report is 13 commits behind head on master.

@@           Coverage Diff           @@
##           master   #17365   +/-   ##
=======================================
  Coverage   98.44%   98.44%           
=======================================
  Files         171      171           
  Lines       22208    22208           
=======================================
  Hits        21863    21863           
  Misses        345      345           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Interesting timing. I have an immediate need for this. I have just written an implementation of #3935 in MicroPython. It implements callbacks and 16-bit addressing, but needs to be called regularly in the main loop to service I2C requests from the controller. It has some other limitations, but is enough for my applications.

I see this failed on STM32F091. I am using STM32L432. I will try it and report back.

Callbacks would be useful, but I can live without them. My inputs need to be debounced or filtered and that logic can update the read registers. For write registers I will maintain a backup copy and diff them periodically to update outputs.

Would the memory_read callback be called before the I2C reply so that the memory can be updated on demand, eg from a pin, ADC, etc?

Is it necessary to name it other than I2CTarget ? If it were named I2CTarget now, with maybe mode=memory it would allow applications to keep working after a full featured I2CTarget was added to MicroPython.

I can build for PYBV10, but not for NUCLEO_L476RG or my L432. I will investigate tomorrow.

@dpgeorge - Alif support? :)

That is a great start for the slave modes of I2C and SPI. I wondered why you added a separate class instead of extending the I2C class with more options, like mode and addr. But this was the implementation is simpler.
I agree with @chrismas9 that some kind of signalling mechanism should exists, telling that a I2C transaction has been finished. That can happen when machine_i2c_target_data_reset() is called, whether by a callback or just a flag that can be polled.

I could start to implement this feature with the MIMXRT port. The NXP lib has support for the slave mode.

I would prefer callbacks to a flag as they allow for a much faster response for I2C reads if the register is updated in the callback. How would a flag work? Would the read function block until the flag is polled, the register updated and the flag is cleared, or would it immediately reply with stale data? The other problem, which I have with my python library, is that the Controller timeout has to be larger than the slowest poll time. I have use cases where such latency would be a problem, eg measuring motor current in a servo controller

A flag would just tell the status of the read/write process and it's state would be returned by a non-blocking call. The number of states is t.b.d., like BUSY, DONE_READING, DONE_WRITING. A callback is faster and more flexible, but would need to carry similar information about what happened.

I have added all STM32 families. I will submit the changes after testing them all on Eval boards.

I was hoping for a blocking read callback that could update a register with fresh data before the reply using clock stretching during the callback. I am making a series of modules which should behave like standard ICs. Most ADCs respond to a read with fresh data, either by clocking a SAR with SCL or using clock stretching until a conversion is finished. Any background loop that has to continuously update registers will result in reading data that is probably 10 to 100ms out of date. I'm not sure it's very useful to have a read callback after the event.

Write callbacks after a write are very useful as long as they contain the range of registers written.

There is an initial version for MIMXRT now at https://github.com/robert-hh/micropython/tree/mimxrt_i2c_target. Needs more testing.

@robert-hh I've added a test to this PR which you should be able to run on an mimxrt board, to validate your implementation.

Note: the API here will most likely change based on feedback.

Thanks. I have seen the test and will run it. And yes, I expect the API to change, but that can be adapted easily.

There were some issues in the wrap tests, that are fixed now. After adapting machine_i2c_target_memory.py with the settings for this board, all tests pass. The code can still be optimized somewhat and tested at the various board types, especially the slower i.mx1011 variants.

@dpgeorge Which wiring would you use for the automated test with the Teensy 4.1. I2C(0) as hardware I2C for the targeth, but which GPIO Pins for the controller port?

I was hoping for a blocking read callback that could update a register with fresh data before the reply using clock stretching during the callback.

This is an important question to discuss further. With a read transaction (point of view from the controller) we have:

1. generate start condition
2. send address byte, with direction flag (read)
3. read byte from the target, so the target must drive SDA
4. generate stop condition

If the target is not ready in step 3 to send the data (eg an ADC that needs to be sampled) then the target could stretch the clock by holding SCL low. Then release it when it has the new data ready.

If we want to support such clock stretching then the API and implementation will be quite different, because we need a way for the Python code to signal that it wants to stretch the clock (stall the bus), and then indicate when it is ready to unstall the bus.

There are real devices out there that do this clock stretching, eg ADS7128 (and it seems that device stretches SCL for quite some time when it's averaging many readings). Other devices like ADS1115 require you to poll a register to see when the ADC conversion is ready. And something like MCP3021 will do the ADC conversion during the first few bits of the data read (and those bits always read 0) so don't need to stretch SCL.

Looking at Zephyr's I2C target API, it does not support clock stretching, so it's not possible to implement a clock stretching target with Zephyr.

Do we need the MicroPython I2C target API to be able to do clock stretching?

Other considerations:

• clock stretching for writes to the target; this could be used for example to implement flow control when a lot of data is written out to the target (it holds SCL low if it's not ready to receive more data)
• support atomic reads from memory device, eg if the I2C controller is reading a 16-bit value from the I2C target, you don't want the I2C target to update that memory in the middle of the read or else the upper and lower bytes of the 16-bit value will be out of sync
• support atomic writes

Things can get complicated pretty quickly when looking at all these things.

Which wiring would you use for the automated test with the Teensy 4.1. I2C(0) as hardware I2C for the targeth, but which GPIO Pins for the controller port?

I'd use SoftI2C for the controller side, so it doesn't matter which pins. Teensy already needs D0/D1 and D2/D3 and D11/D12 connected for tests/extmod_hardware/machine_pwm.py, so maybe can reuse those?

Clock stretching is a very undesirable behavior of I2C.

Do not replicate that. It makes the bus unreliable as it means I2C transfers are not accomplished in fixed time. This kills any MCUs ability to meet a schedule with an I2C shared with multiple devices.

Teensy already needs D0/D1 and D2/D3 and D11/D12 connected

With the test of PWM and UART, these Pins are pairwise connected, D0 to D1, D2 to D3 and D11 to D12. For I2C, SDA (A4) and and SCL(A5) have to be connected each to a GPIO pin. So I suggest to use A3 and A6, being just next to SDA and SCL. At the Teensy, SCL and SDA need an external pull-up resistor for a reliable test. The internal pull-up is not sufficiently strong.

@dpgeorge The MIMXRT implementation is at a stable state with the feature set matching the RP2. It works fine so far.

@chrismas9 I was hoping for a blocking read callback that could update a register with fresh data before the reply using clock stretching during the callback.

Looking at the timing constraints this feature is hard to achieve. Hardware which allows that uses often a double buffer scheme, such that when a value is requested the most recent ADC value is latched to the transmit buffer. Or they need a dedicated command to start a new sampling and offer a status bit that has to be polled, or an INT signal.
What's less critical is a callback that's called after receiving the STOP condition. With that you may be able to emulate a readfrom_mem() by a combo of writeto() and readfrom() calls.

There is an implementation for SAMD at https://github.com/robert-hh/micropython/tree/samd_i2c_target. Works well. Tested so far with SAMD51. Tests with SAMD21 follow.

@dpgeorge I'm still unsure about the wrapping behavior for writing and reading. I could not find a source where this is defined as to be expected. Do you have a reference?

For writing controller->target the target can signal if the memory overflows by sending NAK. Then the controller would stop sending. For receiving there is no such mechanism. So the alternative to wrapping would be sending dummy data like 0x00 or 0xff.

I looked into other ports, whether they can support the I2CTargetMemory class.

• The NRF port it looks promising. The library provides suitable functions.

• ESP8266 does not provides I"C hardware support at all.

• The Renesas port seems to have I2C target mode hardware, but the HAL layer does not offer interfaces for using it.

• The ESP32 situation is crooked for several reasons.

  • The ESP32 API has three versions of I2C support. 1. The legacy API, which is currently used for I2C controller support. 2. The ng API Version 1. 3, The ng API version 2. Option b) is declared as getting expired with idf 6.x. The ng API V2 is available starting version 5.4. So using is at the moment is not possible. The legacy API and ng API must not be used at the same time. So if that is going to be used, the I2C class has to be updated.
  • Implementing the class I2CTargetMemory for trial it turned out that it cannot properly support the target memory mode. The API offers two callback functions or receive and transmit. Each of them is called AFTER the respective bus transactions is finished. That makes it possible to support the writeto_mem() method properly, but fails to support readfrom_mem(). The only option which could be supported is getting the proper content from the previous state of the memory object. After it has been written, one has to call readfrom_mem() twice to get the proper content. It may be useful to have a I2CTargetMemory.write() method which fills the transmit buffer with data from a provided object.
  • One has to declare a suitable large size for the temporary buffer. Otherwise, if on writeto_mem() the sent data is longer than the declared buffer, the amount of data actually provided to the callback is erratic. On readfrom_mem() the reading does not wrap. Instead for the excess data 0xff is sent.

There is still the unexpected behavior that on reading from the Target the ADDR_MATCH handler is called for every byte.
Reason: The condition in line 165 if (data->state == STATE_MEM_ADDR_SELECTED) is never true, since between setting the state to STATE_MEM_ADDR_SELECTED and getting to that line a restart happens in the protocol, setting state to STATE_IDLE.
Edit: It should be sufficient to make the test in line 165 for STATE_IDLE.

There is still the unexpected behavior that on reading from the Target the ADDR_MATCH handler is called for every byte.

I don't see this on the rp2 port. Using RPI_PICO_W and running the test in this PR, the TestIRQ class tests this. At the moment those tests don't full pass, but they show only one ADDR_MATCH per read or write.

Well, I need to rework the rp2 implementation to use a lower-level hardware interface, instead of the higher-level pico_i2c_slave component. That's needed to get events for a simple scan (0-byte write) transaction.

I don't see this on the rp2 port. Using RPI_PICO_W,

Sorry. My old eyes. The second call is for I2C_TARGET_IRQ_MEM_ADDR_MATCH, not I2C_TARGET_IRQ_ADDR_MATCH. Anyhow I see the callback being called twice and data loss. But that's a different problem. It happens when sending data from a separate device at freq=400_000. With freq=100_000 all is fine. And of course it does not happen in the test script set-up, where the data is sent by SoftI2C and being at the same CPU, there cannot be any timing conflict between controller and target.

A nice surprise: the I2C peripheral for alif and rp2 is the same Synopsis silicon IP. So they can have the same low-level implementation and their semantics will match.

I have now reworked the rp2 I2CTarget implementation. It's now the same as the alif one, and passes the included tests.

Downloaded and tested with a RP2 Pico and a second board as Controller and a single callback for ADDR_MATCH. Runs fine at freq=100_000. Works fine with freq=400_000 for data requested from the slave. Behaves weird at freq=400_000 when data is sent from Controller:

• the callback is called twice instead of once.
• single bytes of the transfer are lost. Either the old data i still in the target memory at that place, or a NULL byte is inserted.
  No improvement with that aspect compared to the previous version.
  Without a callback data is transferred properly.

• the callback is called twice instead of once.

• single bytes of the transfer are lost. Either the old data i still in the target memory at that place, or a NULL byte is inserted.

Thanks for testing. I can also see the same issue (using a special test I wrote that uses software SPI for the controller and can detect clock stretching).

Should be fixed now. I had to enable clock stretching for the RX path. And also fix out-of-order events, when the STOP occurs before the RX FIFO is fully drained.

Hi Damien, will the current API be usable for replacing pyb in this case? https://github.com/openmv/openmv/blob/master/scripts/libraries/rpc.py#L502

It looks like fo get_bytes() I can use the read_into() in a loop while tracking time to receive data... However, for put_bytes(), I can't do the same kind of thing because mp_machine_i2c_target_write_bytes() seems to always write 1 byte even if there's no space in the FIFO on alif for example.

Can you adjust mp_machine_i2c_target_write_bytes() to return 0 if there's no space in the FIFO? Otherwise it looks like some type of interrupt driven thing is necessary to write data out. Also, should mp_machine_i2c_target_write_bytes() do more than 1 byte at a time?

It looks like fo get_bytes() I can use the read_into() in a loop while tracking time to receive data...

Yes, you might be able to do that. Although the FIFO is only 16 or 32 bytes deep, so if you expect a lot of data coming in, then it could overflow before you have a chance to read it out.

Can you adjust mp_machine_i2c_target_write_bytes() to return 0 if there's no space in the FIFO?

I tried this, but on rp2 and I assume also alif (they use the same Synopsis IP) the TX (outgoing) FIFO is flushed when there is an address match (also the RX FIFO). So that means if you queue the data in the TX FIFO beforehand, it is all lost once the target is addressed by the controller.

To fix that either needs buffering outside the TX FIFO (ie in RAM), or use an interrupt based mechanism.

Maybe it's possible for you to use I2CTarget in memory device mode. Just set an address size of 0 and a large memory buffer, eg:

buf = bytearray(65536)
i2c_target = I2CTarget(0, addr, mem=buf, mem_addrsize=0)

Then for get_bytes() you just copy out of buf, and for put_bytes() you just copy into buf. The actual transfer is then handled in the background.

Of course, that scheme would require having some way to know that new data has arrived in buf, and that data has been read out of buf. Would having that capbility work for your case?

How much data do you have coming in and going out? Do you need to be able to send large amounts of data from ROM (ie not copy to a temporary RAM buffer)? What exactly is the protocol you use over I2C?

Using an address size of 0 (if that removes the address part) would work for me.

Of course, that scheme would require having some way to know that new data has arrived in buf, and that data has been read out of buf. Would having that capbility work for your case?

Well, I need the method to transfer the requested data size or timeout. So, I'd need to know when the data has all been transferred within a certain time limit.

How much data do you have coming in and going out? Do you need to be able to send large amounts of data from ROM (ie not copy to a temporary RAM buffer)? What exactly is the protocol you use over I2C?

For the OpenMV RPC library things are more complex at the upper layers of the protocol. For the lowest layer for I2C, the target device is just receiving an address match and then an expected number of bytes via I2C or timeout. It also will transmit a known number of bytes after an address match or timeout. The sizes of the buffers change all the time along with the timeouts.

Well, I need the method to transfer the requested data size or timeout. So, I'd need to know when the data has all been transferred within a certain time limit.

With the memory device, you'd set up the data buffer and then just let it do its thing. If you need a timeout I guess you could do something like:

def transfer_bytes(buf, timeout):
    i2c_target = I2CTarget(0, addr, mem=buf, mem_addrsize=0)
    t0 = time.ticks_ms()
    while True:
        if i2c_target.transfer_done():
            # controller read our buffer or wrote into our buffer
            i2c_target.deinit()
            return True
        if time.ticks_diff(time.ticks_ms(), t0) >= timeout:
            # timeout
            i2c_target.deinit()
            return False
        time.sleep(0)

That would handle both reads and writes. The I2C target would disappear between transfers (I think that's how you have it at the moment?).

The sizes of the buffers change all the time along with the timeouts.

What's the approximate maximum buffer size? Is it OK for all buffers to be in RAM?

The largest buffer size can be 2^32 bytes. All would be in RAM or whatever is addressable via byte arrays or memory views. I think I split things up by 65K chunks though in the code.

For:

def transfer_bytes(buf, timeout):
    i2c_target = I2CTarget(0, addr, mem=buf, mem_addrsize=0)
    t0 = time.ticks_ms()
    while True:
        if i2c_target.transfer_done():
            # controller read our buffer or wrote into our buffer
            i2c_target.deinit()
            return True
        if time.ticks_diff(time.ticks_ms(), t0) >= timeout:
            # timeout
            i2c_target.deinit()
            return False
        time.sleep(0)

Could a method be added to machine class to do all this for you in C? Maybe add a poll method which you could select and wait on. It would return or timeout then. Each return could mark the completion of a I2C read/write.

Could a method be added to machine class to do all this for you in C?

What's wrong with having it in Python, eg in a helper module in micropython-lib? That would be more flexible than in C, and not any slower.

The new commits show some improvements. The double call of the ADDR_MATCH callback on data controller -> target is gone. The data corruption is still present, and then two subsequent readfrom_mem() calls return different data.

Log at the controller:

>>> i2c.readfrom_mem(67, 0, 8)
b'01234567'
>>> i2c.writeto_mem(67, 0, "abcdefgh")
>>> i2c.readfrom_mem(67, 0, 8)
b'bcdefg7\x00'
>>> i2c.readfrom_mem(67, 0, 8)
b'\x00bcdefg7'
>>> i2c.readfrom_mem(67, 0, 8)
b'\x00bcdefg7'

Set-up: Controller: MIMXRT1020-EVK, hard I2C, freq=400_000, Target RP2 Pico, mem-object preset with "01234567", callback for ADDR_MATCH, which just prints the irq.flags().
At freq=200_000 and of course below this test passes, even with longer text.

Screen shot of the first readfrom_mem():

Screen shot of writeto_mem():

Screen shot of the last readfrom_mem():

@robert-hh thanks for further testing. I could reproduce your issue with a PYBv1.0 as the controller (at 330kHz), and a Pico W as the target (although I had to add a sleep_us(10) in the I2C IRQ handler to trigger the bug).

The issue was the stop condition being processed before the last incoming byte, which lead to that last byte being missed. And that messed up subsequent transactions. Should be fixed now.

@dpgeorge - A helper in python is fine. Just thinking that the cleanest way to make it to have something that's pollable. https://docs.micropython.org/en/latest/library/select.html

Like, the way the OpenMV's RPC library works now is okay with a blocking loop... but, it needs to be rewritten eventually to support asyncio.

Should be fixed now.

Confirmed. The above test passes at freq=400_000.
At longer messages clock stretching kicks in after 16 bytes.