For your use case, is it necessary that these appear as genuine digitalio.DigitalInOut objects, or could you implement them in Python and duck-type them to look like DigitalInOuts?

I would say that the CYW43 pin stuff was done the way it was due to the idiosyncrasies of these pass-through pins and their use cases.

I am not sure we would want to expand on that style for I/O expanders in general. If we wanted a more general solution, generalizing either microcontroller.Pin or DigitalInOut, maybe with an abstract superclass, and then allowing native or Python implementations would be more flexible. But it would require pervasive changes across the code base. Allowing native support for subtypes is not so simple in the Micropython core, and has to be done explicitly. This was done for displayio in various ways. It also affects performance.

We have had numerous requests for "Pin-alike" or "DigitalInOut-alike" implementations, for things like the MC23017, shift registers, etc. keypad is an example where people would like to use I/O expanders. Adding native code for each kind of I/O expander is going to be a lot of work, and relatively inflexible. Allowing Python implementations is better in the long run.

I'm sure @jepler and @tannewt have things to say about this as well.

Thanks for the quick reply.

The main thing that took me down this path was that a few of the expander pins are intended for (slowly) controlling the secondary lines of SPI displays (CS, DC). I managed to cobble some python code together for testing, but had to do it without the usual SPI display classes as they require Pin/DigitalIO objects. These classes could be hacked to support an IOExpander object I suppose, but having them accept a pin as normal seemed far easier and cleaner for the user.

There's also the case that one of the pins should be initialised and reset at the C level, for safely, rather than relying on user code, so I would be needing to have some form of C driver to talk to the expanders anyway.

The rest of the pins, although it would be nice if they were usable with DigitalIO to simplify the user's understanding, is not strictly required. Perhaps there is even a performance benefit to not doing it this way as pins could then be set and cleared in single Word operations if the user required 🤔

I'm ok adding additional pin types like the CYW43 code does. This makes them available to CP when the VM may not be running. If its only needed by user code in the VM, then I'd push more to make it Python-only.

I wouldn't add it to microcontroller.pin though. That's a lie. :-) If you want to expose them directly then you could expose the expander class through board. Kind of like the built in display.

I've progressed down the CYW43 Pin approach, and am honestly rather happy with the user experience! Having the pins for LEDs and buttons just being accessed using board.LED_A and board.USER_SW for example, feels quite magical. I even updated displayio.FourWire to support these expander pins for DC and CS, and although slow, seeing the REPL on screen using the usual example code was rather exciting!

Newest code added to this PR if anyone wants to look. I also tried a technique in board.c to group pins together, that seems to work rather well.

Because of both these things, for my use-case I don't think it's even necessary to have an IO Expander object that's exposed.
@tannewt, I'm guessing that for your "wouldn't add it to microcontroller.pin" comment, I would need such an object to exist? That wouldn't have any impact on the pin_type ? Rather just change them to appear as tca9555.pin for instance? Which directory should I place such an object in?

A few tangential questions:

• When a digitalio object is created, is there a standard for what the pin state should be? I ask because during the board_init() I set some pins to outputs, which then get set back to inputs when the digitalio objects are made. I was just wondering if I could go against this trend and have it pull its state from the io expander itself. Being an input causes the LEDs to glow dimly, for instance. I'll think this though over the weekend.
• Is there any C side function that gets called when an exception occurs in user code and brings them back to the REPL? I've added code to reset_board() but that only triggers with an explicit stop of a program. Perhaps this can only be done by a CPy object that gets cleaned up?

I've progressed down the CYW43 Pin approach, and am honestly rather happy with the user experience! Having the pins for LEDs and buttons just being accessed using board.LED_A and board.USER_SW for example, feels quite magical. I even updated displayio.FourWire to support these expander pins for DC and CS, and although slow, seeing the REPL on screen using the usual example code was rather exciting!

Awesome! Sounds like the right approach.

Because of both these things, for my use-case I don't think it's even necessary to have an IO Expander object that's exposed. @tannewt, I'm guessing that for your "wouldn't add it to microcontroller.pin" comment, I would need such an object to exist? That wouldn't have any impact on the pin_type ? Rather just change them to appear as tca9555.pin for instance? Which directory should I place such an object in?

I don't think you need a separate object if all useful pins are accessible through board. microcontroller.pin is meant to give access to microcontroller pins not used by the board but potentially useful. (These aren't microcontrolller pins though, that's why I wouldn't put them there.)

A few tangential questions:

* When a digitalio object is created, is there a standard for what the pin state should be? I ask because during the `board_init()` I set some pins to outputs, which then get set back to inputs when the `digitalio` objects are made. I was just wondering if I could go against this trend and have it pull its state from the io expander itself. Being an input causes the LEDs to glow dimly, for instance. I'll think this though over the weekend.

digitalio defaults to input, not pulled. When they are reset (not in use by digitalio), then the board can have a reset state so pins don't cause LEDs to glow. Most ports reset to input, not pulled but that isn't actually optimal for power if the input buffer is enabled (the floating pin will waste buffer power.) So, when reset, you should do input with a pull of your choice. (For example, ESP is up except if the board overrides it.

* Is there any C side function that gets called when an exception occurs in user code and brings them back to the REPL? I've added code to `reset_board()` but that only triggers with an explicit stop of a program. Perhaps this can only be done by a CPy object that gets cleaned up?

reset should be called between code.py and the REPL. If you open the REPL and then do an import of user code, you are still in the same VM and that code should handle the exception.

However, you could also add a finaliser to run before an object is about to be deleted. Generally we have it be the same as deinit(). I2SOut is an example:

and then:

Because of both these things, for my use-case I don't think it's even necessary to have an IO Expander object that's exposed. @tannewt, I'm guessing that for your "wouldn't add it to microcontroller.pin" comment, I would need such an object to exist? That wouldn't have any impact on the pin_type ? Rather just change them to appear as tca9555.pin for instance? Which directory should I place such an object in?

I don't think you need a separate object if all useful pins are accessible through board. microcontroller.pin is meant to give access to microcontroller pins not used by the board but potentially useful. (These aren't microcontrolller pins though, that's why I wouldn't put them there.)

Thanks for the explanation. I understand now what you mean, so have removed the pin defines I added to microcontroller.pin. I did have to use the tca module for defining the other pins though, as not all the pins are picked up by the nested pin groups I set up.

Here's the latest output from help(board) if you're curious:

object <module 'board'> is of type module
  __name__ -- board
  board_id -- pimoroni_yukon
  SLOT1 -- (FAST1=microcontroller.pin.GPIO0, FAST2=microcontroller.pin.GPIO1, FAST3=microcontroller.pin.GPIO2, FAST4=microcontroller.pin.GPIO3, SLOW1=tca.TCA0_3, SLOW2=tca.TCA0_4, SLOW3=tca.TCA0_5, ADC1_ADDR=0, ADC2_TEMP_ADDR=3)
  SLOT2 -- (FAST1=microcontroller.pin.GPIO4, FAST2=microcontroller.pin.GPIO5, FAST3=microcontroller.pin.GPIO6, FAST4=microcontroller.pin.GPIO7, SLOW1=tca.TCA0_0, SLOW2=tca.TCA0_1, SLOW3=tca.TCA0_2, ADC1_ADDR=1, ADC2_TEMP_ADDR=6)
  SLOT3 -- (FAST1=microcontroller.pin.GPIO8, FAST2=microcontroller.pin.GPIO9, FAST3=microcontroller.pin.GPIO10, FAST4=microcontroller.pin.GPIO11, SLOW1=tca.TCA0_8, SLOW2=tca.TCA0_9, SLOW3=tca.TCA0_10, ADC1_ADDR=4, ADC2_TEMP_ADDR=2)
  SLOT4 -- (FAST1=microcontroller.pin.GPIO12, FAST2=microcontroller.pin.GPIO13, FAST3=microcontroller.pin.GPIO14, FAST4=microcontroller.pin.GPIO15, SLOW1=tca.TCA1_7, SLOW2=tca.TCA1_6, SLOW3=tca.TCA1_5, ADC1_ADDR=5, ADC2_TEMP_ADDR=7)
  SLOT5 -- (FAST1=microcontroller.pin.GPIO16, FAST2=microcontroller.pin.GPIO17, FAST3=microcontroller.pin.GPIO18, FAST4=microcontroller.pin.GPIO19, SLOW1=tca.TCA1_15, SLOW2=tca.TCA1_14, SLOW3=tca.TCA1_13, ADC1_ADDR=8, ADC2_TEMP_ADDR=11)
  SLOT6 -- (FAST1=microcontroller.pin.GPIO20, FAST2=microcontroller.pin.GPIO21, FAST3=microcontroller.pin.GPIO22, FAST4=microcontroller.pin.GPIO23, SLOW1=tca.TCA1_10, SLOW2=tca.TCA1_11, SLOW3=tca.TCA1_12, ADC1_ADDR=9, ADC2_TEMP_ADDR=10)
  NUM_SLOTS -- 6
  SDA -- board.SDA
  SCL -- board.SCL
  EX_I2C_SDA -- board.EX_I2C_SDA
  EX_SPI_SCK -- board.EX_I2C_SDA
  GP26_A0 -- board.EX_I2C_SDA
  GP26 -- board.EX_I2C_SDA
  A0 -- board.EX_I2C_SDA
  EX_I2C_SCL -- board.EX_I2C_SCL
  EX_SPI_MOSI -- board.EX_I2C_SCL
  GP27_A1 -- board.EX_I2C_SCL
  GP27 -- board.EX_I2C_SCL
  A1 -- board.EX_I2C_SCL
  INT -- board.INT
  SHARED_ADC -- board.SHARED_ADC
  MAIN_EN -- board.MAIN_EN
  USER_SW -- board.USER_SW
  ADC_ADDR_1 -- board.ADC_ADDR_1
  ADC_ADDR_2 -- board.ADC_ADDR_2
  ADC_ADDR_3 -- board.ADC_ADDR_3
  ADC_MUX_EN_1 -- board.ADC_MUX_EN_1
  ADC_MUX_EN_2 -- board.ADC_MUX_EN_2
  SW_A -- board.SW_A
  SW_B -- board.SW_B
  LED_A -- board.LED_A
  LED_B -- board.LED_B
  LCD_BL -- board.LCD_BL
  LCD_DC -- board.LCD_DC
  LCD_CS -- board.LCD_CS
  CURRENT_SENSE_ADDR -- 12
  TEMP_SENSE_ADDR -- 13
  VOLTAGE_SENSE_ADDR -- 14
  EX_ADC_ADDR -- 15
  I2C -- <function>
  STEMMA_I2C -- <function>

Thank you for answering my other queries too. I haven't had chance to look over them yet, but will do soon.

Hi @dhalbert. I just wanted to let you know that I had some merge conflicts come up due to your recent arg_name changes in: a974402

Specifically, I needed to change these 3 functions from the validate versions to common_hal ones, which in your commit did not have arg_name as a parameter.

I have added that parameter now, but I did have to change the common_hal_digitalio_validate_pin function.

Perhaps you will want to do a separate commit with this change to common_hal_digitalio_validate_pin in? It will be a while until this PR is ready to be merged.

Perhaps you will want to do a separate commit with this change to common_hal_digitalio_validate_pin in? It will be a while until this PR is ready to be merged.

That's a good improvement, thanks. I will add this.

@ZodiusInfuser hi - is this board still in process?

Hi @dhalbert. Yes it's still very much in progress, just taking its time due to the complexity involved and flip-flopping between other duties: https://twitter.com/ZodiusInfuser/status/1673292599552090112?t=IEEiOgdyxdWWXQ05dlDzRw&s=19

not exactly but looking at a commit which adds a new module that is split across shared-modules and shared-bindings may be instructive: f5c637d (there's this guide but it's well out of date: https://learn.adafruit.com/extending-circuitpython/inside-the-virtual-machine)

• Add a SRC_PATTERNS stanza and some SRC_SHARED_MODULE_ALL lines to py/circpy_defns.mk
• Add a default enable/disable Makefile value and a controlling value to CFLAGSS in py/circpy_mpflags.mk
• Add an enable value to port or board specific .mk files

For diagnosis,

• You can make BOARD=... print-ANYVARIABLE to get the actual calculated value of $(ANYVARIABLE) used in make
• You can use make BOARD=... V=commands to see make print the actual commands used; the final link command will list all the files being linked. It's most likely the failure was due to some file not being included at all.

not exactly but looking at a commit which adds a new module that is split across shared-modules and shared-bindings may be instructive: f5c637d (there's this guide but it's well out of date: https://learn.adafruit.com/extending-circuitpython/inside-the-virtual-machine)

• Add a SRC_PATTERNS stanza and some SRC_SHARED_MODULE_ALL lines to py/circpy_defns.mk
• Add a default enable/disable Makefile value and a controlling value to CFLAGSS in py/circpy_mpflags.mk
• Add an enable value to port or board specific .mk files

For diagnosis,

• You can make BOARD=... print-ANYVARIABLE to get the actual calculated value of $(ANYVARIABLE) used in make
• You can use make BOARD=... V=commands to see make print the actual commands used; the final link command will list all the files being linked. It's most likely the failure was due to some file not being included at all.

Thanks for that! I seem to be getting somewhere now (though I'm not specifically sure which thing fixed it). I'll do some tidying up and try committing soon.

On an unrelated note. I noticed this include that seems unnecessary in this file:
https://github.com/adafruit/circuitpython/blob/17ced21e04a901af4c44daabf92ced6683e8c2a6/shared-bindings/digitalio/DigitalInOut.c#L46-48

@jepler An attempt was made at restructuring the code. Builds locally and passes the test here, but please let me know if there are structural things I should improve. There's probably some imports I can remove.

Did a bit more tidy up. I think it's ready for me to convert from a draft, but I'll leave it over the weekend in case I think of any last changes to make.

I'm about to be away for about six weeks so hopefully @tannewt or @dhalbert can provide the necessary review. Thanks for updating the PR!

Thanks @jepler. Have a good break!

Thanks for your review. It seems we have different philosophies when it comes to the implementation of this IO expander.

To me the IO expanders are an implementation limitation of the product, due to the RP2040 not having enough native IO for the functions we needed. As such this PR is geared towards abstracting their existence away, like how was done for CYW. Now, I understand that the CYW support may have been hastily added and thus should not have been used as a template, but the resulting user experience is a good one.

I do understand your points though, and that having the expander structured as you describe would be more broadly valuable to the CircuitPython community.

Sadly though I don't have the remaining dev time to address this before the product launches, given I still have 2 other languages to finish supporting (MicroPython and C++). This is unfortunate, as I have put most of the product's dev time into the CPy support (as observed by this PR being 9 months old), and even considered having Yukon come pre-flashed with it. That could still happen I suppose, if I provide customers with my customfirmware.uf2, but I can see that causing confusion. Unless there was some way to have circuitpython.org point to the builds generated by my repo rather than the main adafruit repo?

Thanks for your review. It seems we have different philosophies when it comes to the implementation of this IO expander.

To me the IO expanders are an implementation limitation of the product, due to the RP2040 not having enough native IO for the functions we needed. As such this PR is geared towards abstracting their existence away, like how was done for CYW. Now, I understand that the CYW support may have been hastily added and thus should not have been used as a template, but the resulting user experience is a good one.

I don't think it's a better user experience when one tries to use the pin for something it isn't meant to work with. It only works with digitalio and therefore should just act like digitalio.

I do understand your points though, and that having the expander structured as you describe would be more broadly valuable to the CircuitPython community.

Sadly though I don't have the remaining dev time to address this before the product launches, given I still have 2 other languages to finish supporting (MicroPython and C++). This is unfortunate, as I have put most of the product's dev time into the CPy support (as observed by this PR being 9 months old), and even considered having Yukon come pre-flashed with it.

To be fair, this feedback is consistent with what I said in January.

That could still happen I suppose, if I provide customers with my customfirmware.uf2, but I can see that causing confusion. Unless there was some way to have circuitpython.org point to the builds generated by my repo rather than the main adafruit repo?

You are welcome to use the CircuitPython code but please do not call it CircuitPython or have CircuitPython as part of the name. We reserve that for code checked into the main repo. That way it gets updated and distributed like all other CircuitPython.

I'll chat with ladyada and see if I can spend a bit of time on our qualia board with the io expander. That'd give you a model for how to do it.

I don't think it's a better user experience when one tries to use the pin for something it isn't meant to work with. It only works with digitalio and therefore should just act like digitalio.

So you're suggesting that rather than having a TcaPin object, since it can only do IO it should skip that step and be a DigitalIO object from the outset? FourWire and such would then accept that for any case, without specifically needing to know about that pin type?

To be fair, this feedback is consistent with what I said in January.

And indeed I did follow that, by creating a new TCA pin type. Initially I added them to microcontroller.pin, but corrected that within a later commit, having them within the tca module instead. So there is no lying about what the pins are. They are also accessible from tuples within board, which is important for this product to keep.

You are welcome to use the CircuitPython code but please do not call it CircuitPython or have CircuitPython as part of the name. We reserve that for code checked into the main repo. That way it gets updated and distributed like all other CircuitPython.

Okay, I understand. I will avoid that then.

I'll chat with ladyada and see if I can spend a bit of time on our qualia board with the io expander. That'd give you a model for how to do it.

That would help immensely! Thank you

I don't think it's a better user experience when one tries to use the pin for something it isn't meant to work with. It only works with digitalio and therefore should just act like digitalio.

So you're suggesting that rather than having a TcaPin object, since it can only do IO it should skip that step and be a DigitalIO object from the outset? FourWire and such would then accept that for any case, without specifically needing to know about that pin type?

Yup! Exactly. Any places where the C code uses digitalinout internally, we should support passing in a digitalinout-like object as well.

To be fair, this feedback is consistent with what I said in January.

And indeed I did follow that, by creating a new TCA pin type. Initially I added them to microcontroller.pin, but corrected that within a later commit, having them within the tca module instead. So there is no lying about what the pins are. They are also accessible from tuples within board, which is important for this product to keep.

Yup, totally ok for board so hold objects besides microcontroller.Pins.

You are welcome to use the CircuitPython code but please do not call it CircuitPython or have CircuitPython as part of the name. We reserve that for code checked into the main repo. That way it gets updated and distributed like all other CircuitPython.

Okay, I understand. I will avoid that then.

Thank you!

I'll chat with ladyada and see if I can spend a bit of time on our qualia board with the io expander. That'd give you a model for how to do it.

That would help immensely! Thank you

👍

I'll chat with ladyada and see if I can spend a bit of time on our qualia board with the io expander. That'd give you a model for how to do it.

That would help immensely! Thank you

Unfortunately, I won't be working on this anytime soon. I'm still happy to advise if you find the time to further modify this code.

I'll chat with ladyada and see if I can spend a bit of time on our qualia board with the io expander. That'd give you a model for how to do it.

That would help immensely! Thank you

Unfortunately, I won't be working on this anytime soon. I'm still happy to advise if you find the time to further modify this code.

Thank you for letting me know. I'll resist this post launch

@ZodiusInfuser Hi - is this board still on track for CircuitPython? Is it Yukon?

@ZodiusInfuser Hi - is this board still on track for CircuitPython? Is it Yukon?

Hi Dan. It is indeed Yukon, but sadly we are no longer pursuing CircuitPython for this board, and just sticking to our Pirate-brand of MicroPython. I'll close this PR.