Topics

esp32 port of iotivity-lite: discovery memory issue


Farid BENAMROUCHE
 

Hello,

I have a working port of iotivity-lite for the esp32 chip.
I would like to submit this port to the community, but before that I have 2 issues to fix. ipv6 is not working properly yet (esp32 stack issue) and one that is the reason of this post:

I'm very resources constrained, and mostly optimized everything now. However I've discovered that some memory are kept by iotivity-lite for 10min for each discovery requests...
The problem is that for each received discovery request, an ssl context is allocated and kept for 10min.
However, if it receives an other discvery request, an additionnal chunk (20KB!) of memory is allocated again for 10min.
So after 3 discovery requests, I'm out of memory...


Why is that mechanism in place? Is it possible to limit the number of pending discovery requests to 1 only? (clean the previous one before processing the new one?)

Isn't it an security issue on all platforms? (if a code sends enough discovery requests as fast as possible it can DDOS any iotivity-lite server)

Is my understanding correct?

Thank you!
Regards


Clarke Stevens
 

Hello,

 

I’m Clarke Stevens and I run the developer Tools group in OCF. I don’t have the answer to your question at the moment, but there are others on this forum that can do that. What I want to tell you is that we a member company that has also recently done a port for ESP32 and they have both security and IPv6 working. I’m including those instructions below. It runs on the ESP32 using FreeRTOS.  They are also able to connect to an OCF cloud.

 

I have also been able to get OCF’s DeviceBuilder tool to work with this and automatically generate working code. All these instructions are included below.

 

I have successfully tested this with the Adafruit Huzzah32 and the Espressif Pico board. I am in the process of putting this example on the IoTivity web site, but it’s not up there yet.

 

Thanks,

-Clarke

 

IoTivity-lite setup for esp32

  • cd ~
  • curl https://openconnectivity.github.io/IOTivity-Lite-setup/install.sh | bash
  • Use the following steps to get the esp32 branch until it is merged

 

Ubuntu

  • I’m not sure if you’ll need to do this, but I had to do it to get permission to use my serial port
    • sudo usermod -a -G tty <username> (then I had to logout and login)
    • sudo usermod -a -G dialout <username> (I’m not sure which of these will work or if you need both)
    • groups (to verify if you have been added to tty and/or dialout groups
  • sudo apt install -y git wget flex bison gperf python3 python3-pip python3-setuptools python3-serial python3-click python3-cryptography python3-future python3-pyparsing python3-pyelftools cmake ninja-build ccache libffi-dev libssl-dev libusb-1.0-0
  • cd ~/iot-lite/iotivity/port/esp32
  • git clone --recursive https://github.com/espressif/esp-idf.git
  • ./esp-idf/install.sh
  • . ./esp-idf/export.sh
  • jump to the common steps below

Windows (note that the windows steps are from Jozef and I have not tested them yet)

Common steps

  • idf.py set-target esp32
  • idf.py menuconfig // set wifi (This will bring up a GUI where you need to set SSID and SSID password under the IoTivity menu item. Exit and save.)
  • ( cd esp-idf/components/mbedtls/mbedtls && git am ../../../../patches/mbedtls/*.patch )
  • ( cd esp-idf && find ../patches/esp-idf/ -type f -name '*.patch' -exec patch -p1 -i {} \; )
  • ( cd esp-idf/components/lwip/lwip && find ../../../../patches/lwip/ -type f -name '*.patch' -exec patch -p1 -i {} \; )
  • idf.py build
  • idf.py erase_flash (optional if you want to erase flash on the esp32)
  • idf.py -p (PORT) flash monitor (on my machine, the PORT is /dev/ttyUSB0)
  • Note: If you are logging in to an account fresh and you’ve already done the above steps, it seem you need to do the following to compile and upload it successfully
    • cd ~/iot-lite/iotivity-lite/port/esp32
    • . ./esp-idf/export.sh
    • idf.py build
    • idf.py erase_flash (optional)
    • idf.py -p (port) flash monitor
      • Note: When in monitor mode, you can use Ctrl + ] to break out (like Ctrl + C)

Scripts and examples for using DeviceBuilder

  • cd ~
  • curl https://openconnectivity.github.io/Project-Scripts/install.sh | bash
  • curl https://raw.githubusercontent.com/openconnectivity/ESP32-Code/main/esp32/install.sh | bash
  • You can then use the following process to build an esp32 project. I usually create a ~/workspace directory and cd ~/workspace
    • create_project.sh esp32test
    • cd esp32test
    • cp ~/ESP32-Code/IotTivity-lite/esp32-example/setup .
    • ./setup
  • Once that is done, you can repeat the following commands to build the device
    • gen.sh (this will run DeviceBuilder to create the device described in the esp32test-config.json file)
    • build.sh (at least once after you login, you will need to do the following:)
      • cd ~/iot-lite/iotivity/port/esp32
      • . ./esp-idf/export.sh
    • reset.sh (to set to RFOTM)
    • run.sh (this will upload the code to the esp32 and start the monitor mode so you can get some feedback)
      • NOTE: You can use Ctrl + ] to exit the monitor (This will stop feedback from the esp32 and get you back to the command line.)

 

 

 

From: "iotivity-dev@iotivity.groups.io" <iotivity-dev@iotivity.groups.io> on behalf of "Farid BENAMROUCHE via groups.io" <fariouche@...>
Reply-To: "fariouche@..." <fariouche@...>
Date: Wednesday, November 4, 2020 at 8:43 AM
To: "iotivity-dev@iotivity.groups.io" <iotivity-dev@iotivity.groups.io>
Subject: [dev] esp32 port of iotivity-lite: discovery memory issue

 

Hello,

I have a working port of iotivity-lite for the esp32 chip.
I would like to submit this port to the community, but before that I have 2 issues to fix. ipv6 is not working properly yet (esp32 stack issue) and one that is the reason of this post:

I'm very resources constrained, and mostly optimized everything now. However I've discovered that some memory are kept by iotivity-lite for 10min for each discovery requests...
The problem is that for each received discovery request, an ssl context is allocated and kept for 10min.
However, if it receives an other discvery request, an additionnal chunk (20KB!) of memory is allocated again for 10min.
So after 3 discovery requests, I'm out of memory...


Why is that mechanism in place? Is it possible to limit the number of pending discovery requests to 1 only? (clean the previous one before processing the new one?)

Isn't it an security issue on all platforms? (if a code sends enough discovery requests as fast as possible it can DDOS any iotivity-lite server)

Is my understanding correct?

Thank you!
Regards


Farid BENAMROUCHE
 

great, thanks.
My port is a bit simpler than yours (no ancillary data, no lwip patches, no ip reconfiguration if network is up/down)
I will give it a try.

But I believe that you will get the same issue as me. You have to exit the obt tool before retrying a new discover to generate a new request id.


Farid BENAMROUCHE
 

I think I understand what is happening:
When discovering a owned device, there is an UDP ssl session that is created to retrieve data (I'm doing an oc_get("/oic/d")).
And the only way to keep track of this connection is to have this idle timer to remove unused connections.

My problem remains the same: is it possible to limit the number of current sessions?


Farid BENAMROUCHE
 

Hello,

Anyone knows how to solve the discovery memory issue?

Thank you
Regards,


Farid BENAMROUCHE
 

Hello,

By the way, I'm now using the esp32 port from "Clarke Stevens", and it is working, but still have the same issues.

By the way, I'm trying to use ipv6 only, with the android iotivity-lite sample application.
Discovery does not work, the sendmsg() function returns an error, errno 118.

This seems to be because the esp32 does not find any route src ipv6 address (I'm using the esp32 as an AP, so my android smartphone is connected directly to the esp32's wifi AP)
I've narrowed it down to udp_sendto_if_chksum() in lwip returning ERR_RTE because ip6_select_source_address() returns a NULL src_ip.
The comment is: No suitable source address was found.

Have you seen this issue?


Clarke Stevens
 

Farid,

 

I don’t remember when I sent you the ESP32 instructions, but there were some issues that Jozef fixed with some patches. I’m enclosing the instructions that show how to install those patches. Also, we’re right in the middle of merging these changes into the main repo, so this will be a bit easier and cleaner shortly.

 

Thanks,

-Clarke

 

 

IoTivity-lite setup for esp32

  • cd ~
  • curl https://openconnectivity.github.io/IOTivity-Lite-setup/install.sh | bash
  • Use the following steps to get the esp32 branch until it is merged

 

Ubuntu

  • I’m not sure if you’ll need to do this, but I had to do it to get permission to use my serial port
    • sudo usermod -a -G tty <username> (then I had to logout and login)
    • sudo usermod -a -G dialout <username> (I’m not sure which of these will work or if you need both)
    • groups (to verify if you have been added to tty and/or dialout groups
  • sudo apt install -y git wget flex bison gperf python3 python3-pip python3-setuptools python3-serial python3-click python3-cryptography python3-future python3-pyparsing python3-pyelftools cmake ninja-build ccache libffi-dev libssl-dev libusb-1.0-0
  • cd ~/iot-lite/iotivity/port/esp32
  • git clone --recursive https://github.com/espressif/esp-idf.git
  • ./esp-idf/install.sh
  • . ./esp-idf/export.sh
  • jump to the common steps below

Windows (note that the windows steps are from Jozef and I have not tested them yet)

Common steps

  • idf.py set-target esp32
  • idf.py menuconfig // set wifi (This will bring up a GUI where you need to set SSID and SSID password under the IoTivity menu item. Exit and save.)
  • ( cd esp-idf/components/mbedtls/mbedtls && git am ../../../../patches/mbedtls/*.patch )
  • ( cd esp-idf && find ../patches/esp-idf/ -type f -name '*.patch' -exec patch -p1 -i {} \; )
  • ( cd esp-idf/components/lwip/lwip && find ../../../../patches/lwip/ -type f -name '*.patch' -exec patch -p1 -i {} \; )
  • idf.py build
  • idf.py erase_flash (optional if you want to erase flash on the esp32)
  • idf.py -p (PORT) flash monitor (on my machine, the PORT is /dev/ttyUSB0)
  • Note: If you are logging in to an account fresh and you’ve already done the above steps, it seem you need to do the following to compile and upload it successfully
    • cd ~/iot-lite/iotivity-lite/port/esp32
    • . ./esp-idf/export.sh
    • idf.py build
    • idf.py erase_flash (optional)
    • idf.py -p (port) flash monitor
      • Note: When in monitor mode, you can use Ctrl + ] to break out (like Ctrl + C)

Scripts and examples for using DeviceBuilder

  • cd ~
  • curl https://openconnectivity.github.io/Project-Scripts/install.sh | bash
  • curl https://raw.githubusercontent.com/openconnectivity/ESP32-Code/main/esp32/install.sh | bash
  • You can then use the following process to build an esp32 project. I usually create a ~/workspace directory and cd ~/workspace
    • create_project.sh esp32test
    • cd esp32test
    • cp ~/ESP32-Code/IotTivity-lite/esp32-example/setup .
    • ./setup
  • Once that is done, you can repeat the following commands to build the device
    • gen.sh (this will run DeviceBuilder to create the device described in the esp32test-config.json file)
    • build.sh (at least once after you login, you will need to do the following:)
      • cd ~/iot-lite/iotivity/port/esp32
      • . ./esp-idf/export.sh
    • reset.sh (to set to RFOTM)
    • run.sh (this will upload the code to the esp32 and start the monitor mode so you can get some feedback)
      • NOTE: You can use Ctrl + ] to exit the monitor (This will stop feedback from the esp32 and get you back to the command line.)

 

 

 

From: "iotivity-dev@iotivity.groups.io" <iotivity-dev@iotivity.groups.io> on behalf of "Farid BENAMROUCHE via groups.io" <fariouche@...>
Reply-To: "fariouche@..." <fariouche@...>
Date: Tuesday, December 8, 2020 at 5:21 AM
To: "iotivity-dev@iotivity.groups.io" <iotivity-dev@iotivity.groups.io>
Subject: Re: [dev] esp32 port of iotivity-lite: discovery memory issue

 

Hello,

By the way, I'm now using the esp32 port from "Clarke Stevens", and it is working, but still have the same issues.

By the way, I'm trying to use ipv6 only, with the android iotivity-lite sample application.
Discovery does not work, the sendmsg() function returns an error, errno 118.

This seems to be because the esp32 does not find any route src ipv6 address (I'm using the esp32 as an AP, so my android smartphone is connected directly to the esp32's wifi AP)
I've narrowed it down to udp_sendto_if_chksum() in lwip returning ERR_RTE because ip6_select_source_address() returns a NULL src_ip.
The comment is: No suitable source address was found.

Have you seen this issue?