micropython · tve · Apr 26, 2020 · Apr 26, 2020 · Apr 26, 2020 · Apr 26, 2020
diff --git a/docs/library/utime.rst b/docs/library/utime.rst
@@ -9,37 +9,34 @@
 The ``utime`` module provides functions for getting the current time and date,
 measuring time intervals, and for delays.
 
-**Time Epoch**: Unix port uses standard for POSIX systems epoch of
-1970-01-01 00:00:00 UTC. However, embedded ports use epoch of
-2000-01-01 00:00:00 UTC.
-
-**Maintaining actual calendar date/time**: This requires a
-Real Time Clock (RTC). On systems with underlying OS (including some
-RTOS), an RTC may be implicit. Setting and maintaining actual calendar
-time is responsibility of OS/RTOS and is done outside of MicroPython,
-it just uses OS API to query date/time. On baremetal ports however
-system time depends on ``machine.RTC()`` object. The current calendar time
-may be set using ``machine.RTC().datetime(tuple)`` function, and maintained
-by following means:
-
-* By a backup battery (which may be an additional, optional component for
-  a particular board).
-* Using networked time protocol (requires setup by a port/user).
-* Set manually by a user on each power-up (many boards then maintain
-  RTC time across hard resets, though some may require setting it again
-  in such case).
-
-If actual calendar time is not maintained with a system/MicroPython RTC,
-functions below which require reference to current absolute time may
-behave not as expected.
+**Time Epoch**: The Unix port uses the POSIX standard epoch of
+1970-01-01 00:00:00 UTC. However, all other ports use an epoch of
+2000-01-01 00:00:00 UTC. The reason for the unconventional epoch
+is that, at least until 2034, the time fits into a 31-bit "small int"
+and thus a call to `time.time()` does not involve memory allocation.
+
+**Maintaining actual calendar date/time** is port-dependent.
+* On operating systems, such as Unix, that manage time on their own
+  MicroPython simply makes the appropriate system calls to retrieve
+  time and it cannot set the time.
+* On ports with an RTOS that can manage time, the Real Time Clock (RTC)
+  is managed by the RTOS and MicroPython leverages the RTOS' functionality
+  to retrieve and set time. On those ports the ``machine.RTC`` object
+  should not be used to set/get time, however the `set_time()` and
+  `adjtime()` methods in this module may be used.
+* On baremetal ports the Real Time Clock (RTC) must be initialized by
+  the application using either the ``machine.RTC`` class or the `set_time()`
+  method. On those ports `adjtime()`` and time zones are not available
+  and `localtime()` as well as `gmtime()` return the same time as set
+  in the RTC.
 
 Functions
 ---------
 
 .. function:: localtime([secs])
 
-   Convert a time expressed in seconds since the Epoch (see above) into an 8-tuple which
-   contains: (year, month, mday, hour, minute, second, weekday, yearday)
+   Convert a time expressed in seconds since the Epoch (see above) into a 9-tuple which
+   contains: (year, month, mday, hour, minute, second, weekday, yearday, isdst)
    If secs is not provided or None, then the current time from the RTC is used.
 
    * year includes the century (for example 2014).
@@ -50,12 +47,56 @@ Functions
    * second  is 0-59
    * weekday is 0-6 for Mon-Sun
    * yearday is 1-366
+   * isdst   is 0=no daylight savings in effect, 1=dst in effect, -1=unknown
+
+.. function:: gmtime([secs])
+
+   Gmtime is identical to localtime except that it returns UTC time.
 
 .. function:: mktime()
 
-   This is inverse function of localtime. It's argument is a full 8-tuple
-   which expresses a time as per localtime. It returns an integer which is
-   the number of seconds since Jan 1, 2000.
+   This is inverse function of localtime. Its argument is a full 9-tuple
+   which expresses a time as per localtime. The values for weekday and yearday are
+   ignored.
+   It returns an integer which is the number of seconds since the Epoch.
+
+.. function:: tzset(zone)
+
+   Set the conversion rules between UTC time and local time used by `mktime()` and
+   `localtime()`.
+   The argument is a string that specifies the name of the zones with/without daylight
+   savings, the standard offset to UTC, and the start/end times of daylight savings.
+   The full rules are described in CPython's `time` module, except that there is no
+   zoneinfo database.
+
+   As an example, the America/Los_Angeles time zone can be specified as
+   `PST+8PDT,M3.2.0/2,M11.1.0/2` where:
+   * PST is the standard time zone name
+   * +8 is the offset to convert from PST to UTC
+   * PDT is the daylight savings time zone name
+   * M3.2.0/2 designates that daylight savings starts at 2am ("/2") on sunday (".0")
+     of the second week (".2") of the third month ("M3.")
+   * M11.1.0/2 designates that daylight savings ends at 2am on sunday of the first week
+     of the eleventh month
+
+   See https://sites.google.com/a/usapiens.com/opnode/time-zones for a lit of zone specs.
+
+.. function:: set_time(secs)
+
+   Sets the current time to the specified number of seconds since the Epoch.
+
+   `set_time()` is an extension to CPython's time module.
+
+.. function:: adjtime(microseconds)
+
+   Gradually adjusts the current time by the number of microseconds specified
+   and returns the adjustment that still remains if one is in progress. The
+   semantics are the same as for the POSIX adjtime call.
+
+   There is an implementation-dependent maximum adjustment, but in general, if
+   the step is large set_time should be used.
+
+   `adjtime()` is an extension to CPython's time module.
 
 .. function:: sleep(seconds)
 
@@ -219,11 +260,11 @@ Functions
 
       In CPython, this function returns number of
       seconds since Unix epoch, 1970-01-01 00:00 UTC, as a floating-point,
-      usually having microsecond precision. With MicroPython, only Unix port
-      uses the same Epoch, and if floating-point precision allows,
-      returns sub-second precision. Embedded hardware usually doesn't have
+      usually having microsecond precision. With MicroPython, some ports
+      use the same Epoch, and if floating-point precision allows,
+      return sub-second precision. Embedded hardware usually doesn't have
       floating-point precision to represent both long time ranges and subsecond
-      precision, so they use integer value with second precision. Some embedded
-      hardware also lacks battery-powered RTC, so returns number of seconds
-      since last power-up or from other relative, hardware-specific point
+      precision, so they use an integer value with second precision. Some embedded
+      hardware also lacks battery-powered RTC, and returns the number of seconds
+      since the last power-up or from another relative, hardware-specific, point
       (e.g. reset).
diff --git a/ports/esp32/Makefile b/ports/esp32/Makefile
@@ -496,7 +496,6 @@ $(BUILD)/$(ESPCOMP)/lwip/%.o: CFLAGS += -Wno-address -Wno-unused-variable -Wno-u
 ESPIDF_LWIP_O = $(patsubst %.c,%.o,\
 	$(wildcard $(ESPCOMP)/lwip/apps/dhcpserver/*.c) \
 	$(wildcard $(ESPCOMP)/lwip/lwip/src/api/*.c) \
-	$(wildcard $(ESPCOMP)/lwip/lwip/src/apps/sntp/*.c) \
 	$(wildcard $(ESPCOMP)/lwip/lwip/src/core/*.c) \
 	$(wildcard $(ESPCOMP)/lwip/lwip/src/core/*/*.c) \
 	$(wildcard $(ESPCOMP)/lwip/lwip/src/netif/*.c) \

diff --git a/ports/esp32/machine_rtc.c b/ports/esp32/machine_rtc.c
@@ -126,9 +126,10 @@ STATIC mp_obj_t machine_rtc_datetime(mp_uint_t n_args, const mp_obj_t *args) {
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(machine_rtc_datetime_obj, 1, 2, machine_rtc_datetime);
 
-STATIC mp_obj_t machine_rtc_init(mp_obj_t self_in, mp_obj_t date) {
-    mp_obj_t args[2] = {self_in, date};
-    machine_rtc_datetime_helper(2, args);
+STATIC mp_obj_t machine_rtc_init(mp_uint_t n_args, const mp_obj_t *args) {
+    if (n_args == 2) {
+        machine_rtc_datetime_helper(n_args, args);
+    }
 
     #if MICROPY_HW_RTC_USER_MEM_MAX > 0
     if (rtc_user_mem_magic != MEM_MAGIC) {
@@ -139,7 +140,7 @@ STATIC mp_obj_t machine_rtc_init(mp_obj_t self_in, mp_obj_t date) {
 
     return mp_const_none;
 }
-STATIC MP_DEFINE_CONST_FUN_OBJ_2(machine_rtc_init_obj, machine_rtc_init);
+STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(machine_rtc_init_obj, 1, 2, machine_rtc_init);
 
 #if MICROPY_HW_RTC_USER_MEM_MAX > 0
 STATIC mp_obj_t machine_rtc_memory(mp_uint_t n_args, const mp_obj_t *args) {

diff --git a/ports/esp32/modutime.c b/ports/esp32/modutime.c
@@ -28,66 +28,196 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <time.h>
 #include <sys/time.h>
+#include <errno.h>
 
 #include "py/runtime.h"
-#include "lib/timeutils/timeutils.h"
 #include "extmod/utime_mphal.h"
 
-STATIC mp_obj_t time_localtime(size_t n_args, const mp_obj_t *args) {
-    timeutils_struct_time_t tm;
-    mp_int_t seconds;
+// time module mostly compatible with CPython's time. For reference, this is the CPython
+// struct_time from https://docs.python.org/3/library/time.html#time.struct_time:
+// Index Attribute Values
+//   0   tm_year   (for example, 1993)
+//   1   tm_mon    range [1, 12]
+//   2   tm_mday   range [1, 31]
+//   3   tm_hour   range [0, 23]
+//   4   tm_min    range [0, 59]
+//   5   tm_sec    range [0, 61]; see (2) in strftime() description
+//   6   tm_wday   range [0, 6], Monday is 0
+//   7   tm_yday   range [1, 366]
+//   8   tm_isdst  0=no, 1=yes, -1=unknown
+// N/A   tm_zone   abbreviation of timezone name
+// N/A   tm_gmtoff offset east of UTC in seconds
+// This module supports the first 9 elements (i.e. a 9-tuple) without names.
+
+// THE EPOCH: this module conforms to the MicroPython "standard" of having the time
+// epoch defined as 2000/1/1 instead of the POSIX std of 1970/1/1. The reason for this
+// decision is that it allows time values up to 2034 to fit into a small int (31 signed bits)
+// and thus time.time() does not require memory allocation. In order to make things work,
+// time values are shifted from one epoch to the other around calls to localtime, gmtime,
+// and mktime.
+#define EPOCH_DELTA 946684800  // seconds between 1970/1/1 and 2000/1/1
+
+// convert a python 9-tuple to a struct tm
+STATIC void time_tm_from_tuple(const mp_obj_t tuple, struct tm *tm) {
+    size_t len;
+    mp_obj_t *elem;
+    mp_obj_get_array(tuple, &len, &elem);
+
+    if (len != 9) {
+        mp_raise_msg_varg(&mp_type_TypeError, MP_ERROR_TEXT("9-tuple required (%d given)"), len);
+    }
+
+    time_t wday = mp_obj_get_int(elem[6]) + 1;
+    if (wday > 7) {
+        wday = 0;
+    }
+    tm->tm_year = mp_obj_get_int(elem[0]) - 1900;
+    tm->tm_mon = mp_obj_get_int(elem[1]) - 1;
+    tm->tm_mday = mp_obj_get_int(elem[2]);
+    tm->tm_hour = mp_obj_get_int(elem[3]);
+    tm->tm_min = mp_obj_get_int(elem[4]);
+    tm->tm_sec = mp_obj_get_int(elem[5]);
+    tm->tm_wday = wday;
+    tm->tm_yday = mp_obj_get_int(elem[7]) - 1;
+    tm->tm_isdst = mp_obj_get_int(elem[8]);
+}
+
+// convert a struct tm to a python 9-tuple
+STATIC mp_obj_t *time_tm_to_tuple(const struct tm *tm) {
+    mp_obj_t tuple[9] = {
+        tuple[0] = mp_obj_new_int(1900 + tm->tm_year),
+        tuple[1] = mp_obj_new_int(tm->tm_mon + 1),
+        tuple[2] = mp_obj_new_int(tm->tm_mday),
+        tuple[3] = mp_obj_new_int(tm->tm_hour),
+        tuple[4] = mp_obj_new_int(tm->tm_min),
+        tuple[5] = mp_obj_new_int(tm->tm_sec),
+        tuple[6] = mp_obj_new_int(tm->tm_wday == 0 ? 6 : tm->tm_wday - 1),
+        tuple[7] = mp_obj_new_int(tm->tm_yday + 1),
+        tuple[8] = mp_obj_new_int(tm->tm_isdst),
+    };
+    return mp_obj_new_tuple(9, tuple);
+}
+
+// time_get_posix_seconds is a helper and either returns the seconds_since_posix_epoch as
+// passed in args or it queries the system clock for that
+STATIC time_t time_get_posix_seconds(size_t n_args, const mp_obj_t *args) {
     if (n_args == 0 || args[0] == mp_const_none) {
         struct timeval tv;
-        gettimeofday(&tv, NULL);
-        seconds = tv.tv_sec;
+        if (gettimeofday(&tv, NULL) != 0) {
+            mp_raise_OSError(errno);
+        }
+        return tv.tv_sec;
     } else {
-        seconds = mp_obj_get_int(args[0]);
+        return mp_obj_get_int(args[0]) + EPOCH_DELTA;
     }
-    timeutils_seconds_since_2000_to_struct_time(seconds, &tm);
-    mp_obj_t tuple[8] = {
-        tuple[0] = mp_obj_new_int(tm.tm_year),
-        tuple[1] = mp_obj_new_int(tm.tm_mon),
-        tuple[2] = mp_obj_new_int(tm.tm_mday),
-        tuple[3] = mp_obj_new_int(tm.tm_hour),
-        tuple[4] = mp_obj_new_int(tm.tm_min),
-        tuple[5] = mp_obj_new_int(tm.tm_sec),
-        tuple[6] = mp_obj_new_int(tm.tm_wday),
-        tuple[7] = mp_obj_new_int(tm.tm_yday),
-    };
-    return mp_obj_new_tuple(8, tuple);
+}
+
+STATIC mp_obj_t time_gmtime(size_t n_args, const mp_obj_t *args) {
+    time_t seconds = time_get_posix_seconds(n_args, args);
+    struct tm tm;
+    gmtime_r(&seconds, &tm);
+    return time_tm_to_tuple(&tm);
+}
+STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(time_gmtime_obj, 0, 1, time_gmtime);
+
+STATIC mp_obj_t time_localtime(size_t n_args, const mp_obj_t *args) {
+    time_t seconds = time_get_posix_seconds(n_args, args);
+    struct tm tm;
+    localtime_r(&seconds, &tm);
+    return time_tm_to_tuple(&tm);
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(time_localtime_obj, 0, 1, time_localtime);
 
 STATIC mp_obj_t time_mktime(mp_obj_t tuple) {
-    size_t len;
-    mp_obj_t *elem;
-    mp_obj_get_array(tuple, &len, &elem);
+    struct tm tm;
+    time_tm_from_tuple(tuple, &tm);
 
-    // localtime generates a tuple of len 8. CPython uses 9, so we accept both.
-    if (len < 8 || len > 9) {
-        mp_raise_msg_varg(&mp_type_TypeError, MP_ERROR_TEXT("mktime needs a tuple of length 8 or 9 (%d given)"), len);
+    time_t seconds = mktime(&tm);
+    if (seconds == -1) {
+        mp_raise_ValueError(MP_ERROR_TEXT("invalid input"));
     }
-
-    return mp_obj_new_int_from_uint(timeutils_mktime(mp_obj_get_int(elem[0]),
-        mp_obj_get_int(elem[1]), mp_obj_get_int(elem[2]), mp_obj_get_int(elem[3]),
-        mp_obj_get_int(elem[4]), mp_obj_get_int(elem[5])));
+    return mp_obj_new_int(seconds - EPOCH_DELTA); // MP_OBJ_NEW_SMALL_INT? e.g. allow pre 1964?
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_1(time_mktime_obj, time_mktime);
 
 STATIC mp_obj_t time_time(void) {
     struct timeval tv;
-    gettimeofday(&tv, NULL);
-    return mp_obj_new_int(tv.tv_sec);
+    if (gettimeofday(&tv, NULL) != 0) {
+        mp_raise_OSError(errno);
+    }
+    return MP_OBJ_NEW_SMALL_INT(tv.tv_sec - EPOCH_DELTA);
 }
 MP_DEFINE_CONST_FUN_OBJ_0(time_time_obj, time_time);
 
+STATIC mp_obj_t time_time_us(void) {
+    struct timeval tv;
+    if (gettimeofday(&tv, NULL) != 0) {
+        mp_raise_OSError(errno);
+    }
+    long long usec = ((long long)tv.tv_sec - EPOCH_DELTA) * 1000000 + (long long)tv.tv_usec;
+    return mp_obj_new_int_from_ll(usec);
+}
+MP_DEFINE_CONST_FUN_OBJ_0(time_time_us_obj, time_time_us);
+
+STATIC mp_obj_t time_tzset(mp_obj_t tz) {
+    // tz is something like PST+8PDT,M3.2.0/2,M11.1.0/2
+    const char *zone = mp_obj_str_get_str(tz);
+    setenv("TZ", zone, 1);
+    tzset();
+    return mp_const_none;
+}
+MP_DEFINE_CONST_FUN_OBJ_1(time_tzset_obj, time_tzset);
+
+STATIC mp_obj_t time_settime(const mp_obj_t secs_in, const mp_obj_t usecs_in) {
+    mp_int_t secs = mp_obj_get_int(secs_in);
+    mp_int_t usecs = mp_obj_get_int(usecs_in);
+    struct timeval tv = { secs + EPOCH_DELTA, usecs };
+    if (settimeofday(&tv, NULL) != 0) {
+        mp_raise_OSError(errno);
+    }
+    return mp_const_none;
+}
+MP_DEFINE_CONST_FUN_OBJ_2(time_settime_obj, time_settime);
+
+STATIC mp_obj_t time_adjtime(const mp_obj_t microseconds_in) {
+    // esp-idf is adjtime is broken in that it returns the current adjustment instead of
+    // the previous adjustement in outdelta. So we need to call it twice.
+    struct timeval tv_old;
+    adjtime(NULL, &tv_old);
+
+    // now make the new adjustment
+    if (microseconds_in != mp_const_none) {
+        // esp32 esp-idf adjtime allows NULL delta to retrieve old delta remaining
+        // not sure this is std...
+        struct timeval tv = { 0, mp_obj_get_int(microseconds_in) };
+        // handle microseconds "overflow"
+        if (tv.tv_usec >= 1000000 || tv.tv_usec <= -1000000) {
+            tv.tv_sec = tv.tv_usec / 1000000;
+            tv.tv_usec = tv.tv_usec % 1000000;
+        }
+        // call adjtime
+        if (adjtime(&tv, NULL) != 0) {
+            mp_raise_ValueError(MP_ERROR_TEXT("adjustment too big"));
+        }
+    }
+
+    return mp_obj_new_int(tv_old.tv_sec * 1000000 + tv_old.tv_usec);
+}
+MP_DEFINE_CONST_FUN_OBJ_1(time_adjtime_obj, time_adjtime);
+
 STATIC const mp_rom_map_elem_t time_module_globals_table[] = {
     { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_utime) },
 
+    { MP_ROM_QSTR(MP_QSTR_gmtime), MP_ROM_PTR(&time_gmtime_obj) },
     { MP_ROM_QSTR(MP_QSTR_localtime), MP_ROM_PTR(&time_localtime_obj) },
     { MP_ROM_QSTR(MP_QSTR_mktime), MP_ROM_PTR(&time_mktime_obj) },
+    { MP_ROM_QSTR(MP_QSTR_tzset), MP_ROM_PTR(&time_tzset_obj) },
     { MP_ROM_QSTR(MP_QSTR_time), MP_ROM_PTR(&time_time_obj) },
+    { MP_ROM_QSTR(MP_QSTR_time_us), MP_ROM_PTR(&time_time_us_obj) },
+    { MP_ROM_QSTR(MP_QSTR_settime), MP_ROM_PTR(&time_settime_obj) },
+    { MP_ROM_QSTR(MP_QSTR_adjtime), MP_ROM_PTR(&time_adjtime_obj) },
     { MP_ROM_QSTR(MP_QSTR_sleep), MP_ROM_PTR(&mp_utime_sleep_obj) },
     { MP_ROM_QSTR(MP_QSTR_sleep_ms), MP_ROM_PTR(&mp_utime_sleep_ms_obj) },
     { MP_ROM_QSTR(MP_QSTR_sleep_us), MP_ROM_PTR(&mp_utime_sleep_us_obj) },