Merge branch 'ethernet' of github.com:d-a-v/Arduino into ethernet

README.md

@@ -3,7 +3,7 @@ Arduino core for ESP8266 WiFi chip
 
 # Quick links
 
-- [Latest release documentation](https://arduino-esp8266.readthedocs.io/en/2.6.3/)
+- [Latest release documentation](https://arduino-esp8266.readthedocs.io/en/2.7.1/)
 - [Current "git version" documentation](https://arduino-esp8266.readthedocs.io/en/latest/)
 - [Install git version](https://arduino-esp8266.readthedocs.io/en/latest/installing.html#using-git-version) ([sources](doc/installing.rst#using-git-version))
 
@@ -28,15 +28,15 @@ ESP8266 Arduino core comes with libraries to communicate over WiFi using TCP and
 
 Starting with 1.6.4, Arduino allows installation of third-party platform packages using Boards Manager. We have packages available for Windows, Mac OS, and Linux (32 and 64 bit).
 
-- Install the current upstream Arduino IDE at the 1.8.7 level or later. The current version is on the [Arduino website](https://www.arduino.cc/en/main/software).
+- Install the current upstream Arduino IDE at the 1.8.9 level or later. The current version is on the [Arduino website](https://www.arduino.cc/en/main/software).
 - Start Arduino and open the Preferences window.
 - Enter ```https://arduino.esp8266.com/stable/package_esp8266com_index.json``` into the *Additional Board Manager URLs* field. You can add multiple URLs, separating them with commas.
 - Open Boards Manager from Tools > Board menu and install *esp8266* platform (and don't forget to select your ESP8266 board from Tools > Board menu after installation).
 
 #### Latest release [![Latest release](https://img.shields.io/github/release/esp8266/Arduino.svg)](https://github.com/esp8266/Arduino/releases/latest/)
 Boards manager link: `https://arduino.esp8266.com/stable/package_esp8266com_index.json`
 
-Documentation: [https://arduino-esp8266.readthedocs.io/en/2.6.3/](https://arduino-esp8266.readthedocs.io/en/2.6.3/)
+Documentation: [https://arduino-esp8266.readthedocs.io/en/2.7.1/](https://arduino-esp8266.readthedocs.io/en/2.7.1/)
 
 ### Using git version
 [![Linux build status](https://travis-ci.org/esp8266/Arduino.svg)](https://travis-ci.org/esp8266/Arduino)
@@ -73,7 +73,7 @@ Documentation for latest development version: https://arduino-esp8266.readthedoc
 
 ### Issues and support ###
 
-[ESP8266 Community Forum](https://www.esp8266.com/u/arduinoanswers) is a well-established community for questions and answers about Arduino for ESP8266. If you need help, have a "How do I..." type question, have a problem with a 3rd party library not hosted in this repo, or just want to discuss how to approach a problem, please ask there.
+[ESP8266 Community Forum](https://www.esp8266.com/u/arduinoanswers) is a well-established community for questions and answers about Arduino for ESP8266. Stackoverflow is also an alternative. If you need help, have a "How do I..." type question, have a problem with a 3rd party library not hosted in this repo, or just want to discuss how to approach a problem, please ask there.
 
 If you find the forum useful, please consider supporting it with a donation. <br />
 [![Donate](https://img.shields.io/badge/paypal-donate-yellow.svg)](https://www.paypal.com/webscr?cmd=_s-xclick&hosted_button_id=4M56YCWV6PX66)
@@ -96,7 +96,7 @@ For minor fixes of code and documentation, please go ahead and submit a pull req
 
 Check out the list of issues that are easy to fix — [easy issues pending](https://github.com/esp8266/Arduino/issues?q=is%3Aopen+is%3Aissue+label%3A%22level%3A+easy%22). Working on them is a great way to move the project forward.
 
-Larger changes (rewriting parts of existing code from scratch, adding new functions to the core, adding new libraries) should generally be discussed by opening an issue first.
+Larger changes (rewriting parts of existing code from scratch, adding new functions to the core, adding new libraries) should generally be discussed by opening an issue first. PRs with such changes require testing and approval.
 
 Feature branches with lots of small commits (especially titled "oops", "fix typo", "forgot to add file", etc.) should be squashed before opening a pull request. At the same time, please refrain from putting multiple unrelated changes into a single pull request.
 
@@ -135,3 +135,5 @@ ESP8266 core files are licensed under LGPL.
 [SoftwareSerial repo](https://github.com/plerup/espsoftwareserial)
 
 [Serial Monitor Arduino IDE plugin](https://github.com/mytrain/arduino-esp8266-serial-plugin) Original discussion [here](https://github.com/esp8266/Arduino/issues/1360), quick download [there](http://mytrain.fr/cms//images/mytrain/private/ESP8266SM.v3.zip).
+
+[FTP Client/Server Library](https://github.com/dplasa/FTPClientServer) 

boards.txt

@@ -4960,15 +4960,15 @@ espinotee.menu.baud.3000000.upload.speed=3000000
 wifinfo.name=WifInfo
 wifinfo.build.board=WIFINFO
 wifinfo.build.variant=wifinfo
-wifinfo.menu.ESPModule.ESP07192=ESP07 (1M/192K SPIFFS)
+wifinfo.menu.ESPModule.ESP07192=ESP07 (1M/192K FS)
 wifinfo.menu.ESPModule.ESP07192.build.board=ESP8266_ESP07
 wifinfo.menu.ESPModule.ESP07192.build.flash_ld=eagle.flash.1m192.ld
 wifinfo.menu.ESPModule.ESP07192.build.flash_size=1M
 wifinfo.menu.ESPModule.ESP07192.build.spiffs_blocksize=4096
 wifinfo.menu.ESPModule.ESP07192.build.spiffs_end=0xFB000
 wifinfo.menu.ESPModule.ESP07192.build.spiffs_start=0xCB000
 wifinfo.menu.ESPModule.ESP07192.upload.maximum_size=827376
-wifinfo.menu.ESPModule.ESP12=ESP12 (4M/1M SPIFFS)
+wifinfo.menu.ESPModule.ESP12=ESP12 (4M/1M FS)
 wifinfo.menu.ESPModule.ESP12.build.board=ESP8266_ESP12
 wifinfo.menu.ESPModule.ESP12.build.flash_ld=eagle.flash.4m1m.ld
 wifinfo.menu.ESPModule.ESP12.build.flash_size=4M

bootloaders/eboot/eboot.c

@@ -12,7 +12,6 @@
 #include <string.h>
 #include "flash.h"
 #include "eboot_command.h"
-#include "spi_vendors.h"
 #include <uzlib.h>
 
 extern unsigned char _gzip_dict;
@@ -115,10 +114,12 @@ int uzlib_flash_read_cb(struct uzlib_uncomp *m)
 }
 
 unsigned char gzip_dict[32768];
+uint8_t buffer2[FLASH_SECTOR_SIZE]; // no room for this on the stack
 
 int copy_raw(const uint32_t src_addr,
              const uint32_t dst_addr,
-             const uint32_t size)
+             const uint32_t size,
+             const bool verify)
 {
     // require regions to be aligned
     if ((src_addr & 0xfff) != 0 ||
@@ -158,8 +159,10 @@ int copy_raw(const uint32_t src_addr,
 	gzip = true;
     }
     while (left > 0) {
-        if (SPIEraseSector(daddr/buffer_size)) {
-            return 2;
+        if (!verify) {
+           if (SPIEraseSector(daddr/buffer_size)) {
+               return 2;
+           }
         }
         if (!gzip) {
             if (SPIRead(saddr, buffer, buffer_size)) {
@@ -179,8 +182,17 @@ int copy_raw(const uint32_t src_addr,
                 buffer[i] = 0xff;
             }
         }
-        if (SPIWrite(daddr, buffer, buffer_size)) {
-            return 4;
+        if (verify) {
+            if (SPIRead(daddr, buffer2, buffer_size)) {
+                return 4;
+            }
+            if (memcmp(buffer, buffer2, buffer_size)) {
+                return 9;
+            }
+        } else {
+            if (SPIWrite(daddr, buffer, buffer_size)) {
+                return 4;
+            }
         }
         saddr += buffer_size;
         daddr += buffer_size;
@@ -190,29 +202,6 @@ int copy_raw(const uint32_t src_addr,
     return 0;
 }
 
-#define XMC_SUPPORT
-#ifdef XMC_SUPPORT
-// Define a few SPI0 registers we need access to
-#define ESP8266_REG(addr) *((volatile uint32_t *)(0x60000000+(addr)))
-#define SPI0CMD ESP8266_REG(0x200)
-#define SPI0CLK ESP8266_REG(0x218)
-#define SPI0C   ESP8266_REG(0x208)
-#define SPI0W0  ESP8266_REG(0x240)
-
-#define SPICMDRDID (1 << 28)
-
-/* spi_flash_get_id()
-   Returns the flash chip ID - same as the SDK function.
-   We need our own version as the SDK isn't available here.
- */
-uint32_t __attribute__((noinline)) spi_flash_get_id() {
-  SPI0W0=0;
-  SPI0CMD=SPICMDRDID;
-  while (SPI0CMD) {}
-  return SPI0W0;
-}
-#endif // XMC_SUPPORT
-
 int main()
 {
     int res = 9;
@@ -235,47 +224,20 @@ int main()
     if (cmd.action == ACTION_COPY_RAW) {
         ets_putc('c'); ets_putc('p'); ets_putc(':');
 
-#ifdef XMC_SUPPORT
-        // save the flash access speed registers
-        uint32_t spi0clk = SPI0CLK;
-        uint32_t spi0c   = SPI0C;
-
-        uint32_t vendor  = spi_flash_get_id() & 0x000000ff;
-        if (vendor == SPI_FLASH_VENDOR_XMC) {
-           uint32_t flashinfo=0;
-           if (SPIRead(0, &flashinfo, 4)) {
-              // failed to read the configured flash speed.
-              // Do not change anything,
-           } else {
-              // select an appropriate flash speed
-              // Register values are those used by ROM
-              switch ((flashinfo >> 24) & 0x0f) {
-                 case 0x0: // 40MHz, slow to 20
-                 case 0x1: // 26MHz, slow to 20
-                      SPI0CLK = 0x00003043;
-                      SPI0C   = 0x00EAA313;
-                      break;
-                 case 0x2: // 20MHz, no change
-                      break;
-                 case 0xf: // 80MHz, slow to 26
-                      SPI0CLK = 0x00002002;
-                      SPI0C   = 0x00EAA202;
-                      break;
-                 default:
-                      break;
-              }
-           }
-        }
-#endif // XMC_SUPPORT
         ets_wdt_disable();
-        res = copy_raw(cmd.args[0], cmd.args[1], cmd.args[2]);
+        res = copy_raw(cmd.args[0], cmd.args[1], cmd.args[2], false);
         ets_wdt_enable();
-
-#ifdef XMC_SUPPORT
-        // restore the saved flash access speed registers
-        SPI0CLK = spi0clk;
-        SPI0C   = spi0c;
-#endif        
+
+        ets_putc('0'+res); ets_putc('\n');
+
+        // Verify the copy
+        ets_putc('c'); ets_putc('m'); ets_putc('p'); ets_putc(':');
+        if (res == 0) {
+            ets_wdt_disable();
+            res = copy_raw(cmd.args[0], cmd.args[1], cmd.args[2], true);
+            ets_wdt_enable();
+            }
+
         ets_putc('0'+res); ets_putc('\n');
         if (res == 0) {
             cmd.action = ACTION_LOAD_APP;

cores/esp8266/Crypto.cpp

@@ -23,10 +23,9 @@
     THE SOFTWARE.
 */
 
-#include <bearssl/bearssl.h>
 #include "Crypto.h"
 #include <TypeConversion.h>
-
+#include <bearssl/bearssl.h>
 #include <assert.h>
 
 namespace TypeCast = experimental::TypeConversion;

cores/esp8266/Crypto.h

@@ -27,6 +27,7 @@
 #define __ESP8266_ARDUINO_CRYPTO_H__
 
 #include <Arduino.h>
+#include <bearssl/bearssl_kdf.h>
 
 namespace experimental
 {
@@ -745,8 +746,7 @@ struct HKDF
 
 private:
 
-    // Use an opaque type to avoid #include <bearssl/bearssl.h> which drags the lib declarations into userland. The global scope prefix is required for compilation to succeed, it seems.
-    ::br_hkdf_context hkdfContext;
+    br_hkdf_context hkdfContext;
 };
 
 

cores/esp8266/FS.h

@@ -266,7 +266,7 @@ using fs::SPIFFSConfig;
 #endif //FS_NO_GLOBALS
 
 #if !defined(NO_GLOBAL_INSTANCES) && !defined(NO_GLOBAL_SPIFFS)
-extern fs::FS SPIFFS;
+extern fs::FS SPIFFS __attribute__((deprecated("SPIFFS has been deprecated. Please consider moving to LittleFS or other filesystems.")));
 #endif
 
 #endif //FS_H

cores/esp8266/Schedule.h

@@ -2,6 +2,7 @@
 #define ESP_SCHEDULE_H
 
 #include <functional>
+#include <stdint.h>
 
 #define SCHEDULED_FN_MAX_COUNT 32
 

cores/esp8266/TZ.h

@@ -1,7 +1,7 @@
 
 // autogenerated from https://raw.githubusercontent.com/nayarsystems/posix_tz_db/master/zones.csv
 // by script <esp8266 arduino core>/tools/TZupdate.sh
-// Mon Dec 16 13:08:18 UTC 2019
+// Thu May  7 19:02:21 UTC 2020
 //
 // This database is autogenerated from IANA timezone database
 //    https://www.iana.org/time-zones

cores/esp8266/core_esp8266_spi_utils.cpp

@@ -52,7 +52,7 @@ static SpiOpResult PRECACHE_ATTR
 _SPICommand(volatile uint32_t spiIfNum,
             uint32_t spic,uint32_t spiu,uint32_t spiu1,uint32_t spiu2,
             uint32_t *data,uint32_t writeWords,uint32_t readWords)
-{ 
+{
   if (spiIfNum>1)
      return SPI_RESULT_ERR;
 
@@ -69,8 +69,11 @@ _SPICommand(volatile uint32_t spiIfNum,
   volatile SpiFlashChip *fchip=flashchip;
   volatile uint32_t spicmdusr=SPICMDUSR;
 
+  uint32_t saved_ps=0;
+
   if (!spiIfNum) {
-     // Only need to precache when using SPI0
+     // Only need to disable interrupts and precache when using SPI0
+     saved_ps = xt_rsil(15);
      PRECACHE_START();
      Wait_SPI_Idlep((SpiFlashChip *)fchip);
   }
@@ -116,6 +119,9 @@ _SPICommand(volatile uint32_t spiIfNum,
   SPIREG(SPI0C) = oldSPI0C;
 
   PRECACHE_END();
+  if (!spiIfNum) {
+      xt_wsr_ps(saved_ps);
+  }
   return (timeout>0 ? SPI_RESULT_OK : SPI_RESULT_TIMEOUT);
 }
 

cores/esp8266/esp8266_peri.h

@@ -250,7 +250,7 @@ extern volatile uint32_t* const esp8266_gpioToFn[16];
 #define UIFF	0 //RX FIFO Full
 
 //UART STATUS Registers Bits
-#define USTX    31 //TX PIN Level
+#define USTX    31 //TX PIN Level (Doesn't seem to work, always reads as 0 for both uarts. HW bug? Possible workaround: Enable loopback UxC0 |= 1<<UCLBE and read USRXD, see https://github.com/esp8266/Arduino/issues/7256 for discussion.)
 #define USRTS   30 //RTS PIN Level
 #define USDTR   39 //DTR PIN Level
 #define USTXC   16 //TX FIFO COUNT (8bit)

cores/esp8266/spiffs/README.md

@@ -49,6 +49,9 @@ What spiffs does not:
  - Presently, it does not detect or handle bad blocks.
  - One configuration, one binary. There's no generic spiffs binary that handles all types of configurations.
 
+## NOTICE
+
+0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1. If you have any worries or questions, it can be discussed in issue [#179](https://github.com/pellepl/spiffs/issues/179)
 
 ## MORE INFO 
 

cores/esp8266/spiffs/spiffs_check.cpp

@@ -163,11 +163,17 @@ static s32_t spiffs_delete_obj_lazy(spiffs *fs, spiffs_obj_id obj_id) {
     return SPIFFS_OK;
   }
   SPIFFS_CHECK_RES(res);
-  u8_t flags = 0xff & ~SPIFFS_PH_FLAG_IXDELE;
+  u8_t flags = 0xff;
+#if SPIFFS_NO_BLIND_WRITES
+  res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
+      0, SPIFFS_PAGE_TO_PADDR(fs, objix_hdr_pix) + offsetof(spiffs_page_header, flags),
+      sizeof(flags), &flags);
+  SPIFFS_CHECK_RES(res);
+#endif
+  flags &= ~SPIFFS_PH_FLAG_IXDELE;
   res = _spiffs_wr(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT,
       0, SPIFFS_PAGE_TO_PADDR(fs, objix_hdr_pix) + offsetof(spiffs_page_header, flags),
-      sizeof(u8_t),
-      (u8_t *)&flags);
+      sizeof(flags), &flags);
   return res;
 }
 
@@ -425,10 +431,17 @@ static s32_t spiffs_lookup_check_validate(spiffs *fs, spiffs_obj_id lu_obj_id, s
           // just finalize
           SPIFFS_CHECK_DBG("LU: FIXUP: unfinalized page is referred, finalizing\n");
           CHECK_CB(fs, SPIFFS_CHECK_LOOKUP, SPIFFS_CHECK_FIX_LOOKUP, p_hdr->obj_id, p_hdr->span_ix);
-          u8_t flags = 0xff & ~SPIFFS_PH_FLAG_FINAL;
+          u8_t flags = 0xff;
+#if SPIFFS_NO_BLIND_WRITES
+          res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_READ,
+              0, SPIFFS_PAGE_TO_PADDR(fs, cur_pix) + offsetof(spiffs_page_header, flags),
+              sizeof(flags), &flags);
+          SPIFFS_CHECK_RES(res);
+#endif
+          flags &= ~SPIFFS_PH_FLAG_FINAL;
           res = _spiffs_wr(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT,
               0, SPIFFS_PAGE_TO_PADDR(fs, cur_pix) + offsetof(spiffs_page_header, flags),
-              sizeof(u8_t), (u8_t*)&flags);
+              sizeof(flags), &flags);
         }
       }
     }