Memory optimization

[tablatronix]

optimize memory usage, namely RAM

• store strings in PROGMEM
• move/declare strings, remove duplicates
• minimize use of Strings
• profile memory for leaks
• optimize html outputting, is it possible to flush to reduce page strings size, does it matter ?

[tablatronix]

Made sure all debug and strings were wrapped in F(), eventually all debug strings will be moved to strings.h with precompiler flags to remove them entirely.

ESP DEBUGGING IS ON

development

BEFORE
Checking program size
text	   data	    bss	    dec	    hex	filename
341992	   9648	  30192	 381832	  5d388	.pioenvs\nodemcuv2_dev\firmware.elf
Uploading 355792 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000

freeheap 37760

data  : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea5b4, len: 8324
bss   : 0x3ffea5b8 ~ 0x3fff1ba8, len: 30192
heap  : 0x3fff1ba8 ~ 0x3fffc000, len: 42072

AFTER

Checking program size
text	   data	    bss	    dec	    hex	filename
345448	   8776	  30200	 384424	  5dda8	.pioenvs\nodemcuv2_dev\firmware.elf
Uploading 358368 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000
+2,576

data  : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea24c, len: 7452 -872
bss   : 0x3ffea250 ~ 0x3fff1848, len: 30200 +8
heap  : 0x3fff1848 ~ 0x3fffc000, len: 42936 +864

free heap 38624 +864

[tablatronix]

data : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea23c, len: 7436
bss : 0x3ffea240 ~ 0x3fff1838, len: 30200
heap : 0x3fff1838 ~ 0x3fffc000, len: 42952
*WM: getCoreVersion(): 00000000
*WM: system_get_sdk_version(): 2.1.0(deb1901)
*WM: system_get_boot_version(): 31
*WM: getFreeHeap(): 38640

[horendus]

Fantastic :)

[tablatronix]

Investigate fragmentation, suspect is page html and params replace strings, user params

[nikeee]

Using the development branch on my ESP32 on Platformio:
With WiFiManager:

RAM:   [===       ]  25.7% (used 84296 bytes from 327680 bytes)
Flash: [======    ]  60.7% (used 1113774 bytes from 1835008 bytes)

And without WiFiManager:

RAM:   [==        ]  19.9% (used 65348 bytes from 327680 bytes)
Flash: [====      ]  39.7% (used 728388 bytes from 1835008 bytes)

(compiled in release mode with optimizations)

Which is 385386 (~385kB) difference in flash.
I remember that the master branch was more lightweight when I used it for an ESP8266.

Is there anything I can do on my side to reduce memory usage?

[tablatronix]

hmm, well all debugging is left in, they are not conditional, so removing all debug statements might help. There might be some cruft in dev since its still alpha that can be removed.. also there is a lot of info page stuff that I intend to make optional somehow, compiler directives for now probably.

I would suspect the css additions.
Remove the images, and go back to percentage tokens, no graph

I can play with it, but it might be a while before I have time atm.

Might want to create an issue just for this optimizations so we can select them out easily.
for now defines work fine and we can branch or pio config low mem options.

There are also some very useful wifi helpers, that could be removed, like textual status() strings etc
all the help page docs..

[nikeee]

That seems like a good idea.

What I was also thinking of is offering the user the option to supply his own HTML and CSS (not just the ability to append CSS, but all if the HTML/CSS). For my current use-case that would be easier than overwriting the existing CSS while being more compact.

[tablatronix]

There IS a define check for the strings, so you can do that at compiler level if you wanted to and replace them elsewhere, problem is its still flash, in order to switch to littlefs/spiffs strings it would require code changes.

[nikeee]

Do you mean this define: https://github.com/tzapu/WiFiManager/blob/development/strings_en.h#L16?

Despite the comment, does defining an own set of strings work by something like this?

#define WIFI_MANAGER_OVERRIDE_STRINGS
const char HTTP_HEAD_START[] PROGMEM = // ...
// ...

#include <WiFiManager.h>

I feel like I'd have more customization options if this function would be virtual, so the WiFiManager could be inherited and this function could be implemented by the library user:

WiFiManager/WiFiManager.cpp

Lines 945 to 963 in 31be25d

	String WiFiManager::getHTTPHead(String title){
	String page;
	page += FPSTR(HTTP_HEAD_START);
	page.replace(FPSTR(T_v), title);
	page += FPSTR(HTTP_SCRIPT);
	page += FPSTR(HTTP_STYLE);
	page += _customHeadElement;
	if(_bodyClass != ""){
	String p = FPSTR(HTTP_HEAD_END);
	p.replace(FPSTR(T_c), _bodyClass); // add class str
	page += p;
	}
	else {
	page += FPSTR(HTTP_HEAD_END);
	}
	return page;
	}

This might not solve the RAM/Flash issue, though.

[tablatronix]

Yeah, this was the first step to abstract and organize the code so it can be refactored better. That func was one of the latest additions with that in mind.

[dalbert2]

For ESP8266 users, IRAM is a critical resource; the SSL library requires a lot of free heap and anything that reduces it is a problem. The WiFiManager is amazing, but it consumes roughly 2.3K of IRAM even when not instantiated.

Are there ways to reduce WiFiManager IRAM usage when not using (or even instantiating) the WiFiManager?

[tablatronix]

I am assuming this is all the strings, specifically the html, try removing the HTTP_STYLE css like icons and stuff and see what you find, i made a minimal version to test but it was a long time ago

https://github.com/tzapu/WiFiManager/blob/master/wm_strings_en.h#L90

[dalbert2]

I submitted a separate issue #1587 addressing at least two problems:

1. string arrays in wm_consts_en.h aren't properly placed in PROGMEM
   I provided examples of how to do it properly and can submit a fix if desired, but the more serious issue is:
2. all of the strings are declared in header files (which are themselves nested)
   so any code that includes WiFiManager.h also creates instances of those strings.
   Why aren't those strings declared in a compilation unit and declared extern in header files (if needed)?

[tablatronix]

.2 but there are include guards..?