Missing Misc.h reference inside vfatfs_api.h

[sanyaade-iot]

Hi Adam5Wu,

First many thanks for putting ESPVFATS on github. I have been looking for thing like so it will be a blessing from you to me and the community.

I have hit a block as I could not find the include file Misc.h reference inside vfatfs.h so Arduino IDE stop compiling. I have google for it but no success.

Please help. Many thanks!

God blesses!!!

Best regards,
Sanyaade

[per1234]

From README.md:

• Requires:
  - ESP8266 Arduino Core Fork
  - ZWUtils-Arduino

You forgot to install the ZWUtils-Arduino library.

[sanyaade-iot]

@per1234 :

I did the two above but I think it is the directory layout that is causing the problems. I had to manually adjust some of the include directives but now I am getting the following errors:

`Arduino: 1.8.5 (Linux), Board: "NodeMCU 1.0 (ESP-12E Module), 80 MHz, 4M (1M SPIFFS), v2 Prebuilt (gcc), Disabled, None, 115200"
"-I/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src" "/tmp/arduino_build_410293/sketch/FATFS_Init.ino.cpp" -o "/tmp/arduino_build_410293/sketch/FATFS_Init.ino.cpp.o"
In file included from /tmp/arduino_modified_sketch_297029/FATFS_Init.ino:10:0:
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:145:7: error: 'bool VFATFSImpl::info(fs::FSInfo&) const' marked override, but does not override
bool info(FSInfo& info) const override;

   ^

/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:150:7: error: 'bool VFATFSImpl::exists(const char*) const' marked override, but does not override
bool exists(const char* path) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:151:7: error: 'bool VFATFSImpl::isDir(const char*) const' marked override, but does not override
bool isDir(const char* path) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:152:9: error: 'size_t VFATFSImpl::size(const char*) const' marked override, but does not override
size_t size(const char* path) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:153:9: error: 'time_t VFATFSImpl::mtime(const char*) const' marked override, but does not override
time_t mtime(const char* path) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:155:14: error: 'fs::FileImplPtr VFATFSImpl::openFile(const char*, fs::OpenMode, fs::AccessMode)' marked override, but does not override
FileImplPtr openFile(const char* path, OpenMode openMode,
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:157:13: error: 'fs::DirImplPtr VFATFSImpl::openDir(const char*, bool)' marked override, but does not override
DirImplPtr openDir(const char* path, bool create) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:186:7: error: 'bool VFATFSFileImpl::truncate()' marked override, but does not override
bool truncate() override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:202:9: error: 'time_t VFATFSFileImpl::mtime() const' marked override, but does not override
time_t mtime() const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:203:7: error: 'bool VFATFSFileImpl::remove()' marked override, but does not override
bool remove() override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:204:7: error: 'bool VFATFSFileImpl::rename(const char*)' marked override, but does not override
bool rename(const char nameTo) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:227:14: error: 'fs::FileImplPtr VFATFSDirImpl::openFile(const char, fs::OpenMode, fs::AccessMode)' marked override, but does not override
FileImplPtr openFile(const char name, OpenMode openMode,
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:229:13: error: 'fs::DirImplPtr VFATFSDirImpl::openDir(const char, bool)' marked override, but does not override
DirImplPtr openDir(const char name, bool create) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:230:7: error: 'bool VFATFSDirImpl::exists(const char) const' marked override, but does not override
bool exists(const char name) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:231:7: error: 'bool VFATFSDirImpl::isDir(const char) const' marked override, but does not override
bool isDir(const char* name) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:232:9: error: 'size_t VFATFSDirImpl::size(const char*) const' marked override, but does not override
size_t size(const char* name) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:233:9: error: 'time_t VFATFSDirImpl::mtime(const char*) const' marked override, but does not override
time_t mtime(const char* name) const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:234:7: error: 'bool VFATFSDirImpl::remove(const char*)' marked override, but does not override
bool remove(const char name) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:235:7: error: 'bool VFATFSDirImpl::rename(const char, const char*)' marked override, but does not override
bool rename(const char* nameFrom, const char* nameTo) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:237:14: error: 'const char* VFATFSDirImpl::entryName() const' marked override, but does not override
const char* entryName() const override {
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:241:9: error: 'size_t VFATFSDirImpl::entrySize() const' marked override, but does not override
size_t entrySize() const override {
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:245:9: error: 'time_t VFATFSDirImpl::entryMtime() const' marked override, but does not override
time_t entryMtime() const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:246:7: error: 'bool VFATFSDirImpl::isEntryDir() const' marked override, but does not override
bool isEntryDir() const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:247:7: error: 'bool VFATFSDirImpl::next(bool)' marked override, but does not override
bool next(bool reset) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:249:14: error: 'fs::FileImplPtr VFATFSDirImpl::openEntryFile(fs::OpenMode, fs::AccessMode)' marked override, but does not override
FileImplPtr openEntryFile(OpenMode openMode,
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:251:13: error: 'fs::DirImplPtr VFATFSDirImpl::openEntryDir()' marked override, but does not override
DirImplPtr openEntryDir() override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:252:7: error: 'bool VFATFSDirImpl::removeEntry()' marked override, but does not override
bool removeEntry() override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:253:7: error: 'bool VFATFSDirImpl::renameEntry(const char*)' marked override, but does not override
bool renameEntry(const char* nameTo) override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:255:9: error: 'time_t VFATFSDirImpl::mtime() const' marked override, but does not override
time_t mtime() const override;
^
/home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS/src/vfatfs_api.h:257:14: error: 'const char* VFATFSDirImpl::name() const' marked override, but does not override
const char* name() const override {
^
Using library ESP8266WiFi at version 1.0 in folder: /home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/libraries/ESP8266WiFi
Using library ESPVFATFS at version 1.0.0 in folder: /home/sanyaade/Compilers/arduino-1.8.5/libraries/ESPVFATFS
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).`

So I am try to look into this as well.

I know your machine directory layout will be different from mine but how to position the 3 directories would save time. Many thanks for your help.

God blesses!!!

Best regards,
Sanyaade

[Adam5Wu]

Hi @sanyaade-iot, to use this library, you will also need to use my fork of Arduino, instead of the official one. The reason is that, stock esp8266 Arduino's FS API interface is too limited, and does not allow user code to access many of the FAT features, such as file modification time, or truncating a file.

My fork (the adam5wu/master branch) extended the stock FS to expose these functions. I try to keep it relatively up-to-date towards the head of the official repo. And I am actively using my fork for all my projects, so any bug I will try to fix ASAP.

[sanyaade-iot]

Hi @Adam5Wu :

Thanks for your support, much appreciated!
I have just navigated into my Arduino IDE/hardware folder
I have just clone: git clone -b adam5wu/master https://github.com/Adam5Wu/Arduino-esp8266.git
cd into it and then into tool and run python get.py. Next restart restart Arduino IDE.

Also I have downloaded into Arduino/libraries:-
1.) Arduino-esp8266-adam5wu
2.) ZWUtils-Arduino
and lastly
3.) ESPVFATFS

I think directory layout is what I have been trying to figure out. Also working out the virtual functions constant override.
Kindly let me know if my above layout of directories/folders are ok. Many thanks!
I am giving a shot at it again as I will like to it working.

Thank you for your support

God blesses!!!

Best regards,
Sanyaade

[sanyaade-iot]

Hi @Adam5Wu :

Going well so far but now I have hit linker problems for undefined VFATFS. Please see below:

"/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar" cru "/tmp/arduino_build_506448/arduino.ar" "/tmp/arduino_build_506448/core/spiffs_hal.cpp.o"
Archiving built core (caching) in: /tmp/arduino_cache_302173/core/core_esp8266com_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None____,FlashErase_none,UploadSpeed_115200_e391011456f0ff44b76356561b53f34e.a

Linking everything together...
"/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc" -CC -E -P -DVTABLES_IN_FLASH "/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/sdk/ld/eagle.app.v6.common.ld.h" -o "/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/sdk/ld/eagle.app.v6.common.ld"
"/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc" -Wl,-Map "-Wl,/tmp/arduino_build_506448/FATFS_Init.ino.map" -g -w -Os -nostdlib -Wl,--no-check-sections -u call_user_start -u _printf_float -u _scanf_float -Wl,-static "-L/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/sdk/lib" "-L/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/sdk/ld" "-L/home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/lib" "-Teagle.flash.4m1m.ld" -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,spi_flash_read -o "/tmp/arduino_build_506448/FATFS_Init.ino.elf" -Wl,--start-group "/tmp/arduino_build_506448/sketch/FATFS_Init.ino.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFi.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFiAP.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFiGeneric.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFiMulti.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFiSTA.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/ESP8266WiFiScan.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/WiFiClient.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/WiFiClientSecure.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/WiFiServer.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/WiFiServerSecure.cpp.o" "/tmp/arduino_build_506448/libraries/ESP8266WiFi/WiFiUdp.cpp.o" "/tmp/arduino_build_506448/libraries/ZWUtils-Arduino/Misc.cpp.o" "/tmp/arduino_build_506448/libraries/ZWUtils-Arduino/Units.cpp.o" "/tmp/arduino_build_506448/arduino.ar" -lhal -lphy -lpp -lnet80211 -llwip2 -lwpa -lcrypto -lmain -lwps -laxtls -lbearssl -lespnow -lsmartconfig -lairkiss -lwpa2 -lstdc++ -lm -lc -lgcc -Wl,--end-group "-L/tmp/arduino_build_506448"
/tmp/arduino_build_506448/sketch/FATFS_Init.ino.cpp.o:(.text.setup+0x40): undefined reference to `VFATFS'
collect2: error: ld returned 1 exit status
Using library ESP8266WiFi at version 1.0 in folder: /home/sanyaade/Compilers/arduino-1.8.5/hardware/esp8266com/esp8266/libraries/ESP8266WiFi
Using library ZWUtils-Arduino at version 1.0.0 in folder: /home/sanyaade/Compilers/arduino-1.8.5/libraries/ZWUtils-Arduino
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).

Kindly give me tips or hints on resolve this issue. Many thanks!
I think if this can be solved then the package will compile.

Hear from you soon!

God blesses!!!

Best regards,
Sanyaade

[Adam5Wu]

ESPVFATFS/src/vfatfs_api.h

Lines 269 to 271 in aa40756

	#if !defined(NO_GLOBAL_INSTANCES) && !defined(NO_GLOBAL_VFATFS)
	extern fs::FS VFATFS;
	#endif

Do you have NO_GLOBAL_INSTANCES defined somewhere?

[sanyaade-iot]

@Adam5Wu :

Good News: I now got it compiled and was able to flash into my ESP8266Ex (i.e ESP-12 4MB) module.
For the benefit of other users of this package, I outline below steps to take:

1. Download and install the latest Arduino IDE for your operating systems platform at: https://www.arduino.cc/en/Main/Software . If you already have the IDE installed then you can skip this step.

2. Next open up your terminal/command prompt, navigate to the installation folder/directory, then go inside the hardware sub-folder/sub-directory. For example mine is: Compilers/arduino-1.8.5/hardware/ on my Ubuntu 16.04 Linux box.

3. Now create a folder named: esp8266com, change into this new created folder (cd esp8266com)

4. Next clone the git repository: Issue: git clone -b adam5wu/master https://github.com/Adam5Wu/Arduino-esp8266.git esp8266

5. Now cd esp8266/tools

6. Run the command: python get.py
   This will download some files including the toolchains need for the Esp8266.

7. Next go into your Arduino/libraries folder: once again, mine is: Compilers/arduino-1.8.5/libraries but your may be Documents\arduino\libraries or whatever. Now clone or download the followings:
   7a: git clone https://github.com/Adam5Wu/ESPVFATFS.git
   7b: https://github.com/Adam5Wu/ZWUtils-Arduino.git

8: Next restart your Arduino IDE and you should be ready to go. Click on the file menu and select examples\ESPVFATFS\FATFS_Init , open the script inside Arduino IDE, compile it flash your ESP8266 module. NOTE: You may need to select your board type and also set the port from Tool\Board and Tools\Port

Lastly: if your Esp8266 module has CH340 then you may need to chage the baudrate to (19200 or 921600 ) CH340 unlike CP2102 does not support 115200. Google this for other baudrate.

It will be nice if this step can be added to the readme/instruction page of ESPVFATFS to aid newcomers like. ESPVFATFS is the only I found on the internet that implements Fat file-system for ESP8266. I can now study it further as I need it for overcoming the limitations of ESP8266. Also I am porting PH7 and other interpreters to ESP8266 so a fat system will give a high advantage over that of SPIFFS. We have a bunch of over 80 to 100 esp8266 in stock for our youth training in our community.
Many thanks to Adam5Wu for all your efforts on VFATFS and other packages on your github.

Thank you!

God blesses!!!

Best regards,
Sanyaade

[Adam5Wu]

Great suggestions @sanyaade-iot
I've been sloppy on docs as there were too many projects and features I work on, will gradually put more efforts on it.

A quick hint hopefully can speed up your progress:

• To get a quick example of how to use the new APIs with FAT FS features, you can look at: https://github.com/Adam5Wu/ESPAsyncWebServer/blob/1bcd031e60c5235d399b27930e85ec4db3b36d24/src/WebHandlers.cpp#L375-L420
  It is part of Web handler which generates directory list, and contains code that uses the relative path operations and reads file modification time.

• Multi-partition support is one of the new features I added recently, which may be quite handy in Edu scenario (e.g. you can keep a golden copy of the file system in 2nd partition, and in case the working one is destroyed, restoration is quick and painless) To use this feature:

  • You need at least 1MB flash space for fs
  • Before call VFATFS.begin() Call VFATPartitions::config(50, 50);
  • This will create 2 partitions, each taking 50% of fs space
  • The default VFATFS object will only work the first partition (index 0)
  • To work on the second one, create a new VFATFSImpl and pass partition index 1

• Finally, I am actively working on project ESPZWAppliance, which takes away lots of pain of the basic and boring development of ESP, for example, how to on-the-fly reconfigure WiFi AP, how to maintain WiFi connection and retry if AP rebooted, how to sync clock and configure time zone, daylight saving, etc.
  With this library, an almost empty sketch can do all these. So that developers just need to focus on their core app logic.
  Maybe it is of interest to you. :)

[sanyaade-iot]

Hi @Adam5Wu :

Thanks for getting back and for the tips and hints on VFATFS API plus your other on-going projects.
Yes! I will be interested in the ESPZWAppliance and any in other projects. The best way to learn is to be involved, participate and contribute towards those projects. I am very well interested.

I will look at your other projects associated with VFATFS including those that you have pointed out. Look deeply at VFATFS API including other links.

BTW: As you are customising esp8266-Arduino, are planning a pull request for it or they are not accepting your modifications/enhancements. I believe your improvements will be of benefits to large users beyond esp8266 including but not limited to other Arduino community like stm32-Arduino, Texas Instruments, PIC and more....
I can also blog and write report on them on hackster.io see my page: https://www.hackster.io/sanyaade1/

Please put me down for your project. I am interested 👍

God blesses!!!

Best regards,
Sanyaade