TR064

[rudolf32]

Hello

I have used this library last year. This year I tried again, but I get an exception28 that rises, because in the function the variable value has no content, it was not initialized. The stream no gets connected and no content is found.

Are there special versions of IDE and Wifi library I have to use?

Thanks for help.

[saak2820]

@rudolf32

Please try my develp branch and Report Back.

https://github.com/saak2820/Arduino-TR-064-SOAP-Library

[rudolf32]

Hello @saak2820

I just tried it, without success:

⸮scandone
17:03:23.582 -> connected with Federation.Home, channel 6
17:03:23.582 -> dhcp client start...
17:03:24.050 -> ............ip:10.4.0.148,mask:255.255.255.0,gw:10.4.0.1
17:03:30.017 -> .
17:03:30.017 -> WiFi connected IP address: 10.4.0.148
17:03:30.017 -> Web Server started at 17:03:29 / 1707671009
17:03:31.143 -> Initialize TR-064 connection
17:03:31.143 -> TR64 started.
17:03:31.143 -> Waiting for Keypress
17:03:33.530 -> pm open,type:2 0
17:03:40.390 -> TR64-Init at 17:03:39
17:03:40.484 -> [TR064][httpRequest] prepare request to URL: http://x.x.x.x:49000/tr64desc.xml
17:03:40.484 -> [HTTP-Client][sendRequest] type: 'GET' redirCount: 0
17:03:43.293 -> [HTTP-Client] connected to x.x.x.x:49000
17:03:43.293 -> [HTTP-Client] sending request header
17:03:43.293 -> -----
17:03:43.293 -> GET /tr64desc.xml HTTP/1.1
17:03:43.293 -> Host: x.x.x.x:49000
17:03:43.293 -> User-Agent: ESP8266HTTPClient
17:03:43.293 -> Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0
17:03:43.293 -> Connection: keep-alive
17:03:43.293 -> Content-Length: 0
17:03:43.293 ->
17:03:43.293 -> -----
17:03:43.293 -> [HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 200 OK
17:03:43.293 -> [HTTP-Client][handleHeaderResponse] RX: 'Cache-Control: max-age=120
17:03:43.293 -> [HTTP-Client][handleHeaderResponse] RX: 'Connection: Keep-Alive
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Length: 14126
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Type: text/xml
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Date: Sun, 11 Feb 2024 16:03:42 GMT
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'ETag: "2E3D0BBxxxxxxxxxx"
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Expires: Sun, 11 Feb 2024 16:05:42 GMT
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Last-Modified: Thu, 01 Jan 1970 00:01:50 GMT
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Mime-Version: 1.0
17:03:43.338 -> [HTTP-Client][handleHeaderResponse] RX: 'Keep-Alive: timeout=60, max=300
17:03:43.385 -> [HTTP-Client][handleHeaderResponse] RX: '
17:03:43.385 -> [HTTP-Client][handleHeaderResponse] code: 200
17:03:43.385 -> [HTTP-Client][handleHeaderResponse] size: 14126
17:03:43.385 -> [TR064][httpRequest] GET...
17:03:43.385 -> [TR064][httpRequest] Response code: 200
17:03:43.385 -> [TR064][initServiceURLs] get the Stream
17:03:43.385 -> [TR064][initServiceURLs] 0 readServiceName: @ readServiceUrl:
17:03:43.385 -> [TR064][initServiceURLs] 1 readServiceName: @ readServiceUrl:
17:03:43.385 -> [TR064][initServiceURLs] 2 readServiceName: @ readServiceUrl:
...
17:03:43.938 -> [TR064][initServiceURLs] 98 readServiceName: @ readServiceUrl:
17:03:43.938 -> [TR064][initServiceURLs] 99 readServiceName: @ readServiceUrl:
17:03:43.938 -> [TR064][initServiceURLs] 100 readServiceName: ⸮⸮⸮??
-> from here only rubbish from memory

17:03:43.986 -> [String] ' @ readSer ... viceUrl: ': OOM: 31 -> 59424 bytes
17:03:43.986 -> @ readServiceUrl:
17:03:43.986 -> Fatal exception 28(LoadProhibitedCause):
17:03:43.986 -> epc1=0x4000e140, epc2=0x00000000, epc3=0x00000000, excvaddr=0x06010000, depc=0x00000000

my own trace shows, that:

bool TR064::xmlTakeParam(String& value, const String& needParam) {
WiFiClient * stream = &tr064client;
stream->Stream::setTimeout(40);

value = ""; would protect from exception

while(stream->connected()) {  -----------------------> it never enters here and returns value without content or memory

stream->connected() is always false.

...

}
return true;

}

Regrads Rudolf

[saak2820]

Hi @rudolf32

I'll have a look at it tomorrow.
Can you make your call without the init method? Example is in the caller.ino

[rudolf32]

Hello @saak2820

not yet

21:35:06.119 -> Calling **9 to Fritzbox by TR64 at 21:35:05
21:35:06.119 -> [TR064]
21:35:06.119 -> [TR064][action] with parameters
21:35:06.165 -> [TR064][action] with extraction
21:35:06.165 -> [TR064][cleanOldServiceName] searching for prefix in servicename: urn:dslforum-org:service:X_VoIP:1
21:35:06.165 -> [String] '<s:Header> ... /s:Header>': Reallocating large String(161 -> 306 bytes)
21:35:06.165 -> [String] '<?xml vers ... s:Body><u:': Reallocating large String(317 -> 336 bytes)
21:35:06.165 -> [String] '<?xml vers ... xmlns:u="': Reallocating large String(346 -> 371 bytes)
21:35:06.165 -> [TR064][action_raw] with parameter, NewX_AVM-DE_PhoneNumber
21:35:06.165 -> [TR064][action_raw] with parametervalue, **9
21:35:06.165 -> [String] '<?xml vers ... X_VoIP:1">': Reallocating large String(381 -> 435 bytes)
21:35:06.213 -> [String] '<?xml vers ... oneNumber>': Reallocating large String(435 -> 481 bytes)
21:35:06.213 -> [TR064][findServiceURL] Services NOT Loaded.
21:35:06.213 -> [TR064][httpRequest] URL is empty, abort http request.
21:35:06.213 -> [TR064][action] Request Failed
21:35:06.213 -> [HTTP-Client][end] tcp is closed
21:35:06.213 -> [HTTP-Client][end] tcp is closed

[rudolf32]

second try like caller.ino

21:42:27.226 -> [TR064][action] with extraction
21:42:27.226 -> [TR064][cleanOldServiceName] searching for prefix in servicename: X_VoIP:1
21:42:27.226 -> [String] '<s:Header> ... /s:Header>': Reallocating large String(161 -> 306 bytes)
21:42:27.226 -> [String] '<?xml vers ... s:Body><u:': Reallocating large String(317 -> 336 bytes)
21:42:27.226 -> [String] '<?xml vers ... xmlns:u="': Reallocating large String(346 -> 371 bytes)
21:42:27.226 -> [TR064][action_raw] with parameter, NewX_AVM-DE_PhoneNumber
21:42:27.273 -> [TR064][action_raw] with parametervalue, **9
21:42:27.273 -> [String] '<?xml vers ... X_VoIP:1">': Reallocating large String(381 -> 435 bytes)
21:42:27.273 -> [String] '<?xml vers ... oneNumber>': Reallocating large String(435 -> 481 bytes)
21:42:27.273 -> [TR064][findServiceURL] Services NOT Loaded.
21:42:27.273 -> [TR064][httpRequest] URL is empty, abort http request.
21:42:27.273 -> [TR064][action] Request Failed
21:42:27.273 -> [HTTP-Client][end] tcp is closed

[rudolf32]

Unfortunately it is looking for the Service String in a not filled array.

THX Rudolf

[rudolf32]

Hello
any new libs to test?
If someone is using this lib and it works, please let me know the version of environment to upload and compile.
THX

[saak2820]

HI Rudolf

in your log there is a line with the following content.

[TR064][initServiceURLs] get the Stream

this comes from the original code.

With the code from the develop branch, the following should be there.

[TR064][initServiceURLs] Searching the XML for serviceType

Please try the sources from my develp branch.
[https://github.com/saak2820/Arduino-TR-064-SOAP-Library]

to use this library, put the files tr064.cpp and tr064.h in your library folder of your Arduino installation.

....\Arduino\libraries\Arduino-TR-064-SOAP-Library\src

[GvDatGithub]

I had the same problem. Crash after tr064_connection.init();
Solution: IDE 1.8.18 and board ( ESP8266 boards ) must not be younger ( newer / more recent ) than version 2.7.4.

[Ted1405]

Okay, this has caused me some headaches. The version of saak2820 is now running on my system, but please don't ask me which exact version (the file date of tr064.cpp is September 26th, 2023, 11:29 AM).

However, I had to make a correction because I kept getting the "stream not available" message during initialization.

The reason for this is in the method:

bool TR064::xmlTakeParamStream(String& value, const String& needParam)

in the check:

if(!stream->available()) {

Since there is no delay beforehand and available() does not block, this check always returned false for me, and as a result, the error was immediately thrown. The delay() calls in the subsequent loop were never reached, even though they would fix the problem.

I have now changed this line to:

if(!stream) {

This ensures that there is a stream available in the first place. The following loop then waits up to 10 times for 50ms for a response... and then it comes.

[Aypac]

Hey @Ted1405 @rudolf32 @saak2820 ,
I'll try to find the time to fix this properly and update the master branch of this - but that won't happen for at least a few weeks. If you have a working version, please feel free to submit a pull request. If it passes the automated tests I'll merge it (note: the automated tests might have to be adjusted).

Background is this: unfortunately things seem to keep breaking (my guess is breaking changes to underlying libraries and IDE versions). Plus it seems near impossible to test all possible board versions (because the same libraries for different boards can have minutely different behavior, as also witnessed by some if-constructs based on board version). I'm a bit frustrated by the complexity that comes from that. Maybe I'll limit the support to a single board, library, and Arduino IDE version.

And thank you @saak2820 again for helping! Really appreciated!

[saak2820]

Hi Ted,

Unfortunately, there is too little information about your error before your change to make an accurate statement.

the available() method should check exactly what you are checking with your change. Can you try again without the change and let us know the result. The version from September 26th, 2023 is the current develop branch

https://github.com/saak2820/Arduino-TR-064-SOAP-Library/tree/develop

int WiFiClient::available() { if (!_client) return 0; ...

[Ted1405]

Okay ... let's break it down ...

Software used: Arduino IDE 2.3.4 all libraries up to date, Windows 10 (64 Bit)
Target hardware: D1 mini Pro (or clone).

My relating code components:

TR064             connection(49000, "192.168.145.1", "Ted", "********");  //a FritzBox 7590AX with 12 smart home devices

void setup() {
   // after initing wifi connection and other stuff
  connection.init();
}

float FBGetTemperature(const char *FbApiAIN) {
  // Offset ist im Temperaturwert enthalten
  float iTemperature = -100;
  String iParams[][2] = {{"NewAIN", FbApiAIN}};
  String iReq[][2] = {{"NewTemperatureIsEnabled", ""}, {"NewTemperatureIsValid", ""}, {"NewTemperatureCelsius", ""}};
  if (connection.action("X_AVM-DE_Homeauto:1", "GetSpecificDeviceInfos", iParams, 1, iReq, 3));
  {
    if (iReq[0][1] == "ENABLED" && iReq[1][1] == "VALID")
    {
      iTemperature = (float)((iReq[2][1]).toInt()) / 10.0;
    }
    Serial.printf("- NewTemperatureIsEnabled: %s\n", iReq[0][1]);
    Serial.printf("- NewTemperatureIsValid:   %s\n", iReq[1][1]);
    Serial.printf("- NewTemperatureCelsius:   %s\n", iReq[2][1]);
  }

With TR064 / 1.2.2 from @Aypac, the following log appears and results in Exception (28), as previously reported by other users:

Build: Thu Feb 06 20:32:27 2025		IDE: 1.6.7
SDK:2.2.2-dev(38a443e)/Core:3.1.2=30102000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

[TR064][httpRequest] prepare request to URL: http://192.168.145.1:49000/tr64desc.xml
[TR064][httpRequest] GET...
[TR064][httpRequest] Response code: 200
[TR064][initServiceURLs] get the Stream 
[TR064][initServiceURLs] 0	readServiceName:  @ readServiceUrl: 
...
[TR064][initServiceURLs] 99	readServiceName:  @ readServiceUrl: 
[TR064][initServiceURLs] 100	readServiceName:  @ readServiceUrl: ��"@P��"@,K�?�"@��
[TR064][initServiceURLs] 101	readServiceName:  @ readServiceUrl: 
[TR064][initServiceURLs] 102	readServiceName:  @ readServiceUrl: 
[TR064][initServiceURLs] 103	readServiceName:  @ readServiceUrl: 
[TR064][initServiceURLs] 104	readServiceName:  @ readServiceUrl: |D�?�I�?h� @���?��"@��"@
[TR064][initServiceURLs] 105	readServiceName: �I�?h� @���?��"@��"@� @ readServiceUrl: 
...
[TR064][initServiceURLs] 269	readServiceName: lJ�?�F�? @ readServiceUrl: �
[TR064][initServiceURLs] 270	readServiceName:  @ readServiceUrl: 

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (28):
epc1=0x4000e140 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00069780 depc=0x00000000

The first reason for this issue is the limitation to 100 elements in tr064.h: String _services[100][2];
The second reason is a missing break condition in tr064.cpp when the array is overrun:

void TR064::initServiceURLs() {
    /* TODO: We should give access to this data for users to inspect the
     * possibilities of their device(s) - see #9 on Github.
     */
// ...
            while(1) {
            }

If I replace the while(1) with an while(i < 100), the crash no longer occurs. However, the service array remains empty, and subsequent requests will trigger another read attempt. The temperature request fails anyway.

[TR064][httpRequest] prepare request to URL: http://192.168.145.1:49000/tr64desc.xml
[TR064][httpRequest] GET...
[TR064][httpRequest] Response code: 200
[TR064][initServiceURLs] get the Stream 
[TR064][initServiceURLs] 0	readServiceName:  @ readServiceUrl: 
...
[TR064][initServiceURLs] 99	readServiceName:  @ readServiceUrl: 
[TR064][action] with extraction
[TR064][cleanOldServiceName] searching for prefix in servicename: X_AVM-DE_Homeauto:1
[TR064][action_raw] with parameter, NewAIN
[TR064][action_raw] with parametervalue, 
[TR064][findServiceURL] searching for service: urn:dslforum-org:service:X_AVM-DE_Homeauto:1
[TR064][httpRequest] URL is empty, abort http request.
[TR064][action]<error> Request Failed 
- NewTemperatureIsEnabled: 
- NewTemperatureIsValid:   
- NewTemperatureCelsius:   

So I asume that the error occures in the method bool TR064::xmlTakeParam(String& value, const String& needParam).
@rudolf32 mentioned:
while(stream->connected()) { -----------------------> it never enters here and returns value without content or memory
I can confirm this. For testing purposes, I added a 100ms delay before the while loop and also replaced stream->connected() with 1 to force iteration. However, this did not noticeably change the behavior.

---

@saak2820:

This is the log of develop branch from yesterday (09/2023):

- NewTemperatureIsEnabled: 
- NewTemperatureIsValid:   
- NewTemperatureCelsius:   
[TR064][initServiceURLs] getting the service detect page
[TR064][httpRequest] prepare request to URL: http://192.168.145.1:49000/tr64desc.xml
[TR064][httpRequest] wifiClient->Stream::getTimeout: 5000

[TR064][httpRequest] GET with SOAPACTION: ''
[TR064][httpRequest] HTTP response code: 200
[TR064][initServiceURLs] Searching the XML for serviceType
[TR064][xmlTakeParamStream] stream not available
[TR064][initServiceURLs] No services found. Try to use the opposite useStreamingForXML flag.
[TR064][initServiceURLs]<error> initServiceUrls failed.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[TR064][action]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[TR064][action] with extraction
[TR064][cleanOldServiceName] searching for prefix in servicename: X_AVM-DE_Homeauto:1
[TR064][action_raw] Parameter: NewAIN => 
[TR064][action_raw]<error> You need to init the library or specify a serviceURL!
[TR064][action]<error> Request failed 
- NewTemperatureIsEnabled: 
- NewTemperatureIsValid:   
- NewTemperatureCelsius:

The initialization process is repeated because the service table is never retrieved.
The key message is "[TR064][xmlTakeParamStream] stream not available" from
bool TR064::xmlTakeParamStream(String& value, const String& needParam)

I added a delay(50); before if(!stream->available), and as a result, the service data is successfully read:

...
[TR064][initServiceURLs] Found 37 services in total.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[TR064][action]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

And the temperature requests return valid values.

Finally I tested the develop branch from today (2025-02-06 / 15:04 CET). This works out of the box. 👍
Thanks @saak2820.