Not Properly Receiving Large Payload.

[kdschlosser]

sorry about the book. I am just trying to provide as much information as possible.

I am trying to send Pronto IR HEX to my Arduino Mega 2560. As we know the code is huge. as true hex the code i am transmitting looks like this if stored on the arduino.

const uint16_t prontoCode[78] = {
0x00,0x6D,0x22,0x03,0xA9,0xA8,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15,
0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x3F,0x15,0x3F,
0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,
0x15,0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,
0x15,0x15,0x15,0x40,0x15,0x15,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15,
0x3F,0x15,0x3F,0x15,0x3F,0x15,0x702,0xA9,0xA8,0x15,0x15,0x15,0xE6E};

as a non hex form same code is as follows again if stored on the arduino.
const char prontoCode[392] = {
"0000 006D 0022 0003 00A9 00A8 0015 003F 0015 003F 0015 003F 0015
0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 003F 0015 003F
0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015
0015 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015
0015 0015 0015 0040 0015 0015 0015 003F 0015 003F 0015 003F 0015
003F 0015 003F 0015 003F 0015 00702 00A9 00A8 0015 0015 0015 00E6E"};

i believe it's 392 for that char. but at any rate o am setting the buffer size to 512, now I am only using 2.2K of the memory on the Arduino, I have not done a Heap Check. but i am sure there is still no problem. I am also using the TCP Events Plugin for EG I could never get this library to work at all with the standard Network Sender/Receiver. Besides having MD5 errors more often than I would like and now this problem. Other than that, works great. Fast too. So I do thank you for making this Library. ok So back to whats happening, Have a Prefix and Suffix Being sent along with the payload of the pronto code. it receives the payload as an event, and no matter what i set the buffers to it receives no more then 21 of the 392. even if i set both the event and payload buffer to 512 it returns true, And at compile time it makes no difference to the memory size, So I am not sure how this is allocated. I would post code, but in order to be relevent I would have to drop the whole thing, about 45K worth. to much to post. But to give an idea. my includes are as follows for the known libraries. MD5 SPI IRLIB Timer EthernetV2_0 EtherEvent and the utility\w5200 along with some I have created

I am using IRLIB for receiving only and using a modified version of the ProntoIR Library by probonopd its on git Hub. also a simple PIR library for shutting down TV ect... also have a Limited ZWave (MiCasaVerde Vera3) Library for controlling lights and such, as well as simple thermostat for controlling fireplace. and a control system for OSD generated by eventghost, and Serial Control of my Harman Kardon AVR. Now I can store this Pronto Code and 3 others like it in char format and have not a problem running it. But there are some things I would like to add onto and want to free up the memory.

Strange why it comes through as an event tho. and If i send a small payload say 10 digits with the large buffer set. the event comes through just fine then but no payload, if I shrink the payload to say 100 works fine. so I am not sure as to what the exact limit is. and if the problem is being caused by TCP Events or with the Library. But I am thinking the Library only because of the changing the buffer it starts to work. I have also tried this without any of the things I mentioned before just plain jane so to speak, and same problem. just the ethernet and the etherevent and MD5 on a different Mega . But if this is able to get those pronto codes sent from elsewhere. woah what a great utility, as there is no library that is already made that works with my samsung TV codes. Only pronto does. and trying to read the code from flash storage is fickle. hit or miss on working so having it sent would be best. works over serial.

Thanks again.
Kevin

[kdschlosser]

Addition, just set a 400 buffer size. it set properly. and can receive a 10 digit, but once i send the pronto code. does the same thing. maybe it's having a problem with the spaces. not sure.

here is a serial output of whats happening

_sent event and Payload from Arduino to EG_ Theatre.Samsng.Power : 1 2

_Received Event from EG_ SamsngPwr

_received payload from EG_ 123456789000

---

_sent event and Payload from Arduino to EG_ Theatre.Samsng.Power : 1 2

_received event from EG_ 0022 00AC 00

same process, just changed the payload data coming from EG is all.
and as you can see there is no received payload, and the event is actually a portion of the payload data. now that is with a 12 for event buffer and a 400 for a payload, on both.

if i increase the event buffer to say 400 when the pronto code comes through as the event, only 21 of the char's print out. and I have it checking for a total of 8 seconds for events after sending to EG, and only the one event shows up. so it's not a buffering to the Ethernet Shield problem. or it could be a limit to the buffer size on the Shield and it just drops the rest of the packets, either way that's pretty useless. it's a seedstudio w5200.

[kdschlosser]

nope not a memory problem either.

5403

Theatre.Samsng.Power

1

Theatre.Samsng.Power : 1

5573

4767

2200AC00AB001500410015

Theatre.Samsng.Power : 1 2

5573 <---- Free Space between HEAP and STACK at sending event to EG

4767 <---- Free Space between HEAP and STACK at receiving pronto Code
this is with a 400 event and a 400 payload
2200AC00AB001500410015 <------- and this is what is received as an event and should be
payload, odd thing is it's not the begining of the payload either, it's a bit in. i removed spaces to see if that was the problem, and it's not, but there are alot of 0's in the begining of the code, and this is exactly after all the 0's

EDIT** Not the 0's either, it's starting at the 15th place into the code, consistently.

[per1234]

Thanks for letting me know of this issue. I can reproduce the problem with any payload longer than 14 characters. I'm working on finding the bug now and will let you know as soon as it's fixed.

[per1234]

I believe I have solved the issue. I have successfully received 400 character payloads now. The updated library is available for download here:
https://github.com/per1234/EtherEvent/archive/master.zip

The problem had 2 parts.

• Bug in calculation of availableEventSubmessageLengthMax caused problems with payloads longer than 14 characters. Fixed in ec12023
• Library was never designed to allow payloads longer than 254 characters. I have modified the library to allow a theoretical payload length of 65534 characters in 1b42470

Please let me know if this makes EtherEvent work for your application.

I am also willing to look into the problem with using the EG Network Event Sender/Receiver plugin if you want to give me more information on your issue(I have just tested with the example sketch and EG example tree and it works for me).

MD5 errors can be caused by too small of timeout value if you are still having this problem try setting a very conservative value by adding EtherEvent.setTimeout(2000) to your setup() to see if that is the cause.

[kdschlosser]

Dude your the man. Thanks for the quick response and fix. I have not tried it yet. In the middle of putting a rather large APC UPS in my home. Hopefully later tonight. Not all to concerned about the standard EG receiver. Just thought I should mention. Idk why is wasn't working. Got not response at all from it. And I didn't do to much digging. Oh one other thing. I really don't have a need for Password protecting my EG. And if I leave the password blank. Get MD5 errors. Blank being "" and if so is there a way to not have to load the MD5 library if I am able to not use a password? Its just more of something I don't Need. I am sure it would be easy to impliment with define seeing if MD5 is loaded or not. Just asking. I could do it. Would take me longer as I am not familiar with the code. And would it be ok if I put something together with the pronto library to make an EG Pronto transmitter/receiver? Something to distribute. All in one thing. It's a neat idea.

[per1234]

I really don't have a need for Password protecting my EG.

An unauthenticated option is high on my to do list for this library. I also don't need authentication and the MD5 library takes a lot of memory and slows event sending/receiving a lot but is necessary to be compatible with the stock Network Event Sender/Receiver and TCPEvents EG plugins. Even with a blank password the EG plugin requires the MD5 authentication so they need to be modified also. I have been focusing more on the hardware aspects of my home automation system lately but now that I know there's at least one other person who would find the option useful I'll try to put it at a higher priority. If you do end up adding the feature please consider making pull requests to EtherEvent and TCPEvents.

would it be ok if I put something together with the pronto library

You are welcome to use my code in any way you want. I put an MIT license on it but really I'm just happy to share my work with the world and don't care how it's used. If you run across any issues or improvements I would appreciate you letting me know as I also use this library in my system.

[kdschlosser]

Sweet, Arduinos weren't really made to decode or encode for that matter. lol. K.I.S.S is a good motto. I personally don't know who would have EG open to a public IP. Set up another EG as a repeater if you are going to do that. But then again, I live in Colorado in the mountains. and closest neighbor is kinda far away. So if someone is trying to "Hack me" I am more concerned about the what for then the how. lol.
I don't live near any kiddos playing hacker either. I am just finishing up the Hardware side of my Home automation project. What i do want to set up is location awareness for inside my home. I set up GeoFencing with EG and the webserver plugin. it works so so. I wish i can find a geofencing software that will do web posts that has a better smoothing for location. because we all have experienced the "driving in the middle of the field" with an automotive GPS. I may just have to script it into EG, but i didn't want to waste the transmits from the phone on bad data. but it's nice to run a series events based on distance from home. like turn the heat on. when i am 10 miles away, and when i am 1000 ft away to turn the lights on. and do things when leaving the fenced area too.

I don't know how to send any kind of a "PM" on git. but I would shoot ya my E-mail. maybe trade ideas and notes on what works and doesn't work. (to much crap doesn't work!!)

but I wanted to set something up using Arduinos PIR sensors BT modules and EG (my lighting control reports to EG) so if any of the 2 things happens in a room to open the shades if day and close them when i leave the room. turn tv off, etc.....BT would be for Phone connectivity (i am under the assumption that BT will connect to the closest transmitter). it would pretty much eliminate the need to touch panels all over the house. use the Arduino to transmit voice recognition data to EG.

[per1234]

Well considering how limited the Arduino's AVR microcontroller is, the authentication is amazingly fast, though of course the newer encryption algorithms are probably slower. I'm just using it on my wired LAN so security is not an issue. It does worry me that I discovered an obvious serious security vulnerability with the Network Event Sender/Receiver Plugin(and TCPEvents) that has been there since 2005, fixed the problem, posted it to the EG forum months ago, and was completely ignored. The password authentication gives people an impression of security but the vulnerability makes the encryption completely useless so they might as well be sending their password in plaintext if they're not using my version of the plugins. I'm working on the unauthenticated EtherEvent option now.

Sounds like you're setting up a nice system! I'm just using Ethernet for my system because I'm old school like that. An alternative to bluetooth could be wifi that seems more consistent since you are already sending data through your network. Another thing I'd really like to add to EtherEvent is ESP8266 compatibility but I don't have a wifi router installed so I haven't even started looking into it yet.

[kdschlosser]

On another note. I forgot about this i just removed the lines from the EtherEvent.cpp pertaining to the remote IP. but there is a fault in it.

EthernetClient.h:27:12: note: candidate expects 1 argument, 0 provided
Line 27: uint8_t *remoteIP(uint8_t remoteIP[]);

that's the modified Library file for the remoteIP

as I tried the remoteIP function and it didn't work.

here is the reference to it from the EtherEvent.cpp File

#ifdef ethernetclientwithremoteIP_h //the include guard from the modified EthernetClient.h
fromIP = ethernetClient.remoteIP(); //Save the IP address of the sender. Requires modified ethernet library
#endif

now I am pretty sure it has to read and tell me if i am wrong, i have only been coding for 5 months now. But i didn't think you could return an array from a function only if the actual variable to store it in is passed to the function, unless you make it static. but that's pointless because it doesn't release the memory. so there is no need to return at all just pass the variable to have it modify the variable passed. and that's all..

unless you do this in the modified EthernetClient.h
in the declaration of the function.

uint8_t *remoteIP(uint8_t *remoteIP = NULL);

or maybe I have just completely done something wrong. all i should have to do is put the modified Ethernet library into my libraries folder. i believe. If i remember correctly i have over written the Arduino Ethernet library with this one. and It didn't work either.

* edit Havent tested other problems with EtherEvent but got rid of the compile error*

[kdschlosser]

and the Debug routine in Etherevent doesn't work pointer problems.

[kdschlosser]

I also seem to have a problem with TCP Events. when EG crashes (not an if but WHEN, LOL). it orphans the Port. and when i reboot EG all i get from TCP Events is "An error occured while sending you event !" and there is no debug routine in the Send Event to explain exactly what the problem is., and to release the port I gotta reboot. I have found a program that sometimes will allow me to kill the orphaned port but not always, it's called currPorts made by NirSoft. It can be a real hassle as I have EG running on my 2012 R2 server which houses a whole comrodery of Virtual machines and a whole bunch of other things including several databases in mysql and MSSQL. so the boot time is rather longish. and because one of the Virtual machines is a Domain controller it pretty much stops everything until it's rebooted. As far as EG goes. it's pretty much a dead stick. no real development has happened to it in years. the creator walked away and hasn't been heard from since. he gave he left it to the "people".

and I am Ethernet also. only true reliable connection. and the ESP modules, are Kick ass.

80 MHZ Risc processor 32 bit with crap tons of memory and like 8 GPIO pins. would run a ported version of EtherEvent no problem. and no need for an arduino. They are working on getting an IR Library to work properly as there is no hardware Interrupts (I think, or they haven't figured out how to access them) I2C works, SPI works, and if You ditch the factory AT firmware and just code C right to it it runs alot better. it's like 3 lines of code to get it to connect. they have added the ESP to the Arduino IDE as a Board in a forked development of the IDE it's on git.

[kdschlosser]

EtherEvent is not functioning. I even used a one of then Example Sketches. It will not Receive and Event. it Can send them but i just get that "An error occured while sending you event !" from TCP Events I have rebooted several times, I can Ping the EtherEvent Arduino, The port is proper. and when it does try to receive it hangs the Arduino. I have tested it upside and down every which way I know how to Eliminate me as the problem. I am going to ditch the Pointer to the Serial and Just have it spit out the Debug to locate the problem, it may not be the pointer that's the problem with the debug, it could be something else in EtherEvent causing the NUL's to be spat out.

found the problem with the Debug, it runs a 2nd Serial.begin with a different BAUD Rate than what i use

[per1234]

EthernetClient.h:27:12: note: candidate expects 1 argument, 0 provided
Line 27: uint8_t *remoteIP(uint8_t remoteIP[]);

What modified Ethernet library are you using? That looks like an old version. It now returns the IPAddress type. Try installing https://github.com/per1234/EthernetMod/archive/W5x00.zip and see if it works.

But i didn't think you could return an array from a function only if the actual variable to store it in is passed to the function

That's correct and the old version did require an array to be passed to the function but you can return an IPAddress because it's not an array. This is how the official Arduino Ethernet library does EthernetUDP.remoteIP().

I also seem to have a problem with TCP Events. when EG crashes

I definitely have some EG crashes but haven't run into the dead port problem. I'm going to work on some improvements on TCPEvents but I'm not so good with python so if you know any I could definitely use some help.

As far as EG goes. it's pretty much a dead stick.

EG could use work for sure but I disagree about it being dead. Even though bitmonster abandoned it there are frequent updates still but I would like to see a lot of things done differently. The main guy pako seems to be really unwelcome to anyone but the core group contributing. There are 2 github repositories for it, both completely abandoned and it's still being developed on Sourceforge which is a total joke. I really like eventghost and would like to put some effort into it but I don't see how. It's a shame because there's a decent community around the project but no real infrastructure to allow them to participate other than the forum which also sucks. I gave Girder a try a while back to see what I was missing and I have never felt so grateful for EventGhost, crashes and all.

ESP modules, are Kick ass

Yes I think that adding support would make EtherEvent useful to many more people. I'd like to add support for running on an ESP but also for using an ESP8266 module connected to an Arduino via serial because it seems like there are still lots of libraries that are not ported to ESP8266..

EtherEvent is not functioning.

Are you still having the problem or was it caused by the debug issue? I just redownloaded and tested the basic and advanced examples and they are working for me.

found the problem with the Debug, it runs a 2nd Serial.begin with a different BAUD Rate than what i use

Yes that's definitely not ideal how I did that I guess maybe it's better to just do the Serial.begin() in the sketch or at least I should document the debug baud rate. I did the Serial.begin() in EtherEvent because often I will want to turn off debug output in my sketch and just have it turned on in EtherEvent only.