ERROR:channel.cc(507) Invalid message size

[michaelbaisch]

As mentioned in #34, I'm trying to get this running on macOS again. One problem I run into is that sometimes it just shows a white screen. I found out when that happens, the associated Helper.app uses a lot of memory for a while, this can go up to a few GB. After a while the Xcode console tells me something like:

[0507/173057.677689:ERROR:channel.cc(507)] Invalid message size: 302285456

After that the memory usage for the helper process goes back to "normal" ~40mb. I have to reload the browser to finally see the webpage.

This happens in maybe about 30% when creating a new browser.

I tried the helper app from the cefclient cef example and the one compiled from the addon. Both behave the same.

I couldn't find anything about that message, and for now I'm not sure whats going on.

[razvanilin]

How's your progress on this? I will give it a go early next week. For now, I'm working on Windows and I managed to get quite a good performance and fixed most of the keyboard input issues.
Have you tried multiple cef builds?

[michaelbaisch]

I'm still in the process of getting my head around the whole CEF business.

I tried to inspect a helper.app that behaves that way, but I didn't have any luck. When choosing that process as target in Instruments (command line) it just quits and I don't get any data. In Xcode I can attach to a helper process but as soon as I try to transfer to Instruments to see what's going on in memory, the process disappears again.

Performance seems to be alright, as good as expected keeping in mind that we still copy from GPU -> CPU -> GPU. I hope we get a way around that soon.

I'm not too concerned about keyboard input issues, right now. Primarily I want to have multiple cef views in 3D space and communicate with JS calls.

That's a good idea I haven't tried any other build, will try the current build tomorrow.

[michaelbaisch]

I've tried the current build form this morning 3.3029.1613. At first I thought it would be better but it's still happening…

Now I took a look at the chromium source to find out where this error is coming from. I found this in mojo/edk/system/channel.cc:

bool Channel::OnReadComplete(size_t bytes_read, size_t *next_read_size_hint) {
  bool did_dispatch_message = false;
  read_buffer_->Claim(bytes_read);
  while (read_buffer_->num_occupied_bytes() >= sizeof(Message::LegacyHeader)) {
    // Ensure the occupied data is properly aligned. If it isn't, a SIGBUS could
    // happen on architectures that don't allow misaligned words access (i.e.
    // anything other than x86). Only re-align when necessary to avoid copies.
    if (!IsAlignedForChannelMessage(
            reinterpret_cast<uintptr_t>(read_buffer_->occupied_bytes()))) {
      read_buffer_->Realign();
    }

    // We have at least enough data available for a LegacyHeader.
    const Message::LegacyHeader* legacy_header =
        reinterpret_cast<const Message::LegacyHeader*>(
            read_buffer_->occupied_bytes());

    if (legacy_header->num_bytes < sizeof(Message::LegacyHeader) ||
        legacy_header->num_bytes > kMaxChannelMessageSize) {
      LOG(ERROR) << "Invalid message size: " << legacy_header->num_bytes;
      return false;
    }
…

This fits the error messages I'm seeing:

const size_t kMaxChannelMessageSize = 256 * 1024 * 1024;  // = 268435456

The interface describes the Channel class like this:

Channel provides a thread-safe interface to read and write arbitrary delimited messages over an underlying I/O channel, optionally transferring one or more platform handles in the process.

When I don't run our update function with CefDoMessageLoopWork(), the bad behavior doesn't happen, so I guess everything around CreateBrowserSync() is fine. The problem could have something to do with the communication between the browser and render process. What we are doing different than the cef examples is using CefDoMessageLoopWork() instead of CefRunMessageLoop().

cef_app.hstates:

When using this function care must be taken to balance performance against excessive CPU usage. It is recommended to enable the CefSettings.external_message_pump option when using this function so that CefBrowserProcessHandler::OnScheduleMessagePumpWork() callbacks can facilitate the scheduling process.

Does anyone think that this problem could be connected to not using external_message_pump? I haven't implemented it because it doesn't seem that easy to get right. I would appreciate any insight.

I'll try to push a branch with my example project tomorrow.

When trying this out, be warned that not all of those helper apps actually stop at at maybe around ~3GB memory and log this error before going back to normal or quitting (Like I wrote initially). Some of them just keep going, I saw some of them going up to >30GB memory. I always have Activity Monitor running to have a look at the helper apps.

[michaelbaisch]

I have created an example project, which shows the issue more easily by being able to create multiple instances:
It's in this branch the example_multi project.
Follow the instructions on #34 to get it working.

I've cleaned up some stuff and tried to remove unnecessary code, just to make sure none of those are messing things up.

Funny thing is that the first instance you create with this code will aways have memory issues. The kind where it doesn't stop at around 3GB and logs the error. But if you just add ofToDataPath("html/index.html", true); to the setup in ofApp.cpp the first one will act normal.
Can't shake the feeling that it's something obvious but I think I tried all the obvious things…

After that you can press n a few times and eventually there will be an instance behaving like described.

[michaelbaisch]

The example project looks something like that on my machine.

[michaelbaisch]

I tested this with the newest 3.3071.1644.g408afd1 version again. This problem still persists. And it is still completely reproducible with the example_multi in this branch. The first instance has always this behavior. And you can still simply alter this by adding

ofToDataPath("html/index.html", true);

this to the setup() function in ofApp.cpp. But simply waiting doesn't work. Which makes no sense to me but it could be some indication for something.
This time I also tested it next to the MacBook Pro Mid 2015 on an oder Mac (iMac Mid 2009, macOS Sierra, 3.33 GHz Intel Core 2 Duo, ATI Radeon HD 4670 256 MB) with the same behavior.

I think there is a good chance to take this addon out of the experimental stage and make it more or less simple for anyone to use on both macOS and windows. The only problem I see is this right now, if anyone has any ideas, I'm open for suggestions.

[michaelbaisch]

I started discussing this in the cef forum here.

[michaelbaisch]

Good news! It seems that I found the problem. It was because of uninitialized variables (w and h) in the renderHander. CEF asks immediately for the size of the page and the renderHandler returned the values of some memory address. This could return a huge number, which would explain the high memory utilization of the render process. And it also explains why commenting or changing some totally unrelated things would change this behavior. This was also an issue on Windows but I only noticed some crashes of render processes in the Release build.

Among other things this was fixed in #39 pull request.