Writing pause/resume for clients

[ypnos]

I use picows as a Websocket client and I occasionally get this output:

Protocol writing pause requested, crossed writing buffer high-watermark
Protocol writing resome requested, crossed writing buffer low-watermark

There is a topic guide for this, but it is written from server perspective. The WSListener interface, however, is available for clients and servers alike.

Just unfortunate wording in the topic guide?

[tarasko]

Yes, the wording is unfortunate. it is the same for server and clients. I'll fix the topic guide

In your case the client is probably sending either a huge message or a lot of smaller messages.

First, socket SNDBUF gets filled, so send syscall returns EAGAIN.
Then, asyncio's write buffer gets filled up to high-watermark, this leads to WSListener.pause_writing callback. After some time, the socket becomes ready for writing, asyncio flushes data from the write buffer, the write buffer size goes below low-watermark and this causes WSListener.resume_writing.

The default implementations of pause_writing and resume_writing in WSListener just print those messages, you can override them if you want to implement some kind of flow control.

Also, I'll fix resome typo :)

[ypnos]

I have one more question about your default implementations. They seem to always print a warning:

# picows.pyx lines 1019-1027 — Protocol class
def pause_writing(self):
    self._logger.warning("Protocol writing pause requested...")
    if self.listener is not None:
        self.listener.pause_writing()  # our override is called AFTER

def resume_writing(self):
    self._logger.warning("Protocol writing resume requested...")  
    if self.listener is not None:
        self.listener.resume_writing()

I would like them to only print a warning if I don't implement pause_writing() and resume_writing() myself. When I implement these methods, I still get console spammed with this messages, and I also don't want to turn of picows warning log level.

So maybe the logger calls should be moved into the WSListener base class?

[tarasko]

Yes, you're right. It should be moved to WSListener pause_writing(). Somehow I thought it is already there, will do

[tarasko]

I forgot to mention that if occasionally sending a huge frame like 20 MB is a normal workflow, it is good to just set a higher high watermark in ws_on_connected.

def on_ws_connected(self, transport: WSTransport):
    transport.underlying_transport.set_write_buffer_limits(high=40*1024*1024)

set_write_buffer_limits

The default high watermark is 256 Kb I think. Given SNDBUF size 2-4 Mb, sending 20 Mb frame will always trigger pause_writing event at least on Linux with the default settings.

The name write buffer limits is a bit confusing. In asyncio the write buffer can grow indefinitely, these limits only indicate when you get notified with pause_writing and resume_writing.

[ypnos]

Thank you very much for your prompt answer and good suggestions. I also looked at your commit and like how you solved the issue!