Page 1 of 1

TCP server disconnection due to slow reading?

Posted: Tue Aug 22, 2017 8:29 pm
by rivasd
Hi! I am trying to write a Python 3 TPC/IP client that also does plotting and processing in real time. The example code found here works pretty well with a few modifications for Python 3. I changed the read() method to simply fetch the next packet instead of using a "duration" argument.

However things start to break down if my program has to perform work in-between the read() calls (between reading each TCP packet). In my case, simply keeping a PyQtgraph open and appending the received data to a numpy matrix in-between the read calls makes the green light on the TCP tab go from "Connected" to "Listening" after a few seconds. Interestingly, even though ActiView show the client as disconnected, my Python script still manages to continue reading data for a few seconds before hanging on the socket.recv() call.

This and other tests on your Matlab client has led me to believe that ActiView disconnects if data is read more slowly than it is collected. I hypothesize that the TCP server has some sort of buffer, and when it reaches its maximum size (because data is read more slowly by my busy program than it is generated), it simply disconnects.

Is this true? Is there any more in-depth documentation on the specifics of the TCP server? what are the conditions for a disconnection?

Also, if I were to detect the closing of the connection and simply re-open a new socket when ActiView disconnects, would data be lost (the remaining data in the server's buffer) ?

Maybe I could trying reading more that the number of bytes per TCP array each read (perhaps until no data is received?) would that work? is it guaranteed to return whole TCP arrays (a byte array that has length multiple of "bytes in TCP array")?

Any link to documentation or technical help would be appreciated

Thank you

Re: TCP server disconnection due to slow reading?

Posted: Wed Aug 23, 2017 12:54 am
by pmac
Hi Rivasd,

The usual approach, in any computer language, to read sockets efficiently is usually
something like this:

1. create a very large input buffer;

2. call 'recv' using that buffer and its dimensioned size;

3. 'recv' fills the buffer with what is immediately available, up to that dimensioned
size and returns the actual number of bytes being returned;

4. your program processes all complete sample sets in the returned buffer;

5. if there is a partial sample set at the end of the recv buffer (there usually is),
be prepared to prefix this partial set to the front of the next buffer returned by 'recv'.

6. repeat from step 2.


Reducing the number of 'recv' calls in this way takes some of the load off the system.

A good tutorial on all this, probably more than you ever wanted to know, is:

Beej's Guide to Network Programming
Using Internet Sockets

http://beej.us/guide/bgnet/output/html/ ... l#sendrecv

I hope this helps,

Paul

Re: TCP server disconnection due to slow reading?

Posted: Wed Aug 23, 2017 3:44 am
by rivasd
Wow Paul,

Thank you so much for taking the time to write this answer! As you guessed, I'm pretty new at such low-level programming. I'll do as you say and read more than one packet at a time, should do the trick!

Still, if anyone from BioSemi can still confirm that my hypothesis about why the TCP server disconnects is correct and give some link to documentation of the workings of this part of ActiView, I think that would benefit all of us!

Thanks again.

Re: TCP server disconnection due to slow reading?

Posted: Mon Aug 28, 2017 2:01 pm
by Coen
TCP/IP is a handshaked protocol. The way it is programmed now is that ActiView can only proceed with reading the next data block from the ringbuffer when the receipt of a TCP/IP package is confirmed by the client. When the client is too slow, then the ringbuffer will not be read with sufficient speed by ActiView. Refer to buffer clock in the lower right corner: green needle (read data) cannot keep up with the red needle (write data). When unread data in the ringbuffer starts to get overwritten with new incoming data (green needle is a full revolution behind the red needle), ActiView stops with an error popup (ringbuffer overflow).

Best regards, Coen (BioSemi)