Why the DaRPCServer could not accept the client connection?

[mokdeve]

Hi All,

I run the example, I found that one DaRPCClient finished it and The server could not accept others client connections, must run the DaRPCServer again.
on my understanding, the server could not be run again, is it right?

[mokdeve]

Because the DaRPCServerGroup could not close the endpoint from client when a disconnection event requested.

[PepperJo]

The server should be able to accept other client connections. You should be able to restart the server process since the listening socket and all client connections will be terminated when the process exits. However, you will not be able to start the DaRPCServer in the same process (without exiting) again if you did not shutdown the server endpoint correctly.

[mokdeve]

Yes,I agree with you, but when you run twice DaRPCClient at the same terminal (client node A),the second run will got the RDMA_CM_EVENT_REJECTED from Server. If open a new terminal (client node A) to run test, everything is ok, that is my confuse.

[patrickstuedi]

I cannot reproduce this. You should be able to keep the server running and have clients come and go. I'm running the server once, then two times the client. Here is the server output:

stu@flex01:~/source/darpc/target$ java -cp darpc-1.7-jar-with-dependencies.jar:darpc-1.7-tests.jar com.ibm.darpc.examples.server.DaRPCServer -a 10.100.0.1 -l 32
running...server 10.100.0.1, poolsize 3, maxinline 0, polling false, recvQueue 16, sendQueue 16, wqSize 16, rpcservice-timeout 0
new connection 1, cluster 0
disconnecting 1, cluster 0
new connection 2, cluster 1
disconnecting 2, cluster 1

And here is the client output:

stu@flex17:/source/darpc/target$ java -cp darpc-1.7-jar-with-dependencies.jar:darpc-1.7-tests.jar com.ibm.darpc.examples.client.DaRPCClient -a 10.100.0.1 -l 32
starting.. threads 1, connections 1, server 10.100.0.1, recvQueue 16, sendQueue16, batchSize 16, mode 0
finished joining worker 0
executionTime 0.01
ops 100.0
throughput 10000.0
stu@flex17:/source/darpc/target$ java -cp darpc-1.7-jar-with-dependencies.jar:darpc-1.7-tests.jar com.ibm.darpc.examples.client.DaRPCClient -a 10.100.0.1 -l 32
starting.. threads 1, connections 1, server 10.100.0.1, recvQueue 16, sendQueue16, batchSize 16, mode 0
finished joining worker 0
executionTime 0.01
ops 100.0
throughput 10000.0

[mokdeve]

Hi patrickstuedi,

Yes , I keep the server running for testing, and add close method at RdmaRpcService the issue fixed.
As for my understand, the server side should be close the endpoint when a disconnection event received, right?

in your test case, is there any different from darpc-1.7 to darpc-1.8 ?

sample code:

@OverRide
public void close(DaRPCServerEndpoint<RdmaRpcRequest, RdmaRpcResponse> endpoint) {
try {
endpoint.close();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("disconnecting " + endpoint.getEndpointId() + ", cluster " + endpoint.clusterId());
}

server:
/usr/java/jdk1.8/bin/java -Dlog4j.configuration=file:/usr/test/log4j.properties -cp darpc-1.8-jar-with-dependencies.jar:disni-2.0-tests.jar:darpc-1.8-tests.jar com.ibm.darpc.examples.server.DaRPCServer -a 10.100.0.1 -l 32768

clinetA:
sudo /usr/java/jdk1.8/bin/java -Dlog4j.configuration=file:/usr/test/log4j.properties -cp darpc-1.8-jar-with-dependencies.jar:darpc-1.8-tests.jar com.ibm.darpc.examples.client.DaRPCClient -a 10.100.0.1 -k 3000 -n 10 -c 5 -l 32768 -m stream-poll

[patrickstuedi]

On Tue, May 7, 2019 at 12:35 PM hanjian2086 ***@***.***> wrote: Hi patrickstuedi, Yes , I keep the server running for testing, and add close method at RdmaRpcService the issue fixed

I didn't add an additional close method, it worked as is.

As for my understand, the server side should be close the endpoint when a disconnection event received, right?

It does, as you can see from the output of my comment in the github issue.

in your test case, is there any different from darpc-1.7 to darpc-1.8 ?

As far as I remember darpc 1.8 is just using the latest version of DiSNI. Please try darpc 1.8 and see if the problem disappears. The close call is just to inform a RPC service about a endpoint disconnect event. If the service has extra state it wants to clean up it can do so in this call. But the benchmark service doesn't have any extra state so it's fine to leave the call empty.

…

sample code: @OverRide <https://github.com/override> public void close(DaRPCServerEndpoint<RdmaRpcRequest, RdmaRpcResponse> endpoint) { *try { endpoint.close(); } catch (IOException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); }* System.out.println("disconnecting " + endpoint.getEndpointId() + ", cluster " + endpoint.clusterId()); } server: /usr/java/jdk1.8/bin/java -Dlog4j.configuration=file:/usr/test/log4j.properties -cp *darpc-1.8*-jar-with-dependencies.jar:disni-2.0-tests.jar:darpc-1.8-tests.jar com.ibm.darpc.examples.server.DaRPCServer -a 10.100.0.1 -l 32768 clinetA: sudo /usr/java/jdk1.8/bin/java -Dlog4j.configuration=file:/usr/test/log4j.properties -cp *darpc-1.8*-jar-with-dependencies.jar:darpc-1.8-tests.jar com.ibm.darpc.examples.client.DaRPCClient -a 10.100.0.1 -k 3000 -n 10 -c 5 -l 32768 -m stream-poll — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEOGIOU7VUB4DJVIJQ2DMMTPUFLNVANCNFSM4HHOZ45A> .

[mokdeve]

Please try darpc 1.8 and see if the problem disappears.

In fact , I have been running darpc 1.8 faced that this issue.

the benchmark service doesn't have any extra state so it's fine to leave the call empty.

Maybe some users will forget to call this method that caused potential resource issues if the darpc also unhandled it.