Benchmark hsnsq and nsq performance

[pharaun]

We want to identify what the performance curves are for nsq so that we can decide if we should use it as a way to stream assets to the client or not.

The basic idea here is to have the standard two dimension benchmark in which:

1. X axis is the send-rate (ie 1msg/sec, 10msg/sec, etc...)
2. Y axis is the message-size (ie 1kb, 10kb, etc...)

Then in each message upon sending we will record the following 3 items:

1. Sequal id (to identify dropped/lost messages)
2. Timestamp of send-time
3. Gibbish for the rest of the message (from a block of memory for dma/mmap speed)

Then upon reception we will record the time the message was received, and record the send-time and sequal id to a csv/logfile for further analysis.

Should be able to make a binary that provides us with a few options such as send rate, message size, and so on for the initial benchmark effort. Binary provided options:

1. hostname/ip to connect to for nsq
2. port to connect to for nsq
3. count of messages to send
4. send-rate in msg/sec
5. message size in kilobytes (min of 1-4kb)
6. RDY size (we are doing 1 for now but should be able to bump this up)

[cantsin]

👍

[mreiferson]

in case you're curious of pure nsqd performance (not specific client libraries) I recently finished up this distributed benchmark automation script, see nsqio/nsq#438