Storing as binary in Couchbase?

[genachka]

Noticed that you're storing the data as binary in Couchbase. In addition to generating graphs from the data with Grafana, I was hoping to get to the stored data to do additional things with the data for reporting and/or analysis unrelated to Grafana using either CB's views of N1QL. Also, with the Full Text search capabilities now being exposed directly in CB 4.5 DP, of course none of that binary data would be useful for that. Why not store the data as JSON in CB?

[ngrossmann]

You cannot append to json data. My main goal with momo was to built something with very good read and write performance and allow data to be collected in 1 second intervals. I used views in the beginning to store which time-series names, I still do that but cache the data in momo. The performance I saw was pretty far away from what I need/expected, so I don't see it as an option for data points.

[genachka]

I think the sub-document functionality could be leveraged for that and N1QL when used with keys (without creating an index) improves performance over views. For my use case, having it in binary format is a show stopper as I don't want to have two databases to deal with (one for Grafana, one for everything else) for the same data set, especially knowing what's coming down with CB 4.5 that we'll want to leverage.

Have you any docs on how the data is stored and how to best query it from within CB? For example, if I wanted to create a PDF document showing the past month's performance of a specific sensor(s) in a specific location, sampled up to every 1 minute then maybe I could still leverage momo.

[ngrossmann]

You can query momo directly e.g.

curl 'http://localhost:8080/series?from=1457592507&interval=1m&q=servers.lisa.cpu-0.cpu-user&to=1457592807'

• q query as in grafana, it's documented in the README
• interval sample interval, e.g 1s, 1m
• from / to time interval in seconds since the epoch.

The port is the port configured in momo (default 8080) not the grafana port (default 3000).

[ngrossmann]

I checked the sub-document feature. Seems it's new in 4.5, as I'm using the community edition it's currently not an option. Anyway I would be interested in seeing a comparison of binary append vs. sub-document append performance.
Implementing this in Momo shouldn't be too difficult, if you want to give it a try look at the MetricActor class.