Elasticsearch用のロガー追加 by Nobu19800 · Pull Request #280 · OpenRTM/OpenRTM-aist-Python

Nobu19800 · 2022-03-29T08:12:32Z

Identify the Bug

fluent-logger-pythonライブラリはforward出力するためのライブラリであり、fluent-bitで対応している他の出力プラグインは使用できない。

Description of the Change

他にFluentd、もしくはFluent Bitを起動して、そこにForward出力、処理して他のプロトコルで出力するのが本来の使い方だと思うが、使用方法が複雑になるので、elasticsearch-pyライブラリを使ってElasticsearch送信には対応した。

Verification

Did you succeed the build?
No warnings for the build?
Have you passed the unit tests?

n-kawauchi · 2026-04-30T06:32:40Z

@Nobu19800

下記ページを参考にして動作確認してみましたがエラーになります。
https://openrtm.org/openrtm/ja/doc/developersguide/advanced_rt_system_programming/fluent_logger_use#toc45

上記ページではelasticsearchはバージョン指定7.8.0がありますが、ecs-loggingは無いためecs-logging-2.3.0がインストールされます。こうすると、下記エラーになりました。

$ python3 ConsoleOut.py -f rtc.conf
　　：
  File "/usr/local/lib/python3.12/dist-packages/ecs_logging/_stdlib.py", line 228, in format_to_ecs
    for field, value in self._extra.items():
                        ^^^^^^^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'items'

rtc.confは下記内容にしています。

logger.enable: YES
logger.log_level: PARANOID
logger.file_name: rtc%p.log,    STDOUT

logger.plugins: /usr/lib/python3/dist-packages/OpenRTM_aist/ext/logger/eslogger/ESLogger.py

logger.logstream.elasticsearch.output0.host: 127.0.0.1
logger.logstream.elasticsearch.output0.port: 9200
logger.logstream.elasticsearch.output0.index: fluentbit

elasticsearchの最新版9.3.0をインストールしての確認もしてみましたが、v8.x 以降ではuse_sslの引数が廃止されたそうで動作しませんでした。この最新版での環境構築と確認は以下の手順で行いました。

$ sudo python3 -m pip install elasticsearch
Successfully installed anyio-4.13.0 elastic-transport-9.2.1 elasticsearch-9.3.0 sniffio-1.3.1 typing-extensions-4.15.0
$ sudo python3 -m pip install ecs-logging
Successfully installed ecs-logging-2.3.0

Elasticsearchは下記公式サイトから「deb x86_64」を選択してダウンロード・インストールする
https://www.elastic.co/jp/downloads/elasticsearch

$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.3.3-amd64.deb
$ sudo dpkg -i elasticsearch-9.3.3-amd64.deb
$ sudo systemctl daemon-reload
$ sudo systemctl enable elasticsearch.service

/etc/elasticsearch/elasticsearch.ymlを確認したところコメント文で下記とあったので編集せずにデフォルト設定のままで使用した。
The default list of hosts is ["127.0.0.1", "[::1]"]

$ sudo systemctl start elasticsearch.service

ConsoleOut.pyを実行

$ python3 ConsoleOut.py -f rtc.conf
      ：
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/ext/logger/eslogger/ESLogger.py", line 69, in __init__
    self._es = Elasticsearch(hosts, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Elasticsearch.__init__() got an unexpected keyword argument 'use_ssl'

n-kawauchi · 2026-05-08T09:16:35Z

@Nobu19800

修正ソースをマージしてdebパッケージを作成しての確認ですが、エラーになります。
動作確認環境では、elasticsearchは7.8.0をインストールしています。

$ python3 -m pip list | grep elasticsearch
elasticsearch         7.8.0
$ python3 -m pip list | grep ecs
ecs-logging           2.3.0

ConsoleOut.py実行時に使用するrtc.confは前回の設定のままで変更しておりません。
実行結果を全て貼り付けます。Ubuntu24.04での確認です。

$ python3 ConsoleOut.py -f rtc.conf
2026-05-08 18:11:26,451 file.manager INFO 2.1.0
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 791, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 461, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 1448, in getresponse
    response.begin()
  File "/usr/lib/python3.12/http/client.py", line 336, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 305, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request
    response = self.pool.urlopen(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 845, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 447, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 791, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 461, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 1448, in getresponse
    response.begin()
  File "/usr/lib/python3.12/http/client.py", line 336, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 305, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/openrtm/pyPR280/ConsoleOut.py", line 154, in <module>
    main()
  File "/home/openrtm/pyPR280/ConsoleOut.py", line 136, in main
    mgr = OpenRTM_aist.Manager.init(sys.argv)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/Manager.py", line 218, in init
    manager.initLogger()
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/Manager.py", line 1693, in initLogger
    self._rtcout.RTC_INFO(
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/SystemLogger.py", line 488, in RTC_INFO
    log.log(str(messages), Logger.INFO, self._logger_name)
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/ext/logger/eslogger/ESLogger.py", line 338, in log
    log.info(msg)
  File "/usr/lib/python3.12/logging/__init__.py", line 1539, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.12/logging/__init__.py", line 1684, in _log
    self.handle(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.emit(record)
  File "/usr/lib/python3/dist-packages/OpenRTM_aist/ext/logger/eslogger/ESLogger.py", line 90, in emit
    self._es.index(index=self._index, body=data)
  File "/usr/local/lib/python3.12/dist-packages/elasticsearch/client/utils.py", line 139, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/elasticsearch/client/__init__.py", line 375, in index
    return self.transport.perform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/elasticsearch/transport.py", line 345, in perform_request
    status, headers_response, data = connection.perform_request(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/elasticsearch/connection/http_urllib3.py", line 245, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

[compat] add erasticsearch logger.

c884dff

Nobu19800 added the enhancement New feature or request label Mar 29, 2022

Nobu19800 requested a review from n-ando March 29, 2022 08:12

Nobu19800 self-assigned this Mar 29, 2022

Merge branch 'master' into feature/elasticsearchlogger

87ca330

n-kawauchi marked this pull request as ready for review April 30, 2026 05:57

Update ESLogger.py

e12c601

n-ando merged commit 5494d77 into OpenRTM:master May 11, 2026
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Elasticsearch用のロガー追加#280

Elasticsearch用のロガー追加#280
n-ando merged 3 commits into
OpenRTM:masterfrom
Nobu19800:feature/elasticsearchlogger

Nobu19800 commented Mar 29, 2022 •

edited

Loading

Uh oh!

n-kawauchi commented Apr 30, 2026

Uh oh!

n-kawauchi commented May 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Nobu19800 commented Mar 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Identify the Bug

Description of the Change

Verification

Uh oh!

n-kawauchi commented Apr 30, 2026

Uh oh!

n-kawauchi commented May 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Nobu19800 commented Mar 29, 2022 •

edited

Loading