AsyncEngine error on storage_engine

[vk496]

Hello and thanks for this amazing project!

I just tried to use it with my PostgreSQL as a storage backend and it threw weird errors. I fixed it by creating a sync_engine exclusively for radar, but wanted to report it.

PS: I'm from my phone, will share the code next day

[536]

Hi, I got a db error too, same thing?

INFO:     127.0.0.1:42336 - "GET /__radar/api/queries?limit=10&slow_only=true&slow_threshold=100 HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1102, in _begin_impl
    self.engine.dialect.do_begin(self.connection)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/duckdb_engine/__init__.py", line 344, in do_begin
    connection.begin()
duckdb.duckdb.TransactionException: TransactionContext Error: cannot start a transaction within a transaction

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 182, in __call__
    with recv_stream, send_stream, collapse_excgroups():
                                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
    raise exc
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 184, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/fastapi_radar/middleware.py", line 49, in dispatch
    return await call_next(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 159, in call_next
    raise app_exc
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 144, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/routing.py", line 78, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/starlette/routing.py", line 75, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 302, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 213, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/fastapi_radar/api.py", line 251, in get_queries
    .all()
     ^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2704, in all
    return self._iter().all()  # type: ignore
           ^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2857, in _iter
    result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
                                                  ^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1242, in _connection_for_bind
    transaction = conn.begin()
                  ^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 856, in begin
    self._transaction = RootTransaction(self)
                        ^^^^^^^^^^^^^^^^^^^^^
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2684, in __init__
    self._connection_begin_impl()
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2702, in _connection_begin_impl
    self.connection._begin_impl(self)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1104, in _begin_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1102, in _begin_impl
    self.engine.dialect.do_begin(self.connection)
  File "/fastapi-radar/.venv/lib/python3.12/site-packages/duckdb_engine/__init__.py", line 344, in do_begin
    connection.begin()
sqlalchemy.exc.OperationalError: (duckdb.duckdb.TransactionException) TransactionContext Error: cannot start a transaction within a transaction
(Background on this error at: https://sqlalche.me/e/20/e3q8)

[vk496]

Yours feels during runtime.

Mine is during init:

│   120 │   if not getattr(current, "__sqlalchemy_greenlet_provider__", False):                  ╭────────────────────────────────────────────────── locals ──────────────────────────────────────────────────╮                                                                                                                                          │
│   121 │   │   _safe_cancel_awaitable(awaitable)                                                │ awaitable = <coroutine object AsyncConnection.connect at 0x7e860ad52430>                                   │                                                                                                                                          │
│   122 │   │                                                                                    │   current = <greenlet.greenlet object at 0x7e8619f4d6c0 (otid=0x7e8619f50d80) current active started main> │                                                                                                                                          │
│ ❱ 123 │   │   raise exc.MissingGreenlet(                                                       ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                                          │
│   124 │   │   │   "greenlet_spawn has not been called; can't call await_only() "                                                                                                                                                                                                                                                                       │
│   125 │   │   │   "here. Was IO attempted in an unexpected place?"                                                                                                                                                                                                                                                                                     │
│   126 │   │   )                                                                                                                                                                                                                                                                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)

PS: It happen with the following code:

db_connector.engine: AsyncEngine

radar = Radar(app, db_engine=db_connector.engine.sync_engine, storage_engine=db_connector.engine.sync_engine)
    radar.create_tables()

So Im fixing it right now with this code:

radar_storage_engine = create_engine(
        db_connector.DATABASE_URL,
        pool_pre_ping=True,
    )

    radar = Radar(app, db_engine=db_connector.engine.sync_engine, storage_engine=radar_storage_engine)
    radar.create_tables()

PS2: The DB is PostgreSQL using psycopg

[doganarif]

#35