Add support for reading from in-memory table from Polars

[DeflateAwning]

It would be awesome if you could read from an in-memory table from Ruby-Polars, similar to how you can in Python.

Here are some related readings which may be helpful:

• https://duckdb.org/docs/stable/extensions/arrow.html
• https://duckdb.org/2021/12/03/duck-arrow.html
• https://arrow.apache.org/docs/python/memory.html#pyarrow-buffer

[suketa]

Does https://github.com/red-data-tools/red-arrow-duckdb help us?

[suketa]

@DeflateAwning
Currently I don't know how to read in-memory data directly, but you can read Polars data using temporary file.

require 'polars-df'
require 'duckdb'

df = Polars::DataFrame.new(
  {
    a: [1, 2, 3],
    b: %w[one two three]
  }
)

df.write_parquet('/tmp/data.parquet')

DuckDB::Database.open do |db|
  db.connect do |conn|
    conn.query("CREATE TABLE data AS SELECT * FROM read_parquet('/tmp/data.parquet')")
    conn.query('SELECT * FROM data').each do |row|
      puts row.inspect
    end
  end
end

I'll continue to investigate how to read in-memory data directly.

[DeflateAwning]

Yeah, I'm aware of the parquet option. It's quite a bit slower though, so not ideal for webserver deployments.

Red Arrow DuckDB sounds promising - that might be a big part of the solution.