Skip to content

Flink error: More buffers requested available than totally available. #29

@ziXet

Description

@ziXet

I get the following error when I want to build the Texas state tiles.

Command:
java -jar sharedstreets-builder-0.3.1.jar --input osm/texas-latest.osm.pbf --output ./texas_tiles

Error:

03/04/2021 21:16:20 CHAIN Join(Join at buildWays(OSMDataStream.java:179)) -> Map (Map at buildWays(OSMDataStream.java:179))(6/16) switched to FAILED
java.lang.IllegalArgumentException: More buffers requested available than totally available.
at org.apache.flink.runtime.operators.hash.MutableHashTable.ensureNumBuffersReturned(MutableHashTable.java:1333)
at org.apache.flink.runtime.operators.hash.MutableHashTable.initTable(MutableHashTable.java:1156)
at org.apache.flink.runtime.operators.hash.MutableHashTable.buildTableFromSpilledPartition(MutableHashTable.java:921)
at org.apache.flink.runtime.operators.hash.MutableHashTable.prepareNextPartition(MutableHashTable.java:631)
at org.apache.flink.runtime.operators.hash.MutableHashTable.nextRecord(MutableHashTable.java:666)
at org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashJoinIterator.callWithNextKey(NonReusingBuildFirstHashJoinIterator.java:116)
at org.apache.flink.runtime.operators.JoinDriver.run(JoinDriver.java:222)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:490)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:355)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)

There is not much explanation about this error on the Internet.

I already tried to increase the Flink taskmanager heap size.
taskmanager.memory.size
taskmanager.heap.size

Environment:
java -version
openjdk version "1.8.0_282" OpenJDK Runtime Environment (build 1.8.0_282-8u282-b08-0ubuntu1~16.04-b08) OpenJDK 64-Bit Server VM (build 25.282-b08, mixed mode)

CPU: 16 - Memory 60GB

Any help would be appreciated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions