Skip to content
This repository was archived by the owner on Dec 13, 2023. It is now read-only.
This repository was archived by the owner on Dec 13, 2023. It is now read-only.

Postgres indexing problem #252

@rpe4a

Description

@rpe4a

Describe the bug
I try to use postgres with indexing insted of elastic. And execute a simple test workflow but conductor can't update task_index and has an error:

docker-postgres-1  | 2023-07-26 18:00:56.578 UTC [2661] ERROR:  value too long for type character varying(32)
docker-postgres-1  | 2023-07-26 18:00:56.578 UTC [2661] STATEMENT:  INSERT INTO task_index (task_id, task_type, task_def_name, status, start_time, update_time, workflow_type, json_data)VALUES ($1, $2, $3, $4, $5, $6, $7, $8::JSONB) ON CONFLICT (task_id) DO UPDATE SET task_type = EXCLUDED.task_type, task_def_name = EXCLUDED.task_def_name, status = EXCLUDED.status, update_time = EXCLUDED.update_time, json_data = EXCLUDED.json_data
conductor-server   | 267641 [http-nio-8080-exec-20] ERROR com.netflix.conductor.core.dal.ExecutionDAOFacade [] - Error updating task: c979373e-f80a-4de9-a992-7ae712c0b079 in workflow: 09c197b2-d91a-4776-9aea-2766f622afb3
conductor-server   | com.netflix.conductor.core.exception.NonTransientException: ERROR: value too long for type character varying(32)
conductor-server   |    at com.netflix.conductor.postgres.dao.PostgresBaseDAO.getWithRetriedTransactions(PostgresBaseDAO.java:147) ~[conductor-postgres-persistence-3.13.8.jar!/:3.13.8]
conductor-server   |    at com.netflix.conductor.postgres.dao.PostgresBaseDAO.queryWithTransaction(PostgresBaseDAO.java:209) ~[conductor-postgres-persistence-3.13.8.jar!/:3.13.8]
conductor-server   |    at com.netflix.conductor.postgres.dao.PostgresIndexDAO.indexTask(PostgresIndexDAO.java:101) ~[conductor-postgres-persistence-3.13.8.jar!/:3.13.8]
conductor-server   |    at com.netflix.conductor.core.dal.ExecutionDAOFacade.updateTask(ExecutionDAOFacade.java:514) ~[conductor-core-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server   |    at com.netflix.conductor.service.ExecutionService.poll(ExecutionService.java:172) ~[conductor-core-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]

The main problem is that task_index table has field task_type which is type varchar(32) but conductor tries to insert there wrong value.
On the first screenshoot I see Task type = SIMPLE and Task name = TestWorkerDynamicSubWorkflowForkInput but on the second screenshoot on the postgres base I see Task type = TestTransferChannelWorker1 and Task name = TestTransferChannelWorker1. And here is a problem:

  • the filed type must be more then varchar(32)
    or
  • you must write right value to this field(task type). Not a task name or something like this.

Details
Conductor version: 3.13.8
Persistence implementation: Postgres
Queue implementation: Redis
Lock: Redis or Zookeeper?
Workflow definition:
Task definition:
Event handler definition:

To Reproduce
Steps to reproduce the behavior:

  1. Use postgres index insted of elastic
  2. Create workflow with task which has name more then 32 symbols;
  3. Run it
  4. See conductor server logs to catch the error.

Expected behavior
Update operation will be successful.

Screenshots
image
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions