Pipelines no longer working in dev server after updating to latest cloud sdk

[snstanton]

I have a mapreduce pipeline that worked in SDK 122 that now fails in the dev server with the following error after updating to SDK 126:

Traceback (most recent call last):
  File "/vl/local/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/Users/stanton/src/vlshare/appserver/src/vl/base.py", line 645, in wrapper
    return handler(*args, **kwargs)
  File "/Users/stanton/src/vlshare/appserver/src/vlops/handlers.py", line 319, in any
    result = func(self.request)
  File "/Users/stanton/src/vlshare/appserver/src/vlops/admin/files.py", line 301, in _getZone
    result = launchPipeline(GetZonePipeline(zone.key.urlsafe()))
  File "/Users/stanton/src/vlshare/appserver/src/vl/pipe_utils.py", line 62, in launchPipeline
    pipeline.start(base_path='/ops/pipeline')
  File "/Users/stanton/src/vlshare/appserver/src/lib/pipeline/pipeline.py", line 673, in start
    self, idempotence_key, str(e)))
PipelineSetupError: Error starting vlops.admin.files.GetZonePipeline(*('aghkZXZ-Tm9uZXIXCxIKWm9uZUVudGl0eRiAgICAgMCvCgyiAQZnbG9iYWw',), **{})#342c4ceab7084adfb389b35b45960fdd: 

As far as I can tell the pipeline is having problems submitting tasks to the task queue. Note that the pipeline is not running in the default service. Also, everything works as expected in the production environment, so this is specifically a problem with the dev server.

Any ideas? What additional logging would be helpful here?

[Erfa]

I have the exact same problem, with the original App Engine SDK. I created a small repo that shows the problem: https://github.com/Erfa/mapreduce-test

[javpaw]

I'm having the same problem, It's not working on my dev server.

[brunoripa]

Same for me. Also, i have been suggested to downgrade to 122.0.0, but the issue is still there.

[timmehmainframe]

This is fixed by: GoogleCloudPlatform/appengine-pipelines@2f1930b

[georgewhewell]

above 'fix' didn't work for me- I added to my app.yaml:

module: default
version: someversion

and it started working. there has not been new release on pypi since that commit, so presumably this is broken out of the box for anyone with updated sdk.

[psundar]

The fix mentioned by @timmehmainframe worked. I commented the two lines mentioned below under src/pipeline/util.py:
if module == "default":
return version
Thank you.

[bjohnson-va]

For me, specifically, the fix was to add:

module: default
version: default

to app.yaml.

The version had to be default. I'm not going to pretend I understand why.

[magoarcano]

I only added to the app.yaml:
version: default
And it works. But app engine documentation at https://cloud.google.com/appengine/docs/standard/python/config/appref#version says:

The version ID can contain lowercase letters, digits, and hyphens. It cannot begin with the prefix ah- and the names default and latest are reserved and cannot be used.

[diogovk]

Indeed this is happening, and changing version to default works. (GoogleAppEnginePipeline version 1.9.22.1)

What would a definite solution to this problem would look like so I can make a PR?