Reference the Defaults dict

[mihai-stancu]

Hello,

Thank you for this tool, it's awesome.

Is there a way to reference the full list of Defaults to achieve something like:

apiVersion: v1
kind: ConfigMap

metadata:
  namespace: !Var namespace
  name: !Var name

data: !Var AllTheDefaults

I've been trying

• !LookupAll "$" -- this should be the reference to the root dict but it errors out with "maximum recursion depth exceeded"
• !LookupAll "*" -- this would likely create a new list of values from the root dict (I want the names too)
• !Index {over: !LookupAll "$", ...} -- maximul recursion again

[japsu]

I think the maximum recursion depth error is due to #15.

Am I reading this correctly, that instead of the defaults, you would actually like to reference the full dict of variables as it stands at the time and place of reference?

There can be multiple sources of variables, and even multiple instances of !Defaults. Generally the order of precedence (smaller number is stronger precedence) is

1. local bindings with !With, !Loop etc.,
2. variables defined with --define/-D,
3. environment variables, if --include-env/-e was specified,
4. variable files defined with --var-file/-f, and finally
5. default values defined with !Defaults

[mihai-stancu]

Hi,

Yes, I would like to access the contextualized version of the root object -- now that you emphasize the full logic behind it I can intuit it's not directly accessible and needs to be derived on the fly based on a variable name you're trying to determine a value for (and it falls back through the layers of the defaults).

[mihai-stancu]

Let me explain my usecase so we can determine if I'm going about this all wrong and theres a much better way to do this via emrichen.

I'm running a small script of my own which takes environment-scoped values and determines which value is relevant for the current scope.

# Input file for my script

backend:WebService:
  replicas: 1
  replicas@prod: 10
  replicas@prod-eu: 15
  replicas@prod-us: 20
  ports:
    http: 80

mailpit:WebService@test:
  replicas: 1
  ports:
    smtp: 1025
    http: 8025

When invoked with a specific environment my code calculates which of the above prameters are relevant and outputs the final version of configuration for that environment. For example for env=prod-eu:

# Output of my script

backend:WebService:
  replicas: 15
  ports:
    http: 80

Then it takes each k8s workload in the output and:

• Identifies an emrichen template (in our case that's WebService.tpl.yml)
• executes emrichen -D env=prod-eu -f tmp_defaults_file.yml < WebService.tpl.yml
• ...
• Profit!

The tmp_defaults_file.yml is generated on the fly with the exact params for that service:

# tmp_defaults_file.yml
replicas: 15
ports:
  http: 80

[mihai-stancu]

With that I mind I tried to handle generating a ConfigMap with my script:

config:ConfigMap:
  DB_NAME: my_app

  DB_HOST@prod-us: the-us.db.tld
  DB_USER@prod-us: someuser

  DB_HOST@prod-eu: the-eu.db.tld
  DB_USER@prod-eu: someotheruser

Which would yield the final parameters:

config:ConfigMap:
  DB_NAME: my_app

  DB_HOST: the-eu.db.tld
  DB_USER: someotheruser

And when getting invoked via emrichen it would look something like:

# defaults_file.yml

DB_NAME: my_app

DB_HOST: the-eu.db.tld
DB_USER: someotheruser

With a template something like:

apiVersion: v1
kind: ConfigMap

data: !Var root

[mihai-stancu]

As a workaround obviously I could wrap the variables I write into the emrichen defaults file.

config:
  DB_NAME: my_app
  
  DB_HOST: the-eu.db.tld
  DB_USER: someotheruser

But I wasn't sure if that was really necessary.

[mihai-stancu]

Note, wrapping the vriables in a top-level variable such as config means that no deep merge occurs.

So if I have:

!Defaults
config:
   a: a
   b: b
---
show_a: !Lookup config.a
show_b: !Lookup config.b

With a defaults file:

config:
  b: bb

The order of precedence makes the external default file overwrite the value of $.config and the output is an error declaring that "Lookup could not find config.a".

[mihai-stancu]

I think the maximum recursion depth error is due to #15

With this in mind I tried to make sure all of the defaults in the defaults file are bare values that don't need any enrichment at all. The error continues to occur.

Here's a stripped down version that reproduces the issue:

emrichen -D env=prod-eu -f defaults.yml < ConfigMap.tpl.yml

# defaults.yml
namespace: default
DB_NAME: main

# ConfigMap.tpl.yml

apiVersion: v1
kind: ConfigMap

data: !Index
  over: !LookupAll "$"
  index_as: key
  as: value
  by: !Var key

However if I iterate on !LookupAll "*" then it works (omits the keys but works without a recursion error).
I get all of the expected values for each parameter + an extra value that looks like a file resource: <stdin>.