Internal variables

[theoephraim]

Sometimes it is helpful to have intermediate storage of some data while you are transforming it. Putting that into a normal environment variable works, but it feels a little weird because we don't actually need to make it available outside the scope of the resolution.

In general we should probably add more functionality to have more control over which vars are available in different contexts - for example don't inject my secret zero (e.g. OP_SERVICE_ACCOUNT_TOKEN) that was used to fetch other secrets into my application code.

But there is a distinct case where a var is never used outside of varlock, and the idea here is to introduce a new concept of a "variable" to handle this. Defining it is probably a root decorator

These vars would be able to be used like env var refs within functions, so we probably need a way to distinguish between the two, probably a prefix/symbol of some kind. Given that regular env vars are usually referenced via the shorthand $ENV_VAR_NAME, these vars could be something like $$VARNAME or *VARNAME.

To define the var it could be a root decorator call @var(VARNAME, value)

Alternatively it could be defined more like a regular env var *VARNAME=foo, or maybe this whole concept is just a decorator @internalVar on existing items that says to never expose it outside of the varlock resolution process.

[SeeThruHead]

maybe internal vars dont have to be access the same way as existings vars

maybe @set("MY VAR", "bob") and @get("MY VAR") is how you always use them to completely avoid ambiguity

[theoephraim]

One benefit of making it more env-var like is that you still get @type to add validation rules, @sensitive, descriptions, etc... and it goes into the usual dependency graph without special handling.

Making it something else entirely could be useful but you lose out on all that good stuff... this is making me think more about just letting you mark things as internal somehow and/or using a naming convention (something like $var=... becomes internal, reference via $$var)

[theoephraim]

could also just be something like @internalVar decorator and then its up to you to decide if you want to use a different naming convention. just making them all lowerCamel would probably be clear enough for most people

[SeeThruHead]

i like the decorator one, no change to the actual env var itself

[theoephraim]

I think my main holdup is that saying something is "internal" is not always clear. Like you may have a var that you dont want injected into your application code, but you do want to inject it during build time, or into some other project script with varlock run. So while these "vars" are never injected and purely internal, we also need ways of selectively including/excluding specific items in some contexts, and these 2 concepts can get a bit blurry.

[SeeThruHead]

i think defining the var inside a comment makes it clear its not actually part of the conceptual .env file
and is internal

and why not make all the other decorators "just work" on internal vars defined in a comment?

[theoephraim]

yeah its totally possible, it'll just require changes to the language/parser itself because of how comments/decorators are attached to config items. Not necessarily a bad thing, but every change comes with additional mental tax, so need to make sure its worth it if we can accomplish the same using existing concepts.

# @required
# $VAR=foo # this now must _end_ the comment block
ITEM=bar