Tagging Hypershift clusters for tracking by krishvoor · Pull Request #157 · cloud-bulldozer/perfscale-managed-services

krishvoor · 2023-06-07T14:14:20Z

No description provided.

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

jtaleric

lgtm

krishvoor · 2023-06-07T15:11:10Z

When we run this wrapper from our forked repo: this will look like this:


2023-06-07 14:56:38.359 DEBUG rosa-hosted-wrapper - _build_cluster: Attempting cluster installation
2023-06-07 14:56:38.359 DEBUG rosa-hosted-wrapper - _build_cluster: Output directory set to /tmp/bea96cda-815c-4d0d-804e-dc5d44aa2b9b/chktag-njc-0001
2023-06-07 14:56:38.361 DEBUG rosa-hosted-wrapper - _build_cluster: ['/root/rosa/rosa', 'create', 'cluster', '--cluster-name', 'chktag-njc-0001', '--tags=User:krishvoor', '--replicas', '3', '--hosted-cp', '--sts', '--mode', 'auto', '-y', '--output', 'json', '--oidc-config-id', 'abcd', '--subnet-ids', 'subnet-XXX,subnet-YYY,subnet-ZZZ,subnet-xxx,subnet-yyy,subnet-zzz', '--properties', 'provision_shard_id:8c3bcd1f-xxxx', '--version', '4.12.15', '--operator-roles-prefix', 'chktag-njc']

morenod · 2023-06-07T16:45:08Z

You can do the same just passing the tag you want on the extra params.

This change requires that the command will be launched from a git based folder, which is not mandatory at this moment

morenod

If we want to add this, we need to consider the option of running wrapper from outside a git based folder.

Anyway, I still recommend to use the extra-params option instead of this

krishvoor · 2023-06-08T10:38:11Z

You can do the same just passing the tag you want on the extra params.

We need this tag <User:GITHUB_USERNAME> for better consolidation of cloud-expenses as defined here, and users will not have to pass tags for every-run

However, will implement supplying extra-tags via argsparse in case.

krishvoor · 2023-06-08T16:08:36Z

+    # Get the Git remote URL for the current working directory
+    old_wd = os.getcwd()
+    current_wd = os.path.dirname(os.path.abspath(__file__))
+    os.chdir(current_wd)


@morenod This tackles the issue: even if the user triggers the wrapper script outside the GitHub repo.

dry923 · 2023-06-09T14:52:21Z

+    else:
+        GITHUB_USERNAME = "cloud-bulldozer"
+
+    cluster_cmd = [rosa_cmnd, "create", "cluster", "--cluster-name", cluster_name, f"--tags=User:{GITHUB_USERNAME}", "--replicas", str(worker_nodes), "--hosted-cp", "--sts", "--mode", "auto", "-y", "--output", "json", "--oidc-config-id", oidc_config_id]


@krishvoor I honeslty don't think that the github user is the most useful tag to apply here. I'd rather we take in a string from the CLI to tag whatever we need it to be, ie some financial tracking, username, or whatever

@krishvoor I honeslty don't think that the github user is the most useful tag to apply here.

@dry923 I hear you, today the cloud expenses are tracked via GitHub user details just like the one defined here via ROSA DAGs and have proved useful.

I'd rather we take in a string from the CLI to tag whatever we need it to be, ie some financial tracking, username, or whatever

Agreed, will implement changes through with this as an additional change.

morenod · 2023-06-09T15:11:20Z

Im not saying I want to tag, but if we are going to do it, we need to also add this tags on:

vpcs created by terraform
oidc-config objects
operator-roles
machinepool

krishvoor · 2023-06-09T18:51:17Z

Im not saying I want to tag, but if we are going to do it, we need to also add this tags on:

vpcs created by terraform

oidc-config objects

operator-roles

machinepool

Thanks, @morenod this would mean it will be easy to find all "resources" in the AWS belonging to the same cluster.

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

krishvoor · 2023-06-15T15:55:43Z

@morenod @dry923: Based on the discussions above, I have implemented
--extra-tags parameter will also tag operator-roles, machinepool, oidc-config objects & vpcs.
as w/ --wildcard_options we will only be limited to tag resources created during the _build_cluster() function.

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

vishnuchalla · 2023-06-15T19:29:46Z

@krishvoor Can you please attach the steps in detail on how these changes were tested and verified? I am somehow unable to reproduce the steps mentioned in README.md

vishnuchalla · 2023-06-15T19:37:34Z

                        logging.debug("Creating cluster on VPC %s, with subnets: %s" % (vpc_info[0], vpc_info[1]))
                    try:
-                        thread = threading.Thread(target=_build_cluster, args=(ocm_cmnd, rosa_cmnd, cluster_name_seed, args.must_gather_all, args.provision_shard, args.create_vpc, vpc_info, args.workers_wait_time, args.add_cluster_load, args.cluster_load_duration, jobs, workers, my_path, my_uuid, loop_counter, es, args.es_url, args.es_index, args.es_index_retry, service_cluster, sc_kubeconfig_path, all_clusters_installed, oidc_config_id, args.workload_type, args.kube_burner_version, args.e2e_git_details, args.git_branch, operator_roles_prefix))
+                        thread = threading.Thread(target=_build_cluster, args=(ocm_cmnd, rosa_cmnd, cluster_name_seed, args.must_gather_all, args.provision_shard, args.create_vpc, vpc_info, args.workers_wait_time, args.add_cluster_load, args.cluster_load_duration, jobs, workers, my_path, my_uuid, loop_counter, es, args.es_url, args.es_index, args.es_index_retry, service_cluster, sc_kubeconfig_path, all_clusters_installed, oidc_config_id, args.workload_type, args.kube_burner_version, args.e2e_git_details, args.git_branch, operator_roles_prefix, tagging))


I think we can use **kwargs or dictionaries here instead of passing these many arguments in the params.

krishvoor · 2023-07-19T14:41:26Z

Closing this in favor of #162

krishvoor and others added 2 commits June 7, 2023 19:43

Tag the Hypershift Clusters for cloud spend

d4faddf

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

Merge branch 'cloud-bulldozer:main' into tagging

302f934

krishvoor requested review from dry923 and morenod June 7, 2023 14:14

vishnuchalla self-requested a review June 7, 2023 14:36

jtaleric approved these changes Jun 7, 2023

View reviewed changes

morenod suggested changes Jun 7, 2023

View reviewed changes

Considers the option of running wrapper from outside the git repo

5a82e35

krishvoor commented Jun 8, 2023

View reviewed changes

dry923 reviewed Jun 9, 2023

View reviewed changes

krishvoor requested a review from smalleni June 9, 2023 18:51

dry923 mentioned this pull request Jun 14, 2023

Add option for budget approval ticket #159

Open

krishvoor and others added 2 commits June 14, 2023 18:48

Merge branch 'cloud-bulldozer:main' into tagging

68ba4ab

Updates to oidc & machinepool & operator-roles

5e29672

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

Updates the tagging on VPCs

8566fea

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

vishnuchalla reviewed Jun 15, 2023

View reviewed changes

krishvoor closed this Jul 19, 2023

Conversation

krishvoor commented Jun 7, 2023

Uh oh!

jtaleric left a comment

Choose a reason for hiding this comment

Uh oh!

krishvoor commented Jun 7, 2023

Uh oh!

morenod commented Jun 7, 2023

Uh oh!

morenod left a comment

Choose a reason for hiding this comment

Uh oh!

krishvoor commented Jun 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krishvoor Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

dry923 Jun 9, 2023

Choose a reason for hiding this comment

Uh oh!

krishvoor Jun 9, 2023

Choose a reason for hiding this comment

Uh oh!

morenod commented Jun 9, 2023

Uh oh!

krishvoor commented Jun 9, 2023

Uh oh!

krishvoor commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vishnuchalla commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vishnuchalla Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krishvoor commented Jul 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

krishvoor commented Jun 8, 2023 •

edited

Loading

krishvoor commented Jun 15, 2023 •

edited

Loading

vishnuchalla commented Jun 15, 2023 •

edited

Loading

vishnuchalla Jun 15, 2023 •

edited

Loading