Gdxdsd 8240 google search python39

[doughon]

This PR does the following

Updates the google_search Microservice to use Python 3.9

Please note that some files were modified for testing purposes. You can see these modified files attached in the ticket.

Review files

Review the following files to make sure you understand the changes:

1. google_search.py
2. Pipfile

Note: Pipfile.lock contains changes too but do not need to be manually reviewed. They are caused automatically by the initial creation of the Python 3.9 virtual environment and are based off of the changes in the Pipfile.

Login to development environment

1. Run the following command to login to EC2

$ awsmfa prod ######
Generating new IAM STS Token ...
STS Session Token generated and saved in profile mfa successfully.
$ microservice_ssm

2. Run the following command to change to the development branch:

cd /home/microservice/branch/GDXDSD-8240_google_search_python39/google_search

Testing instructions:

1. Create the Python 3.9 virtual environment:

/home/microservice/.local/bin/pipenv install --ignore-pipfile

Output:

Creating a virtualenv for this project...
Pipfile: /home/microservice/branch/GDXDSD-8240_google_search_python39/google_search/Pipfile
Using /usr/local/bin/python3.9 (3.9.25) to create virtualenv...
⠇ Creating virtual environment...created virtual environment CPython3.9.25.final.0-64 in 447ms
  creator CPython3Posix(dest=/home/microservice/.local/share/virtualenvs/google_search-KXJvjyfw, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/microservice/.local/share/virtualenv)
    added seed packages: pip==26.0.1, setuptools==82.0.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/microservice/.local/share/virtualenvs/google_search-KXJvjyfw
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Installing dependencies from Pipfile.lock (d9836a)...

2. Run the following command to test the microservice with python 3.9

/home/microservice/.local/bin/pipenv run python3.9 google_search.py -o credentials_search.json -a credentials_search.dat -c config.d/GDXDSD-8240_config_search.json 

Output:

Report: /home/microservice/branch/GDXDSD-8240_google_search_python39/google_search/google_search.py

Config: config.d/GDXDSD-8240_config_search.json

Google Search derived table build not attempted
Microservice started at: 2026-05-27 10:41:45-0700 (PDT), ended at: 2026-05-27 10:41:45-0700 (PDT), elapsing: 0:00:00.000044.

Sites to process: 2
Successful API calls: 2
Failed API calls: 0
Failed loads to RedShift: 0


Objects loaded to S3 and copied to RedShift:
1: s3://sp-ca-bc-gov-131565110619-12-microservices/client/doug_test/GDXDSD-8240/google_gdx/googlesearch-hcapparticipants.gov.bc.ca-2026-05-23-2026-05-25.csv
2: s3://sp-ca-bc-gov-131565110619-12-microservices/client/doug_test/GDXDSD-8240/google_gdx/googlesearch-healthymindsbc.gov.bc.ca-2026-05-23-2026-05-25.csv

3. Check to make sure the data was loaded into the test table successfully

$ microservice_rs
snowplow=> select count(*) from test.gdxdsd8240_googlesearch;
 count 
-------
   850
(1 row)
snowplow=>

4. Truncate the test table in preparation for the crontab test

snowplow=> truncate table test.gdxdsd8240_googlesearch;
TRUNCATE TABLE and COMMIT TRANSACTION
snowplow=> \q
$

5. Delete the objects out of S3 client: https://ca-central-1.console.aws.amazon.com/s3/buckets/sp-ca-bc-gov-131565110619-12-microservices?region=ca-central-1&prefix=client/doug_test/GDXDSD-8240/google_gdx/&showversions=false
6. Schedule the following with an appropriately updated timing in crontab to test the microservice with python 3.9:

15 17 27 5 * cd /home/microservice/branch/GDXDSD-8240_google_search_python39/google_search/ && /home/microservice/.local/bin/pipenv run python3.9 google_search.py -o credentials_search.json -a credentials_search.dat -c config.d/GDXDSD-8240_config_search.json | $CUSTOM_SUBJECT "ONE TIME RUN - GDXDSD-8240 - google_search - `date`" $MAILTO

Tip: the job can be found easier by searching for "GDXDSD-8240"

7. Check to make sure the data was loaded into the test table successfully

$ microservice_rs
snowplow=> select count(*) from test.gdxdsd8240_googlesearch;
 count 
-------
   850
(1 row)
snowplow=>

8. Truncate the test table

snowplow=> truncate table test.gdxdsd8240_googlesearch;
TRUNCATE TABLE and COMMIT TRANSACTION
snowplow=> \q
$

[ozdemirozcelik]

works as expected