Virtual_Environments_Notes.txt

--------------------
Virtual Environments
--------------------

------
Python
------
PipEnv, PyEnv and VirtualEnv

PipEnv, PyEnv and VirtualEnv are very different tools that work in different
ways to do different things.

-------
Summary
-------
PyEnv      (pyenv)      - Use to install different versions of Python
PipEnv     (pipenv)     - Use in place of pip to install/lock packages
VirtualEnv (virtualenv) - Used by PipEnv

-------------
Documentation
-------------
[PyEnv: Simple Python Version Management](https://github.com/pyenv/pyenv)
[Pipenv: Python Development Workflow for Humans](https://pypi.org/project/pipenv/)
[Pipenv: A Guide to the New Python Packaging Tool](https://realpython.com/pipenv-guide/)
[VirtualEnv: Virtual Python Environment builder](https://pypi.org/project/virtualenv/)
[Pipenv & Virtual Environments](https://docs.python-guide.org/dev/virtualenvs/)


-----
PyEnv
-----
PyEnv is a bash extension - will not work on Windows - that intercepts your
calls to python, pip, etc., to direct them to one of several of the system
python tool-chains. So you always have all the libraries that you have
installed in the selected python version available - as such it is good for
users who have to switch between different versions of python.

------ ------ ------ ------ ------ ------
[pyenv](https://github.com/pyenv/pyenv)
------ ------ ------ ------ ------ ------
is used to isolate Python versions. For example, you may want to test your
code against Python 2.6, 2.7, 3.3, 3.4 and 3.5, so you'll need a way to switch
between them. Once activated, it prefixes the PATH environment variable with
~/.pyenv/shims, where there are special files matching the Python commands
(python, pip). These are not copies of the Python-shipped commands; they are
special scripts that decide on the fly which version of Python to run based on
the PYENV_VERSION environment variable, or the .python-version file, or the
~/.pyenv/version file. pyenv also makes the process of downloading and
installing multiple Python versions easier, using the command pyenv install.

pyenv: Installation
-------------------
# get it
  # (MAC OS)
    $ brew install pyenv
    $ brew install pyenv-virtualenv
  # (non MAC OS)
    $ git clone https://github.com/pyenv/pyenv ~/repos/pyenv
# link it (non MAC OS)
  # (non MAC OS)
  $ ln -s ~/repos/pyenv ~/.pyenv
# add pyenv bin to PATH (non MAC OS)
  $ vi ~/.bash_profile
  #-----
   pyenv_repo=$HOME/repos/pyenv
   if [ -d $pyenv_repo ]; then
      export PYENV_ROOT=$pyenv_repo
      pyenv_bin=$PYENV_ROOT/bin
      [[ -d $pyenv_bin && ! $PATH =~ ^$pyenv_bin:|:$pyenv_bin:|:$pyenv_bin$ ]] \
         && export PATH="$pyenv_bin:$PATH"
   fi
  #-----
# add `pyenv init` to shell to enable shims and autcompletion
# (place at the end of ~/.bash_profile)
  $ vi ~/.bash_profile
  #-----
  [ $(command -v pyenv) ] && eval "$(pyenv init -)"
  #-----
# restart the shell
  $ source ~/.bash_profile   # OR $ exec "$SHELL"
# install desired Python versions
  $ pyenv install 3.8.2
# use `pyenv` to set/use the version of Python desired
  $ pyenv global|local|shell VERSION [VERSION2 VERSION3...]
# more help
  $ pyenv help   # OR pyenv commands

pyenv: Usage
-------------------
$ pyenv versions                 # show python versions & virtual environments
$ pyenv install VERSION          # install version of python
$ pyenv virtualenv VERSION NAME  # create virtual env NAME with python VERSION
$ pyenv activate NAME            # use virtual env NAME
$ pyenv deactivate               # stop using virtual env NAME
$ pyenv uninstall NAME|VERSION   # uninstall specific virtual env/python version
$ pyenv virtualenvs              # list virtual environments
$ pyenv virtualenv-delete NAME   # uninstall specific virtual environment

------
PipEnv
------
by Kenneth Reitz (the author of requests), is the newest project in this list.
It aims to combine Pipfile, pip and virtualenv into one command on the
command-line. The virtualenv directory typically gets placed in
~/.local/share/virtualenvs/XXX, with XXX being a hash of the path of the
project directory. This is different from virtualenv, where the directory is
typically in the current working directory.

------ ------ ------ ------ ------ ------
pipenv  (pypi.python.org/pypi/pipenv) [realpython.com/pipenv-guide]
------ ------ ------ ------ ------ ------
The Python Packaging Guide recommends pipenv when developing Python
applications (as opposed to libraries). There does not seem to be any plans to
support venv instead of virtualenv (#15). Confusingly, its command-line
option --venv refers to the virtualenv directory, not venv, and similarly, the
environment variable PIPENV_VENV_IN_PROJECT affects the location of the
virtualenv directory, not venv directory (#1919).

pipenv: Installation/Use
------------------------
$ brew install pipenv

  OR

$ pip install [--user] pipenv       # install pipenv
$ pipenv --python VERSION           # specify which verson of Python to use (creates env)
$ pipenv shell                      # creates a virtual environment
$ pipenv install PACKAGE[==VERSION] # creates/updates Pipfile and Pipfile.lock
$ pipenv install DEV_PACKAGE --dev  # install dev package
$ pipenv lock                       # creates/updates Pipfile.lock
$ pipenv lock -r > requirements.txt # creates requirements.txt output/file
$ pipenv install --ignore-pipfile   # use Pipfile.lock to install
$ pipenv install --dev              # install depedencies needed for development
$ pipenv open PACKAGE               # open the PACKAGE in the default editor
$ pipenv run COMMAND                # run a command in the virtual environment
$ pipenv check                      # check for security vulnerabilities
$ pipenv uninstall PACKAGE          # uninstall a package, updating Pipfile(s)
$ pipenv --venv                     # where is the virtual environment
$ pipenv --where                    # where is the project home

pipenv: Workflow
------------------------
- Set up new environment (from scratch - create Pipfiles)
   - use `pyenv` to install/select desired python version
   - use `pipenv --python VERSION` to set the project's desired python version
   - install packages (see below)
- Set up new environment (from existing Pipfiles)
   - use `pyenv` to install/select specified python version in Pipfile
   - use `pipenv sync --dev` to install from Pipfile.lock
      - if a package does not install, use
        `pipenv install [--dev] PACKAGE` (using specs defined in Pipfile)
      - if a package still does not install, use
        `pipenv install [--dev] PACKAGE --verbose` (checking for conflicts/issues)
- Installing packages
   - Sometime installation seems to hang, try hitting [enter] a couple of times
   - If installing from private repos, set `PIP_EXTRA_INDEX_URL` with credentials
   - Running install verbosely and using `pipenv graph` is pretty useful for
     resolving dependency hell
   - Method 1
      - install with `pipenv install [--dev] PACKAGE[(==|~=|>|<|>=|<=)VERSION[,(<|<=)VERSION]]`, e.g.
        pipenv install PACKAGE                          # any version
	pipenv install PACKAGE~=MAJ.MIN.PATCH           # any maj.min version (patch can vary)
	pipenv install PACKAGE~=MAJ.MIN                 # any major version (minor.patch can vary)
	pipenv install PACKAGE==MAJ.MIN.PATCH           # exact version
	pipenv install PACKAGE<=MAJ.MIN.PATCH           # capped version
	pipenv install PACKAGE>=MAJ.MIN.PATCH,<MAJ.MIN  # ranged version
      - `pipenv install` automatically updates the Pipfle.lock file
   - Method 2
      - edit Pipfile
      - update Pipfile.lock with `pipenv lock [--dev]`
      - install with `pipenv sync [--dev]`
      - if a package does not install, use Method 1 using specs defined in Pipfile
      - if the package still does not install, use the `--verbose` option with
        the install command and check for version/dependency conflicts or any
	other issues


Private Repo access
-------------------
- Example of Pipfile entry for private Azure DevOps Artifacts (repo)

   # File: Pipfile
   [[source]]
   name = "pypi-private"
   url = "https://pkgs.dev.azure.com/COMPANY/PROJECT/_packaging/FEED/pypi/simple"
   verify_ssl = true

   [packages]
   cool_company_package = {version = "==1.2.3", index = "pypi-private"}

- Get ADO token

   $ ADO_ARTIFACTS_TOKEN=$(keyring -b artifacts_keyring.ArtifactsKeyringBackend get https://pkgs.dev.azure.com/$ADO_COMPANY/$ADO_PROJECT/_packaging/$ADO_FEED/pypi/simple/ VssSessionToken 2> /dev/null)

- Set PIP_EXTRA_INDEX_URL

   $ export PIP_EXTRA_INDEX_URL="https://any_user:$ADO_ARTIFACTS_TOKEN@pkgs.dev.azure.com/$ADO_COMPANY/$ADO_PROJECT/_packaging/$ADO_FEED/pypi/simple"

pipenv: Automatic Loading of `.env`
------------------------
https://pipenv.pypa.io/en/latest/advanced/#automatic-loading-of-env

If a `.env` file is present in your project, `pipenv shell` and `pipenv run`
will automatically load it, for you:

pipenv: Adanced Usage of Pipenv
-------------------------------
https://pipenv.pypa.io/en/latest/advanced/

The use of ~= is preferred over the == identifier as
the latter prevents pipenv from updating the packages:

$ pipenv install "PACKAGE~=major[.minor[.micro]]"
$ pipenv update [PACKAGE]

For example
# locks the major version of the package
(this is equivalent to using ==2.*)
$ pipenv install "requests~=2.2"

----------
VirtualEnv
----------
http://docs.python-guide.org/en/latest/dev/virtualenvs/

virtualenv (https://pypi.python.org/pypi/virtualenv) is a tool to create
isolated Python environments. virtualenv creates a folder which contains all
the necessary executables to use the packages that a Python project would need.

VirtualEnv is pure python so works everywhere, it makes a copy of, optionally
as specific version of, python and pip local to the activate environment which
may or may not include links to the current system tool-chain, if it does not
you can install just a known subset of libraries into that environment. As such
it is almost certainly much better for testing and deployment as you know
exactly which libraries, at which versions, are used and a global change will
not impact your module.

------ ------ ------ ------ ------ ------
virtualenv (https://pypi.python.org/pypi/virtualenv)
------ ------ ------ ------ ------ ------
is a very popular tool that creates isolated Python environments for Python
libraries. If you're not familiar with this tool, I highly recommend learning
it, as it is a very useful tool, and I'll be making comparisons to it for the
rest of this answer.

It works by installing a bunch of files in a directory (eg: env/), and then
modifying the PATH environment variable to prefix it with a custom bin
directory (eg: env/bin/). An exact copy of the python or python3 binary is
placed in this directory, but Python is programmed to look for libraries
relative to its path first, in the environment directory. It's not part of
Python's standard library, but is officially blessed by the PyPA (Python
Packaging Authority). Once activated, you can install packages in the virtual
environment using pip.

virtualenv: Installation
------------------------
Install virtualenv via pip:

$ pip install virtualenv

Basic Usage
-----------
# In summary

$ cd PROJECT_FOLDER
$ virtualenv VIRTUAL_ENV_NAME                        # create virtual environment
$ virtualenv -p /usr/bin/python2.7 VIRTUAL_ENV_NAME  # set the python version
$ source ./VIRTUAL_ENV_NAME/bin/activate             # use virtual env
$ pip install -r requirements.txt                    # install packages
$ deactivate                                         # return to system environment

# In more detail

# Create a virtual env
$ cd PROJECT_FOLDER
$ virtualenv VIRTUAL_ENV_NAME	# creates a folder in the current dir

# to use a Python interpreter of your choice
$ virtualenv -p /usr/bin/python2.7 VIRTUAL_ENV_NAME

# or change globally in ~/.bashrc
$ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.7

# begin using the virtual environment
$ source /PATH/TO/PROJECT_FOLDER/VIRTUAL_ENV_NAME/bin/activate

# install packages as usual, e.g.
$ pip install PACKAGE

# when done, and to return to system's default environment
$ deactivate

# to disable the evironment in the prompt
VIRTUAL_ENV_DISABLE_PROMPT=YES


----------------
Additional Tools
----------------

There are a number of tools that it is worth mentioning, and considering, as
they can help with the use of one or more of the above:

VirtualEnvWrapper (https://pypi.python.org/pypi/virtualenvwrapper)
-----------------
VirtualEnvWrapper Manage and simplify the use and management of
VirtualEnv - Cross Platform.

pyenv-virtualenv (https://github.com/yyuu/pyenv-virtualenv)
----------------
pyenv-virtualenv, installed by pyenv-installer
(https://github.com/yyuu/pyenv-installer), which gives PyEnv tools for managing
and interfacing to VirtualEnv - with this you can have a base installation that
includes more than one version of python and create isolated environments
within each of them - Linux Only. Suggested by Johann Visagie

PyInstaller (http://www.pyinstaller.org/)
-----------
PyInstaller can take your python code, possibly developed & tested under
VirtualEnv, and bundle it up so that it can run one platforms that do not have
your version of python installed - Note that it is not a cross compiler you
will need a Windows (virtual-)machine to build Windows installs, etc., but it
can be handy even where you can be sure that python will be installed but
cannot be sure that the version of python and all the libraries will be
compatible with your code.

Environments:
------------

Ansible_1.x
-----------
VIRTUAL_ENV=/usr/local/home/praco/envs/Ansible_1.x
PATH=/usr/local/home/praco/envs/Ansible_1.x/bin:/home/praco/repos/pyenv/shims:
   /home/praco/repos/pyenv/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:
   /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:
   /sbin:/home/praco/repos/phacility/arcanist/bin

$ sudo yum install libffi-devel		# NOT NEEDED (?)
$ sudo pip install -U \
     ansible==1.9.1 credstash==1.3.1 httplib2==0.9.2 awscli==1.9.12 \
     boto==2.38.0 botocore==1.3.12
$ sudo pip install --ignore-installed PyCrypto==2.6.1

ansible (1.9.1)
awscli (1.9.12)
boto (2.38.0)
botocore (1.3.12)
credstash (1.3.1)
httplib2 (0.9.2)
pycrypto (2.6.1)

Ansible_2.x
-----------
VIRTUAL_ENV=/usr/local/home/praco/envs/Ansible_2.x
PATH=/usr/local/home/praco/envs/Ansible_2.x/bin:/home/praco/repos/pyenv/shims:
   /home/praco/repos/pyenv/shims:/home/praco/repos/pyenv/bin:
   /usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:
   /bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:
   /home/praco/repos/phacility/arcanist/bin

ansible (2.1.1.0)
ansible-playbook-debugger (0.2.4)
ansible-role-manager (0.4)
awscli (1.10.62)
boto (2.42.0)
boto3 (1.4.0)
botocore (1.4.52)
credstash (1.11.0)
pycrypto (2.6.1)

Ansible_2.2
-----------
VIRTUAL_ENV=/usr/local/home/praco/envs/Ansible_2.2

vmedix
-----------
- VIRTUAL_ENV=/usr/local/home/praco/envs/vmedix
- python 2.7.9
- pip list
- --------
   ansible (2.2.0.0)
   boto (2.40.0)
   boto3 (1.4.4)
   credstash (1.11.0)

Node tools
----------
nvm - version manager
   - brew install nvm
   - following instructions from brew
   - nvm --version | --help
npm - package manager
   - nvm install VERSION
   - nvm alias default VERSION
   - nvm use VERSION | system | default
   - nvm ls
   - node version
npx - npm package runner

tags: pipenv, pyenv, venv, virtualenv, nvm