Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 27 additions & 22 deletions act/discovery/nasa.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,15 @@ def download_mplnet_data(
https://mplnet.gsfc.nasa.gov/product-info/
Level 1 data should never be used for publication.
product : str
MPLNet Product (NRB, CLD, PBL, AER).
NRB - Lidar signals; volume depolarization ratos, diagnostics
CLD - Cloud Heights, thin cloud extinction and optical depths, cloud
phase
AER - Aerosol heights; extinction, backscatter, and aerosol
depolarization ratio profiles; lidar ratio
PBL - Surface-Attached Mixed Layer Top and estimated mixed layer AOD
MPLNet Product (NRB, CLD, PBL, AER).

* NRB - Lidar signals; volume depolarization ratios, diagnostics
* CLD - Cloud Heights, thin cloud extinction and optical depths, cloud
phase
* AER - Aerosol heights; extinction, backscatter, and aerosol
depolarization ratio profiles; lidar ratio
* PBL - Surface-Attached Mixed Layer Top and estimated mixed layer AOD

Information on the MPLNet Products can be found here:
https://mplnet.gsfc.nasa.gov/product-info/
year : str
Expand Down Expand Up @@ -177,20 +179,21 @@ def get_mplnet_meta(
Returns a list of meta data from the NASA MPL Network Data
https://mplnet.gsfc.nasa.gov/mplnet_web_services.cgi?metadata


Parameters
----------
sites : str
How to return MPLNET Site Information
all - produces output on all sites (active and inactive)
active - produces output file containing only active sites
(if year, month, or day are not set then uses today's date)
inactive - produces output file containing only inactive sites
(if year, month, or day are not set then uses today's date)
planned - produces output file containing only planned sites
site_name - produces output file containing only requested site
collection - produces output file containing sites in pre-defined
collections (e.g. field campaigns or regions)

* all - produces output on all sites (active and inactive)
* active - produces output file containing only active sites
(if year, month, or day are not set then uses today's date)
* inactive - produces output file containing only inactive sites
(if year, month, or day are not set then uses today's date)
* planned - produces output file containing only planned sites
* site_name - produces output file containing only requested site
* collection - produces output file containing sites in pre-defined
collections (e.g. field campaigns or regions)

year : str
Four digit Year for desired product download (YYYY).
Note Level 1 and 1.5 products are available for
Expand All @@ -205,11 +208,13 @@ def get_mplnet_meta(
in a zip file.
method : str
Method for returning JSON list of MPLNET GALION format parameters.
station - returns GALION JSON with only station and PI contact info
data - return GALION JSON with data elements, station, date and PI
contact information
print_to_screen : Boolean
If true, print MPLNET site identifiers to screen

* station - returns GALION JSON with only station and PI contact info
* data - returns GALION JSON with data elements, station, date and
PI contact information

print_to_screen : bool
If true, print MPLNET site identifiers to screen.
"""

# Add headers
Expand Down
25 changes: 12 additions & 13 deletions act/io/ameriflux.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ def read_ameriflux(
A dictionary containing current variable names and units to be added to that variable.
Default is None and uses current units from Ameriflux's unit database.


Returns
-------
ds : xarray.Dataset
Expand Down Expand Up @@ -462,40 +461,40 @@ def convert_to_ameriflux(
**kwargs,
):
"""

Returns `xarray.Dataset` with stored data and metadata from a user-defined
query of ARM-standard netCDF files from a single datastream. Has some procedures
to ensure time is correctly fomatted in returned Dataset.
to ensure time is correctly formatted in returned Dataset.

Parameters
----------
ds : xarray.Dataset
Dataset of data to convert to AmeriFlux format
variable_mapping : dict
Dictionary of variables mappings. The key should be the name of the variable
Dictionary of variables mappings. The key should be the name of the variable
in the Dataset with the values being dictionaries of the AmeriFlux name and units.
For example:
For example::

var_mapping = {
'co2_flux': {'name': 'FC', 'units': 'umol/(m^2 s)'},
}

soil_mapping : dict
Dictionary of soil variables mappings following the same formatting as variable_mapping.
It is understood that the AmeriFlux name may be the same for some variables. This
It is understood that the AmeriFlux name may be the same for some variables. This
script attempts to automatically name these measurements. If a variable is not dimensioned
by a depth nor has a sensor_height attribute, it will automatically assume that it's
at the first depth in the depth_profile variable.
depth_profile : list
List of depths that the variables will be mapped to. If a depth is not in this list,
List of depths that the variables will be mapped to. If a depth is not in this list,
the index chosen will be the one closest to the depth value.
include_missing_variables : boolean
If there variables that are completely missing (-9999) chose whether or not to include
them in the DataFrame.
include_missing_variables : bool
If there are variables that are completely missing (-9999) choose whether or not to
include them in the DataFrame.

Returns
-------
df : pandas.DataFrame (or None)
Returns a pandas dataframe for easy writing to csv

df : pandas.DataFrame or None
Returns a pandas dataframe for easy writing to csv.
"""
# Use ARM variable mappings if none provided
if variable_mapping is None:
Expand Down
30 changes: 17 additions & 13 deletions act/qc/bsrn_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -446,30 +446,34 @@ def bsrn_comparison_tests(
Value used in longwave down less than longwave up test.
LWdn_gt_LWup_component : int or float
Value used in longwave down greater than longwave up test.
use_dask : boolean
use_dask : bool
Option to use Dask for processing if data is stored in a Dask array

References
----------
Long, Charles N., and Ellsworth G. Dutton. "BSRN Global Network recommended QC tests, V2. x." (2010).
Long, Charles N., and Y. Shi. “An Automated Quality Assessment and Control Algorithm for Surface
Radiation Measurements.” (2008) https://doi.org/10.2174/1874282300802010023
Radiation Measurements.” (2008) https://doi.org/10.2174/1874282300802010023

Examples
--------
.. code-block:: python

ds = act.io.arm.read_arm_netcdf(act.tests.EXAMPLE_BRS, cleanup_qc=True)
ds.qcfilter.bsrn_comparison_tests(
gbl_SW_dn_name='down_short_hemisp',
glb_diffuse_SW_dn_name='down_short_diffuse_hemisp',
direct_normal_SW_dn_name='short_direct_normal',
glb_SW_up_name='up_short_hemisp',
glb_LW_dn_name='down_long_hemisp_shaded',
glb_LW_up_name='up_long_hemisp',
use_dask=True)
"""
An example on how to add BSRN comparison tests:

.. code-block:: python

ds = act.io.arm.read_arm_netcdf(act.tests.EXAMPLE_BRS, cleanup_qc=True)
ds.qcfilter.bsrn_comparison_tests(
gbl_SW_dn_name='down_short_hemisp',
glb_diffuse_SW_dn_name='down_short_diffuse_hemisp',
direct_normal_SW_dn_name='short_direct_normal',
glb_SW_up_name='up_short_hemisp',
glb_LW_dn_name='down_long_hemisp_shaded',
glb_LW_up_name='up_long_hemisp',
use_dask=True
)

"""
if isinstance(test, str):
test = [test]

Expand Down
9 changes: 4 additions & 5 deletions act/qc/clean.py
Original file line number Diff line number Diff line change
Expand Up @@ -939,16 +939,15 @@ def clean_cf_qc(self, variables=None, sep='__', **kwargs):
def fix_incorrect_variable_bit_description_attributes(self):
"""
Method to correct incorrectly defined quality control variable attributes.
There are some datastreams with the attribute names incorrectly having 'qc_'
prepended to the attribute name. This will fix those attributes so the cleanqc
There are some datastreams with the attribute names incorrectly having ``qc_``
prepended to the attribute name. This will fix those attributes so the ``cleanqc``
method can correctly read the attributes.

If the variable long_name starts with the string "Quality check results on"
and a variable attribute follows the pattern qc_bit_#_description the 'qc_' part of
the variable attribute will be removed.
and a variable attribute follows the pattern ``qc_bit_#_description`` the ``qc_``
part of the variable attribute will be removed.

"""

attr_description_pattern = r'^qc_bit_([0-9]+)_description$'
attr_assessment_pattern = r'^qc_bit_([0-9]+)_assessment$'

Expand Down
28 changes: 14 additions & 14 deletions act/qc/qctests.py
Original file line number Diff line number Diff line change
Expand Up @@ -1781,42 +1781,42 @@ def add_relative_variability_test(
):
"""
Method to calculate the variability of a variable relative to
other variables that are passed in. The process includes
1.) Calculating a rolling standard deviation for all variables passed in
2.) For each variable it will subtract the mean standard deviation of all
the other variables
3.) Variability relative to the mean greater than the threshold passed in
will be flagged.
other variables that are passed in. The process includes:

1. Calculating a rolling standard deviation for all variables passed in.
2. For each variable it will subtract the mean standard deviation of all
the other variables.
3. Variability relative to the mean greater than the threshold passed in
will be flagged.

Parameters
----------
var_names : list
List of data variable names. At least 2 are required.
List of data variable names. At least 2 are required.
threshold : int or float
Threshold value to use in test.
test_assessment : str
Optional single word describing the assessment of the test.
Will set a default if not set.
test_number : int
Optional test number to use. If not set will ues next
Optional test number to use. If not set will use next
available test number.
flag_value : boolean
flag_value : bool
Indicates that the tests are stored as integers
not bit packed values in quality control variable.
prepend_text : str
Optional text to prepend to the test meaning.
Example is indicate what institution added the test.
Example is to indicate what institution added the test.
window : int or float
Window to calculate rolling standard deviation
Window to calculate rolling standard deviation.
min_periods : int
Minimum number of samples to use for rolling standard deviation

Minimum number of samples to use for rolling standard deviation.

Returns
-------
test_info : tuple
A tuple containing test information including var_name, qc variable name,
test_number, test_meaning, test_assessment
test_number, test_meaning, test_assessment.

"""
test_meaning = 'Standard deviation of variable is greater than the mean standard deviation of other like variables'
Expand Down
4 changes: 2 additions & 2 deletions act/retrievals/radiation.py
Original file line number Diff line number Diff line change
Expand Up @@ -289,10 +289,10 @@ def calculate_irradiance_stats(
Name of the second irradiance variable
diff_output_variable : str
Variable name to store the difference results
Defaults to 'diff_' + variable
Defaults to ``diff_ + variable``
ratio_output_variable : str
Variable name to store the ratio results
Defaults to 'ratio_' + variable
Defaults to ``ratio_ + variable``

Returns
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,6 @@
'python': ('https://docs.python.org/3/', None),
'numpy': ('https://docs.scipy.org/doc/numpy/', None),
'scipy': ('https://docs.scipy.org/doc/scipy/reference/', None),
'pandas': ('https://pandas.pydata.org/pandas-docs/stable', None),
'pandas': ('https://pandas.pydata.org/docs/', None),
'matplotlib': ('https://matplotlib.org', None),
}
2 changes: 1 addition & 1 deletion examples/discovery/plot_improve.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""
IMPROVE Data
-----------
------------

This example shows how to get IMPROVE data for the
system located at ARM's Southern Great Plains site.
Expand Down
Loading