Latent space documentation by punkduckable · Pull Request #15 · llnl/GPLaSDI

punkduckable · 2024-10-24T15:34:21Z

I added documentation (comments + doc strings + type hints + formatting) to latent_space.py

The code is functionally almost identical, save for a few minor tweaks:

Removed the MultiHeadAttention activation function option. This isn't really an activation function, even though it is listed as such in the torch api. It is used to implement an attention layer in a transformer block. It introduces a whole new set of parameters (for the key, query, and value weight matrices) and is really designed to map finite sequences to finite sequences. I do not think it makes sense to include this in the MLP class as an activation option, so I removed it.

Removed apply_attention. Since there is no more multiheadedattention activation option, this function should be removed.

Otherwise, all changes are cosmetic (e.g., comments + formatting).

I also fixed a small bug in gplasdi.py: np.Inf no longer exists. It is deprecated. It has been replaced with np.inf. I fixed this.

dreamer2368 · 2024-10-24T16:35:10Z

This PR will be rebased and merged after PR #11.
There are code conflicts in PR, which happens after other PRs get merged to main branch. Since this is already based on sphinx branch (PR #11), technically it does not have a conflict, only that github does not automatically resolve the past commits. Once PR #11 is merged, rebasing the last 3 commits from the (merged) main branch would not require any editing.

dreamer2368 · 2024-10-28T19:53:16Z

@punkduckable , you did not resolve any conflict while you're rebasing. See there are bunch of conflict messages everywhere in the code.

This won't happen if you rebase the branch only with your commits. You had to do the following git commands:

git checkout latent_space_documentation
# interactive rebasing. pick only the last 3 commits, drop everything else.
git rebase -i main latent_space_documentation

But since you already rebased without resolving, rebasing may still leave those unresolved conflict messages, which needs to be resolved manually. If you need help in this overall, I can help you in a web-ex.

dreamer2368 · 2024-10-29T15:55:13Z

+    combination and then return a list housing the latent space ICs.
+
+
+    -----------------------------------------------------------------------------------------------


I think this format is not recognized by sphinx and makes param_grid description hidden in the API website (see this webpage).

Remove the line 51 and reduce line 53 the same as line 52. and writing line 55-60 as below will resolve the issue:

Arguments --------- param_grid: :obj:`numpy.ndarray` A 2d numpy.ndarray object of shape (number of parameter combination) x (number of parameters). ...

It'd be great if you can also specify type, but I don't think it's necessary.

I thought I was following numpy style in src/lasdi/timing.py, but it was actually google style. Since your comment is already similar to numpy, we can stick to numpy style. I'll fix src/lasdi/timing.py in future.

dreamer2368 · 2024-10-29T15:56:09Z

+    autoencoder: The actual autoencoder object that we use to map the ICs into the latent space.
+
+
+    -----------------------------------------------------------------------------------------------


Same as above, change line 63-66 as below:

Returns ------- Z0 : :obj:`numpy.ndarray` A list of numpy ndarray objects whose i'th element holds the latent space initial condition ...

I also removed the multheadedattention activation type (plus the associated apply_attention function) because they did not make sense in this class. I have now documented the MLP class, but need to add documentation to the autoencoder class.

I added comments + doc strings to the Autoencoder class.

gplasdi had an instance of np.Inf (which is depricated). I also fixed a typo in latent_spaces.py

dreamer2368 · 2024-10-29T16:04:09Z

+
+
+# -------------------------------------------------------------------------------------------------
+# initial_conditions_latent function


In order to be parsed in API document as one-line description, this line can be moved to line 43 with an additional empty line. Something like below:

def initial_condition_latent(param_grid : np.ndarray, physics : Physics, autoencoder : torch.nn.Module) -> list[np.ndarray]: """initial_conditions_latent function This function maps a set of initial conditions for the fom to initial conditions for the

dreamer2368 · 2024-10-29T16:06:31Z

+
+
+# -------------------------------------------------------------------------------------------------
+# MLP class


Similar to initial_condition_latent. Move line 99 below line 102 with """ """ comment as below:

class MultiLayerPerceptron(torch.nn.Module): """ MLP class """ def __init__( self,

dreamer2368 · 2024-10-29T16:07:36Z

-            self.fcs += [torch.nn.Linear(layer_sizes[k], layer_sizes[k + 1])]
-        self.fcs = torch.nn.ModuleList(self.fcs)
-        self.init_weight()
+        -------------------------------------------------------------------------------------------


Same as above for function argument.

dreamer2368 · 2024-10-29T16:12:48Z

+        below the threshold. 
+
+
+        -------------------------------------------------------------------------------------------


Same as above for returns.

dreamer2368 · 2024-10-29T16:32:36Z

+        """
+
+        # Run checks.
+        # Make sure the reshape index is either 0 (input to 1st layer) or -1 (output of last 


I don't think we need this long explanation for two lines 172-173. Line 160-163 is simply repeating what the code is doing. I suggest:

If the reshape index and shape are provided for input (index 0) or output (index -1), we will either flatten the input ndarray or reshape the output 1d array into the provided ndarray shape. Either way, the specified ndarray shape (reshape_shape) should match the size of input/output layer.

dreamer2368 · 2024-10-29T16:35:38Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above for arguments.

dreamer2368 · 2024-10-29T16:35:51Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above for returns.

dreamer2368 · 2024-10-29T16:44:25Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above for arguments.

dreamer2368 · 2024-10-29T16:44:39Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above for returns.

dreamer2368 · 2024-10-29T16:46:26Z

+
+
+# -------------------------------------------------------------------------------------------------
+# Autoencoder class


Same as for MLP class, move line 305 below line 308 with """ ... """ comment.

dreamer2368 · 2024-10-29T16:50:57Z

-        self.use_multihead = False
+        super(MultiLayerPerceptron, self).__init__()
+
+        # Note that layer_sizes specifies the dimensionality of the domains and co-domains of each


This comment describes a member variable, but it is not parsed by API document. This can be parsed if the comment is written after the code with """ ... """ comment as below:

self.n_layers : int = len(layer_sizes) - 1; """ Note that layer_sizes specifies the dimensionality of the domains and co-domains of each layer. Specifically, the i'th element specifies the input dimension of the i'th layer, ... """

dreamer2368 · 2024-10-29T16:53:08Z

+        # while the final element specifies the dimensionality of the co-domain of the final layer.
+        # Thus, the number of layers is one less than the length of layer_sizes.
+        self.n_layers       : int                   = len(layer_sizes) - 1;
+        self.layer_sizes    : list[int]             = layer_sizes


It'd be great if you can add description for each class member variables, but it's not a must.

dreamer2368 · 2024-10-29T18:19:09Z

+        # layer.
        if (self.reshape_index == 0):
-            # make sure the input has a proper shape
+            # Make sure the input has a proper shape. There is a lot going on in this line; let's


I think most of these comments are somewhat repetitive of what is written earlier in arguments/returns section. I think we can reduced it into one sentence as below:

For k-dimension self.reshape_shape, make sure the last k dimension of x has the same shape.

dreamer2368 · 2024-10-29T18:21:16Z

-            # we use torch.Tensor.view instead of torch.Tensor.reshape,
-            # in order to avoid data copying.
+
+            # Now that we know the final k dimensions of x have the correct shape, let's squeeze 


Same as above, how about this sentence?

For k-dimension self.reshape_shape, reshape and flatten the last k dimension of x into 1d array that fits the first layer. Note that we use torch.Tensor.view instead of torch.Tensor.reshape in order to avoid data copying.

dreamer2368 · 2024-10-29T18:25:43Z

        if (self.reshape_index == -1):
-            # we use torch.Tensor.view instead of torch.Tensor.reshape,
-            # in order to avoid data copying.
+            # In this case, we need to split the last dimension of x, the output of the final


Maybe use key words only?

split the last dimension of x, the output of the final layer, to match the reshape_shape. Note that we use torch.Tensor.view instead of torch.Tensor.reshape in order to avoid data copying.

dreamer2368 · 2024-10-29T19:19:38Z

+
 class Autoencoder(torch.nn.Module):
+    def __init__(self, physics : Physics, config : dict) -> None:
+        """


Currently this is somehow not parsed by API. I found that we need to turn on an option. We can just leave as is and wait until I push a new PR for it, or it'd be great if you can add one liner in line 28 of docs/source/conf.py:

autoapi_python_class_content = 'both'

dreamer2368 · 2024-10-29T19:20:49Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above for arguments

dreamer2368 · 2024-10-29T19:22:28Z

+
+        physics: A "Physics" object that holds the fom solution frames. We use this object to 
+        determine the shape of each fom solution frame. Recall that each Physics object has a 
+        corresponding PDE. We 


Is there a sentence missed here at the end?

dreamer2368 · 2024-10-29T19:23:26Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above for Returns.

dreamer2368 · 2024-10-29T19:26:34Z

+        # each point).
+        self.qgrid_size : list[int]     = physics.qgrid_size;
+
+        # The product of the elements of qgrid_size is the number of dimensions in each fom 


Move these comments after 364 with """ ... """ comment so that it can be parsed into API.

dreamer2368 · 2024-10-29T19:31:25Z

-        self.decoder = MultiLayerPerceptron(layer_sizes[::-1], act_type,
-                                            reshape_index=-1, reshape_shape=self.qgrid_size,
-                                            threshold=threshold, value=value, num_heads=num_heads)
+        # A Physics object's qgrid_size is a list of integers specifying the shape of each frame of 


Move these comments after 364 with """ ... """ comment so that it can be parsed into API.

I personally think these belong to Physics class comments, but we can leave it here. It'd be great if we can refer readers to Physics class.

dreamer2368 · 2024-10-29T19:32:00Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above for arguments

dreamer2368 · 2024-10-29T19:32:27Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above for returns

dreamer2368 · 2024-10-29T19:32:39Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above

dreamer2368 · 2024-10-29T19:32:44Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above

dreamer2368 · 2024-10-29T19:33:09Z

+
+
+        -------------------------------------------------------------------------------------------
+        Arguments


Same as above

dreamer2368 · 2024-10-29T19:33:17Z

+
+
+        -------------------------------------------------------------------------------------------
+        Returns


Same as above

dreamer2368

@punkduckable , I reviewed through the comments. Most of them look good, just need some format changes to be parsed into API.

I thought I commented src/lasdi/timing.py per numpy style, but it was google style. I'll change this later.

punkduckable requested review from chldkdtn and dreamer2368 October 24, 2024 15:34

dreamer2368 mentioned this pull request Oct 24, 2024

Modularizing encoder/decoder in Autoencoder class #10

Merged

punkduckable force-pushed the latent_space_documentation branch from 773df78 to ddbac34 Compare October 28, 2024 19:26

dreamer2368 reviewed Oct 29, 2024

View reviewed changes

Robert Stephany added 3 commits October 29, 2024 08:58

Started documenting latent_space.py

78c328b

I also removed the multheadedattention activation type (plus the associated apply_attention function) because they did not make sense in this class. I have now documented the MLP class, but need to add documentation to the autoencoder class.

Finished adding documentation to latent_space.py

4fa50fe

I added comments + doc strings to the Autoencoder class.

Fixed bugs

782f57a

gplasdi had an instance of np.Inf (which is depricated). I also fixed a typo in latent_spaces.py

punkduckable force-pushed the latent_space_documentation branch from cc77c3f to 782f57a Compare October 29, 2024 15:59

dreamer2368 reviewed Oct 29, 2024

View reviewed changes

dreamer2368 mentioned this pull request Oct 29, 2024

Added documentation to gp.py, input.py, and param.py #16

Open

		combination and then return a list housing the latent space ICs.


		-----------------------------------------------------------------------------------------------

		autoencoder: The actual autoencoder object that we use to map the ICs into the latent space.


		-----------------------------------------------------------------------------------------------



		# -------------------------------------------------------------------------------------------------
		# initial_conditions_latent function



		# -------------------------------------------------------------------------------------------------
		# MLP class

		below the threshold.


		-------------------------------------------------------------------------------------------



		-------------------------------------------------------------------------------------------
		Arguments



		-------------------------------------------------------------------------------------------
		Returns



		# -------------------------------------------------------------------------------------------------
		# Autoencoder class

Uh oh!

Conversation

punkduckable commented Oct 24, 2024

Uh oh!

dreamer2368 commented Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dreamer2368 commented Oct 28, 2024

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreamer2368 left a comment

dreamer2368 commented Oct 24, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading

dreamer2368 Oct 29, 2024 •

edited

Loading