MAP Holons Data Manager
The following diagram depicts the internal architecture of the MAP Holons Data Manager
Notice the separation of the updates (create, delete, update) from read-only queries.
Since everything in MAP is represented as holons (nodes) or relationships (links), MAP data can be viewed as an information graph. As its foundation, the MAP Holons Data Manager relies on Holon References and a Holon Cache. The Holon Cache Manager provides a cache of holon state and manages the retrieval of holons from their local or external HolonSpace. All read access to the local Holons Integrity Zome is mediated by the Holon Cache Manager. The MAP Data Manager supports create, update and delete functions for this information graph via Holon Storage Functions. The Query Engine provides a general graph query execution layer inspired by graph database query languages. The Query Engine relies on the Holons and SmartLinks provided by the MAP Holons Integrity Zome. It also leverages SmartLinks to support filtering, sorting, and pagination of query results. Creation and removal of these links is managed by the SmartLink Manager in response to requests from Holon Storage Functions.
A Holon is an “in-memory” representation of a HolonNode. It wraps the HolonNode, but adds some additional fields to make it easier to work with and also to help manage the lifecycle state of the Holon.
A Holon plays different roles within the architecture.
- It serves as a Builder that allows a new HolonNode to be incrementally populated with property values and related to other holons prior to committing it to the persistent store.
- It also serves as a Builder that allows a previously saved and fetched Holon to be updated incrementally and then committed to the persistent store.
- It serves as an in-memory data store during query processing and a dance provider (i.e., it provides implementation logic for the dances (actions) offered by that HolonType.
The lifecycle state of a Holon is maintained in its state property. The implementation of the MAP Holons Coordinator Zome functions are responsible for maintaining the state of a Holon.
The HolonNode always reflects the latest saved state of that HolonNode. Thus, for Holons that have not yet (ever) been persisted, HolonNode will be None and the holon's state will be New. It retains that state until it is committed. Assuming the commit succeeds, the holon's saved_node will be updated to contain the HolonNode just created. Since the state of the in-memory representation of the Holon now matches that of the saved_node, its state will be set to Fetched. If the Holon is subsequently updated, its state will be set to Changed and it will remain in that state until it is again successfully committed.
The process of creating a new (persistent) HolonNode consists of creating an empty in-memory Holon object, incrementally building up its state, and then committing its state to the persistence tier (i.e., creating its HolonNode instance in its HolonSpace's DHT). Accumulated changes can be committed at any time (provided the Holon is in a valid state). For example, both of the following scenarios are possible:
-
Fine-Grained Commit Example:
- Create an empty Holon (
state = New) - Populate values for its required properties and relationships (
state = New) - Commit the Holon -- (creates the HolonNode and SmartLinks) (
state = Fetched) - Populate a few more property values (
state = Changed) - Commit the Holon -- (updates the HolonNode) (
state = Fetched) - Populate a relationship (
state = Changed) - Commit the Holon -- (adds new SmartLinks) (
state = Fetched) - Populate a relationship (
state = Changed) - Populate a few more property values (
state = Changed) - Commit the Holon -- (updates the HolonNode again and adds new SmartLink) (
state = Fetched)
- Create an empty Holon (
-
Coarse-Grained Commit Example:
- Create an empty Holon (
state = New) - Populate values for its required properties and relationships (
state = New) - Populate a few more property values (
state = New) - Populate a relationship (
state = New) - Populate a relationship (
state = New) - Commit the Holon -- (creates the HolonNode and adds multiple SmartLinks)(
state = Fetched)
- Create an empty Holon (
This flexibility in the commit-model allows a variety of use cases to be supported and offers maximum freedom to human agents to decide their own commit points.
PropertyMaps act like dictionaries that map property names to property values. The representation of a property_map is identical in both the HolonNode and Holon structs, but their contents are only identical while the holon is in a Fetched state. In the New state, there is no saved_node (so there is no HolonNode to have a property_map). In the Changed state, the HolonNode's property_map will reflect the latest saved state of the HolonNode, whereas the Holon's property_map will accumulate the updates performed since the last commit.
Holons can be related to each other via holon relationships. A Holon's outbound relationships (i.e., relationships for which this holon is the source) are accessed via the Holon's RelationshipMap. This is a BTreeMap that maps the relationship name to values for that relationship as represented via the RelationshipTarget enum. The variant selected for a specific outbound relationship depends upon the cardinality of the relationship, as specified in the RelationshipDescriptor for that relationship. NOTE: RelationshipDescriptors for the outbound relationships can be retrieved from the holon's HolonDescriptor.
If the relationship's cardinality is exactly one, (i.e., max_target_cardinality and min_target_cardinality both = 1), the value is represented as a HolonReference.
If the relationship's cardinality is zero or one, (i.e., max_target_cardinality =1 and min_target_cardinality = 0), the value is represented as an Option<HolonReference>.
If the relationship's max_target_cardinality >1, the RelationshipTarget is a HolonCollection to which other holons may be added or removed.
For holons in a Fetched state, adding (or removing) a related holon to the holon's relationship_map (or an associated HolonCollection) will change the holon's state to Changed.
When a holon is committed, the relationships in its relationship_map will be written to the persistence tier as SmartLinks via the SmartLink Manager component.
HolonReferences are used to refer to local or external holons.
Target holons that are in the same HolonSpace as the source holon can be identified by just their HolonId. Target holons in a different HolonSpace require both a proxy_id and an external_holon_id. The HolonReference enum encapsulates both types, with variants mapping to the different types of references. Note that both LocalHolonReference and ExternalHolonReference include a holon_ref: Option<&Holon> field. The population of the HolonReference fields varies depending upon the referenced holon's state:
- if the referenced holon
New(i.e., not yet created), the HolonReference'sholon_idwill beNoneandholon_refwill point to the holon being staged for creation. - if the referenced holon has been created, the HolonReference's
holon_idwill beSome(HolonId)andholon_refwill be populated only when the referenced holon has actually been fetched and cached.
Recall that within the MAP Holons Integrity Zome, multi-valued relationships (i.e, relationships where a single source holon may be related to many target holons) are represented as SmartLinks. This means they are not actually stored in the HolonNode Entry itself. in the MAP Holons Coordinator Zome, the members of multi-valued relationships are represented as HolonCollections.
During the construction or editing of a holon (i.e., state = New or Changed), StagedCollection is used to contain references to related holons. When the source holon is committed, the outbound relationships in its StagedCollection are written to the persistent store as SmartLinks using the SmartLink Manager component.
When a source holon is fetched from the HolonCache, SmartCollections are created (in memory) for each of its multi-valued relationships. However, their elements vector is initially empty. Each SmartCollection has a query_spec it can pass to the MAP QueryEngine to retrieve the SmartReferences for that relationship. The results is populated in its elements field.
Each SmartReference includes a HolonReference that contains the local or external identifiers for that holon. The SmartReference also contains values for some of the properties of the target holon. These values allow filtering and sorting to be performed on the SmartCollection without having to retrieve the target holons themselves. If the target holon IS retrieved it can be stored in the holon field of the LocalHolonReference or ExternalHolonReference.