Objects can appear both as existing and deleted simultaneously

[joesiltberg]

From what I understand this implementation doesn't implement any functionality for sharing different subsets of the SS12000 data model to different clients. So to limit what is shared with Skolverket we need to only ingest the data we wish to share.

This means it's a valid scenario that a Person (for instance) is added, deleted and added again (with the same UUID), as time goes by and the subset we wish to share changes.

In such a scenario (added, deleted, added) the object will (in the provisioning-reference-api implementation) appear when querying /persons and also when querying /deletedEntities. Then whether or not the consuming system will keep or delete the object would be undefined and can depend on the order which these endpoints are queried.

[GustafNilstadius]

There's no expressed support in the current SS12000 standard for RBAC, as for now filtering would have to be done in a custom implementation based on the claims in the JWT. The provisioning-reference-api hence doesn't support filtering based on clients (yet).

This means it's a valid scenario that a Person (for instance) is added, deleted and added again (with the same UUID), as time goes by and the subset we wish to share changes.
In such a scenario (added, deleted, added) the object will (in the provisioning-reference-api implementation) appear when querying /persons and also when querying /deletedEntities. Then whether or not the consuming system will keep or delete the object would be undefined and can depend on the order which these endpoints are queried.

Reusing a universally unique identifier doesn't follow best practice and is very hard to handle. But you're right, if you create, delete and create the same object (same UUID) in a very short time span a race condition does occur and the object is in a undefined state. Same applies to the conventional REST API provided for provisioning.
PUT /persons (and all other data types) is idempotent, for updating a object it's recommended to use PUT and not POST -> DELETE -> POST.
Possible fixes could be to notify the DeletedEntities service on create or update of an object to make ensure that the UUID is removed from the DeletedEnteties collection and thous allowing for UUID reuse. Alternatively querying the DeletedEntities service to check if the UUID occurs in the collection and respond with 409 Duplicate and refuse the create/update and not allowing UUID reuse.

[joesiltberg]

There's no expressed support in the current SS12000 standard for RBAC

This is true. Proponents of the method of transfer specified in SS12000 do claim it does support it (implicitly, "up to the implementation to take care of"). This has been Skolverkets position from what I understand ( @NiklasLeideSkv ) since they are happy with the standard from a GDPR perspective.

filtering would have to be done in a custom implementation based on the claims in the JWT

Filtering is easy enough for the initial reading, it gets trickier when the whole life cycle is considered, and when the subset of information we share changes during an established integration (this issue is one example).

If authorization is implemented by limiting what we make available through the ingest API, basically limiting what's in the MongoDB, and thus only support one data model and one client per deployment, authorization becomes easy on the SS12000 level (this issue with DeletedEntities might be the only problem really). But from your comment above it sounds like that's not what you envision for someone who wishes to support authorization..?

Reusing a universally unique identifier doesn't follow best practice and is very hard to handle.

I think this statement sheds some light over why the current SS12000 specification (while being perfectly adequate for many applications) is not suitable for account provisioning. We need to be able to share some information, stop sharing it and then share it again, even though that information has remained locally throughout. This is nothing strange in e.g. SCIM, we're not really re-using a UUID, when we tell a service provider to delete some information we're not saying that that object has necessarily stopped existing locally, just that we don't want you to have it anymore.

If we want to avoid this (add, delete, add with the same UUID) we need to either say that:

1. Delete only means that the object is really deleted locally, and must not be used to limit what information the client should keep

or

2. The SS12000 implementation needs to come up with new UUIDs if this happens, and maintain client specific UUIDs

1 makes the standard unusable from a GDPR perspective. I would be surprised if anyone wants 2.

But you're right, if you create, delete and create the same object (same UUID) in a very short time span a race condition does occur

It is a "very short time span", but only if you rely on the subscriptions to work and the clients to act within a "very short time span". The server also needs to be always accessible and the subscriptions can't be paused temporarily.

A very short time span is also not an unlikely scenario. For instance if our authorization is based on only sharing students in certain groups, a student moving to a different class could mean it is briefly without membership in one of the shared groups.

Same applies to the conventional REST API provided for provisioning.

It does, provided multiple asynchronous jobs are active in parallel. IIRC I've been advised by @nighal to only do one bulk job at a time, which should prevent this issue.

for updating a object it's recommended to use PUT and not POST -> DELETE -> POST.

I agree PUT should be used for regular updates. This issue is about what happens when we wish to reduce the subset of information we've shared, and later increase it again.

[GustafNilstadius]

The decision to use SS12000 has been made. We have to keep this strictly technical. Everything non technical you'll have to go to another forum as specified in the CONTRIBUTING.md document.

My take is that we are taking about the the ability to notify subscribers (clients) to delete an object without deleting it locally (in the provisioning-reference-api), maybe also a way to quietly do CRUD on objects (i.e no webhook is triggered). Seems like a easy enhancement with a added query parameter to the ingest API endpoints, maybe even a config that specifies what the default behavior should be.
This in conjunction with my previous proposed solution for limiting/handling UUID reuse.

Possible fixes could be to notify the DeletedEntities service on create or update of an object to make ensure that the UUID is removed from the DeletedEnteties collection and thous allowing for UUID reuse. Alternatively querying the DeletedEntities service to check if the UUID occurs in the collection and respond with 409 Duplicate and refuse the create/update and not allowing UUID reuse.

It seems like it would cover all the concerns. Thoughts?

[joesiltberg]

I was merely addressing your statements, but if you don't wish to discuss this further I'm content with the statement that "The provisioning-reference-api hence doesn't support filtering based on clients (yet)", which seems to imply that my scenario of sharing an object, un-sharing it and later sharing it is out of scope currently.

If you do want to address that I think your first suggestion ("notify the DeletedEntities service on create or update of an object to make ensure that the UUID is removed from the DeletedEnteties collection and thous allowing for UUID reuse") is the way to go. This way your implementation would on the protocol level (as far as I know) behave as would be expected of an implementation that supports authorization. You could even claim that you support authorization, albeit in a simple way by using the ingest API to limit what the client sees (and deploying multiple instances of the server if multiple clients are to be supported), but it would behave correctly on the protocol level from the clients perspective (and supporting multiple clients in the same server would be an implementation detail).

I don't see a need to change the ingest API for this, your first solution would suffice, unless you intend to implement full blown authorization for multiple clients within one deployment. Then it would be cleaner to implement that in a general way rather than special query parameters to the ingest API to deal with the scenario in this issue.