[Registry] Using Key methods for MetaKeys

[Felix Berlakovich]

Hi!

Question 1:
I recently stumbled upon the following issue:

I have a Key with MetaKeys below "conflict/". For example the Key has the following MetaKeys:

conflict/operation/our
conflict/operation/their
...

Now I need to find all the MetaKeys below "conflict/". The easiest way would be to use Key ("conflict").isBelow(currentMetaKey). Unfortunately this is not possible because the Key creation does not allow Keys with a name prefix other than "user" or "system". Is there any feasible workaround to reuse the methods of Key for arbitrary named MetaKeys? At the moment I use something like currentMeta.getFullName().find("conflict/") == 0, but this is definitely just an ugly workaround.

Related question 2:
I think the use of isBelow is not very intuitive. I think the isBelow property should refer to the key on which it is called instead of the passed argument. For example:

Key ("user/config/key/below").isBelow (Key ("user/config")) == true
Key ("user/config/key/below").isBelow (Key ("user/config/key/below/deeper")) == false

This way the call could be read as an English sentence. Nonetheless this interface would not any longer mirror the parameter order of the C interface.

Regards,

Felix

[Markus Raab]

Hello,

Post by Felix Berlakovich
I have a Key with MetaKeys below "conflict/". For example the Key has the
conflict/operation/our
conflict/operation/their
...
Now I need to find all the MetaKeys below "conflict/". The easiest way
would be to use Key ("conflict").isBelow(currentMetaKey). Unfortunately
this is not possible because the Key creation does not allow Keys with a
name prefix other than "user" or "system". Is there any feasible
workaround to reuse the methods of Key for arbitrary named MetaKeys? At
the moment I use something like
currentMeta.getFullName().find("conflict/") == 0, but this is definitely
just an ugly workaround.

Yes, that is a good question.

The currently best solution for this problem is to use an array. So the
detector of the conflict generates the keys conflict/#0, conflict/#1..
Then no actual iteration is needed, but the user can directly do a lookup to
the value of interest, e.g. keyGetMeta(key, "conflict/#/our"). # is then
replaced with an index #0, #1 for every conflict.

To know the length of the array, the warnings use the convention that in the
rootkey ("conflict" in this case) stores the number of elements. We can discuss
if this is useful and should document this as a guideline.

Additionally, we should also discuss if it is better to use the same naming
conventions for arrays in key sets and meta data. In key sets, where iteration
guarantees the order, the index is prefixed with _ to preserve the ascii-order,
e.g. #9 and then #_10.

Post by Felix Berlakovich
I think the use of isBelow is not very intuitive. I think the isBelow
property should refer to the key on which it is called instead of the
Key ("user/config/key/below").isBelow (Key ("user/config")) == true
Key ("user/config/key/below").isBelow (Key
("user/config/key/below/deeper")) == false
This way the call could be read as an English sentence. Nonetheless this
interface would not any longer mirror the parameter order of the C
interface.

You are competely right and I also stumbled over this issue. I did not solve
this issue with a keyIsBelow2 in C up to now because I still hope that there
is a good solution for the more general problem of hierarchical relation
between keys. keyRel() did not really work out and has other problems on its
own.

I would really love to see a good proposal for this.

For C++, however, we can change the implementation of isBelow, isDirectBelow
and isBelowOrSame because we do not have API/ABI compatibility constraints up
to now, but that change might need changes in SWIG/GObject bindings.

best regards,
Markus

[Markus Raab]

Hello,

Using an array is not really an option here as I want to store different
kinds of information below "conflicts" (not necessarily related or ordered
keys).

You simply can add conflict/#/type that contains which type of conflict it is.
If you store types so different that they are hard to use, we should consider
to have more than one hierarchy (not to store everything below conflict). Or is
this exactly your suggestion?

Especially I have different places which set "conflict/*" metakeys
and one place where I want to delete any keys below "conflict/" without
even knowing which conflict keys exist.

Would be interesting to see the code where you need it, but you are right,
keyIsBelow does not work on metakeys and that would be useful.

Shortly after writing the first
I want to reuse the existing merging algorithm for merging MetaKeys.

Manuel even suggested to pass the KeySet of meta keys directly to the user.
This would make it trivial to reuse the algorithm.

But then it would not be allowed anymore to switch the implementation, e.g. to
a hashtable.

This means that I have to provide a parentKey for each (Meta)Keyset, but the
parentKey for arbitrary MetaKeySets would be "". Again this is not a valid
Key name.
I solve this issue by temporary rebasing all the MetaKeys to
"user/" and strip the prefix after merging. However, this again feels like
an ugly workaround.
Wouldn't it be an option to allow the creation of arbitrary named Keys?
Maybe with special restrictions like a restricted set of allowed
operations?

That is a very good suggestion!

There are even other use cases for further opening to other names:
1.) When keys could have a starting "/" we would not need ksLookupByName()
and ksLookup() would be enough.
2.) For the contextual values keys starting with "/" would be needed to.

What about having KEY_META_NAME and KEY_CASCADING_NAME flags for keyNew that
allows to create keys with those names. As a restriction those keys should not
be appended in keysets (except within key*Meta functions). That might slow
down ksAppend, though.

Completely arbitrary, however, would also allow double slashes and /../ that
might be confusing. But that is actually not checked for meta keys at the
moment.

What do you think?

best regards,
Markus

[Markus Raab]

Hello,

Post by Markus Raab
Would be interesting to see the code where you need it, but you are
right, keyIsBelow does not work on metakeys and that would be useful.

For each key that causes a conflict during the merge, the corresponding
* Which operation was performed on our version
* Which operation was performed on their verison
* Maybe the names of all conflicting metakeys (this may be an array)
* State tracking information for the resolution strategies
All this information should be tracked in metakeys. When a conflict is
finally resolved, ALL the mergeconflict related metakeys should be deleted
(otherwise it would still be present in the merge result). The class
deleting these metakeys is different from the class(es) recording the
metakeys. Preferably the deleting class should not require any knowledge
about which metakeys exactly were stored. So as I see it the cleanest
solution would be to find all metakeys below "conflict/" (i.e. all
metakeys with a name like "conflict/*") and delete them. This would
completely decouple the recording class(es) from the deleting class. I
don't think that an array is suitable for recording all this information.

You are right. I thought that we are just talking about the communication
between the merge algorithm and the user, but it seems like that you are
already talking about the automatical resolution strategies in the chain of
responsibility design pattern.

The user, however, should have a simple enough way to find out which keys
created which conflicts.

while ((currentMeta = key.nextMeta()))
{
// TODO: this is just a workaround because keys with a
prefix other than // user/ or system/ cannot be created and therefore
isBelow cannot be used if (currentMeta.getName().find("conflict/") == 0)
{
// TODO: this is just a workaround for the meta
deletion bug #8 // key.setMeta(currentMeta.getName(), NULL);
ckdb::keySetMeta(key.getKey(), currentMeta.getName().c_str(), 0); }
}
Irrespective of whether my approach of solving the conflict management is
good (this would be another discussion), I am quite confident that this or
similar requirements will reemerge.

Obviously, in the moment we passed out the metakeys (even readonly) it was
clear that all key* related methods are expected to work.

Post by Markus Raab

Shortly after writing the first
I want to reuse the existing merging algorithm for merging MetaKeys.

Manuel even suggested to pass the KeySet of meta keys directly to the
user. This would make it trivial to reuse the algorithm.
But then it would not be allowed anymore to switch the implementation,
e.g. to a hashtable.

I think that retrieving ALL metakeys would be convenient sometimes, but as
already stated raises many implementation questions. What I wanted to
point out here was that this is yet another occurrence of the key naming
restriction issue. Anyway, I think a discussion about how to handle
metakey access in a convenient way would be valuable as well.

Yes, it would be; but on the other hand users could mess up the meta keyset,
that currently deliberately has following limitations:
- no null values
- no binary data
- no modification of references (COW)
- no guarantee of ordering

Post by Markus Raab
1.) When keys could have a starting "/" we would not need
ksLookupByName() and ksLookup() would be enough.
2.) For the contextual values keys starting with "/" would be needed to.
What about having KEY_META_NAME and KEY_CASCADING_NAME flags for
keyNew that allows to create keys with those names. As a restriction
those keys should not be appended in keysets (except within key*Meta
functions). That might slow down ksAppend, though.
Completely arbitrary, however, would also allow double slashes and /../
that might be confusing. But that is actually not checked for meta keys
at the moment.
What do you think?

The suggestion with KEY_META_NAME and KEY_CASCADING_NAME sounds very good.
Maybe we should discuss this at the next meeting? I can work around the
issue for now.

Ok, let us avoid narrow decisions and discuss the whole topic.

best regards,
Markus