Markus Raab
2014-09-02 20:52:00 UTC
Hello,
In this release we changed 578 files in 473 commits
(68596 insertions(+), 59260 deletions(-) compared to Elektra 0.8.7).
We assume thats the largest change set for any of Elektra's releases
up to now. It happened only within a bit more than a month up
(0.8.7 was released 28.07.2014).
## New features
GSoC finished successfully (thanks Ian and Felix)
See http://community.libelektra.org/wp for the latest results.
So Elektra now has a 3-way merging framework that is superior
to text-based merging in many scenarios (e.g. moving configuration
options within a file or with in-line comments) iff a storage plugin
creates key names that are not only line numbers.
We love to get Feedback!
Writing plugins is now even more comfortable.
A plugin writer tutorial was written (thanks Ian):
https://github.com/ElektraInitiative/libelektra/blob/master/doc/tutorials/plugins.md
The documentation was completely reworked:
http://doc.libelektra.org/api/0.8.7/html/group__plugin.html
And two new macros allow printf formating for warnings and errors
(ELEKTRA_ADD_WARNINGF and ELEKTRA_SET_ERRORF).
The ini plugin was greatly improved (tested with samba configurations
and added to ALL plugins) and the hosts plugin was rewritten to support
ipv6 properly (thanks to Felix).
The constants plugin was added and allows introspection of Elektra's
cmake variables. Because such non-file based plugins (e.g. also uname)
do not need resolving, the plugin noresolver was added. It supersedes
the success plugin.
Elektra now allows to correctly fsync its configuration files
(sync plugin) and the folders where files are stored (resolver plugin).
Just make sure to add the "sync" plugin using kdb mount.
The resolver plugin now reads from passwd and no longer needs
environment variables. Additionally, the resolver plugin was prepared
to support other variants by so called compilation variants.
The error plugin now allows, next to list all possible errors, to
provoke errors when opening plugins. We fixed some issues related
to plugins having errors when they initialize themselves.
So following plugins were added: sync noresolver line ini constants
Nearly all plugins now have a README.md for further information
(thanks to Ian). An overview of all plugin is on with links to them:
https://github.com/ElektraInitiative/libelektra/blob/master/src/plugins/
The kdb tools were greatly improved (thanks to Felix):
- added remount tool
- umount now also accepts mountpath
- mount allows to specify different resolvers
- import now can use merge strategies
- check without arguments checks key database
- mount is now more verbose when validation fails
New/improved scripts/make targets (note that scripts can be executed by
kdb <script>):
- mounting, unmounting scripts were added
- generate template for a new plugin was improved
- configure-debian was added
- added targets run_all and run_memcheck
- bash completion file now installed
- ucf integration
- merging scripts were added for the usage with ucf
- scripts doing internal checks on source of plugins
## Compatibility
This time we had to break compatibility. We did not change the ABI (your
application still will be able to use Elektra 0.8.8) and we did not
change the API (your application still will compile against Elektra). We
changed the third part of our interface: the semantic interface.
The problems were following:
keyAddBaseName/keySetBaseName did something obvious when no special
characters were in the baseName. But once there were, there are two
different interpretations what it should do:
1.) add/set a basename, so escape characters that are not canonical
in the basename
2.) add all parts of the name given (with slashes)
The methods were used in both ways, so it was obvious that something is
very wrong. We decided that it should do what the name says, that is
add/set a basename (variant 1).
The variant 2, to add any name was added and is called keyAddName() and
added as proposal.
(Thank Felix for implementations and Manuel for investigations)
When keys are renamed after adding to a keyset is a bad thing because it
destroys the order of the keyset. This is now avoided by keyLock.
Use keyDup() to get rid of such locks.
Another, even larger, change is also about ordering of keys in keysets.
Elektra now internally has an null-terminated unescaped keyname.
Ordering of keysets will always happen on this name. The keyCmp() tool
can be used to check this order. It works very efficiently with
memcmp() and never gets confused by ASCII ordering of / (because / is
0 in the unescaped keyname).
The syntax, semantics and conventions of key names is now documented in
detail:
http://doc.libelektra.org/api/0.8.8/html/group__keyname.html
ksNew() does now return a keyset with a properly set cursor (ksRewind).
Because its always possible that software relies on bugs the
better way to deal with such a situation (as the keySetBaseName()
situation described above) is to provide the same function twice.
Manuel said he will create a prototype to introduce symbol versioning
in Elektra. With that, old customers would still receive the old
behaviour, but people compiling against a new version would get the new
behaviour. So in one of the next releases we will also avoid semantic
interface changes when there is a valid use case for it (there is none
if the program e.g. crashes).
Symbol versioning also allows to compile against old versions on
purpose if you do not want the new behaviour.
We have prepared an ABI-test suite, that also checks behaviour,
for that purpose, but we also improved testing in other parts:
- (New Test strategy)[/doc/TESTING.md]
- New resolver tests for conflicts (needs tty)
If you try to execute test_ks from 0.8.7 with libelektra 0.8.8 it will
crash, but not because of any incompatibility, but because of strcmp in
the test itself gets a null pointer. The pointer is now null, because
ksNew correctly rewinds its internal cursor (see above). Amusingly,
it says on that line 94 in test_ks.c:
// TODO: why is the cursor here?
## API Proposals
see above for more information:
keyAddName .. add key name without escaping, like keySetName
keyUnescapedName .. get access to null-separated unescaped name
keyLock .. to allow to secure keys against modifications
some new ideas:
keySetStringF .. printf format-style changing of the key string
elektraKeySetName .. to allow to set meta + cascading keys
elektraArrayIncName() now works correctly with empty arrays embedded in
other arrays (yajl+line plugin)
elektraArrayValidateName() was also added, thanks to Felix.
These methods are declared in the file kdbproposal.h
but do not guarantee any forms of compatibility (they might
even be removed).
## Issues
Many issues were resolved as you can see in github:
https://github.com/ElektraInitiative/libelektra/issues
Alone for the milestone 0.8.8 we closed 17 issues, including
those mentioned in "Compatibility". Other issues (not all were
tracked on github):
- fix undefined errors in kdbOpen() or kdbClose()
- Now Python 2+3 work in parallel
- python2 interpreter is found correctly (also on Arch)
- Sentinel now makes sure that you cannot forget KS_END to end ksNew
- Fixes for architecture-specific problems by Pino
- fix .pc file
- fix compilation problem with KDB_MAX_PATH_LENGTH
- tmpnam to mkstemp (security)
- make test data naming consistent (thanks Pino)
- use LIB_SUFFIX for TARGET_TOOL_EXEC_FOLDER thanks to Kai Uwe
- Fix search for boost (thank Pino)
## Other Stuff ##
Thanks to Pino Toscano Elektra 0.8.7-4 is now available in Debian
Testing: https://packages.debian.org/search?keywords=elektra
So it is only a matter of time that other (debian-based) distributions
will follow and replace the dusty Elektra 0.7.
Debian Continuous Integration http://ci.debian.net/packages/e/elektra
(thanks Pino)
greatly complement our tests running on http://build.libelektra.org:8080/
Elektra's buildserver also was improved:
- now also compiles with icc
- runs make run_memcheck
- checks if plugins are added correctly in-source
- runs ABI + behavioural tests
Raffael Pancheri now made a merge request for qt-gui
https://github.com/ElektraInitiative/libelektra/pull/103/files
in which copy, paste and delete of keys already works.
It is, however, still work in progress.
Manuel Mausz made great progress in script-based Elektra plugins. He is
also working on glib+gobject-introspection based bindings.
He investigated some issues, e.g. a crash of the python binding which
was only triggered if python3 is build with a specific flag/module
combination, see:
https://github.com/ElektraInitiative/libelektra/issues/25
## Get It! ##
You can download the release from:
http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.8.tar.gz
size: 1644441
md5sum: fe11c6704b0032bdde2d0c8fa5e1c7e3
sha1: 16e43c63cd6d62b9fce82cb0a33288c390e39d12
sha256: ae75873966f4b5b5300ef5e5de5816542af50f35809f602847136a8cb21104e2
already built API-Docu can be found here:
http://doc.libelektra.org/api/0.8.8/html/
Best regards,
Markus
In this release we changed 578 files in 473 commits
(68596 insertions(+), 59260 deletions(-) compared to Elektra 0.8.7).
We assume thats the largest change set for any of Elektra's releases
up to now. It happened only within a bit more than a month up
(0.8.7 was released 28.07.2014).
## New features
GSoC finished successfully (thanks Ian and Felix)
See http://community.libelektra.org/wp for the latest results.
So Elektra now has a 3-way merging framework that is superior
to text-based merging in many scenarios (e.g. moving configuration
options within a file or with in-line comments) iff a storage plugin
creates key names that are not only line numbers.
We love to get Feedback!
Writing plugins is now even more comfortable.
A plugin writer tutorial was written (thanks Ian):
https://github.com/ElektraInitiative/libelektra/blob/master/doc/tutorials/plugins.md
The documentation was completely reworked:
http://doc.libelektra.org/api/0.8.7/html/group__plugin.html
And two new macros allow printf formating for warnings and errors
(ELEKTRA_ADD_WARNINGF and ELEKTRA_SET_ERRORF).
The ini plugin was greatly improved (tested with samba configurations
and added to ALL plugins) and the hosts plugin was rewritten to support
ipv6 properly (thanks to Felix).
The constants plugin was added and allows introspection of Elektra's
cmake variables. Because such non-file based plugins (e.g. also uname)
do not need resolving, the plugin noresolver was added. It supersedes
the success plugin.
Elektra now allows to correctly fsync its configuration files
(sync plugin) and the folders where files are stored (resolver plugin).
Just make sure to add the "sync" plugin using kdb mount.
The resolver plugin now reads from passwd and no longer needs
environment variables. Additionally, the resolver plugin was prepared
to support other variants by so called compilation variants.
The error plugin now allows, next to list all possible errors, to
provoke errors when opening plugins. We fixed some issues related
to plugins having errors when they initialize themselves.
So following plugins were added: sync noresolver line ini constants
Nearly all plugins now have a README.md for further information
(thanks to Ian). An overview of all plugin is on with links to them:
https://github.com/ElektraInitiative/libelektra/blob/master/src/plugins/
The kdb tools were greatly improved (thanks to Felix):
- added remount tool
- umount now also accepts mountpath
- mount allows to specify different resolvers
- import now can use merge strategies
- check without arguments checks key database
- mount is now more verbose when validation fails
New/improved scripts/make targets (note that scripts can be executed by
kdb <script>):
- mounting, unmounting scripts were added
- generate template for a new plugin was improved
- configure-debian was added
- added targets run_all and run_memcheck
- bash completion file now installed
- ucf integration
- merging scripts were added for the usage with ucf
- scripts doing internal checks on source of plugins
## Compatibility
This time we had to break compatibility. We did not change the ABI (your
application still will be able to use Elektra 0.8.8) and we did not
change the API (your application still will compile against Elektra). We
changed the third part of our interface: the semantic interface.
The problems were following:
keyAddBaseName/keySetBaseName did something obvious when no special
characters were in the baseName. But once there were, there are two
different interpretations what it should do:
1.) add/set a basename, so escape characters that are not canonical
in the basename
2.) add all parts of the name given (with slashes)
The methods were used in both ways, so it was obvious that something is
very wrong. We decided that it should do what the name says, that is
add/set a basename (variant 1).
The variant 2, to add any name was added and is called keyAddName() and
added as proposal.
(Thank Felix for implementations and Manuel for investigations)
When keys are renamed after adding to a keyset is a bad thing because it
destroys the order of the keyset. This is now avoided by keyLock.
Use keyDup() to get rid of such locks.
Another, even larger, change is also about ordering of keys in keysets.
Elektra now internally has an null-terminated unescaped keyname.
Ordering of keysets will always happen on this name. The keyCmp() tool
can be used to check this order. It works very efficiently with
memcmp() and never gets confused by ASCII ordering of / (because / is
0 in the unescaped keyname).
The syntax, semantics and conventions of key names is now documented in
detail:
http://doc.libelektra.org/api/0.8.8/html/group__keyname.html
ksNew() does now return a keyset with a properly set cursor (ksRewind).
Because its always possible that software relies on bugs the
better way to deal with such a situation (as the keySetBaseName()
situation described above) is to provide the same function twice.
Manuel said he will create a prototype to introduce symbol versioning
in Elektra. With that, old customers would still receive the old
behaviour, but people compiling against a new version would get the new
behaviour. So in one of the next releases we will also avoid semantic
interface changes when there is a valid use case for it (there is none
if the program e.g. crashes).
Symbol versioning also allows to compile against old versions on
purpose if you do not want the new behaviour.
We have prepared an ABI-test suite, that also checks behaviour,
for that purpose, but we also improved testing in other parts:
- (New Test strategy)[/doc/TESTING.md]
- New resolver tests for conflicts (needs tty)
If you try to execute test_ks from 0.8.7 with libelektra 0.8.8 it will
crash, but not because of any incompatibility, but because of strcmp in
the test itself gets a null pointer. The pointer is now null, because
ksNew correctly rewinds its internal cursor (see above). Amusingly,
it says on that line 94 in test_ks.c:
// TODO: why is the cursor here?
## API Proposals
see above for more information:
keyAddName .. add key name without escaping, like keySetName
keyUnescapedName .. get access to null-separated unescaped name
keyLock .. to allow to secure keys against modifications
some new ideas:
keySetStringF .. printf format-style changing of the key string
elektraKeySetName .. to allow to set meta + cascading keys
elektraArrayIncName() now works correctly with empty arrays embedded in
other arrays (yajl+line plugin)
elektraArrayValidateName() was also added, thanks to Felix.
These methods are declared in the file kdbproposal.h
but do not guarantee any forms of compatibility (they might
even be removed).
## Issues
Many issues were resolved as you can see in github:
https://github.com/ElektraInitiative/libelektra/issues
Alone for the milestone 0.8.8 we closed 17 issues, including
those mentioned in "Compatibility". Other issues (not all were
tracked on github):
- fix undefined errors in kdbOpen() or kdbClose()
- Now Python 2+3 work in parallel
- python2 interpreter is found correctly (also on Arch)
- Sentinel now makes sure that you cannot forget KS_END to end ksNew
- Fixes for architecture-specific problems by Pino
- fix .pc file
- fix compilation problem with KDB_MAX_PATH_LENGTH
- tmpnam to mkstemp (security)
- make test data naming consistent (thanks Pino)
- use LIB_SUFFIX for TARGET_TOOL_EXEC_FOLDER thanks to Kai Uwe
- Fix search for boost (thank Pino)
## Other Stuff ##
Thanks to Pino Toscano Elektra 0.8.7-4 is now available in Debian
Testing: https://packages.debian.org/search?keywords=elektra
So it is only a matter of time that other (debian-based) distributions
will follow and replace the dusty Elektra 0.7.
Debian Continuous Integration http://ci.debian.net/packages/e/elektra
(thanks Pino)
greatly complement our tests running on http://build.libelektra.org:8080/
Elektra's buildserver also was improved:
- now also compiles with icc
- runs make run_memcheck
- checks if plugins are added correctly in-source
- runs ABI + behavioural tests
Raffael Pancheri now made a merge request for qt-gui
https://github.com/ElektraInitiative/libelektra/pull/103/files
in which copy, paste and delete of keys already works.
It is, however, still work in progress.
Manuel Mausz made great progress in script-based Elektra plugins. He is
also working on glib+gobject-introspection based bindings.
He investigated some issues, e.g. a crash of the python binding which
was only triggered if python3 is build with a specific flag/module
combination, see:
https://github.com/ElektraInitiative/libelektra/issues/25
## Get It! ##
You can download the release from:
http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.8.tar.gz
size: 1644441
md5sum: fe11c6704b0032bdde2d0c8fa5e1c7e3
sha1: 16e43c63cd6d62b9fce82cb0a33288c390e39d12
sha256: ae75873966f4b5b5300ef5e5de5816542af50f35809f602847136a8cb21104e2
already built API-Docu can be found here:
http://doc.libelektra.org/api/0.8.8/html/
Best regards,
Markus