Sorry, we don't support your browser.  Install a modern browser

Immutable IDs for pages#16

A system of having immutable ID’s for pages. Right now UUID’s are basically slugs, which can be changed by editors. The consequence is that the stored references to these pages break.

There’s already a plugin that tries to solve this:

2 years ago
Changed the title from "Immutable ID's for pages" to "Immutable IDs for pages"
2 years ago
Changed the status to
2 years ago

I’ve been using the AutoID plugin recently and where very happy with it. For our own implemention, I would – however – prefer to use the UUID version 4 standard (random numbers), as it is a well-established standard, that ensures that there is enough entropy to avoid collisions. Only the downside of random numbers could be, that this could cause merge conflicts in git, when 2 dev setups are assigning different UUIDs to the same page, when edited from the panel.

To circumvent this, the UUIDs for any page/file should be generated as soon, as the corresponding page/file has been created in the panel/via API, even if the UUID is not needed at that time.

2 years ago

I get that it’s a hard problem: you have take into account multiple scenario’s (e.g. “directly drafting content in the filesystem via the Finder” or “performance for fetching pages via the Kirby internal page(), pages()and->find()` functionality” or “merge collisions when content is generated at multiple origins and then pushed into a repo somehow”).

There are already some ideas (e.g. Bruno’s autoid plugin that saves the id directly in the content file - but this one needs performance “hacks” e.g. for indexing I think), but their might be others. E.g. I’ve lately thought about what npm/composer do with their lockfiles: maybe we could use something like that too? Then we have something that contains an index, with key-values that can be used in the internal ->find() and ->index() functions or to store these “id’s” as relations in content files. It can be updated quickly on “slug-change”-events in the panel, and theoretically we can even have some kind of maintenance function that checks periodically if this file exists and if all pages are added to it. It can also be put in version control systems and will throw a merge conflict when there would be an issue. On the other hand, it’s a potential SPoF when you lose or corrupt this file.

2 years ago

@Bart Vandeputte IMHO, storing the ID directly in the content file will give us the least amount of problems, because as you noted, having everything stored in a separate index could easily lead to catastrophic failure, once this file gets corrupted. But having everything stored within content files as a single source of truth is very save, unless 2 different installations of a project will generate a UUID for the same page/file. But that is less likely, when content gets a UUID assigned as soon as it is uploaded/created.

In that case, the lookup table just serves as a cache and could even be self-correcting. Each lookup can be easily verified by:

  1. Kirby get a request, e.g. $site->lookup('MY-UUID') (method name lookup just for example).
  2. Kirby queries the lookup table to see, if that UUID exists there:

If the UUID exists in the lookup table: Kirby fetches the corresponding page/file and checks if its ID matches the cache.

If so, Kirby returns the fetched page or file object. If the ID is not present in the lookup table or the corresponding page/file was not found, Kirby will traverse the site tree until it finds the requested page/file. If that also fails, return null.

Cases, in which the lookup table could get out-of-sync would be:

  1. Content files have been moved/created directly from the file system. The index can correct things on-the-fly, as the lookup() method is used. This means, the index file does not need to be 100% correct at any point. If e.g. the UUID for a page/file is never looked-up, it does not matter if its lookup entry remains broken until needed.
  2. A new content file has been created on another installation and the index has not been updated yet. In this case, the lookup table can just make a single lookup and add the corresponding page to the index.
  3. Something has been deleted from the file-system and does not exist in the content folder any more. In this case, the lookup table can just delete the corresponding entry.
  4. If a lot of content was changed and it would not make any sense to update the lookup table “entry-by-entry”, it would easily be possible to re-create it from scratch by traversing the site tree.

If a database is used to store content instead (with a dedicated UUID column), the lookup table could even be skipped, though it might still be relevant for performance reasons. I’m not so much of a database expert, but I remember that caching can often speed up things a lot, especially when the database is hosted on another server.

2 years ago

I agree Fabian. The generated (?) cached lookup file (or redis cache or whatever) could be used as a performance trick to avoid slow reference lookup. But this would also mean it guarantees 100% accuracy at each moment in time.

2 years ago

I think Fabian already outlined this very nicely. I’m also not too worried about out-of-sync indexes. AFAIK relational databases have the exact same problem. I.e. it’s quite easy and possible to wreck a MySQL database as well. With redundant indexes and caches there’s always a risk of corruption.

2 years ago

This is a great news, with a good relation logic we can really use Kirby for anything.

a year ago

While this feature being developped, how do you guys handle relationships with that mutable slug ?

9 months ago

See link in first post

9 months ago

Any news if this will make it into core anytime soon?

7 months ago

I hope this feature for 3.7. It would make me choose kirby for way more projects :)

7 months ago

This or being able to use autoid with the pages field would solve most of the relationship issues I think. It would certainly make Kirby perfect.

7 months ago

IDs for structures would be also very handy. If one is selecting something from structure then th key is the order number. It’s very easy to then come to the structure and reorder items - changing their keys in this case.
Blocks already have some ID system i assume the implementation must be very similar.

4 months ago

we are running into related issues with every slightly more complex project and are highly seeking this feature!

2 months ago