Sorry, we don't support your browser.  Install a modern browser

Immutable IDs for pages#16

A system of having immutable ID’s for pages. Right now UUID’s are basically slugs, which can be changed by editors. The consequence is that the stored references to these pages break.

There’s already a plugin that tries to solve this: https://github.com/bnomei/kirby3-autoid

2 years ago
Changed the title from "Immutable ID's for pages" to "Immutable IDs for pages"
2 years ago
Changed the status to
Planned
2 years ago
7

I’ve been using the AutoID plugin recently and where very happy with it. For our own implemention, I would – however – prefer to use the UUID version 4 standard (random numbers), as it is a well-established standard, that ensures that there is enough entropy to avoid collisions. Only the downside of random numbers could be, that this could cause merge conflicts in git, when 2 dev setups are assigning different UUIDs to the same page, when edited from the panel.

To circumvent this, the UUIDs for any page/file should be generated as soon, as the corresponding page/file has been created in the panel/via API, even if the UUID is not needed at that time.

2 years ago

I get that it’s a hard problem: you have take into account multiple scenario’s (e.g. “directly drafting content in the filesystem via the Finder” or “performance for fetching pages via the Kirby internal page(), pages()and->find()` functionality” or “merge collisions when content is generated at multiple origins and then pushed into a repo somehow”).

There are already some ideas (e.g. Bruno’s autoid plugin that saves the id directly in the content file - but this one needs performance “hacks” e.g. for indexing I think), but their might be others. E.g. I’ve lately thought about what npm/composer do with their lockfiles: maybe we could use something like that too? Then we have something that contains an index, with key-values that can be used in the internal ->find() and ->index() functions or to store these “id’s” as relations in content files. It can be updated quickly on “slug-change”-events in the panel, and theoretically we can even have some kind of maintenance function that checks periodically if this file exists and if all pages are added to it. It can also be put in version control systems and will throw a merge conflict when there would be an issue. On the other hand, it’s a potential SPoF when you lose or corrupt this file.

2 years ago
2

@Bart Vandeputte IMHO, storing the ID directly in the content file will give us the least amount of problems, because as you noted, having everything stored in a separate index could easily lead to catastrophic failure, once this file gets corrupted. But having everything stored within content files as a single source of truth is very save, unless 2 different installations of a project will generate a UUID for the same page/file. But that is less likely, when content gets a UUID assigned as soon as it is uploaded/created.

In that case, the lookup table just serves as a cache and could even be self-correcting. Each lookup can be easily verified by:

  1. Kirby get a request, e.g. $site->lookup('MY-UUID') (method name lookup just for example).
  2. Kirby queries the lookup table to see, if that UUID exists there:

If the UUID exists in the lookup table: Kirby fetches the corresponding page/file and checks if its ID matches the cache.

If so, Kirby returns the fetched page or file object. If the ID is not present in the lookup table or the corresponding page/file was not found, Kirby will traverse the site tree until it finds the requested page/file. If that also fails, return null.

Cases, in which the lookup table could get out-of-sync would be:

  1. Content files have been moved/created directly from the file system. The index can correct things on-the-fly, as the lookup() method is used. This means, the index file does not need to be 100% correct at any point. If e.g. the UUID for a page/file is never looked-up, it does not matter if its lookup entry remains broken until needed.
  2. A new content file has been created on another installation and the index has not been updated yet. In this case, the lookup table can just make a single lookup and add the corresponding page to the index.
  3. Something has been deleted from the file-system and does not exist in the content folder any more. In this case, the lookup table can just delete the corresponding entry.
  4. If a lot of content was changed and it would not make any sense to update the lookup table “entry-by-entry”, it would easily be possible to re-create it from scratch by traversing the site tree.

If a database is used to store content instead (with a dedicated UUID column), the lookup table could even be skipped, though it might still be relevant for performance reasons. I’m not so much of a database expert, but I remember that caching can often speed up things a lot, especially when the database is hosted on another server.

2 years ago
1

I agree Fabian. The generated (?) cached lookup file (or redis cache or whatever) could be used as a performance trick to avoid slow reference lookup. But this would also mean it guarantees 100% accuracy at each moment in time.

2 years ago

I think Fabian already outlined this very nicely. I’m also not too worried about out-of-sync indexes. AFAIK relational databases have the exact same problem. I.e. it’s quite easy and possible to wreck a MySQL database as well. With redundant indexes and caches there’s always a risk of corruption.

2 years ago

we are running into related issues with every slightly more complex project and are highly seeking this feature!

4 months ago
1
Changed the status to
Unassigned
2 months ago
D

When features are planned for a long time, it mostly means that they are complex for us or that a plugin already solves them in a good way and therefor we try to tackle other problems first. We just removed the planned status from tickets on Nolt. It really doesn’t help if it is assigned for a long time.

https://discord.com/channels/525634039965679616/525641819854471168/980743726165000192

2 months ago

Just to clarify: Removing the “planned” status does not mean we are not planning to add it at all. We know that this feature is important to a lot of you and it is on our list. The votes are more than clear. As described on Discord, it is one of the really complex ones and Bruno’s plugin is already pretty fantastic. We are on it, but it didn’t make it into 3.7. That’s why it seems more fair to remove the status than to suggest that it is just around the corner.

2 months ago
1
Changed the status to
In progress
a month ago
9

wohoo

a month ago

Great!

a month ago

😍

4 days ago