Storage in LisaGUI
On filesystems:
In LisaGUI, a filesystem volume can exist as a JavaScript object in memory, or as an IndexedDB database. These implementations are encapsulated by a class instance for each volume, and a controller which handles filesystem operations and the creation and deletion of volumes:
Yes, I really should call them "Volumes" or "FileSystemVolumes" or something, but "Disk" is shorter and more fun. Both disks and the files on them are identified by UUIDs, termed a "diskID" for a disk and a "fileID" for a file. FileIDs have three different mappings:
- metadata: Objects with properties including a file's name, type, dates created and modified, cached file sizes, saved window positioning, etc.
- data: Objects with actual file data, organized by version. The contents of each version are dependent on the file type.
- childSets: A set of other fileIDs representing a file's children within the directory structure. Not all files have an entry; the presence of an entry is what determines whether a file is a directory.
A file's metadata object contains a reference to its parent. This is mostly for convenience. The reference is used if the file is ever orphaned and needs to be reconnected to its parent.
The DiskRAM class uses JavaScript Map objects to store this data. For the DiskIDB class, each mapping is stored within an IndexedDB object store. The IndexedDB API itself is a bit clunky; I ended up having to write a tiny library of wrapper functions for DiskIDB to use.
The disk controller handles the creation and management of IndexedDB databases (and RAM disks), and ensures the uniqueness of each diskID. For a while, I was actually juggling around DiskABS class objects in various parts of the system, but I recently did a large refactor which cleaned all that up. All disk operations are now neatly funneled through the controller.
Changing a file's parent requires writing to the child, and both the old and new parents. That same refactor involved the batching of these types of write operations, moving all calls within a single IndexedDB transaction. The atomicity greatly reduces the chance of leaving the system in a broken state (a file might otherwise be orphaned if an operation is interrupted at just the right moment).
Lets briefly discuss LisaGUI's Application layer.
Applications are defined as objects, and their functionality is passed in through constructor arguments. The ones we care about are:
- An initialization function, which runs when the Application is first added to the system, and defines shared functions and constants for that app.
- A "setupContext" function. When a file is opened, an "AppContext" is created to represent a process for that file, and this function is run. The scope of setupContext contains the bulk of an Application's code.
When an AppContext is created, it receives a reference to the disk controller via its constructor, and uses it privately to handle the logic around saving and loading its file's data. The application-specific code sees none of it. All setupContext receives in the way of arguments is a reference to the AppContext itself. That reference also gives it access to the application's shared data object, so it can reference the values defined in the aforementioned initialization function.
So an AppContext can save itself or load itself, but that's it. The exception, as you may have guessed, is ElevatedAppContexts, which are only created for certain privileged apps. ElevatedAppContexts are given an extra argument: a special "privateFunctions" object from the System decorated with a variety of functions and objects which are otherwise encapsulated within the System, including the disk controller. The most notable elevated app is the Desktop Manager (the file manager). Naturally, it needs special access to the disk controller to actually handle all of the disk operations the user initiates by dragging files and folders around the system.
On disk repair:
Recently, I added disk repair functions; selecting a disk and choosing the "Repair disk" option in the Housekeeping menu actually does something now! Repairing a disk runs two passes over it - the first ensures all the entries in each disk are valid objects with a minimum set of required properties. The next pass actually checks the hierarchical structure of the filesystem, starting at the roots of the disk. There are, in fact, multiple roots:
- The first is the main filesystem root which is represented to the user as a disk icon on the desktop.
- The second root is the desktop. It runs in it's own ElevatedAppContext created by the Desktop Manager, and is what you switch to when no window is open, or if you click on the desktop. The children of the desktop directory are the preferences, the wastebasket folder, and the clipboard. The desktop file itself is never objectified to the user and is purposefully kept hidden.
- The third root is a temporary directory used during certain disk operations.
A recursive traversal function is run over each root. The function keeps track of which files have been traversed using a shared Set; traversal is what indicates a file has a valid position in the hierarchy. For each directory, we check the metadata of its children to make sure they properly point back to it. However, if one of the children has been previously traversed, it must belong to another parent, and so it's removed from the current directory's childSet.
Once the three roots are checked, any files not yet traversed are orphaned by definition and must be placed back into the hierarchy. At this point, we simply run a check on all the disk's fileIDs, and skip over any previously traversed files. This part is a bit tricky; we don't know anything about the structure of the orphaned files. If a large folder on your disk is orphaned, that's an arbitrarily large disconnected subtree which must be reattached.
The solution is to find the top of the subtree of each orphaned file by using the orphan's "parent" property and working our way up the tree as best we can, keeping track as we go to avoid getting caught in a loop. Ultimately, we either reach:
- a file who's parent is already in the hierarchy, directly indicating the place the subtree can be reattached
- a dead end via a loop or a file with no "parent" property, meaning we can simply reattach the subtree to the root of the disk.
Once reattached, the subtree is traversed using the same recursive function mentioned earlier, ensuring all files in the subtree are correctly placed.
At this point, we continue the orphan loop. Once it finishes, all files are accounted for.
To give a more concrete example, if three large directories become detached from the main hierarchy, then the orphan loop will only ever encounter three non-traversed files. Each time it runs, we trace our way to one of those three directories, and about 1/3rd of the remaining files are placed in the hierarchy (assuming the directories have roughly the same number of descendants).