What to say…
greeting in formal manner
Hello everyone. Welcome to the presentation of my diploma thesis about the planning and architecture and very minimal prototype implementation of the mize data engine.
EDIT: which as matteo wants me to put it is the implementation of my ideal software world.
table of contents
There are two big chapters to this presentation: The big WHY the topic of data management is of such interest to me.
so much so that I decided to do my diploma thesis about it, as i was warned about was a bad idea.
and then the HOW i plan to do data management in a better way
WHY
There is something that bothers me about the way software is made and supposed to be used.
and a good example can be found right at my doorstep: The energy system of my families homes.
3 Households in my family have invested into a solar and energy system. at my one in particular we have an Inverter from GoodWe, Battery from Lithtec, a Heat Pump from Heliotherm. and so aon. All from different brands and all marketed as somehow “smart” or IoT ready or energy innovation…
every device has their own way of somehow talking to the cloud of the respective brands with a delay of 5 to 15 minutes, because aparently their networks work with sound and not light… the batteries cloud is still under construction… but already it has a wifi AP running that your CAN’t TURN OFF… so does the inverter, even though it uses our wifi network to talk to the cloud. Oh and to not forget the dryer and oven also have some app functionality. It’s a huuuuuuuuuuuuuuuuuge mess… the term i would use for those devices is “stupid” and not “smart”
You can not do the simplest automations that you would need two of the devices for like charging the car, when the inverter has access power. whenever My two uncles always come to me with automation ideas like that (as i’m the computer guy of the family) and i just have to tell them, that there is now way to do this in this mess of a data landscape.
next_slide: I want software to work better together, especailly in the embedded space. where there is the biggest mess.
next_slide: As a prompt Linux Power user, when in a terminal, I get a glimpse an old philosophy for application development. The so called Unix philosophy. Especially the point on “Write programms to work together”. Most terminal applications really can work together.
HOW
let’s move on to the HOW part. HOW I plan to improve this situation.
at it’s core the problem is about every application managing data in their own way and my proposed solution is a data management library, which is general enought so it can be used for any application. I also call this library The Mize Data Engine.
An already established way of managing data, especially for storing it… are Filesystems. They are also very familiar to users and developers.
However there are a handful of problems with Filesystems, that mize tries to improve upon.
- The actual format of the data in files is usually implemented differently by every application, breaking interopability.
- Two applications can’t modify the same file at the same time.
- There is no standard way for an application to know when some other application modified a file.
- People have multiple devices and any data should be accesible on any device, which no Filesystem properly supports.
next_slide: Instead of files stored on a Filesystem, with mize you have Items that are fully managed by the data engine.
One weired concept with mize is, that an Item is both a file and a folder. ignoring metadata a File is essentially a collection of bytes and a folder is a collection of named links to other files or folders. An Item has both… a collection of bytes and a collection of named links to other items.
Instead of filetypes mize has a whole additive type system, where an item has many types at once to fit the specific needs for the application, other applications using the same item can just ignore the types they don’t know about and add their own specific ones.
The data engine has a custom protocol, which allows items to be accessed on any device just as they were saved on that device, even if they aren’t.
This protocol and the data engine also propagate changes to items to wherever needed.
another important aspect to the library being usable for any application besides a general enough data model is portability… that being the ability to use the library from any programming language on any platform. A lot of time went into figuring that out… but it is possible. The very interesting details of that do however not fit into the time of the presentation.
ending
so all i have left to say not is that I believe the interopability gained by such a system could improve the capabilities of everyday software in ways end users can’t even imagine to be possible.
extra: portability
Applications are written in a variaty of different programming languages and for a variaty of different target systems. so how do you make a library, which can be used by any language on every platform.
About the many languages: every language supports loading and using a C-library… which are those dlls you might know from the windows os…, even Javascript in the browser, thanks to asm.js or now WebAssembly. The Rust build-system with minor modifications can emit such a C-library. Add to that bindings in the language in question, which is a small bit of code, that makes interacting with the C-library more plesant.
About the many systems or platforms, the most important part was to split the library into platform dependent code and code, that works on any platform. and seccond to that is creating build infrastructure, which can compile the library for any system, which for mize is done with the nix package manager. The details of how that, would however far break the boundaries of this presentation.