We are designing the upgrade mechanism for:
- Efficiency - We use next level technology git to manage the entire root filesystem
- Help Webconverger Limited the business to move effectively to a subscription model
- We can easily check the integrity of the filesystem, good for security
- Git is fast & efficient for upgrades, also good for security
- We can safely downgrade and roll back to an earlier version
- Every commit accountable and transparent
The upgrade features requires persistency, i.e. install to a writable medium
- USB-HDD (.img) version which can upgrade itself or do a install - related issue
- ISO (.iso) version which can live boot and a install
Webconverger uses git to maintain the filesystem.
- Webconverger downloads a config and looks for
fetch-revision=BRANCH, otherwise defaults to master
- On boot it fetches upto that branch, unless
update-revision=SHAis specified, which is used for mounting exact commits
- If there is an update, the kernels upon /live/image/live are updated and a new git-revision based on the fetch branch is applied
- On REBOOT the script /usr/share/initramfs-tools/scripts/live in the initrd git-fs mounts the git-revision SHA1 set upon the
17.x UPDATE: Mounting scripts move to /lib/live/boot and
/usr/share/initramfs-tools/hooks is how they are included.
Note: After changing anything in
/usr/share/initramfs-tools/, re-generate the initrd and commit them like so:
update-initramfs -u -k all
In future we hope to utilise the boot-once option in extlinux, where by if the machine failed to boot from a kernel error, a subsequent boot will revert to the previous "known working" git-revision and kernel.
How do I roll back / downgrade ?
This is largely for debugging, using
update-revision=. Supported clients will "fast forward" out of trouble.
No upgrades on Live version since it isn't a writable disk
Two reboots for an update, <48hr delay
Webconverger currently only fetches on boot, so if an update is pushed, an already running system needs effectively two boots, one to fetch, and one to apply the changes to come up on the revision.
In practice a lot of machines are shutdown overnight. So for example an update that is pushed on a Monday afternoon, would be downloaded on the Tuesday boot and applied on the Wednesday boot.
We want to get this time down. For example an option that syncronously checks for new commits upon https://github.com/Webconverger/webc/commits/master, fetches and reboots immediately for security concious deployments.
noupgrade boot API disables upgrades
Disabling upgrades will disable Webconverger from doing a
git fetch on boot.
Notes on validation
- using hashes as update-revision, you'll do a sort of implicit validation
- signed tags might be useful, but we'll have to have some way to circumvent that so we can push debug revisions without having to tag all of them
- use https:// instead of git://
Quick notes on testing before a push
- Commit to master, make a note of sha1
git push origin master:testbefore pushing to master, push to a remote branch test
fetch-revision=test debugto your test instance of Webconverger
- Ensure sha1 was fetched
- Reboot and check the update was applied
- Test the update one more time
git pushthe chroot to master to let everyone else enjoy the update
- Perhaps tweet the change
- Clean up
git push origin :testand delete remote branch on https://github.com/Webconverger/webc
Note one should probably work from a branch, e.g.
git branch flashtest; git push origin flashtest -u and then merge to master for a push
TODOs and issues
- auto reboot
- dialog to choose install device
- TODOs in the code
- shallow fetches need to be investigated, since it can now sometimes end up downloading the complete history
- Cleanup of the .git directory needs to be investigated
- failover bootloader stuff
- make fetch happen periodically from /etc/inittab to reduce upgrade delay
- use https:// for firewalled deployments ?
For more background, see these gitupgrades notes.
Does git-fs use a lot of RAM?
Our memory requirements have gone up and by our calculations, it's about 200M of memory usage for git-fs, which isn't a lot.