I am thinking about how this "package management" should work in practice. I want to be able to download and update stuff using the game UI, but I also want to be able to update/manage stuff that the player put into the game data folder themselves.
I think one "mod" can contain multiple files (e.g. Caedes.ocf and Caedes.ocg). How would one find out which files in the game data folder belong to which mod? How to show the user which mods are installed?
If I used some sort of database to manage installed mods, I could not fluently manage manually added (e.g. downloaded from network games) packages.
And yes, those questions are the current blocker in my implementation of the ingame client for Kanibal's Larry.
> And yes, those questions are the current blocker in my implementation of the ingame client for Kanibal's Larry.
No new pictures :(
> Rest of story...
I don't understand what is exactly the problem. What is wrong with putting everything in the appdata folder and then either updating in your UI or when you join a game that hosts a newer version (i.e. you get a query to update or to not join). All stuff that is not on Larry could be added to the appdata folder by hand.
I think it is essential that people use Version.txt in their .ocd's .ocf's etc. so that the engine can determine whether an update is there.
Should I use the filenames? Or a checksum?
If I find matching filenames, how do I know which version of the mod they belong to? And ideally all that should be pretty fast
>No new pictures :(
That's because I didn't do anything yet!
>Well, imagine I have the files A.ocd, B.ocf and C.ocs in my game data folder. How do I know that A and B belong to mod X?
I don't know what this information is necessary for exactly. Update mod X? The information could be read from Scenario.txt: Everything that is part of Scenario.txt in C.ocs if obviously part of that mod, somehow.
A problem that I can imagine is the following: B.ocf needs version 1 of A.ocd, but C.ocs needs version 2 of A.ocd. Which one do you download?
In case of mods downloaded from Larry/CCAN 2.0/... I could think of letting clients download from that platform instead of using the current way of downloading and saving scenarios. That would at least ensure clients have the same version and if clients need to install a mod they could do this the same way as they would normally do when installing mods.
Maybe I better give an example for that:
Lets assume the host is hosting "DM_Knueppeln.c4s.ocs" from Knueppeln 0.8 and it has been installed from the integrated mod list. So the masterserver will know the host uses Knueppeln 0.8 from the mod list and clients will be able to see this and check whether Knueppeln 0.8 is installed or not. Assuming now it's not installed, any client which still needs it can download and install Knueppeln 0.8 from Larry (let's use Larry as the successor to CCAN in this case). Having downloaded and installed the Knueppeln package, this can be noted by the client in its package management.
Now let's do another scenario, but this time the scenario is not from any of the mods from the mods list. In this case the scenario and all the stuff it requires will be shared between clients normally (the 'classic' way). This might also be an alternative in case of Larry being offline for whatever reason.
Seing this situation I would probably tend to make a subfolder in the OC appdata folder containing version and package management for all the mods downloaded from Larry. Could look like this:
| | |-0.8 stable
| | | |-Knueppeln.c4d.ocd
| | | \-Knueppeln.c4f.ocf
| | \-0.9 beta
| | |-Knueppeln.c4d.ocd
| | \-Knueppeln.c4f.ocf
| \-Tower of despair
| \-0.5 stable
That's just the way how I would solve the versioning problem. This would at least allow the players to have multiple versions of a package installed without having conflicts on a mod being installed/present in a wrong version. Though it might need the mod developers to update dependencies on their scenarios.
For those mods which were manually added (either by network game saving or by manually putting the files to the OC appdata folder) I would say they might be shown in the list as well. Maybe they could be marked as manually installed or "unidentifiable", so the player can see this file(s) don't belong to any of the installed mods from the list.
Hope this post didn't get too confusing
Having this information "hacked" into the group headers makes the c4group interface more complicated. We would also lose the information on unpack+pack. Plus you couldn't keep the information in the version control but would have to pass it through the release pipelines somehow.
1. Fast pack/unpack. Avoid double compression of images and music. (Unpack performance is fine as-is, but there is a noticeable delay when hosting from a development tree.)
2. Transparent delta updates from the lobby. During development, it's very common to iterate quickly on a couple of scripts. Having to wait every time for the unchanged images and music is annoying.
3. Transparent web downloads from the lobby. Avoid the limited upstream from home ISPs when possible.
4. Versioning and dependency information. Version.txt is fine as API, but we'll want to do more with that information. There is some value in a fixed-format header field for external tools compared to unpacking and parsing a free-form text file.
I already have some ideas on what a new package format could look like. I'm going to write it down in the next days...
Additional notes for 1:
Compared to standard compression tools, c4group is unfortunately very much on the slow side of things:
% time tar caf Objects.ocd.tar.gz Objects.ocd
bsdtar caf Objects.ocd.tar.gz Objects.ocd 1.38s user 0.02s system 99% cpu 1.400 total
% time tar caf Objects.ocd.zip Objects.ocd
bsdtar caf Objects.ocd.zip Objects.ocd 1.31s user 0.04s system 99% cpu 1.354 total
% time tar cf Objects.ocd.tar Objects.ocd
bsdtar cf Objects.ocd.tar Objects.ocd 0.01s user 0.03s system 99% cpu 0.041 total
% time c4group Objects.ocd -p
c4group Objects.ocd -p 5.01s user 0.14s system 99% cpu 5.144 total
% ls -lS
-rw-r--r-- 1 lukas lukas 45582848 Jun 25 21:47 Objects.ocd.tar
-rw-r--r-- 1 lukas lukas 33014223 Jun 25 21:54 Objects.ocd.zip
-rw-r--r-- 1 lukas lukas 32753064 Jun 25 21:47 Objects.ocd
-rw-r--r-- 1 lukas lukas 32316248 Jun 25 21:46 Objects.ocd.tar.gz
So c4group is more than four times slower than bsdtar while producing larger files. Compression saves roughly 12 MB.
With these stats, I'd argue that current c4group spends more time (de-)compressing the groups than is saved by having to load less data from disk. (We can still do compression for network transfers. An online archive like Larry could save gzipped archives and serve these directly with
The more important data point would be how fast it is to load from the tar to memory versus the time to load from c4group to memory, because it determines the startup time for every single scenario. This would be disk IO time plus unpacking time. I'm pretty sure we can find a better format, but I also imagine that we would lose a lot if we can no longer pre-organize everything to be in the correct order for reading.
>I'm pretty sure that for the unpacking, the bottleneck is the way disk IO is handled by c4group (I think it's creating temp files and renaming them).
I got SSDs, so I don't care about disk I/O, and I think in a few years everybody will (only) have SSDs. So I guess we can still optimize for CPU and win a lot in the long run
I imagine we could be a lot faster if e.g. we pre-unpack groups into memory in a separate thread while the game is loading.
>I think in a few years everybody will (only) have SSDs.
Perhaps if they get less expensive and more space.
Do you own a USB flash drive? Are you aware they're so much more expensive than hard drives and do not have a lot of space?
Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill