Not logged inOpenClonk Forum
Up Topic Development / Developer's Corner / Mac Builds
- - By Mortimer [de] Date 2013-09-04 20:45
Concept:
-Some OSX machine sitting at my home which is in standby mode most of the time but can be woken up via WOL by accessing deenosaurierlair.de using ssh with special port 22222
-User is openclonk, does not get full login shell but some restricted thingie (Source code, node.js content ahead) meant for being flexible enough to produce
arbitrary builds built from arbitrary git branches checked out from main repository

==Example==
Some allowable commands separated by ';':
ssh -p 22222 openclonk@deenosaurierlair.de "git checkout master;git pull;xcodebuild -configuration Release -target openclonk"
Return code of ssh invocation is failure code or 0 if all commands finished successfully.
stdout should be build messages and such.

In case of success, another connection can be made to pipe the zipped build somewhere:
ssh -p 22222 openclonk@deenosaurierlair.de "zip_no_c4groups Release" > mac-release.zip

In the zip folder there is the skeleton of the Mac app bundle (a folder) into which Objects.ocd, System.ocd etc have to be put, exact folder is <bundle>.app/Contents/Resources
The end result should be a runnable Mac application

Stupid idea? I have no clue about network security? Mac builds are stupid anyways? Discuss.
Attachment: osxbuildling.js (6k)
Reply
Parent - - By Newton [vn] Date 2013-09-05 03:25
We have two options here:

1. Either the current autobuild server uses deenosaurier as a service to have a mac-build built, then downloads it from there and uploads it to openclonk.org via FTP, updates the version in the DB etc.
2. Or, deenosaurier is notified in the same commit hook as the win/linux autobuild server from git and does everything itself, no remote control: build mac-build, upload it to FTP, update version in the DB

Option 2 might be easier to code, but as yourself I lean towards option 1 since I think it is easier to manage plus the code from there can be reused. Also, we have only one autobuild server then, so only one system that has write access to FTP and DB. The other argument that speaks for option 1 is that on the win/linux autobuild server (you know, we really need a name for it ;-)), the OC* folders have already been created, so no need to do it twice.

However, for option 1 you need to stick your head together with Clonk-Karl or perhaps Isilkor if he has been wise enough to appoint a co-admin on the autobuild-stuff because he seems to be currently absent from the forum. Otherwise, just email him. There is a repository for the autobuild scripts, you will need access to that. (Perhaps just for reference, you should upload the osxbuildings.js there as well so that we have it somewhere in a repository.)
From that repository, you can also see how the other build targets are achieved, so building the final code for the osx-autobuilds might be as easy as copy&pasting from the linux build targets.

Clonk-Karl: Would it be sensible by the way to move the autobuild-scripts to a repository on git.openclonk.org? It would be good to have all reachable in one place, IIRC there is no sensitive information in the repository itself, is there?
Parent - By Isilkor Date 2013-09-05 12:03
ck's code is just packaging finished autobuilds; the actual build master/slave isn't currently publicly available but I'll clean up the code in the near future and push it somewhere.
Reply
Parent - - By Clonk-Karl [de] Date 2013-09-05 12:11

> However, for option 1 you need to stick your head together with Clonk-Karl


I think I have tried this before, but then lost it from the radar somehow... anyway my time is pretty limited at the moment, so I'm not sure I can incorporate this soon.

> Clonk-Karl: Would it be sensible by the way to move the autobuild-scripts to a repository on git.openclonk.org?


I certainly wouldn't mind. Someone would need to create a repository on git.openclonk.org and push the stuff there.
Reply
Parent - - By Newton [vn] Date 2013-09-11 15:40
Could you do that, Isilkor?
Parent - By Newton [vn] Date 2013-09-05 03:27

>Stupid idea? I have no clue about network security? Mac builds are stupid anyways? Discuss.


Also, start being proud on your achievements! I find this whole WOL stuff ingenious!
Parent - - By Newton [vn] Date 2013-09-12 12:55
Just a few abbreviations I want to introduce before this post: deeno = deenosaurier, Mortimer's build mac service; autobuild = Isilkor's autobuild scripts on nosebud; nightly = CK's development snapshot scripts

Based on CK's and Isilkor's reply I would suggest the following structure to enable even mac autobuilds and make the deenosaurier fit into the current build structure. The basic idea is to make Isilkor's autobuildscript handle deenosaurier like just another build slave:

On the git commit hook, autobuild makes deeno build a mac binary with the given git id, e.g. openclonk-mac-{gitid}, pipes the output that deeno gives into a log file and on success, copies the binary via scp onto nosebud into the same place where the other logs and binaries are. This should happen parallel to triggering it's own build slaves for win and linux so autobuild doesn't need to wait for deeno to finish his build. Deeno on the other hand should automatically clean up old binaries (e.g. older than a day) after the build.
This way, we can have mac-autobuilds.
For the development snapshots, nightly just has to be extended to create snapshots not for 4 but 5 different platforms. Thus, the "building the skeleton of a mac app" of deeno is moved to nightly.

Sounds good? Am I overlooking something?
Parent - - By Isilkor Date 2013-09-13 12:18
I need to rejigger the autobuild master, because at the moment it assumes every slave can build the exact same architectures, so don't expect the mac builds to happen instantly.
Reply
Parent - By Newton [th] Date 2014-04-21 11:35
So, whats the status?

I'd really like to have this finally off the stove (off the watchlist) since I can't imagine that in the status it is now (Mortimer's part done, CK and Newton waiting for autobuilds to be available) it will take much effort to get it running.
Parent - - By Mortimer [us] Date 2013-09-13 16:26
Sounds good, but I am in the US for the next 2 weeks so I might not be readily available although I might have time in the evening to fix issues preventing the scheme from working.
Reply
Parent - - By Isilkor Date 2013-09-13 20:38
No that's fine, in two weeks I may have fixed the build master. (Knowing myself, I'll probably have procrastinated instead.)
Reply
Parent - - By Mortimer [de] Date 2013-11-09 00:53
*Poke*
Reply
Parent - By Isilkor Date 2013-11-09 10:46
Did I say two weeks? I meant two quarters. *cough*
Reply
Parent - - By Newton [vn] Date 2013-09-14 08:29
Nice btw, what are you doing in the USA?
Parent - By Mortimer [us] Date 2013-09-14 14:16
Just having a vacation, seeing New York, San Francisco and stuff.
Reply
Parent - - By Newton [th] Date 2013-10-16 05:19
Hey guys, whats the status?

I am currently redesigning the downloads and snapshots page. Would be great if I could already include the macs in there.
Parent - By Mortimer [de] Date 2013-10-16 16:22
The next thing I will get to.
Reply
Parent - By Clonk-Karl [de] Date 2013-10-16 17:14
No progress on my side so far... we don't have the mac autobuilds yet, right?
Reply
- - By Newton [th] Date 2013-10-16 06:19
This has nothing to do directly with the mac builds, I post this here because this could perhaps be done by the way with the release scripts available in repos now:

For the redesign of the downloads page, I want to include a link to the source code package of the current version that is automatically created alongside the other architectures. Till now the source tarball wasn't created automatically but Isilkor had to build and upload it manually each time. As the source code package is now a essential part of the release (pkern needs it for Debian packaging), I would like to get rid of this step.

Looking at the source, there is already a TODO for this ;-)

Isilkor, you used to create these tarballs. Do you have a script for that which you could insert into the release scripts? I had a look into the source and I would have a hard time implementing it since I don't really know Python plus didn't set up a debugging environment, so perhaps you or Clonk-Karl could volunteer for this?
Parent - - By Isilkor Date 2013-10-16 14:02

> Do you have a script for that which you could insert into the release scripts?


$ git archive --prefix=openclonk-release-$VERSION-src/ -o openclonk-release-$VERSION-src.tar.gz v$VERSION should work
Reply
Parent - - By Newton [th] Date 2013-10-16 18:19
I'd prefer
$ git archive --prefix=openclonk-$VERSION-src/ -o openclonk-$VERSION-src.tar.bz2 v$VERSION
to have it in the same format as the other packages. But how to define that git should output it as bz2? In the manpage it is mentioned under "Configuration"
Parent - By Isilkor Date 2013-10-17 10:22
Make sure git supports it ($ git archive -l). If not, you'll have to generate a plain .tar archive and pipe that through bzip2 ($ git archive --prefix=openclonk-$VERSION-src/ v$VERSION | bzip2 > openclonk-$VERSION-src.tar.bz2)
Reply
Up Topic Development / Developer's Corner / Mac Builds

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill