Unfortunately, I am being forced to switch off public access to the nightly builds. I had not foreseen the huge interest, and do not have the infrastructure to handle it. The nightly builds are currently hosted on a private server I have on a web hotel, and I did most definitely not buy the net capacity nor the machine to host this kind of traffic.
The last few days the machine has served 10-15 gigabytes per day, and then only counting nightly builds.
Me and Manuel are discussing different ways to solve this, and will most likely figure a way around the problem soonish. Until then, the nightly builds are password protected so that at least the devs have somewhere to go.
If you have suggestions you're welcome to publish them as comments here. Some things we've discussed so far are:
- Launchpad (but that only works for debs, so would exclude tarballs and the windows builds)
- Sourceforge (only allows downloads of a single files a time from a pool of mirrors, won't work with deb repositories)
- Hosting at a university (but my department boss looked rather doubtful when I mentioned it involved 15gb traffic per day :) )
I dont know if this makes sense.. but one way is to reduce the too many unneeded files (or have the extra files as "additional" download.. so the nightly builds will only have the necessary changes: mainly *.exe, *.py and the new targets after alpha4).. This may sound messy.. but it sure reduces the bandwith by a very significant amount.
ReplyDeleteBuilding alpha4 I get only 20Mb. But your Builds are around 40Mb.. is this due to new targets?
Yes, separating the build into several packages is (on a longer term) a good idea, at least when the static data files stop changing so much. At the moment, the data parts change at least as much as the code parts though, so right now it would probably cause more problems than it solves to separate the packages.
ReplyDeleteOn a longer term, splitting the distribution into makehuman and makehuman-data would probably make a lot of sense.
I can't really say why the nightlies are so much bigger than the alpha, but a lot of work has been done with targets after the alpha (for example breast tool, nose tool, hairs...).
As a further comment to the above, there's a nice diagram in a forum post showing the balance between data and code. For hard numbers, the installation size is 120 mb. Of this 105 mb is data, and 13 mb is "utils" (tools to make new targets etc) and 2 mb is "other stuff" which includes the actual executable.
ReplyDeleteOf the data, 86 mb are targets.
Interesting! But, you know, every morning it's been a blast to download the Nightly build to see what new cool stuff has been added. It was only yesterday that the hip morphing tool worked the way I expected it to.
ReplyDeleteIt is such a great piece of software. I've contributed a model based on the January 28 Ubuntu build to the gallery. Many thanks for all your hard work.
Upload weekly builds to some hoster like Rapidshare? It would involve some manual work, but would also move the bandwidth problem somewhere else.
ReplyDeleteI am against rapidshare, they are terrible to free users now, if you are going to do something like that, beeter go with some other fielhosting service.
ReplyDeleteI have used https://www.dropbox.com and till now its best i have seen.
You migh even contact them and ask for better plan in exchange for holding their banner on site.
Other possible solution is to ask someone from some big opensource project for help (blender).
Also are you sure that real people are downloading these builds? Maybe adding some captcha could help?
Hosting it on google projects??
Uh, i was wrong, it was http://drop.io/ not dropbox, sry
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteMonthlies and xdelta files?
ReplyDeleteOr just a visible (bittorrent?) link for a monthly release on the front page. If there's a recent, endorsed development release, accesses to nightlies go down a lot, I bet.
This comment has been removed by the author.
ReplyDeleteyou could use diffrent download options.
ReplyDeletemega downloads,bit torrent, tons of them out there, still can't figure out how to get the windows build to work myself so only dl'ed the one myself :p
Ok, some news updates:
ReplyDeleteSeems we're going to split the distribution.
- We'll post dumb tarballs and zips on google code once we finish the upload script. We won't keep any history nightlies but instead overwrite the same statically named file again and again. (this should also allow people to make a scripted download even if they're not using debs)
- We'll set up a debian repository on launchpad to also support smarter downloading of nightlies. I've started this process, but also got an offer of help from someone who had done it before, so I'm feeling confident this should work out in the end.
- I'm also seriously considering the suggestion of splitting the distribution into two or three packages (makehuman-data, makehuman-executable and possibly makehuman-utils... they'd be 85, 2 and 15 megabytes large of installed size)
All in all this will take a few days, mainly depending on how long launchpad takes to finish the import request.
Happy New Year~~!!!............................................................
ReplyDeleteThe bit torrent site http://www.mininova.org/ , a former pirate site now gone legit (or so I've read) could probably do this without any problem. I don't know how difficult it is to upload to a torrent site as I've never done it, but it might be worth looking into. And thank you so much for the wonderful program, which I just started to use. It's great to see how good GNU software is becomming.
ReplyDeleteDavid
Early experiments show that google code can be used to host files via scripted uploads.
ReplyDeletehttp://code.google.com/p/makehuman/downloads/list
Once we've validated all looks sane I'll enable the nightly script too. But at least you have today's files there for now.
Wow! Since last week, I can now: alter scale
ReplyDeleteon the hips; alter the nose (which I need, as I am sculpting an Innu--Eskimo--face) ; export hair as mesh or curve. That's why the nightly builds were being downloaded so hard. Makehuman is on its way to being a major success.
Joel, the nicest thing for most Linux users and distro maintainers to have is a date or version appended source snapshot of the build, a nice make / cmake / ... build script, and the actual build log from you guys (which is most useful if it includes which libraries the program was built against, and the commands issued by your build server). If these are coherently available for every release including the nightlies and full releases, that's easiest to work with.
ReplyDeleteIf you do it like this, it will also not be an issue at all if you take the non-program content out of the source snapshot. Just version that content, too, and we'll be able to figure out from the logs which additional tarball we need, and this can then directly be represented in a distro's packaging.
As for the dumb tarballs on google, I'd only bother to put up (mostly?) statically linked binaries for deployment in an average linux user's home dir or on an average windows install that way. Having dynamically linked binaries or source code up like that will just result in a support nightmare.
Just use a file hosting service like rapidshare, megaupload or mediafire. They're pretty good and people who want nightlies can live with less-than-one-minute delays before starting their downloads. There, problem solved...
ReplyDelete