Hi guys!
I had this thought recently when using my mediafire account- it sucks... probably it doesn't suck more than average filehosting account, but still...
The idea (in fact two of them) I came up is the following:
Distribute evrything via bittorrent :)
For now I can't see any major downsides of the concept - but I think it'll be most useful for bigger distributions that build daily builds and/or support many phones. It could considerably ease the load on their uplinks thanks to the way bt works.
How I imagine it to work:
The person who would like to contribute would downoad the client, install it on his server/pc, register to the central repository, and in the end set a few config options (bandwidth he wants to contribute, how much disk space, and to what projects/phones/developers) and that's it. The client would then ask the central repo for some torrents to mirror and it'll run by itself (periodically asking the repo for new downloads or removals) - hopefully without any maintanance ever needed.
I imagine "the client" to be as simple as possible - maybe even a one shell script using curl to communicate with the repo. And of course some torrent client - if the project develops, support for many clients (and windows) can be added.
The server would do all the thinking as it'd have the most complete information about the entire "pool" - how much space is available to which projects, how many dowloads of each torrent there are etc. so it'd be easier for it to decide who should mirror what (of course within their defined filters). But in the beginning I don't think it would have to be very complicated either - if more clients connect, the algorithm can be made smarter.
The thing is - many of your users would be happy to contribute in some way to rom development, but usually they lack the skill necessary. But they could contribute hardware/bandwidth, especially that it doesn't require any effort from them and gives them the satisfaction of contributing.
The other thing is distributed build system based on similar principles: provide people with a configured VM image of a building system for various virtualization technologies and operating systems and that's it. The initial setup would be made as simple as possible for the end user. Having one vmimage would guarantee that everybody has consistent build environments.
This would of course be useful only to large android rom distributions that support multiple devices - building 20+ nightlies/a day (and distributing them) can be quite a challenge and requires considerable processing power and bandwidth, but if they had a farm of build servers (even small), this wouldn't be a problem and they could commit their hardware for pure development/debuging efforts.
Of course the system would inform the developer if his build failed somewhere and provided a log, so that he (or a person responsible) can investigate. The system could also do some automated recovery, i.e.: build fails -> clean ccache and rebuild, if fails, restart on another machine and if this fails, inform the developer. The system could also detect a faulty build machine (that fails some builds that run successfuly elsewhere) and exclude it from the pool (and of course send an email to machine owner). And of course ready builds would be reported to the repo server and distributed via torrent.
The whole trick is that for both distribution and compilation none individual contribution has to be large - users can dedicate 1GB of mirror space and it'll allow to host 4-5 rom builds. For build machines, they don't need to have a 6-core monster because it doesn't matter if it builds 1,2 or 3 hours as long as it builds (my 4x3.2GHz machine builds a rom in <90mins). What's more, they don't have to run those severs 24/7 - the system will know which machines are always on and which are not and distribute the load taking this information into account.
When the community grows beyond some critical mass, the system will work despite people droping in and out of the "grid"...
Just a loose idea... :) Tell me what you think.
I had this thought recently when using my mediafire account- it sucks... probably it doesn't suck more than average filehosting account, but still...
The idea (in fact two of them) I came up is the following:
Distribute evrything via bittorrent :)
For now I can't see any major downsides of the concept - but I think it'll be most useful for bigger distributions that build daily builds and/or support many phones. It could considerably ease the load on their uplinks thanks to the way bt works.
How I imagine it to work:
The person who would like to contribute would downoad the client, install it on his server/pc, register to the central repository, and in the end set a few config options (bandwidth he wants to contribute, how much disk space, and to what projects/phones/developers) and that's it. The client would then ask the central repo for some torrents to mirror and it'll run by itself (periodically asking the repo for new downloads or removals) - hopefully without any maintanance ever needed.
I imagine "the client" to be as simple as possible - maybe even a one shell script using curl to communicate with the repo. And of course some torrent client - if the project develops, support for many clients (and windows) can be added.
The server would do all the thinking as it'd have the most complete information about the entire "pool" - how much space is available to which projects, how many dowloads of each torrent there are etc. so it'd be easier for it to decide who should mirror what (of course within their defined filters). But in the beginning I don't think it would have to be very complicated either - if more clients connect, the algorithm can be made smarter.
The thing is - many of your users would be happy to contribute in some way to rom development, but usually they lack the skill necessary. But they could contribute hardware/bandwidth, especially that it doesn't require any effort from them and gives them the satisfaction of contributing.
The other thing is distributed build system based on similar principles: provide people with a configured VM image of a building system for various virtualization technologies and operating systems and that's it. The initial setup would be made as simple as possible for the end user. Having one vmimage would guarantee that everybody has consistent build environments.
This would of course be useful only to large android rom distributions that support multiple devices - building 20+ nightlies/a day (and distributing them) can be quite a challenge and requires considerable processing power and bandwidth, but if they had a farm of build servers (even small), this wouldn't be a problem and they could commit their hardware for pure development/debuging efforts.
Of course the system would inform the developer if his build failed somewhere and provided a log, so that he (or a person responsible) can investigate. The system could also do some automated recovery, i.e.: build fails -> clean ccache and rebuild, if fails, restart on another machine and if this fails, inform the developer. The system could also detect a faulty build machine (that fails some builds that run successfuly elsewhere) and exclude it from the pool (and of course send an email to machine owner). And of course ready builds would be reported to the repo server and distributed via torrent.
The whole trick is that for both distribution and compilation none individual contribution has to be large - users can dedicate 1GB of mirror space and it'll allow to host 4-5 rom builds. For build machines, they don't need to have a 6-core monster because it doesn't matter if it builds 1,2 or 3 hours as long as it builds (my 4x3.2GHz machine builds a rom in <90mins). What's more, they don't have to run those severs 24/7 - the system will know which machines are always on and which are not and distribute the load taking this information into account.
When the community grows beyond some critical mass, the system will work despite people droping in and out of the "grid"...
Just a loose idea... :) Tell me what you think.
Aucun commentaire:
Enregistrer un commentaire