Dragonfly is an intelligent P2P based file distribution system. It resolves issues like low-efficiency, low-success rate and waste of network bandwidth in large-scale file distribution scenarios such as application deployment, large-scale cache file distribution, data file distribution, image distribution etc.At Alibaba, the system transfers 2 billion times and distributes 3.4PB of data every month, it has become one of the most important piece of infrastructure at Alibaba. The reliability is up to 99.9999%.
DevOps takes a lot of benefits from container technologies, but at the same time, it also brings a lot of challenges: the efficiency of image distribution, especially when you have a lot of applications and require image distribution at the same time. Dragonfly works extremely well with both Docker and Pouch, and actually we are compatible with any other container technologies without any modifications of container engine.
It delivers up to 57 times the throughput of native docker and saves up to 99.5% the out bandwidth of registry.
Dragonfly makes it simple and cost-effective to set up, operate, and scale any kind of files/images/data distribution.
The project is an open source version of the dragonfly and more internal features will be gradually opened.
|Test Environment |||--------------------|-------------------||Dragonfly server|2 * (24core 64GB 2000Mb/s)||File Source server|2 * (24core 64GB 2000Mb/s)||Client|4core 8GB 200Mb/s||Target file size|200MB||Executed Date|2016-04-20|
For Dragonfly, no matter how many clients issue the file downloading, the average downloading time is always around 12 seconds.And for wget, the downloading time keeps increasing when you have more clients, and as the amount of wget clients reaches 1200, the file source will crash, then it can not serve any client.
Dragonfly is available under the Apache 2.0 License.
If you need commercial support of Dragonfly, please contact us for more information: 云效.
Dragonfly is already integrated with AliCloud Container ServicesIf you need commercial support of AliCloud Container Service, please contact us for more information: Container Service
git clone alibaba-Dragonfly_-_2018-06-14_09-56-44.bundle