Abstract
MapReduce has become the dominant programming model in a cloud-based data processing environment, such as Hadoop. First In First Out (FIFO) is the default job scheduling policy of Hadoop, but it cannot guarantee that the job will be completed by a specific deadline. Research has been focused on developing deadline-based MapReduce schedulers by using the non-preemptive scheduling approach. However, compared with the non-preemptive scheduling approach, the preemptive scheduling approach has some advantages, such as the total completion time and slot utilization. In this paper, we first formulated the preemptive scheduling problem under deadline constraint, and then we proposed preemptive scheduling algorithms. To our knowledge we implemented the first real preemptive job scheduler to meet deadlines on Hadoop. The experimental results indicate that the preemptive scheduling approach is promising, which is more efficient than the non-preemptive one for executing jobs under a certain deadline. Copyright © 2012 IEEE.
Original language | English |
---|---|
Title of host publication | Proceedings of 2012 Eighth International Conference on Semantics, Knowledge and Grids, SKG 2012 |
Place of Publication | Danvers, MA |
Publisher | IEEE |
Pages | 72-79 |
ISBN (Print) | 9780769547947 |
DOIs | |
Publication status | Published - 2012 |