ADAPTIVE MEMORY ALLOCATION MODEL IN MULTI-CORE MACHINES CLUSTER

There are many scheduling approaches in computing research, most of which are loaded with assumptions, in this work we try to bring some different approach of an adaptive model of a cluster that virtually changes its capacities and resource to solve an ‘ancient’ problem.

We used as a case study, where we construct a topology of N homogenous multicore machines formed as a logical cluster, with a shared memory resource R, and S number of processing cores each. These machines execute a stream of jobs J_j, from multiple queues.

Fig. 1 Logical flow of jobs.

As we can see from Fig. 1, the scheduler has to do the match-making between jobs demands and the available resources of cores and memory. Managing both cores and memory, which constrain each other, i.e. two cores on the same machine have access to shared memory of 4 GB, is a complex task. We propose, as described on Fig. 2, to create an abstraction of resources which represent three virtual clusters of resources, and to manage them separately.

Fig. 2 Abstraction of cluster resources.

We create three virtual clusters, with adaptive number of cores, that already have the sufficient suitable amount of memory for the incoming jobs. The incoming jobs will be placed in three different queues, and if we consider that one of the queues is overloaded, we can simply provision more cores to the relevant pool. It allows us to design a simple provisioning method, where we need to define only the rules when to provision more pools to a queue.

Fig. 3 Flow of jobs with virtual clusters.

In this work, we addressed a scheduling problem in a cluster of computers, where each machine has two cores and a shared memory of 4 GB. The model is innovative since it makes no assumption on job arrivals or processing times. In addition, the resources were represented as designated pools of computing resources with pre-assigned memory capacities of 1, 2 or 3 GB, thus allowing easier provisioning of cores, and allow us to focus on the scheduling strategy. We have shown that the model is adaptive, and it is adequately robust.

This model can serve as a framework for developing some more complex models in the future. The model in this work defined only three type of jobs with memory requirements, while in real-life the memory requirement have a continuous nature. Future works may explore a more sophisticated representation of the core and memory allocation “duo”. Moreover, some future research may address the provision strategies, explicitly the thresholds by which the changes in provisioning take place. For example, having a dynamic threshold which changes with the statistics of the system, to allow more sophisticated switch between states.