I was evaluating 'mesos' as one of the options for scheduling distributes application resources on a cluster. I did a quick prototype of mesos based application framework using python to check its viability for our use case. I captured the prototype details below hoping it may come in handy for anyone considering mesos as an alternative for scheduling distributed applications.
Let me clarify that by mesosI am referring to the scheduler itself and not the mesos ecosystem like mesosphere, CNI, etc. My requirement is to schedule processes and/or containers of a distributed applications across a set of VM or bare metal.
Mesos is a lightweight and scalable framework which is easy to install and integrate with most of the applications. Many applications like x,y,z, use mesos under the hood for its scheduling.
A quick primer on mesos architecture
Mesos provides a loosely coupled interface to collate available resources and to schedule tasks as per application framework/scheduler’s directive.
Mesos primarily have a mastertask and a set of agent/slave tasks. Mesos slave/agent tasks report the resources available on its machine to the master task. Master task collates the resources and publishes available resources to Application Scheduler. Application scheduler accepts the offer and specifies the task details (process/container) to be created. Mesos master picks up an appropriate mesos agent and directs it to spawn the task. Mesos slave/agent spawns the requested task and monitors the state. When the task state or resource state changes it reports it back to master.
For more details about mesos scheduler refer to this page: Mesos Architecture (https://mesos.apache.org/documentation/latest/architecture/ )
Setting up Mesos application framework/scheduler
Add Mesos repository
$ sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
Installing & Configuring Master node
Install mesos and zookeeper
$ sudo yum -y install mesos mesosphere-zookeeper $ sudo service zookeeper start $ sudo service mesos-master start $ sudo systemctl enable mesos-master.service $ sudo systemctl disable mesos-slave.service
Configure mesos master
#Master IP of mesos sudo su echo "192.168.1.1" > /etc/mesos-master/ip #Host name of master node echo "mesos-master" > /etc/mesos-master/hostname #Cluster name echo "mesos-cluster" > /etc/mesos-master/cluster
Configure zookeeper on master
echo "1" >> /var/lib/zookeeper/myid echo "server.1=192.168.1.1:2888:3888" > /etc/zookeeper/conf/zoo.cfg echo "zk://192.168.1.1:2181/mesos" > /etc/mesos/zk
Note: To add additional master nodes, add additional ID in myid file for each master node and "server.n" config in zoo.cfg.
Restart service and make suremesos master is running:
$ service zookeeper restart $ service mesos-master restart $ service mesos-slave restart $ curl 127.0.0.1:5050
Installing & Configuring Slave nodes
sudo yum -y install mesos sudo service mesos-slave start sudo systemctl enable mesos-slave.service sudo systemctl disable mesos-master.service
Configure mesos slave
echo "zk://192.168.1.1:2181/mesos" > /etc/mesos/zk echo "192.168.1.2" > /etc/mesos-slave/ip
Note: Repeat the above config for all slave nodes and set appropriate IP address in its configuration.
Install python library pymesosfor interfacing with mesos master
yum groupinstall'Development Tools' yum install python curl "https://bootstrap.pypa.io/get-pip.py" | python yum install python-devel pip install pymesos