I was evaluating 'mesos' as one of the options for scheduling distributes application resources on a cluster. I did a quick prototype of mesos based application framework using python to check its viability for our use case. I captured the prototype details below hoping it may come in handy for anyone considering mesos as an alternative for scheduling distributed applications.

Let me clarify that by mesosI am referring to the scheduler itself and not the mesos ecosystem like mesosphere, CNI, etc. My requirement is to schedule processes and/or containers of a distributed applications across a set of VM or bare metal.

Mesos is a lightweight and scalable framework which is easy to install and integrate with most of the applications. Many applications like x,y,z, use mesos under the hood for its scheduling.

A quick primer on mesos architecture

Mesos provides a loosely coupled interface to collate available resources and to schedule tasks as per application framework/scheduler’s directive.

Mesos primarily have a mastertask and a set of agent/slave tasks. Mesos slave/agent tasks report the resources available on its machine to the master task. Master task collates the resources and publishes available resources to Application Scheduler. Application scheduler accepts the offer and specifies the task details (process/container) to be created. Mesos master picks up an appropriate mesos agent and directs it to spawn the task. Mesos slave/agent spawns the requested task and monitors the state. When the task state or resource state changes it reports it back to master.

For more details about mesos scheduler refer to this page: Mesos Architecture (https://mesos.apache.org/documentation/latest/architecture/ )

Setting up Mesos application framework/scheduler

Add Mesos repository

$ sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm

Installing & Configuring Master node

Install mesos and zookeeper

$ sudo yum -y install mesos mesosphere-zookeeper
$ sudo service zookeeper start
$ sudo service mesos-master start
$ sudo systemctl enable mesos-master.service
$ sudo systemctl disable mesos-slave.service

Configure mesos master

#Master IP of mesos
sudo su
echo "192.168.1.1" > /etc/mesos-master/ip 
#Host name of master node
echo "mesos-master" > /etc/mesos-master/hostname
#Cluster name
echo "mesos-cluster" > /etc/mesos-master/cluster

Configure zookeeper on master

echo "1" >> /var/lib/zookeeper/myid
echo "server.1=192.168.1.1:2888:3888" > /etc/zookeeper/conf/zoo.cfg
echo "zk://192.168.1.1:2181/mesos" > /etc/mesos/zk
Note: To add additional master nodes, add additional ID in myid file for each master node and "server.n" config in zoo.cfg.

Restart service and make suremesos master is running:

$ service zookeeper restart
$ service mesos-master restart
$ service mesos-slave restart
$ curl 127.0.0.1:5050

Installing & Configuring Slave nodes

Install mesos

sudo yum -y install mesos 
sudo service mesos-slave start
sudo systemctl enable mesos-slave.service
sudo systemctl disable mesos-master.service

Configure mesos slave

echo "zk://192.168.1.1:2181/mesos" > /etc/mesos/zk
echo "192.168.1.2" > /etc/mesos-slave/ip
Note: Repeat the above config for all slave nodes and set appropriate IP address in its configuration.

Install PyMesos

Install python library pymesosfor interfacing with mesos master

yum groupinstall'Development Tools'
yum install python
curl "https://bootstrap.pypa.io/get-pip.py" | python 
yum install python-devel
pip install pymesos

Sample Code

https://github.com/smurli/pymesos-sample