It is still an ongoing work not ready for production, but the upstream version of OpenShift origin has already an experimental support for running OpenShift Origin using system containers. The “latest” Docker image for origin, node and openvswitch, the 3 components we need, are automatically pushed to docker.io, so we can use these for our test. The rhel7/etcd system container image instead is pulled from the Red Hat registry.

This demo is based on these blog posts www.projectatomic.io/blog/2016/12/part1-install-origin-on-f25-atomic-host/ and www.projectatomic.io/blog/2016/12/part2-install-origin-on-f25-atomic-host/ with some differences for the provision of the VMs and obviously running system containers instead of Docker containers.

The files used for the provision and the configuration can also be found here: https://github.com/giuseppe/atomic-openshift-system-containers, if you find it easier than copy/paste from a web browser.

In order to give it a try, we need the latest version of openshift-ansible for the installation. Let’s use a known commit that worked for me.

1
2

$ git clone https://github.com/openshift/openshift-ansible.git
$ git checkout a395b2b4d6cfd65e1a2fb45a75d72a0c1d9c65bc

To provision the VMs for the OpenShift cluster, I’ve used this simple Vagrantfile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

BOX_IMAGE = "fedora/25-atomic-host"
NODE_COUNT = 2

# Workaround for https://github.com/openshift/openshift-ansible/pull/3413 (which is not yet merged while writing this)
SCRIPT = "sed -i -e 's|^Defaults.*secure_path.*$|Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin|g' /etc/sudoers"

Vagrant.configure("2") do |config|
  config.vm.define "master" do |subconfig|
    subconfig.vm.hostname = "master"
    subconfig.vm.network :private_network, ip: "10.0.0.10"
  end
  
  (1..NODE_COUNT).each do |i|
    config.vm.define "node#{i}" do |subconfig|
      subconfig.vm.hostname = "node#{i}"
      subconfig.vm.network :private_network, ip: "10.0.0.#{10 + i}"
    end
  end

  config.vm.synced_folder "/tmp", "/vagrant", disabled: 'true'
  config.vm.provision :shell, :inline  => SCRIPT
  config.vm.box = BOX_IMAGE

  config.vm.provider "libvirt" do |v|
    v.memory = 1024
    v.cpus = 2
  end

  config.vm.provision "shell" do |s|
    ssh_pub_key = File.readlines(ENV['HOME'] + "/.ssh/id_rsa.pub").first.strip
    s.inline = <<-SHELL
      mkdir -p /home/vagrant/.ssh /root/.ssh
      echo #{ssh_pub_key} >> /home/vagrant/.ssh/authorized_keys
      echo #{ssh_pub_key} >> /root/.ssh/authorized_keys
      lvextend -L10G /dev/atomicos/root
      xfs_growfs -d /dev/mapper/atomicos-root
    SHELL
  end
end

The Vagrantfile will provision three virtual machines based on the `fedora/25-atomic-host` image. One machine will be used for the master node, the other two will be used as nodes. I am using static IPs for them so that it is easier to refer to them from the Ansible playbook and to require DNS configuration.

The machines can finally be provisioned with vagrant as:

1

# vagrant up --provider libvirt

At this point you should be able to login into the VMs as root using your ssh key:

1
2
3
4

for host in 10.0.0.{10,11,12};
do
    ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@$host "echo yes I could login on $host"
done
y y y e e e s s s I I I c c c o o o u u u l l l d d d l l l o o o g g g i i i n n n o o o n n n 1 1 1 0 0 0 . . . 0 0 0 . . . 0 0 0 . . . 1 1 1 0 1 2

Our VMs are ready. Let’ install OpenShift!

This is the inventory file used for openshift-ansible, store it in a file origin.inventory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
etcd

# Set variables common for all OSEv3 hosts
[OSEv3:vars]
ansible_user=root
ansible_become=yes
ansible_ssh_user=vagrant
containerized=true
openshift_image_tag=latest
openshift_release=latest
openshift_router_selector='router=true'
openshift_registry_selector='registry=true'
openshift_install_examples=False

deployment_type=origin

###########################################################
#######SYSTEM CONTAINERS###################################
###########################################################
system_images_registry=docker.io
use_system_containers=True
###########################################################
###########################################################
###########################################################

# enable htpasswd auth
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]
openshift_master_htpasswd_users={'admin': '$apr1$zgSjCrLt$1KSuj66CggeWSv.D.BXOA1', 'user': '$apr1$.gw8w9i1$ln9bfTRiD6OwuNTG5LvW50'}

# host group for masters
[masters]
10.0.0.10 openshift_hostname=10.0.0.10

# host group for etcd, should run on a node that is not schedulable
[etcd]
10.0.0.10 openshift_ip=10.0.0.10

# host group for worker nodes, we list master node here so that
# openshift-sdn gets installed. We mark the master node as not
# schedulable.
[nodes]
10.0.0.11 openshift_hostname=10.0.0.11 openshift_schedulable=true openshift_node_labels="{'region': 'primary', 'router':'true'}"
10.0.0.12 openshift_hostname=10.0.0.12 openshift_schedulable=true openshift_node_labels="{'region': 'primary', 'registry':'true'}"

The new configuration required to run system containers is quite visible in the inventory file. `use_system_containers=True` is required to tell the installer to use system containers, `system_images_registry` specifies the registry from where the system containers must be pulled.

And we can finally run the installer, using python3, from the directory where we forked ansible-openshift:

1

$ ansible-playbook -e 'ansible_python_interpreter=/usr/bin/python3' -v -i origin.inventory ./playbooks/byo/config.yml

After some time, if everything went well, OpenShift should be installed.

To copy the oc client to the local machine I’ve used this command from the directory with the Vagrantfile:

1

$ scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i .vagrant/machines/master/libvirt/private_key [email protected]:/usr/local/bin/oc /usr/local/bin/

As non root, let’s login into the cluster:

1

$ oc login --insecure-skip-tls-verify=false 10.0.0.10:8443  -u user -p OriginUser

1
2
3
4
5

Login successful.

You don't have any projects. You can try to create a new project, by running

    oc new-project <projectname>

1
2
3
4
5
6
7
8

$ oc new-project test
Now using project "test" on server "https://10.0.0.10:8443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git

to build a new example application in Ruby.

1

$ oc new-app https://github.com/giuseppe/hello-openshift-plus.git
F C i i b d s S B R o r m m u e e u u u u A A T P W e a a i p r c i n n n h o A a g g l l v c l d D T E i r O R t e e d o i e d i o h v s t t N i s s c y c s o D m c e e s h I n t t o m e s s c o a k r i e N g r r n e c c g e r y m 8 r G e e f n " h s k e r e a 0 : r a a i t h e t e s t g 8 c e m m g c e d a r s b u i e 0 o I s o l u t t u l m , n m o " " " n l l u i r i t e w t a u f h h f o e s m e l i i 8 a g r e e e i - d ' a a d n " l 8 i e c d l l g o , g m g f l 8 n e o l l p t e u e 8 e " s r o o " e u o w s i d b r f a - - h n s 1 i i m o e w s e " o o e s e v f l n a r i d p p l h i 8 l g g a d l c o c e e l i e e e : e l a r r n n o f o w c b s l p n a e s s - t c 1 e o w a l b " a h h o - y 1 u i t o e a t i i p p l o c r l e y c r e f f e l o u ( r c l s e l c u d t t n u g r 6 e e t d o e n - - s s s a b " a s s p p h " a d t c e i d s l l i - p a e o c n a u u f c f p y d d p h b t s s s t r . s e u a d a h " " - e b a s n e l i t p a c o s f h g p a s h c c l t / l r e e l n e r r u e h d " o d s o c s e e s d e ) f m y e e ' a a " l e t a m d r r t t l f d h o e v o e e c o r o t n n b i o d d r - o r t i e t y c t e o m a p m w e ' a p : s a c s t e D l : g b o e t u e n o a / e u n r h s d s c t / i f v r e h k e g s l i i o r i e s i t d g c u f r t t r e g w t " h e w " h h - H u a i h " i p u t b m l e h t c l b h . l l e h h u a c " l l e s f t o h b o l m ' o m e e - o h a r w / l o - o y t i g l t p o s o " l i o r e p t n f l u - i n e n o t e s o g s n a t r d t e p g h s m a o r p e e i h e b c r a p n r f i e k a c e s e t f " " k / h d - t h p i h i p - e e t t e f l p l r s h l t u l l m e l - s u o i p o p " s - t r s - l " o t o o o u p e g u p s e d r r e : n e c n l s b s e s a h y s h t i . i i e f y m f s t o a t t - u g - " p r e p l l u c u s l s " u . s g t i e t r w a i d l m l i n b i e s t c r r a e t a o t r e d

After some time, we can see our service running:

1

oc get service
N h A e M l E l o - o p e n s h i f t - p l u s C 1 L 7 U 2 S . T 3 E 0 R . - 2 I 0 P 4 . 1 4 0 E & X l T t E ; R n N o A n L e - > I P P O R T 8 ( 0 S 8 ) 0 / T C P , 8 8 8 8 / T C P A G E 4 6 m

Are we really running on system containers? Let’s check it out on master and one node:

The atomic command upstream has a breaking change so with future versions of atomic we will need -f backend=ostree to filter system containers, as clearly ostree is not a runtime.

1
2
3
4

for host in 10.0.0.{10,11};
do
    ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@$host "atomic containers list --no-trunc -f runtime=ostree"
done
C e o C o o O t r O r p N c i N i e T d g T g n A i A i v I n I n s N - N - w E m E n i R a R o t s d c I t I e h D e D r 1 1 I 9 9 I 1 1 M 2 2 M 9 9 A . . A 2 2 G 1 1 G . . E 6 6 E 1 1 8 8 6 6 . . 8 8 1 1 . . . . 1 1 1 1 . . 3 3 1 1 : : 3 3 5 5 : : 0 0 5 5 0 0 0 0 0 0 0 0 / 0 0 o / / p p r o e e h p n n e e s s l n h h 7 s i i / h f f e i t t t f / / c t n o d / o p o d e r e n i : v g l s i a w n t i : e t l s c a t h t : e l s a t t e C / / s O u u t M s s M r r / / A / / C u u N b l O s s D i o M r r n c M / / / a A l l e l N o o t / D c c c b a a d i l l - n / / e / b b n s i i v y n n . s / / s t s s h e y y m s s / - t t u c e e s o m m r n - - / t c c b a o o i i n n n n t t / e a a e r i i t - n n c w e e d r r r a - - p w w p r r e a a r p p . p p s e e h r r . . C 2 2 s s R 0 0 h h E 1 1 A 7 7 2 2 T - - C 0 0 E 0 0 R 1 1 D 2 2 E 7 7 - - A - - 2 2 T 0 0 3 3 E 2 2 D - - 1 1 2 2 1 1 3 3 : : 0 1 1 1 1 0 1 1 : : S r r 1 1 T u u 7 8 A n n T n n r r E i i S u u n n T n n g g A n n T i i E n n g g R o o U s s N t t T r r o o I e e R s s M e e U t t E N r r T e e I e e M E

And to finally destroy the cluster:

1

vagrant destroy