add homework 9.4, add some preparations

readme.md

@@ -44,3 +44,4 @@
 * [8.6. Создание собственных modules](/src/homework/08-ansible/8.6)
 * [9.1. Жизненный цикл ПО](/src/homework/09-ci/9.1)
 * [9.3. CI\CD](/src/homework/09-ci/9.3)
+* [9.4 Jenkins](/src/homework/09-ci/9.4)

src/homework/09-ci/9.4/infrastructure/.gitignore

@@ -0,0 +1,3 @@
+files/*.pub
+roles/*
+inventory/hosts.yml

src/homework/09-ci/9.4/infrastructure/files/CentOS-Base.repo

@@ -0,0 +1,47 @@
+# CentOS-Base.repo
+#
+# The mirror system uses the connecting IP address of the client and the
+# update status of each mirror to pick mirrors that are updated to and
+# geographically close to the client.  You should use this for CentOS updates
+# unless you are manually picking other mirrors.
+#
+# If the mirrorlist= does not work for you, as a fall back you can try the
+# remarked out baseurl= line instead.
+#
+#
+
+[base]
+name=CentOS-$releasever - Base
+mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+exclude=postgresql*
+
+#released updates
+[updates]
+name=CentOS-$releasever - Updates
+mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+exclude=postgresql*
+
+#additional packages that may be useful
+[extras]
+name=CentOS-$releasever - Extras
+mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#additional packages that extend functionality of existing packages
+[centosplus]
+name=CentOS-$releasever - Plus
+mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
+gpgcheck=1
+enabled=0
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

src/homework/09-ci/9.4/infrastructure/files/pg_hba.conf

@@ -0,0 +1,89 @@
+# PostgreSQL Client Authentication Configuration File
+# ===================================================
+#
+# Refer to the "Client Authentication" section in the PostgreSQL
+# documentation for a complete description of this file.  A short
+# synopsis follows.
+#
+# This file controls: which hosts are allowed to connect, how clients
+# are authenticated, which PostgreSQL user names they can use, which
+# databases they can access.  Records take one of these forms:
+#
+# local      DATABASE  USER  METHOD  [OPTIONS]
+# host       DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
+# hostssl    DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
+# hostnossl  DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
+#
+# (The uppercase items must be replaced by actual values.)
+#
+# The first field is the connection type: "local" is a Unix-domain
+# socket, "host" is either a plain or SSL-encrypted TCP/IP socket,
+# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a
+# plain TCP/IP socket.
+#
+# DATABASE can be "all", "sameuser", "samerole", "replication", a
+# database name, or a comma-separated list thereof. The "all"
+# keyword does not match "replication". Access to replication
+# must be enabled in a separate record (see example below).
+#
+# USER can be "all", a user name, a group name prefixed with "+", or a
+# comma-separated list thereof.  In both the DATABASE and USER fields
+# you can also write a file name prefixed with "@" to include names
+# from a separate file.
+#
+# ADDRESS specifies the set of hosts the record matches.  It can be a
+# host name, or it is made up of an IP address and a CIDR mask that is
+# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that
+# specifies the number of significant bits in the mask.  A host name
+# that starts with a dot (.) matches a suffix of the actual host name.
+# Alternatively, you can write an IP address and netmask in separate
+# columns to specify the set of hosts.  Instead of a CIDR-address, you
+# can write "samehost" to match any of the server's own IP addresses,
+# or "samenet" to match any address in any subnet that the server is
+# directly connected to.
+#
+# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
+# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
+# Note that "password" sends passwords in clear text; "md5" or
+# "scram-sha-256" are preferred since they send encrypted passwords.
+#
+# OPTIONS are a set of options for the authentication in the format
+# NAME=VALUE.  The available options depend on the different
+# authentication methods -- refer to the "Client Authentication"
+# section in the documentation for a list of which options are
+# available for which authentication methods.
+#
+# Database and user names containing spaces, commas, quotes and other
+# special characters must be quoted.  Quoting one of the keywords
+# "all", "sameuser", "samerole" or "replication" makes the name lose
+# its special character, and just match a database or username with
+# that name.
+#
+# This file is read on server startup and when the server receives a
+# SIGHUP signal.  If you edit the file on a running system, you have to
+# SIGHUP the server for the changes to take effect, run "pg_ctl reload",
+# or execute "SELECT pg_reload_conf()".
+#
+# Put your actual configuration here
+# ----------------------------------
+#
+# If you want to allow non-local connections, you need to add more
+# "host" records.  In that case you will also need to make PostgreSQL
+# listen on a non-local interface via the listen_addresses
+# configuration parameter, or via the -i or -h command line switches.
+
+
+
+# TYPE  DATABASE        USER            ADDRESS                 METHOD
+
+# "local" is for Unix domain socket connections only
+local   all             all                                     trust
+# IPv4 local connections:
+host    all             all             127.0.0.1/32            md5
+# IPv6 local connections:
+host    all             all             ::1/128                 md5
+# Allow replication connections from localhost, by a user with the
+# replication privilege.
+local   replication     all                                     trust
+host    replication     all             127.0.0.1/32            md5
+host    replication     all             ::1/128                 md5

src/homework/09-ci/9.4/infrastructure/inventory/group_vars/jenkins.yml

@@ -0,0 +1,9 @@
+---
+user_group: "{{ jenkins_user_group }}"
+user_name: "{{ jenkins_user_name }}"
+jenkins_user_name: jenkins
+jenkins_user_group: jenkins
+java_packages:
+  - java-11-openjdk-devel
+  - java-11-openjdk
+jenkins_agent_dir: /opt/jenkins_agent/

src/homework/09-ci/9.4/infrastructure/inventory/hosts.yml.example

@@ -0,0 +1,19 @@
+---
+all:
+  hosts:
+    jenkins-master-01:
+      ansible_host: <master>
+    jenkins-agent-01:
+      ansible_host: <agent>
+  children:
+    jenkins:
+      children:
+        jenkins_masters:
+          hosts:
+            jenkins-master-01:
+        jenkins_agents:
+          hosts:
+              jenkins-agent-01:
+  vars:
+    ansible_connection_type: paramiko
+    ansible_user: <user>

src/homework/09-ci/9.4/infrastructure/site.yml

@@ -0,0 +1,153 @@
+---
+- name: Preapre all hosts
+  hosts: all
+  tasks:
+    - name: Create group
+      become: true
+      group:
+        name: "{{ user_group }}"
+        state: present
+    - name: Create user
+      become: true
+      user:
+        name: "{{ user_name }}"
+        generate_ssh_key: true
+      register: ssh_key_info
+    - name: Install JDK
+      become: true
+      package:
+        name: "{{ java_packages }}"
+        state: present
+
+- name: Get Jenkins master installed
+  hosts: jenkins_masters
+  tasks:
+    - name: Get repo Jenkins
+      become: true
+      get_url:
+        url: https://pkg.jenkins.io/redhat-stable/jenkins.repo
+        dest: /etc/yum.repos.d/jenkins.repo
+        validate_certs: false
+    - name: Add Jenkins key
+      become: true
+      rpm_key:
+        key: https://pkg.jenkins.io/redhat-stable/jenkins.io.key
+        state: present
+        validate_certs: false
+    - name: Install epel-release
+      become: true
+      yum:
+        name: epel-release
+        state: present
+    - name: Install Jenkins and requirements
+      become: true
+      yum:
+        name:
+          - jenkins
+          - git
+        state: present
+      register: jenkins_installed
+    - name: Ensure jenkins agents are present in known_hosts file
+      become: true
+      become_user: "{{ jenkins_user_name }}"
+      known_hosts:
+        name: "{{ hostvars[item].ansible_host }}"
+        state: present
+        key: "{{ lookup('pipe', 'ssh-keyscan {{ hostvars[item].ansible_host }}') }}"
+        hash_host: true
+      with_items: "{{ groups.jenkins_agents }}"
+    - name: Start Jenkins
+      become: true
+      systemd:
+        name: jenkins
+        state: restarted
+        enabled: true
+      when: jenkins_installed is changed
+
+- name: Prepare jenkins agent
+  hosts: jenkins_agents
+  become: true
+  become_user: "{{ jenkins_user_name }}"
+  tasks:
+    - name: Add master publickey into authorized_key
+      authorized_key:
+        user: "{{ jenkins_user_name }}"
+        state: present
+        key: "{{ hostvars['jenkins-master-01']['ssh_key_info']['ssh_public_key'] }}"
+    - name: Create agent_dir
+      become_user: root
+      file:
+        path: "{{ jenkins_agent_dir }}"
+        state: directory
+        owner: "{{ jenkins_user_name }}"
+        group: "{{ jenkins_user_group }}"
+    - name: Add docker repo
+      become_user: root
+      get_url:
+        url: https://download.docker.com/linux/centos/docker-ce.repo
+        dest: /etc/yum.repos.d/docker-ce.repo
+        validate_certs: false
+    - name: Install some required
+      become_user: root
+      yum:
+        name:
+          - git
+          - python3
+          - docker-ce
+          - docker-ce-cli
+          - containerd.io
+        state: present
+    - name: Update pip
+      become_user: root
+      pip:
+        executable: pip3
+        name: pip
+        extra_args: --upgrade
+    - name: Install Ansible
+      become_user: root
+      pip:
+        executable: pip3
+        name:
+          - selinux
+          - ansible<3.0
+        state: present
+    - name: Reinstall Selinux
+      become_user: root
+      pip:
+        executable: pip3
+        name:
+          - selinux
+        state: forcereinstall
+    - name: Add local to PATH
+      lineinfile:
+        dest: ~/.bashrc
+        regexp: export PATH=$PATH:/home/jenkins/.local/bin
+        line: export PATH=$PATH:/home/jenkins/.local/bin
+    - name: Create docker group
+      become_user: root
+      group:
+        name: docker
+        state: present
+    - name: Add jenkinsuser to dockergroup
+      become_user: root
+      user:
+        name: "{{ jenkins_user_name }}"
+        groups: docker
+    - name: Restart docker
+      become_user: root
+      systemd:
+        name: docker
+        state: restarted
+        enabled: true
+    - name: Install agent.jar
+      get_url:
+        url: "http://{{ hostvars['jenkins-master-01']['ansible_host'] }}:8080/jnlpJars/agent.jar"
+        dest: "{{ jenkins_agent_dir }}"
+      register: install_agent
+      until: install_agent is succeeded
+      retries: 10
+      timeout: 10
+    - name: Install pip molecule
+      ansible.builtin.pip:
+        executable: pip3
+        name: molecule

src/homework/09-ci/9.4/readme.md

@@ -0,0 +1,57 @@
+Выполнение [домашнего задания](https://github.com/netology-code/mnt-homeworks/blob/MNT-13/09-ci-04-jenkins/README.md)
+по теме "9.3. Jenkins".
+
+## Q/A
+
+### Задание 1
+
+> Подготовка к выполнению
+> 
+> 1. Создать 2 VM: для jenkins-master и jenkins-agent.
+> 2. Установить jenkins при помощи playbook'a.
+> 3. Запустить и проверить работоспособность.
+> 4. Сделать первоначальную настройку.
+
+[Playbook](./infrastructure/site.yml) для установки `jenkins`. Добавлена установка `molecule` 
+в сравнении с оригинальным содержимым.
+
+// todo Доработать playbook для запуска добавления github.com в known_hosts
+// todo Доработать playbook для установки python версии >= 3.8 (+pip)
+
+Запуск:
+
+```shell
+ansible-playbook -i inventory/hosts.yml site.yml
+```
+
+```text
+<...>
+PLAY RECAP *********************************************************************************************
+jenkins-agent-01           : ok=18   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
+jenkins-master-01          : ok=11   changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
+```
+
+![jenkins_start](./img/jenkins_start.png)
+
+После первоначальной настройки будет выведен dashboard с настройками `jenkins`:
+![jenkins_dashboard](./img/jenkins_dashboard.png)
+
+// todo описать отключение master от сборок
+// todo описать подключение agent к сборкам
+
+### Задание 2
+
+> Основная часть
+> 
+> 1. Сделать Freestyle Job, который будет запускать `molecule test` из любого вашего репозитория с ролью.
+
+В качестве репозитория для запуска тестов будет использован [netology-devops-ansible-vector](https://github.com/Dannecron/netology-devops-ansible-vector).
+
+
+> 2. Сделать Declarative Pipeline Job, который будет запускать `molecule test` из любого вашего репозитория с ролью.
+> 3. Перенести Declarative Pipeline в репозиторий в файл `Jenkinsfile`.
+> 4. Создать Multibranch Pipeline на запуск `Jenkinsfile` из репозитория.
+> 5. Создать Scripted Pipeline, наполнить его скриптом из [pipeline](./pipeline).
+> 6. Внести необходимые изменения, чтобы Pipeline запускал `ansible-playbook` без флагов `--check --diff`, если не установлен параметр при запуске джобы (prod_run = True), по умолчанию параметр имеет значение False и запускает прогон с флагами `--check --diff`.
+> 7. Проверить работоспособность, исправить ошибки, исправленный Pipeline вложить в репозиторий в файл `ScriptedJenkinsfile`.
+> 8. Отправить ссылку на репозиторий с ролью и Declarative Pipeline и Scripted Pipeline.