From c4ad5d2340e08c68fd7ef15198fdbb2a1f01548c Mon Sep 17 00:00:00 2001 From: dannc Date: Wed, 27 Jul 2022 10:55:01 +0700 Subject: [PATCH] add incomplete homework 8.3 --- readme.md | 1 + .../08-ansible/8.3/playbook/.gitignore | 2 + .../8.3/playbook/group_vars/all/vars.yml | 3 + .../playbook/group_vars/clickhouse/vars.yml | 6 + .../playbook/group_vars/lighthouse/vars.yml | 5 + .../8.3/playbook/group_vars/vector/vars.yml | 6 + .../8.3/playbook/inventory/prod.yml.example | 13 ++ .../08-ansible/8.3/playbook/readme.md | 35 ++++ src/homework/08-ansible/8.3/playbook/site.yml | 149 ++++++++++++++++++ .../8.3/playbook/templates/nginx.conf.j2 | 28 ++++ .../templates/nginx.lighthouse.conf.j2 | 10 ++ .../8.3/playbook/templates/vector.config.j2 | 1 + .../8.3/playbook/templates/vector.service.j2 | 7 + src/homework/08-ansible/8.3/readme.md | 19 +++ 14 files changed, 285 insertions(+) create mode 100644 src/homework/08-ansible/8.3/playbook/.gitignore create mode 100644 src/homework/08-ansible/8.3/playbook/group_vars/all/vars.yml create mode 100644 src/homework/08-ansible/8.3/playbook/group_vars/clickhouse/vars.yml create mode 100644 src/homework/08-ansible/8.3/playbook/group_vars/lighthouse/vars.yml create mode 100644 src/homework/08-ansible/8.3/playbook/group_vars/vector/vars.yml create mode 100644 src/homework/08-ansible/8.3/playbook/inventory/prod.yml.example create mode 100644 src/homework/08-ansible/8.3/playbook/readme.md create mode 100644 src/homework/08-ansible/8.3/playbook/site.yml create mode 100644 src/homework/08-ansible/8.3/playbook/templates/nginx.conf.j2 create mode 100644 src/homework/08-ansible/8.3/playbook/templates/nginx.lighthouse.conf.j2 create mode 100644 src/homework/08-ansible/8.3/playbook/templates/vector.config.j2 create mode 100644 src/homework/08-ansible/8.3/playbook/templates/vector.service.j2 create mode 100644 src/homework/08-ansible/8.3/readme.md diff --git a/readme.md b/readme.md index 2792da7..bede02f 100644 --- a/readme.md +++ b/readme.md @@ -38,3 +38,4 @@ * [7.6. Написание собственных провайдеров для Terraform](/src/homework/07-terraform/7.6) * [8.1. Введение в Ansible](/src/homework/08-ansible/8.1) * [8.2. Работа с Playbook](/src/homework/08-ansible/8.2) +* [8.3. Использование Yandex Cloud](/src/homework/08-ansible/8.3) diff --git a/src/homework/08-ansible/8.3/playbook/.gitignore b/src/homework/08-ansible/8.3/playbook/.gitignore new file mode 100644 index 0000000..9da7295 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/.gitignore @@ -0,0 +1,2 @@ +inventory/prod.yml +files/* diff --git a/src/homework/08-ansible/8.3/playbook/group_vars/all/vars.yml b/src/homework/08-ansible/8.3/playbook/group_vars/all/vars.yml new file mode 100644 index 0000000..8da3075 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/group_vars/all/vars.yml @@ -0,0 +1,3 @@ +--- +ansible_user_id: 1000 +ansible_user_gid: 1000 diff --git a/src/homework/08-ansible/8.3/playbook/group_vars/clickhouse/vars.yml b/src/homework/08-ansible/8.3/playbook/group_vars/clickhouse/vars.yml new file mode 100644 index 0000000..383879b --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/group_vars/clickhouse/vars.yml @@ -0,0 +1,6 @@ +--- +clickhouse_version: "22.3.3.44" +clickhouse_packages: + - clickhouse-client + - clickhouse-server + - clickhouse-common-sta diff --git a/src/homework/08-ansible/8.3/playbook/group_vars/lighthouse/vars.yml b/src/homework/08-ansible/8.3/playbook/group_vars/lighthouse/vars.yml new file mode 100644 index 0000000..9991a0a --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/group_vars/lighthouse/vars.yml @@ -0,0 +1,5 @@ +--- +nginx_username: dannc +lighthouse_vcs: https://github.com/VKCOM/lighthouse.git +lighthouse_location: /var/www/lighthouse +lighthouse_access_log_name: lighthouse diff --git a/src/homework/08-ansible/8.3/playbook/group_vars/vector/vars.yml b/src/homework/08-ansible/8.3/playbook/group_vars/vector/vars.yml new file mode 100644 index 0000000..3f1964b --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/group_vars/vector/vars.yml @@ -0,0 +1,6 @@ +--- +vector_version: 0.23.0 +vector_config_dir: /var/lib/vector +vector_config: + data_dir: /var/lib/vector + # todo добавить конфигурацию https://vector.dev/docs/reference/configuration/ diff --git a/src/homework/08-ansible/8.3/playbook/inventory/prod.yml.example b/src/homework/08-ansible/8.3/playbook/inventory/prod.yml.example new file mode 100644 index 0000000..af57fae --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/inventory/prod.yml.example @@ -0,0 +1,13 @@ +--- +clickhouse: + hosts: + clickhouse-01: + ansible_host: +vector: + hosts: + vector-01: + ansible_host: +lighthouse: + hosts: + lighthouse-01: + ansible_host: diff --git a/src/homework/08-ansible/8.3/playbook/readme.md b/src/homework/08-ansible/8.3/playbook/readme.md new file mode 100644 index 0000000..c00f2f1 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/readme.md @@ -0,0 +1,35 @@ +# Clickhouse and vector playbook + +Данный playbook устанавливает `clickhouse`, `vector` и `lighthouse` (доступ через webserver `nginx`) на хосты, +перечисленные в inventory. +Для каждой утилиты может быть указан свой хост для установки. + +## Parameters + +### Clickhouse + +- `clickhouse_version` - версия `clickhouse`, которая будет установлена +- `clickhouse_packages` - конкретные приложения из стека `clickhouse`, которые будут установлены + +### Vector + +- `vector_version` - версия `vector`, которая будет установлена + +### Lighthouse + +// todo + +## Tags + +### Clickhouse + +- `clickhouse` - установка и запуск только `clickhouse` + +### Vector + +- `vector` - установка только `vector` +- `vector_check_version` - запуск только `task` для проверки текущей установленной версии `vector` + +### Lighthouse + +// todo diff --git a/src/homework/08-ansible/8.3/playbook/site.yml b/src/homework/08-ansible/8.3/playbook/site.yml new file mode 100644 index 0000000..74ec98d --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/site.yml @@ -0,0 +1,149 @@ +--- +- name: clickhouse + hosts: clickhouse + tasks: + - block: + - name: clickhouse | get distrib + ansible.builtin.get_url: + url: "https://packages.clickhouse.com/rpm/stable/{{ item }}-{{ clickhouse_version }}.noarch.rpm" + dest: "./{{ item }}-{{ clickhouse_version }}.rpm" + with_items: "{{ clickhouse_packages }}" + rescue: + - name: clickhouse | get distrib + ansible.builtin.get_url: + url: "https://packages.clickhouse.com/rpm/stable/clickhouse-common-static-{{ clickhouse_version }}.x86_64.rpm" + dest: "./clickhouse-common-static-{{ clickhouse_version }}.rpm" + - name: clickhouse | install packages + become: true + ansible.builtin.yum: + name: + - clickhouse-common-static-{{ clickhouse_version }}.rpm + - clickhouse-client-{{ clickhouse_version }}.rpm + - clickhouse-server-{{ clickhouse_version }}.rpm + - name: clickhouse | start service + become: true + ansible.builtin.service: + name: clickhouse-server + state: restarted + - name: clickhouse | create database + ansible.builtin.command: "clickhouse-client -q 'create database logs;'" + register: create_db + failed_when: create_db.rc != 0 and create_db.rc !=82 + changed_when: create_db.rc == 0 + tags: + - clickhouse + +- name: lighthouse + hosts: lighthouse + tasks: + - name: lighthouse | install dependencies + become: true + ansible.builtin.yum: + name: + - git + - epel-release + - name: lighthouse | create nginx site dir + become: true + ansible.builtin.file: + mode: "755" + owner: "{{ ansible_user_id }}" + group: "{{ ansible_user_gid }}" + path: "/var/www" + state: "directory" + - name: lighthouse | create nginx log dir + become: true + ansible.builtin.file: + mode: "755" + owner: "{{ ansible_user_id }}" + group: "{{ ansible_user_gid }}" + path: "/var/log/nginx" + state: "directory" + - name: lighthouse | clone repository + ansible.builtin.git: + repo: "{{ lighthouse_vcs }}" + dest: "{{ lighthouse_location }}" + # todo nginx forbidden 403 to the lighthouse/index.html + - name: lighthouse | install nginx + become: true + ansible.builtin.yum: + name: nginx + state: present + - name: lighthouse | nginx template config + become: true + ansible.builtin.template: + src: "templates/nginx.conf.j2" + dest: "/etc/nginx/nginx.conf" + - name: lighthouse | nginx lighthouse config + become: true + ansible.builtin.template: + src: "templates/nginx.lighthouse.conf.j2" + dest: "/etc/nginx/conf.d/lighthouse.conf" + - name: lighthouse | start nginx service + become: true + ansible.builtin.service: + name: nginx + state: started + tags: + - lighthouse + +- name: vector + hosts: vector + tasks: + - name: vector | install archive manager + become: true + ansible.builtin.yum: + name: + - tar + - name: vector | get distrib + ansible.builtin.get_url: + url: "https://packages.timber.io/vector/{{ vector_version }}/vector-{{ vector_version }}-x86_64-unknown-linux-musl.tar.gz" + dest: "./vector-{{ vector_version }}-x86_64-unknown-linux-musl.tar.gz" + - name: vector | unpack distrib + ansible.builtin.unarchive: + src: "./vector-{{ vector_version }}-x86_64-unknown-linux-musl.tar.gz" + dest: "./" + remote_src: true + - name: vector | install + become: true + ansible.builtin.copy: + src: "vector-x86_64-unknown-linux-musl/bin/vector" + dest: "/usr/local/bin/" + remote_src: true + mode: "755" + - name: vector | check installed version + ansible.builtin.shell: + cmd: vector --version + register: result + changed_when: + - 'vector_version not in result.stdout' + tags: + - vector_check_version + - name: vector | create data dir + become: true + ansible.builtin.file: + mode: "755" + owner: "{{ ansible_user_id }}" + group: "{{ ansible_user_gid }}" + path: "{{ vector_config_dir }}" + state: "directory" + - name: vector | template config + ansible.builtin.template: + src: "templates/vector.config.j2" + dest: "{{ vector_config_dir }}/vector.yaml" + - name: vector | register as service + become: true + ansible.builtin.template: + src: "templates/vector.service.j2" + dest: "/etc/systemd/system/vector.service" + owner: "{{ ansible_user_id }}" + group: "{{ ansible_user_gid }}" + mode: "0644" + backup: true + - name: vector | start service + become: true + ansible.builtin.service: + name: vector + state: started + daemon_reload: true + tags: + - vector diff --git a/src/homework/08-ansible/8.3/playbook/templates/nginx.conf.j2 b/src/homework/08-ansible/8.3/playbook/templates/nginx.conf.j2 new file mode 100644 index 0000000..0ee6dce --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/templates/nginx.conf.j2 @@ -0,0 +1,28 @@ +user {{ nginx_username }}; + +worker_processes auto; +error_log /var/log/nginx/error.log; +pid /run/nginx.pid; + +events { + worker_connections 1024; +} + +http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + types_hash_max_size 4096; + + include /etc/nginx/mime.types; + default_type application/octet-stream; + + include /etc/nginx/conf.d/*.conf; +} diff --git a/src/homework/08-ansible/8.3/playbook/templates/nginx.lighthouse.conf.j2 b/src/homework/08-ansible/8.3/playbook/templates/nginx.lighthouse.conf.j2 new file mode 100644 index 0000000..af77598 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/templates/nginx.lighthouse.conf.j2 @@ -0,0 +1,10 @@ +server { + listen 80; + + access_log /var/log/nginx/{{ lighthouse_access_log_name }}.log; + + location / { + root {{ lighthouse_location }}; + index index.html; + } +} diff --git a/src/homework/08-ansible/8.3/playbook/templates/vector.config.j2 b/src/homework/08-ansible/8.3/playbook/templates/vector.config.j2 new file mode 100644 index 0000000..9fe2a85 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/templates/vector.config.j2 @@ -0,0 +1 @@ +{{ vector_config | to_nice_yaml }} diff --git a/src/homework/08-ansible/8.3/playbook/templates/vector.service.j2 b/src/homework/08-ansible/8.3/playbook/templates/vector.service.j2 new file mode 100644 index 0000000..b2ee2a0 --- /dev/null +++ b/src/homework/08-ansible/8.3/playbook/templates/vector.service.j2 @@ -0,0 +1,7 @@ +[Unit] +Description=Vector service +[Service] +User={{ ansible_user_id }} +Group={{ ansible_user_gid }} +ExecStart=/usr/local/bin/vector --config-yaml {{ vector_config_dir }}/vector.yaml --watch-config +Restart=always diff --git a/src/homework/08-ansible/8.3/readme.md b/src/homework/08-ansible/8.3/readme.md new file mode 100644 index 0000000..7153c58 --- /dev/null +++ b/src/homework/08-ansible/8.3/readme.md @@ -0,0 +1,19 @@ +Выполнение [домашнего задания](https://github.com/netology-code/mnt-homeworks/blob/MNT-13/08-ansible-02-playbook/README.md) +по теме "8.3. Использование Yandex Cloud". + +## Q/A + +### Задание 1 + +> Подготовка к выполнению +> 1. Подготовьте в Yandex Cloud три хоста: для [clickhouse](https://clickhouse.com/), для [vector](https://vector.dev) и для [lighthouse](https://github.com/VKCOM/lighthouse) + +Предыдущая итерация playbook с установкой `clickhouse` и `vector` перенесена в [playbook](./playbook) из [домашней работы 8.2](/src/homework/08-ansible/8.2). +Дополнительно для `vector` добавлены новые шаги с конфигурированием и запуском как systemd-сервис. + +Новая группа хостов добавлена в [inventory/prod.yml.example](./playbook/inventory/prod.yml.example). + +### Задание 2 + +> Основная часть +> 1. Допишите playbook: нужно сделать ещё один play, который устанавливает и настраивает lighthouse.