RabbitMQ dagi Queuelarni Clickhousega to'g'ridan to'g'ri yozish

Assalomu alaykum. Saytda anchadan beri maqola yozmadik buning o`ziga yarasha sabablari bor. Ishlar va shaxsiy rivojlanishni kuchayitish uchun saytga qaragani unchalik vaqt bo'lmadi.

Hullas maqolagaga o'tksak. Ishxonada Laravel error loglarni pramoy Redisga saqlab shundan so`ng bittta bitta o`qib RabbitMQ ga yuborib undan to`g`ridan to`gri ClickHouse ga yozish topshirig`iri berilgan edi. Bu haqida O'zbek tilida u yoqda tursin Rus va Ingliz tillarida ham pichioqqa ilinarli malumot topa olmadim. Shunig uchun O'zbek kontentini yaxshilash va o`z hissamni qo`shish maqsadida va yoki o`zim uchun keyinchalik o'qishim uchun maqola sifatida ushbu qilgan usulimni maqola sifatida yozib qo`ymoqchiman.

Menda RabbitMQ avvaldan o`rnatilgan edi. Shuning uchun Clickhouse ni yaratishdan boshladim.
srv/docker/click_house/docker-compose.yml degan fayl yaratib shu yml faylga tashlaysiz.

version: '3'

services:
  click_server:
    image: clickhouse/clickhouse-server
    ports:
      - "8123:8123"
    volumes:
      - ./dbfiles:/var/lib/clickhouse
      - ./config/houseadmin.xml:/etc/clickhouse-server/users.d/houseadmin.xml

  click_client:
    image: clickhouse/clickhouse-client
    entrypoint:
      - /bin/sleep
    command:
      - infinity

srv/docker/click_house/config/houseadmin.xml degan fayl yaratib shu xml faylga tashlaysiz.

<clickhouse>
    <users>
      <default remove="remove"></default>
      <houseadmin>
          <profile>default</profile>
            <networks>
                  <ip>127.0.0.1</ip>
                  <ip>::/0</ip>
            </networks>
          <password>{PAROL}</password>
          <quota>default</quota>
      </houseadmin>
    </users>
</clickhouse>

keyin srv/docker/click_house/ papkasida terminal orqali quyidagi buyruqni yuboring.

docker compose up -d

SHundan so`ng Docker sizga ClickHouse serveri va Clientini o`rnatib beradi. (Docker o`rnatishni googledan qarang).

Endi Clickhouse Client ichiga kirib Tablitsa yaratishimiz kerak buning uchun terminalga ushbu buyruqni yozing.

sudo docker compose exec click_server clickhouse-client --password {PAROL} --user {USERNAME}

Endi ClickHouse versiyasini tekshirib olamiz.

SELECT VERSION();

Men o`rnatgan versiya **23.12.2.59** ekan

Endi RabbitMQ dan to`g`ridan to`g`ri malumotlarni o'qib olish uchun Tablitsa yaratamiz.

CREATE TABLE rabbit_error_connect (
  `message` Nullable(String), 
  `context` Nullable(String), 
  `level` UInt32, 
  `level_name` String, 
  `channel` String, 
  `datetime` DateTime('Asia/Tashkent'), 
  `extra` Nullable(String), 
  `service` String
) ENGINE = RabbitMQ SETTINGS 
    rabbitmq_address = 'amqp://{RABBIT_READER_USER}:{RABBIT_READER_PASSWORD}@{RABBIT_READER_IP}:{RABBIT_READER_PORT}/{RABBIT_READER_VHOST}',
    rabbitmq_exchange_name = 'default', 
    rabbitmq_commit_on_select=1,
    rabbitmq_format = 'JSONEachRow';

Bu tablitsa RabbitMQdan "Queue" paydo bo'lgan vaqtda rabbit_error_connect. tablitsasiga yozib qolinadi va o`sha Queue o`chirib yuboriladi. Ko'proq ma'lumot olish uchun qarang.

Yomon tomoni bu bu rabbit_error_connect tablitsasiga siz SELECT so`rovini yoza olmaysiz.

Buning uchun huddi shunday boshqa Tablitsa yaratishingiz kerak faqat boshqa nom bilan

CREATE TABLE error_logs (
  `message` Nullable(String), 
  `context` Nullable(String), 
  `level` UInt32, 
  `level_name` String, 
  `channel` String, 
  `datetime` DateTime('Asia/Tashkent'), 
  `extra` Nullable(String), 
  `service` String
) ENGINE = MergeTree() 
ORDER BY datetime;

Va bu ikkala Tablitsalarni bir biriga bog`lash uchun MATERIALIZED VIEW yaratishingiz kerak.

CREATE MATERIALIZED VIEW material_view_logs TO error_logs AS 
SELECT message, context, level, level_name, channel, datetime, extra, service 
FROM rabbit_error_connect;

Bo`ldi. Shundan so`ng siz SELECT qilsangiz bo'ladi.

SELECT * FROM error_logs;

Ha aytgancha Rabbit orqali jo`natishni aytmadim.

Men Rabbitdan ma'lumotni jo`natish uchun RabbitMQning PHP kutubhonasidan foydalandim.

Uni o`rnatib olamiz.

composer require php-amqplib/php-amqplib

So`ngra /app/Utils/ErrorSendRabbitMqClient.php fayl yaratib quyidagi kodni yozamiz.

<?php

namespace App\Utils;

use Illuminate\Support\Facades\Log;
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

class ErrorSendRabbitMqClient
{
    private AMQPStreamConnection                                                $connection;
    private \PhpAmqpLib\Channel\AbstractChannel|\PhpAmqpLib\Channel\AMQPChannel $channel;
    private                                                                     $callback_queue;
    private                                                                     $response;
    private                                                                     $corr_id;
    private                                                                     $queueName;

    private const CONFIG = [
        'error_log' => [
            'host'     => '{SIZING_ICHKI_RABBIT_IP_HOST}',
            'port'     => '{RABBIT_PORT}',
            'user'     => '{RABBIT_USER}',
            'password' => '{RABBIT_PAROL}',
            'vhost'    => '{RABBIT_VHOST}',
            'queue'    => '{RABBIT_QUEU_NOMI}',
        ],
    ];

    public function __construct(string $configName)
    {
        $config          = self::CONFIG[$configName];
        $this->queueName = $config['queue'];

        try {
            $this->connection = new AMQPStreamConnection($config['host'], $config['port'], $config['user'], $config['password'], $config['vhost']);
            $this->channel    = $this->connection->channel();
            $this->channel->queue_declare("", false, false, true, false);
        } catch (\Throwable $exception) {
            Log::error($exception);
            $this->channel?->close();
            try {
                $this->connection?->close();
            } catch (\Exception $e) {
                Log::error($e);

            }
        }
    }

    private function closeConnection(): void
    {
        try {
            $this->channel->close();
            $this->connection->close();
        } catch (\Throwable $exception) {
            Log::error($exception);
        }
    }

    private function closeConnectionAndChannel($payload): void
    {
        Log::error('rabbitMQ');
        try {
            Log::error($payload);
            $this->channel->close();
            $this->connection->close();
        } catch (\Throwable $exception) {
            Log::error($exception);
        }

    }

    /**
     * @param array $payload
     * @return void
     */
    public function sending(array $payload): void
    {
        $msg = new AMQPMessage(json_encode($payload),);
        try {
            $this->channel->basic_publish($msg, 'default', $this->queueName);
            $this->closeConnection();
        } catch (\Throwable $exception) {
            $this->closeConnectionAndChannel($payload);

            Log::channel('daily')->error($exception->getMessage());
        }
    }

    public static function send(array $params): void
    {
        (new self('error_log'))->sending($params);
    }
}

So'ng shu Classdagi send() funksiyasini istalgan joydan chaqiramiz.

use App\Utils\ErrorSendRabbitMqClient;

$data = [
    "message"    => "Bu test cuun",
    "context"    => "6565",
    "level"      => 200,
    "level_name" => "INFO",
    "channel"    => "Redis Logger",
    "datetime"   => "2024-01-11 13:08:15",
    "extra"      => "",
    "error"      => true,
    "service"    => "Servis nomi",
];

ErrorSendRabbitMqClient::send($data);

Shunday ma'lumot jo`natasiz va u RabbitMQga yuboradi.

Va bo'ldi endi ishlashi kerak. Agarda sizda biror muammo bolsa Telegram Guruhimizda so'rang.

Maqola UzHackerSW.uz sayti uchun maxsus tayyorlandi. Ruxsatsiz maqolani ko‘chirish qatiyan taqiqlanadi. Maqola o`g`rincha ko`chirilganda websaytga havola ko`rsatilishi shart!

Bundanda qiziqarli ma'lumotlarga ega bo'lish uchun saytimizga tez-tez kirib turing! Biz eng yaxshilarini sizlarga ilinamiz