Mikrotik Logs

Complementary to MKTXP, this project also adds some extra capabilities such an centralized Mikrotik log processing based on a syslog-ng / promtail / Loki stack.

Alt text

Syslog-ng

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: syslog-ng
spec:
  replicas: 1
  selector:
    matchLabels:
      app: syslog-ng
  template:
    metadata:
      labels:
        app: syslog-ng
    spec:
      containers:
        - name: syslog-ng
          image: balabit/syslog-ng:4.8.0
          ports:
            - containerPort: 601
              protocol: TCP
            - containerPort: 514
              protocol: UDP
          volumeMounts:
            - name: config-volume
              mountPath: /etc/syslog-ng/syslog-ng.conf
              subPath: syslog-ng.conf
            - name: syslog-ng-logs
              mountPath: /var/log # Syslog-NG will write logs here
      volumes:
        - name: config-volume
          configMap:
            name: syslog-ng-config
        - name: syslog-ng-logs
          persistentVolumeClaim:
            claimName: syslog-ng-logs-pvc

configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: syslog-ng-config
data:
  syslog-ng.conf: |
    @version: current
    @include "scl.conf"
    source s_local {
    internal();
    };

    source s_network {
    default-network-drivers(
    );
    };
    destination d_local {
        file("/var/log/messages");
    };
    destination d_loki {
       syslog("promtail-syslog.monitoring.svc.cluster.local" transport("tcp") port(1514)); 
    };
    log {
        source(s_local);
        source(s_network);
        destination(d_local);
        destination(d_loki);
    };

networking.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    org: empire
  name: syslog-ng-service
spec:
  selector:
    app: syslog-ng
  ports:
    - name: tcp-syslog
      protocol: TCP
      port: 601
      targetPort: 601
    - name: udp-syslog
      protocol: UDP
      port: 514
      targetPort: 514
  type: LoadBalancer
  loadBalancerIP: 10.10.10.169

storage.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: syslog-ng-logs-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: synology-iscsi-storage
  resources:
    requests:
      storage: 2Gi

Promtail

promtail.yaml
server:
      http_listen_port: 9080
      grpc_listen_port: 0

    clients:
    - url: http://loki.monitoring:3100/loki/api/v1/push

    positions:
      filename: /tmp/positions.yaml
    target_config:
      sync_period: 10s
    scrape_configs:
    - job_name: syslog
      syslog:
        listen_address: 0.0.0.0:1514
        idle_timeout: 60s
        label_structured_data: yes
        labels:
          job: "syslog"
      relabel_configs:
        - source_labels: ['__syslog_message_hostname']
          target_label: 'routerboard'
        - source_labels: ['__syslog_connection_hostname']
          target_label: 'syslog_host'
    - job_name: pod-logs
      kubernetes_sd_configs:
        - role: pod
      pipeline_stages:
        - docker: {}
      relabel_configs:
        - source_labels:
            - __meta_kubernetes_pod_node_name
          target_label: __host__
        - action: labelmap
          regex: __meta_kubernetes_pod_label_(.+)
        - action: replace
          replacement: $1
          separator: /
          source_labels:
            - __meta_kubernetes_namespace
            - __meta_kubernetes_pod_name
          target_label: job
        - action: replace
          source_labels:
            - __meta_kubernetes_namespace
          target_label: namespace
        - action: replace
          source_labels:
            - __meta_kubernetes_pod_name
          target_label: pod
        - action: replace
          source_labels:
            - __meta_kubernetes_pod_container_name
          target_label: container
        - replacement: /var/log/pods/*$1/*.log
          separator: /
          source_labels:
            - __meta_kubernetes_pod_uid
            - __meta_kubernetes_pod_container_name
          target_label: __path__

promtail-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: promtail-syslog
  namespace: monitoring
spec:
  selector:
    name: promtail
  ports:
    - protocol: TCP
      port: 1514 # The port Syslog-NG will use to send logs
      targetPort: 1514 # Make sure this matches the port Promtail listens to
  type: ClusterIP

my DevOps Odyssey

“Σα βγεις στον πηγαιμό για την Ιθάκη, να εύχεσαι να ‘ναι μακρύς ο δρόμος, γεμάτος περιπέτειες, γεμάτος γνώσεις.” - Kavafis’ Ithaka.



Mikrotik Logs

2025-06-29

Series:lab

Categories:Kubernetes

Tags:#k3s, #mikrotik, #monitoring, #lab


Mikrotik Logs: