topip - simple terminal haproxy log analyzer

It's meant to give you a quick overview of your haproxy logs, like:

What ips are making the most connections?
What vhost consumes the most bandwidth?
Which files are most downloaded?
Is there any bot that abuses your webserver?

Installation

Binaries

topip has prebuilt binaries for Linux and macOS on amd64 and arm64.

You can install it easily by using these commands:

wget https://git.rznet.fr/tchivert/topip/releases/download/latest/topip-<os>-<arch> -O /usr/local/bin/topip
chmod +x /usr/local/bin/topip

Exemple for an amd64 linux machine:

wget https://git.rznet.fr/tchivert/topip/releases/download/latest/topip-linux-amd64 -O /usr/local/bin/topip
chmod +x /usr/local/bin/topip

Debian-based distributions

You can install topip with a deb package from my repository:

sudo curl https://dl.rznet.fr/api/packages/tchivert/debian/repository.key -o /etc/apt/keyrings/rznet-tchivert.asc
echo "deb [signed-by=/etc/apt/keyrings/rznet-tchivert.asc] https://dl.rznet.fr/api/packages/tchivert/debian stable main" | sudo tee -a /etc/apt/sources.list.d/rznet.list
sudo apt update

sudo apt install topip

Or by downloading directly the latest .deb:

wget https://git.rznet.fr/tchivert/topip/releases/download/latest/topip-amd64.deb
sudo dpkg -i topip_1.2-1_amd64.deb

Usage

Usage: topip [-f logfile] [-t conn|bytes] [-n num] [-p pattern] [-i all|ip|vhost|request] [-d]
  -d	Enable debug mode
  -f string
    	Path to the HAProxy log file (use - for stdin) (default "/var/log/haproxy/haproxy.log")
  -i string
    	Information to display (all, ip, vhost, req) (default "all")
  -n int
    	Number of IP addresses and vhosts to display (default 20)
  -p string
    	Filter by request pattern
  -t string
    	Sort by connections or bytes sent (conn or bytes) (default "conn")

Recommended haproxy log-format:
  log-format "%ci %b/%s %ST %B %Tt %sq/%bq %{+Q}r %hr %hs"

For example:

$ topip -f /var/log/haproxy/haproxy.log -t bytes -n 5          
Address         Count      Bytes    Time    User Agent      Last Request
---------       ------  ---------   -----   -------------   -------------
2a01:e34::0     59      8.3 GB      2m      Mozilla/5.0     /endeavouros/iso/EndeavourOS_Endeavour_neo-2024.09.22.iso
62.35.00.00     4       5.5 GB      1m      Wget/1.21.3     /endeavouros/iso/EndeavourOS_Endeavour_neo-2024.09.22.iso
2a01:e0a::0     4       5.5 GB      1m      Mozilla/5.0     /endeavouros/iso/EndeavourOS_Endeavour_neo-2024.09.22.iso
144.2.00.00     7       4.8 GB      2m      curl/8.11.1     /endeavouros/repo/state
194.34.00.00    2       2.9 GB      11m     Mozilla/5.0     /endeavouros/iso/EndeavourOS_Endeavour_neo-2024.09.22.iso

Vhost               Count   Bytes       Time    Last Request
---------           ------  ---------   -----   -------------
mirror.rznet.fr     117305  261.3 GB    221h    /archlinux/iso/2024.11.01/archlinux-2024.11.01-x86_64.iso
git.rznet.fr        141     4.5 MB      59s     /tchivert/tbin/raw/commit/018ffb503/.gitea/workflows/build.yml
rznet.fr            70      414.6 KB    23s     /

Request                                                                     Count   Bytes       Time
---------                                                                   ------  ---------   -----
/endeavouros/iso/EndeavourOS_Endeavour_neo-2024.09.22.iso                   3947    219.9 GB    80h
/archlinux/iso/2024.11.01/archlinux-2024.11.01-x86_64.iso                   55574   17.3 GB     127h
/archlinux/core/os/x86_64/linux-6.12.7.arch1-1-x86_64.pkg.tar.zst           23      2.9 GB      8m
/archlinux/extra/os/x86_64/extra.db                                         4114    2.6 GB      22m
/endeavouros/repo/endeavouros/x86_64/eos-qogir-icons-6-1-any.pkg.tar.zst    282     1.2 GB      27m

License

This project is licensed under the MIT License. See the LICENSE file for more information.

4.0 KiB Raw Permalink Blame History