{"id":176,"date":"2025-04-25T11:37:27","date_gmt":"2025-04-25T09:37:27","guid":{"rendered":"https:\/\/vittrup-graversen.dk\/?p=176"},"modified":"2026-03-28T12:11:14","modified_gmt":"2026-03-28T10:11:14","slug":"monitoring-cyber-threats-with-crowdsec-victoriametrics-and-grafana","status":"publish","type":"post","link":"https:\/\/vittrup-graversen.dk\/index.php\/2025\/04\/25\/monitoring-cyber-threats-with-crowdsec-victoriametrics-and-grafana\/","title":{"rendered":"Monitoring Cyber Threats with CrowdSec, VictoriaMetrics, and Grafana"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Introduction<\/h2>\n\n\n\n<p>Keeping an eye on who\u2019s trying to attack your servers can be invaluable for sysadmins, self-hosters, and homelab enthusiasts. In this guide, I\u2019ll set up a <strong>low-code, open-source cyber threat monitoring stack<\/strong> using <strong>CrowdSec<\/strong> (as the intrusion detection engine), <strong>VictoriaMetrics<\/strong> (as a time-series database for metrics), and <strong>Grafana<\/strong> (as the visualization layer). This combination provides rich real-time insights into attacks on your infrastructure \u2013 for example, you can see which countries and networks (ASNs) most of your attackers come from, what attack scenarios (SSH brute-force, web exploits, etc.) are being triggered, and how many threats are being blocked over time. By leveraging these tools, you get the kind of cyber threat statistics that default CrowdSec metrics alone don\u2019t provide \u2013 all in a visual dashboard.<\/p>\n\n\n\n<p>The process outlined here was deeply inspired by the article <em>\u201c<a href=\"https:\/\/freefd.github.io\/articles\/8_cyber_threat_insights_with_crowdsec_victoriametrics_and_grafana\/\" target=\"_blank\" rel=\"noreferrer noopener\">Cyber threat insights with CrowdSec, VictoriaMetrics and Grafana<\/a>\u201d<\/em> and the Grafana dashboard <strong>\u201cCrowdSec Cyber Threat Insights\u201d<\/strong> available on <a href=\"https:\/\/grafana.com\/grafana\/dashboards\/21689-crowdsec-cyber-threat-insights\/\" target=\"_blank\" rel=\"noreferrer noopener\">Grafana\u2019s website<\/a>\u200b. My goal is to document the end-to-end setup so that you can replicate it on your own infrastructure. I will use <strong>three servers or containers<\/strong>: one running CrowdSec (on the machine we want to monitor\/protect), one running VictoriaMetrics (to store metrics pushed from CrowdSec), and one running Grafana (to query VictoriaMetrics and display dashboards).<\/p>\n\n\n\n<p><strong>Why this setup?<\/strong> It allows you to maintain full control over your data and get actionable, real-time intelligence on attacks. Instead of relying on CrowdSec\u2019s cloud Console for insights or its limited built-in Prometheus metrics (which lack detailed threat context), you can self-host an equivalent solution\u200b. The Grafana dashboard will highlight things like top attacker countries, a world map of threat origins, the most frequent attack scenarios hitting your systems, and more \u2013 giving you a clear visual overview of your \u201cthreat landscape\u201d at a glance.<\/p>\n\n\n\n<p>In the following sections, I\u2019ll walk through the setup step by step, from installing each component to configuring CrowdSec to push data into VictoriaMetrics and finally viewing the results in Grafana. Let\u2019s get started!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Architecture Overview<\/h2>\n\n\n\n<p>Before diving in, here\u2019s a quick overview of how the pieces work together:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CrowdSec (Security Engine)<\/strong> \u2013 Runs on the server you want to monitor. It analyzes logs and detects malicious behavior (e.g. SSH brute force, port scans), and generates <strong>decisions<\/strong> (such as bans). We will configure CrowdSec to send these decisions as metrics to VictoriaMetrics using its HTTP notification plugin.<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>VictoriaMetrics (Metrics Store)<\/strong> \u2013 A high-performance time-series database (compatible with Prometheus API) that will <strong>ingest metrics<\/strong> from CrowdSec. CrowdSec will push data to VictoriaMetrics\u2019s HTTP API endpoint (<code>\/api\/v1\/import<\/code>), effectively logging each security decision as a metric data point.<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Grafana (Visualization Dashboard)<\/strong> \u2013 Queries the metrics from VictoriaMetrics and displays them in a dashboard. We will use a pre-built Grafana dashboard (<a href=\"https:\/\/grafana.com\/grafana\/dashboards\/21689-crowdsec-cyber-threat-insights\/\" target=\"_blank\" rel=\"noreferrer noopener\">ID <code>21689<\/code><\/a>) tailored for CrowdSec threat insights, which expects the metrics we\u2019re sending (like <code>cs_lapi_decision<\/code> events). Grafana\u2019s interface will allow filtering, exploring, and visualizing the CrowdSec data over time.<\/li>\n<\/ul>\n\n\n\n<p>These components communicate as follows: CrowdSec (on each protected host) \u2192 <strong>HTTP push<\/strong> \u2192 VictoriaMetrics (central metrics DB) \u2192 <strong>PromQL queries<\/strong> \u2192 Grafana (dashboard for analysis). This decoupled design means you can monitor multiple CrowdSec-protected machines by pointing them all to a single VictoriaMetrics instance if desired:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"599\" src=\"https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/VictoriaMetrics-1024x599.jpeg\" alt=\"\" class=\"wp-image-184\" srcset=\"https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/VictoriaMetrics-1024x599.jpeg 1024w, https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/VictoriaMetrics-300x175.jpeg 300w, https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/VictoriaMetrics-768x449.jpeg 768w, https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/VictoriaMetrics.jpeg 1360w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"has-white-background-color has-background\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 1: Install and Run VictoriaMetrics (Metrics Storage)<\/h2>\n\n\n\n<p>First, set up the VictoriaMetrics server, which will collect and store the security events from CrowdSec. We\u2019ll use Docker to keep things simple (you can also use a binary or system package if you prefer):<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Run the VictoriaMetrics container<\/strong> \u2013 On your designated metrics server, run:<br><br><code>docker run -d --name victoriametrics -p 8428:8428 victoriametrics\/victoriametrics<\/code><br><br>This pulls the latest VictoriaMetrics single-node image and exposes it on port <strong>8428<\/strong> (the default port for VictoriaMetrics HTTP API and UI). In a few seconds, the service should be up and listening.<\/li>\n\n\n\n<li><strong>Verify it\u2019s running<\/strong> \u2013 You can confirm by visiting <code>http:\/\/&lt;VM_server_IP>:8428<\/code> in a browser. VictoriaMetrics provides a built-in web UI (at <code>\/vmui<\/code>) for queries and status\u200b. If all is well, you should see a page titled \u201cVMUI\u201d or similar. Another quick check is running <code>docker logs victoriametrics<\/code> to see if it started without errors.<\/li>\n\n\n\n<li><strong>(Optional) Persist Data<\/strong> \u2013 By default, the above run command stores data in the container. For a real deployment, you might want to mount a volume for <code>\/victoria-metrics-data<\/code> to preserve data across restarts. For this guide, we\u2019ll assume a simple ephemeral setup is fine.<\/li>\n<\/ol>\n\n\n\n<p>VictoriaMetrics doesn\u2019t require much initial configuration \u2013 it\u2019s ready to receive data. We\u2019ll proceed to set up Grafana next, then come back to configuring CrowdSec to push metrics here.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 2: Install Grafana (Visualization Layer)<\/h2>\n\n\n\n<p>Next, we\u2019ll set up Grafana to visualize the data. You can either use Docker (quick for testing) or install it on a Linux server (Debian\/Ubuntu) directly. I\u2019ll outline both approaches:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Option A: Docker<\/strong> \u2013 On the Grafana server, run:<br><br><code>docker run -d --name grafana -p 3000:3000 grafana\/grafana-oss<\/code><br><br>This will start a Grafana instance on port <strong>3000<\/strong>. Log in by visiting <code>http:\/\/&lt;Grafana_server_IP&gt;:3000<\/code> (the default login is <strong>admin\/admin<\/strong>, which you\u2019ll be prompted to change).<\/li>\n\n\n\n<li><strong>Option B: Native Install (Debian\/Ubuntu)<\/strong> \u2013 Grafana provides .deb packages and APT repository. In short, you would add Grafana\u2019s GPG key and repository, then install the package. For example, on Ubuntu\/Debian:<br><br><code>sudo apt-get install -y apt-transport-https software-properties-common<br>wget -q -O - https:\/\/packages.grafana.com\/gpg.key | sudo apt-key add -<br>sudo add-apt-repository \"deb https:\/\/packages.grafana.com\/oss\/deb stable main\"<br>sudo apt-get update &amp;&amp; sudo apt-get install -y grafana<br>sudo systemctl enable --now grafana-server<\/code><br><br>This installs Grafana and starts the service on port 3000. Access it via browser as above.<\/li>\n<\/ul>\n\n\n\n<p><strong>Add VictoriaMetrics as a Data Source:<\/strong> Once Grafana is running, we need to connect it to VictoriaMetrics as a data source. Grafana treats VictoriaMetrics like a Prometheus data source (since VictoriaMetrics supports the PromQL API). In Grafana\u2019s web UI:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Log in and go to <strong>Configuration \u2192 Data Sources \u2192 Add data source<\/strong>.<\/li>\n\n\n\n<li>Select <strong>Prometheus<\/strong> as the data source type.<\/li>\n\n\n\n<li>Set the URL to your VictoriaMetrics instance. For example: <code>http:\/\/&lt;VM_server_IP&gt;:8428<\/code> (use the host\/IP where VM is running, port 8428)\u200b. <em>Note:<\/em> You <strong>do not<\/strong> need to add <code>\/api\/v1<\/code> in the URL \u2013 Grafana will handle the proper endpoints.<\/li>\n\n\n\n<li>Click <strong>Save &amp; Test<\/strong>. Grafana should confirm that the data source is working (it will perform a test query). If successful, you\u2019re ready to use VictoriaMetrics data in Grafana.<\/li>\n<\/ol>\n\n\n\n<p>At this point, Grafana is set up and knows how to query VictoriaMetrics. Now we\u2019ll import the pre-built dashboard for CrowdSec data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 3: Import the CrowdSec Cyber Threat Insights Dashboard<\/h2>\n\n\n\n<p>Grafana allows importing dashboards that others have created. The dashboard we want to use is <strong>\u201cCrowdsec Cyber Threat Insights\u201d<\/strong> (available on Grafana\u2019s community dashboards site, ID <code>21689<\/code>). This dashboard was specifically created to visualize CrowdSec metrics in VictoriaMetrics. It includes charts and tables for things like \u201cCyberthreats Over Last Month\u201d, \u201cTop 10 Cyberthreat Countries\u201d, a world map of threat origins, and real-time lists of recent attacks\/decisions.<\/p>\n\n\n\n<p>To import the dashboard: In Grafana UI, go to <strong>Plus (+) icon \u2192 Import<\/strong>. Then either:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enter the dashboard ID <strong><code>21689<\/code><\/strong> and click \u201cLoad\u201d, or<\/li>\n\n\n\n<li>Paste the URL: <code>https:\/\/grafana.com\/grafana\/dashboards\/21689-crowdsec-cyber-threat-insights<\/code> and click \u201cLoad\u201d, or<\/li>\n\n\n\n<li>Download the JSON from Grafana\u2019s site and upload it.<\/li>\n<\/ul>\n\n\n\n<p>Grafana will then ask you to select the data source for the dashboard\u2019s queries. Choose the VictoriaMetrics\/Prometheus data source you added in the previous step (e.g. \u201cPrometheus \u2013 http:\/\/:8428\u201d). Complete the import, and Grafana will add the dashboard to your list.<\/p>\n\n\n\n<p>Now, when you open the <strong>CrowdSec Cyber Threat Insights<\/strong> dashboard, it may be mostly empty (since we haven\u2019t fed any data yet). But you should see the layout of panels. For example, the top section has a table of \u201cAffected Hosts\u201d and counts of attacks over the last month, a pie chart of top attacker countries, and a world map of cyberthreats. Lower down, there\u2019s a \u201cRealtime Cyberthreats\u201d table listing recent decisions (with columns like timestamp, country, IP, decision action, scenario), and some stats like total count of events. This dashboard is quite comprehensive \u2013 it even has interactive filters (you can click on a country in the pie chart to filter the whole dashboard for that country, etc.). We\u2019ll see it in action once data is flowing.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p><em>Example Grafana dashboard view (CrowdSec Cyber Threat Insights) \u2013 once data is ingested, you\u2019ll see panels showing top attacker countries, a world map of attack sources, recent attack logs, etc., all derived from CrowdSec metrics.<\/em> (This dashboard comes from Grafana Labs ID 21689.)<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"462\" src=\"https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/grafana_dashboard.png\" alt=\"\" class=\"wp-image-178\" srcset=\"https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/grafana_dashboard.png 1000w, https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/grafana_dashboard-300x139.png 300w, https:\/\/vittrup-graversen.dk\/wp-content\/uploads\/2025\/04\/grafana_dashboard-768x355.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p>At this stage, Grafana and VictoriaMetrics are ready. The remaining work is to install CrowdSec on the target server and configure it to push its decisions to VictoriaMetrics.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 4: Install and Enroll CrowdSec on the Monitored Server<\/h2>\n\n\n\n<p>Now we set up <strong>CrowdSec<\/strong> on the server that we want to monitor (this could be a public-facing VPS, a local server, etc. \u2013 anywhere you have logs of services and want to detect attacks). If you already have CrowdSec running on your server, you can skip the installation steps and move to configuration.<\/p>\n\n\n\n<p><strong>Install CrowdSec:<\/strong> The CrowdSec team provides packages for easy installation. On a Debian\/Ubuntu system, you can enable the CrowdSec repository and install it via APT. For example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Add CrowdSec repository (via packagecloud script)\ncurl -s https:\/\/packagecloud.io\/install\/repositories\/crowdsec\/crowdsec\/script.deb.sh | sudo bash\n\n# Install CrowdSec\nsudo apt install crowdsec<\/code><\/pre>\n\n\n\n<p>Within a minute or two, CrowdSec should be installed and running as a system service (it starts automatically)\u200b. You can verify by checking the service status: <code>systemctl status crowdsec<\/code> (look for \u201cactive (running)\u201d). By default, CrowdSec will start parsing logs and detecting common attack scenarios (the installation comes with a \u201cbase collection\u201d of parsers and scenarios for SSH, web servers, etc., and uses a local SQLite database to store decisions).<\/p>\n\n\n\n<p><strong>Enroll in CrowdSec Console (optional):<\/strong> CrowdSec has a centralized Console (dashboard) and community blocklist sharing. Enrolling your instance with the CrowdSec Console isn\u2019t strictly required for our monitoring setup, but it is recommended if you want to contribute to\/benefit from CrowdSec\u2019s community threat intelligence. To enroll, you would create an account on app.crowdsec.net (if you haven\u2019t already) and retrieve an enrollment key. Then on your server, run a command like <code>sudo cscli console enroll &lt;your_enrollment_key&gt;<\/code>. This will register your CrowdSec instance with your Console account. (If you skip this, your CrowdSec still works locally, you just won\u2019t see it in the CrowdSec web Console \u2013 which is fine for our purposes.)<\/p>\n\n\n\n<p>At this point, CrowdSec is running and detecting attacks on your server. It\u2019s likely already banning some IPs (decisions) if, for example, you have an SSH server and someone is hammering it. However, by default CrowdSec\u2019s <strong>notification output<\/strong> is minimal \u2013 it won\u2019t automatically send these events to our VictoriaMetrics. We need to configure CrowdSec\u2019s notification plugin to forward decisions to our metrics store. We\u2019ll do that next.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 5: Configure CrowdSec to Push Metrics to VictoriaMetrics<\/h2>\n\n\n\n<p>This is the most critical part \u2013 we\u2019ll set up CrowdSec\u2019s <strong>HTTP notification plugin<\/strong> so that every time CrowdSec generates a new decision (e.g. bans an IP), it sends a JSON payload describing that decision to VictoriaMetrics. VictoriaMetrics will ingest these as time-series data points. The Grafana dashboard we imported is built to query and display those points.<\/p>\n\n\n\n<p>CrowdSec\u2019s notification system is configured via two files: <code>profiles.yaml<\/code> (which controls <em>when<\/em> to send notifications and to which plugin) and an HTTP plugin config (which controls <em>how<\/em> and <em>where<\/em> to send them). We will modify\/add to these files.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Enable the HTTP notification in profiles:<\/strong> Open CrowdSec\u2019s profiles configuration, typically at <code>\/etc\/crowdsec\/profiles.yaml<\/code>. Find the section for <code>notifications<\/code>. By default, you might see an example that is commented out. We need to ensure there is a notification entry for our HTTP plugin. For example, in <code>profiles.yaml<\/code> you can add (or uncomment) lines under the appropriate profile to include <code>http_default<\/code> as a notification channel. I&#8217;ve been tangling around with my setup and ended up with:<br><br><code>name: default_ip_remediation <\/code><br><code>filters: <\/code><br>      &#8211; <code>Alert.Remediation == true &amp;&amp; Alert.GetScope() == \"Ip\" <\/code><br><code>decisions: <\/code><br>     &#8211; <code>type: ban <\/code><br>     &#8211; <code>duration: 4h <\/code><br><code>notifications: <\/code><br>     &#8211; <code>http_default <\/code><br><code>on_success: break<br><\/code><\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-preformatted has-small-font-size\"><strong>(Note: I have trouble with above formatting. Ensure that it's valid yaml!)<\/strong><\/pre>\n\n\n\n<p>This tells CrowdSec to use the notification plugin named <strong>\u201chttp_default\u201d<\/strong> for alerts that match this profile. In the default CrowdSec configuration, the last profile usually matches all alerts\/decisions, so adding <code>http_default<\/code> there will make sure <em>every<\/em> decision triggers our HTTP plugin. (If you installed CrowdSec fresh, there may already be an <code>http_default<\/code> notification in profiles.yaml that\u2019s just commented out \u2013 simply remove the <code>#<\/code> to enable it\u200b or use mine as above).<\/p>\n\n\n\n<p>2. <strong>Configure the HTTP plugin (http.yaml):<\/strong> Now edit the file <code>\/etc\/crowdsec\/notifications\/http.yaml<\/code>. If it doesn\u2019t exist, create it. This is where we define the HTTP request that CrowdSec will make for each alert\/decision. We will set it up to call VictoriaMetrics. A minimal configuration for our purpose will look like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>type: http\nname: http_default\nmethod: POST\nurl: http:\/\/&lt;Victoriametrics_IP&gt;:8428\/api\/v1\/import\nheaders:\n  Content-Type: application\/json\n#  Authorization: \"Basic &lt;base64encodedcreds&gt;\"   # (not needed unless VM requires auth)\nformat: &gt; \n  {{- range $alert := . -}}\n  {{- range .Decisions -}}\n  {\"metric\":{\"__name__\":\"cs_lapi_decision\",\n             \"instance\":\"&lt;YOUR_INSTANCE_NAME&gt;\",\n             \"country\":\"{{$alert.Source.Cn}}\",\n             \"asname\":\"{{$alert.Source.AsName}}\",\n             \"asnumber\":\"{{$alert.Source.AsNumber}}\",\n             \"latitude\":\"{{$alert.Source.Latitude}}\",\n             \"longitude\":\"{{$alert.Source.Longitude}}\",\n             \"iprange\":\"{{$alert.Source.Range}}\",\n             \"scenario\":\"{{.Scenario}}\",\n             \"type\":\"{{.Type}}\",\n             \"scope\":\"{{.Scope}}\",\n             \"ip\":\"{{.Value}}\"\n            },\n   \"values\":&#91;1],\n   \"timestamps\":&#91;{{now|unixEpoch}}000]}\n  {{- end }}\n  {{- end -}}\n<\/code><\/pre>\n\n\n\n<p>Let\u2019s break down what\u2019s happening here:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>type\/name:<\/strong> We define an HTTP type notification named <code>http_default<\/code> (the name must match what we put in profiles.yaml).<\/li>\n\n\n\n<li><strong>url:<\/strong> This is the endpoint to send data to. We use VictoriaMetrics\u2019s <code>\/api\/v1\/import<\/code> HTTP endpoint\u200b. Replace <code>&lt;Victoriametrics_IP&gt;<\/code> with the address (or hostname) of your VM server. If VictoriaMetrics uses a different port or HTTPS, adjust accordingly. (For a local test where CrowdSec and VM are on the same machine, you could use <code>http:\/\/localhost:8428\/api\/v1\/import<\/code>.)<\/li>\n\n\n\n<li><strong>headers:<\/strong> We set <code>Content-Type: application\/json<\/code> because we\u2019ll send JSON. We leave out authorization since our VM instance isn\u2019t behind auth. If you secured your VictoriaMetrics with basic auth or an API token, you\u2019d include an <code>Authorization<\/code> header here. Otherwise, <strong>make sure to remove or comment out the <code>Authorization<\/code> line entirely<\/strong>\u200b.<\/li>\n\n\n\n<li><strong>format:<\/strong> This is a Go template that produces the JSON payload. It iterates over each alert and each decision in that alert (usually one alert contains one decision in our context) and formats a JSON object. The JSON is structured in a way VictoriaMetrics expects for the import API (essentially in <strong>JSON Lines<\/strong> format, one metric per line). In the above template:\n<ul class=\"wp-block-list\">\n<li><code>\"__name__\": \"cs_lapi_decision\"<\/code> sets the metric name for all events. We use <strong><code>cs_lapi_decision<\/code><\/strong> as the metric name (short for \u201cCrowdSec Local API decision\u201d). This is the same metric name the Grafana dashboard expects.<\/li>\n\n\n\n<li><code>\"instance\": \"&lt;YOUR_INSTANCE_NAME&gt;\"<\/code> is a label to identify which server this data came from \u2013 you can put anything like <code>\"web01\"<\/code> or a hostname here. For example, in the template from the inspiration article it\u2019s set to <code>host00.domain.tld<\/code>\u200b. Use a value that makes sense for you (especially if you will have multiple servers sending to one VM).<\/li>\n\n\n\n<li>The other fields like <code>country<\/code>, <code>asname<\/code>, <code>asnumber<\/code>, etc. are grabbing information from CrowdSec\u2019s alert object (<code>$alert.Source<\/code>). CrowdSec enriches alerts with GeoIP and ASN info, so these will include the country code, the ISP\/organization name, the AS number of the source IP, latitude\/longitude, and the IP range (prefix) if available. Similarly, <code>scenario<\/code> is the CrowdSec scenario that triggered (e.g. <code>crowdsecurity\/ssh-bf<\/code>), <code>type<\/code> will usually be \u201cban\u201d (or challenge, etc.), <code>scope<\/code> is typically \u201cIp\u201d, and <code>ip<\/code> is the malicious IP address itself\u200b. All these become labels on the metric.<\/li>\n\n\n\n<li>We set <code>\"values\": [1]<\/code> and a timestamp (epoch in milliseconds) for each decision\u200b. Using a value of 1 for each event is a simple trick: it means each record contributes a count of 1. This way, when Grafana visualizes the data, it can count events, sum them, etc., treating them as time-series occurrences\u200b.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>The end result is that every decision (ban) from CrowdSec will generate a metric data point named <code>cs_lapi_decision{instance=\"X\", country=\"Y\", scenario=\"Z\", ...} = 1<\/code>. In VictoriaMetrics, if you query this metric you\u2019ll be able to filter and aggregate by these labels \u2013 which is exactly what our Grafana dashboard does.<\/p>\n\n\n\n<p>3. <strong>Apply the configuration:<\/strong> Once you have updated <code>profiles.yaml<\/code> and <code>http.yaml<\/code>, save the files. Double-check that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>In <code>profiles.yaml<\/code>, the <code>http_default<\/code> notification is uncommented and in the right place (under the profile that should trigger on decisions).<\/li>\n\n\n\n<li>In <code>http.yaml<\/code>, the placeholders have been replaced with real values (your VM URL, an instance name, etc.), and there are no syntax errors (YAML spacing matters). If you copied the template, ensure the indentations are correct.<\/li>\n<\/ul>\n\n\n\n<p>4. <strong>Restart CrowdSec:<\/strong> For the changes to take effect, restart the CrowdSec service. On Linux: <code>sudo systemctl restart crowdsec<\/code>. This will cause CrowdSec to load the new notification configuration. You can check CrowdSec\u2019s log (<code>\/var\/log\/crowdsec.log<\/code>) to see if it mentions loading the HTTP plugin without errors.<\/p>\n\n\n\n<p>After restart, CrowdSec will now attempt to send an HTTP POST to VictoriaMetrics for every new decision. To ensure it\u2019s working, we should test it with a sample event.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 6: Testing the Integration (Verifying Data Flow)<\/h2>\n\n\n\n<p>With everything in place, it\u2019s time to test whether our pipeline works: CrowdSec \u2192 VictoriaMetrics \u2192 Grafana.<\/p>\n\n\n\n<p><strong>Trigger a test decision:<\/strong> The easiest way to do this is to manually add a ban through CrowdSec. This simulates a detected threat. We can use CrowdSec\u2019s command-line tool (<code>cscli<\/code>) to insert a decision. For example, on the CrowdSec server run:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo cscli decisions add --ip 1.2.3.4 --duration 4h --reason \"test-scenario\" --scope Ip<\/code><\/pre>\n\n\n\n<p>Replace <code>1.2.3.4<\/code> with any IP of your choice (perhaps your own IP or a dummy). This command tells CrowdSec to treat <em>1.2.3.4<\/em> as if it was detected attacking and to ban it for 4 hours, with a reason label of &#8220;test-scenario&#8221;. When you execute this, CrowdSec will create a new decision internally\u200b<a href=\"https:\/\/docs.crowdsec.net\/docs\/cscli\/cscli_decisions_add#:~:text=cscli%20decisions%20add%20,value%20foobar\" target=\"_blank\" rel=\"noreferrer noopener\">docs.crowdsec.net<\/a>. Importantly, because we enabled the HTTP notification, it should immediately send our JSON payload to VictoriaMetrics.<\/p>\n\n\n\n<p><strong>Check VictoriaMetrics:<\/strong> To confirm VictoriaMetrics received the data, you can query it. If you still have the VictoriaMetrics UI open (<code>http:\/\/&lt;VM_server&gt;:8428\/vmui<\/code>), try running a query for the metric name. In the VM UI\u2019s query box, enter:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cs_lapi_decision&#91;1h]<\/code><\/pre>\n\n\n\n<p>and execute it (you might set a time range to the last 5 minutes to narrow it). If everything worked, you should see at least one data point. You can also query with a filter, for example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cs_lapi_decision{ip=\"1.2.3.4\"}&#91;5m]<\/code><\/pre>\n\n\n\n<p>to specifically look for the metric with the IP label you added. In the VM UI\u2019s <strong>JSON<\/strong> tab, you\u2019d see an output confirming the data, e.g. something like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&#91;\n  {\n    \"metric\": {\n      \"__name__\": \"cs_lapi_decision\",\n      \"instance\": \"web01\",\n      \"country\": \"??\",\n      \"asname\": \"&lt;ASN name&gt;\",\n      \"asnumber\": \"&lt;ASN number&gt;\",\n      \"scenario\": \"test-scenario\",\n      \"type\": \"ban\",\n      \"scope\": \"Ip\",\n      \"ip\": \"1.2.3.4\"\n    },\n    \"values\": &#91; 1 ],\n    \"timestamps\": &#91; 1692873600000 ]\n  }\n]<\/code><\/pre>\n\n\n\n<p>The exact labels will vary; for a private IP or an IP not in the GeoIP database, <code>country<\/code> might be empty or <code>??<\/code>. If you used a real public IP, CrowdSec\u2019s GeoIP enrichment should have filled in the country, ASN, etc., which would show up here. The key is that we have a <code>cs_lapi_decision<\/code> data point in VictoriaMetrics. <\/p>\n\n\n\n<p><strong>Check Grafana Dashboard:<\/strong> Now head back to Grafana and view the imported <strong>CrowdSec Cyber Threat Insights<\/strong> dashboard. Make sure the time range at the top includes \u201cLast 5 minutes\u201d (or whichever interval covers the current time) and hit the refresh icon. You should start to see data appearing. For instance:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The \u201cRealtime Cyberthreats\u201d table should now list the decision we added. It will show the timestamp, the <strong>Country<\/strong> (if resolved, or it might be blank if unknown), <strong>ASN<\/strong> info, the <strong>IP<\/strong> (<code>1.2.3.4<\/code>), Decision (<code>ban<\/code>), and Scenario (<code>test-scenario<\/code>). This confirms that Grafana (via VictoriaMetrics) is receiving the data.<\/li>\n\n\n\n<li>Other panels like \u201cTop 10 Cyberthreat Countries\u201d might not show much with just one event (or it might list an \u201cunknown\u201d country if that IP wasn\u2019t resolved). As more events come in, those will populate.<\/li>\n\n\n\n<li>If the map or pie chart remains empty from a single test, don\u2019t worry \u2013 those need a bit more data (e.g., at least one real geo-locatable IP). You can repeat the test with a couple of different IP addresses and reasons to generate sample points (for example, try one with <code>--ip 5.6.7.8 --reason \"crowdsecurity\/ssh-bf\"<\/code> to mimic an SSH brute-force scenario from another IP).<\/li>\n<\/ul>\n\n\n\n<p>Over time, as CrowdSec runs and detects real attacks, this dashboard will become incredibly insightful. You\u2019ll start seeing which hosts were attacked the most in the last month, which countries dominate your threat feed, and a scrollable list of all recent threats. Grafana\u2019s interactive features (built into this dashboard) allow you to click on a country or ASN in the tables and filter the entire dashboard for that selection \u2013 so you can drill down on, say, \u201cshow me all attacks from China\u201d with one click\u200b. There are also data links in some panels that can take you to external lookup tools (for example, clicking an ASN might offer a link to a regional internet registry lookup\u200b or CrowdSec\u2019s Threat Intelligence search).<\/p>\n\n\n\n<p><strong>Using the VictoriaMetrics GUI and Grafana Explore:<\/strong> Besides the dashboard, remember that you can use VictoriaMetrics\u2019s <strong>VMUI<\/strong> for ad-hoc queries (as we did) and Grafana\u2019s <strong>Explore<\/strong> mode to query and inspect the data. This can be useful for troubleshooting or crafting custom queries. For instance, in Grafana Explore you could run a PromQL query like <code>sum(rate(cs_lapi_decision[1h])) by (country)<\/code> to see which countries have the most attacks in the last hour (assuming enough data). The combination of VMUI for quick JSON output and Grafana for graphing gives you a lot of flexibility to verify and analyze the metrics.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>By integrating CrowdSec with VictoriaMetrics and Grafana, we\u2019ve built a powerful monitoring solution that turns raw security events into meaningful visuals and stats in real time. This setup provides immediate situational awareness of your cyber threats: you can identify trends (e.g., a surge of SSH attacks from a particular country), see the impact of your defenses (which IPs got banned and when), and gain context on each attacker (via geoIP and ASN info). All of this is achieved with open-source tools running on your own servers, making it an attractive solution for those who prefer self-hosting and full data control.<\/p>\n\n\n\n<p><strong>Who benefits?<\/strong> If you\u2019re a system administrator, a devops engineer, or a homelabber running internet-facing services, this integration gives you a \u201cThreat Observatory\u201d dashboard for your infrastructure. You\u2019ll know, at a glance, what\u2019s happening security-wise \u2013 something that typically required expensive SIEM solutions or managed services. Now you have actionable insights: for example, you might notice a spike in a particular attack scenario and decide to tighten a firewall rule, or you might use the data to report abuse to ISPs if you see repeated offenders.<\/p>\n\n\n\n<p>Moreover, this setup complements CrowdSec\u2019s prevention capabilities. CrowdSec will still <strong>block<\/strong> malicious IPs in real-time via bouncers (don\u2019t forget to install a CrowdSec bouncer like the firewall bouncer, so that those \u201cban\u201d decisions are enforced on your server). What our monitoring stack adds is the <strong>observability<\/strong> aspect \u2013 the ability to see and understand those security events at scale and historically.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>don\u2019t forget to install a CrowdSec bouncer like the firewall bouncer, so that those \u201cban\u201d decisions are enforced on your server<\/p>\n<\/blockquote>\n\n\n\n<p>In summary, we set up CrowdSec as the brain detecting threats, VictoriaMetrics as the memory storing those threat events, and Grafana as the eyes visualizing the patterns. With minimal configuration and using largely off-the-shelf components, we achieved a real-time cybersecurity insights dashboard\u200b. This empowers you to be proactive and informed about attacks on your systems. Happy monitoring, and may your logs forever be full of insightful data rather than uninvited guests!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">References &amp; Useful Links:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>CrowdSec Official Website \u2013 <a href=\"http:\/\/crowdsec.net\" target=\"_blank\" rel=\"noreferrer noopener\">crowdsec.net<\/a> (for documentation and Console access)<\/li>\n\n\n\n<li>VictoriaMetrics Official Docs \u2013 <a href=\"http:\/\/victoriametrics.com\" target=\"_blank\" rel=\"noreferrer noopener\">victoriametrics.com<\/a> (for details on VM UI, API, etc.)<\/li>\n\n\n\n<li>Grafana Labs \u2013 <a href=\"http:\/\/grafana.com\" target=\"_blank\" rel=\"noreferrer noopener\">grafana.com<\/a> (for documentation on using Grafana, importing dashboards, etc.)<\/li>\n\n\n\n<li><em><a href=\"https:\/\/freefd.github.io\/articles\/8_cyber_threat_insights_with_crowdsec_victoriametrics_and_grafana\/\" target=\"_blank\" rel=\"noreferrer noopener\">Cyber threat insights with CrowdSec, VictoriaMetrics and Grafana<\/a><\/em> (inspiration for this guide)<\/li>\n\n\n\n<li>CrowdSec documentation on HTTP notifications &#8211; \u200b<a href=\"https:\/\/docs.crowdsec.net\/docs\/notification_plugins\/http#:~:text=In%20your%20profile%20file%20,%2C%20uncomment%20the%20section\" target=\"_blank\" rel=\"noreferrer noopener\">docs.crowdsec.net<\/a> (for advanced config and troubleshooting)<\/li>\n\n\n\n<li>Grafana Dashboard ID 21689: <a href=\"https:\/\/grafana.com\/grafana\/dashboards\/21689-crowdsec-cyber-threat-insights\/\" target=\"_blank\" rel=\"noopener\"><em>CrowdSec Cyber Threat Insights<\/em> <\/a>(available on Grafana Labs dashboards)<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Monitoring cyber threats with CrowdSec, VictoriaMetrics, and Grafana. A practical guide to open source security monitoring.<\/p>\n","protected":false},"author":1,"featured_media":188,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[9,19],"tags":[22,20,21],"class_list":["post-176","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-homelab","category-security","tag-crowdsec","tag-grafana","tag-victoriametrics"],"acf":[],"_links":{"self":[{"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/posts\/176","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/comments?post=176"}],"version-history":[{"count":14,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/posts\/176\/revisions"}],"predecessor-version":[{"id":1076,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/posts\/176\/revisions\/1076"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/media\/188"}],"wp:attachment":[{"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/media?parent=176"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/categories?post=176"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vittrup-graversen.dk\/index.php\/wp-json\/wp\/v2\/tags?post=176"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}